IEDB Analysis Resource

MHC I Automated Server Benchmarks


This is a live ranking of MHC I servers based on performance, which continues to be reevaluated over time. The weekly IEDB releases are automatically checked for datasets large enough to add to the benchmarks. The benchmark metrics in the table below will only be updated on releases where such new data is becoming available.

Accumulated overall ranking scores

Ranking scores based on data sets submitted to the IEDB for the last at least 5 references.

Server 2021-06-11 2021-05-21 2021-04-16 2021-03-05 2021-02-19 2021-01-29 2021-01-15 2021-01-01 2020-12-25 2020-12-18 2020-12-11 2020-11-13 2020-10-02 2020-09-11 2020-07-10 2020-04-24 2020-03-13 2019-11-01 2019-10-04 2019-08-30 2019-06-21 2019-06-07 2019-05-24 2019-03-15 2019-02-01 2018-05-11 2018-01-26 2017-09-01 2017-03-23 2016-12-09 2016-05-03 2016-02-19 2015-08-07 2015-07-31 2015-07-17 2015-06-26 2015-06-19 2015-05-15 2015-02-06 2015-01-16 2014-10-31 2014-06-20 2014-05-23 2014-03-28 2014-03-21
mhcflurry 1.2.0 84 84 53 59 52 35 79 29 58 58 39 15 0 0 0 0 - - - 71 90 85 80 80 70 63 61 61 70 69 80 88 88 90 92 92 92 93 93 83 55 90 88 86 85
NetMHCpan 2.8 75 73 51 51 55 49 52 51 52 48 55 65 63 65 69 72 56 63 61 60 57 47 42 46 53 66 68 66 67 63 66 67 72 55 56 53 54 50 52 51 43 65 67 61 60
NetMHCcons 72 67 73 68 72 73 62 66 69 57 60 70 68 70 72 73 67 73 81 84 76 68 59 60 67 73 78 80 78 73 70 61 68 66 68 69 71 67 65 64 71 56 63 64 63
ANN 3.4 68 65 78 76 80 84 70 68 76 58 61 62 58 59 67 66 78 78 88 78 62 62 54 58 68 75 80 81 78 82 74 65 77 67 69 65 68 64 63 59 59 58 65 46 48
SMMPMBEC 66 55 71 79 65 71 56 41 55 73 78 67 64 61 57 50 73 73 38 29 34 32 35 47 52 58 65 65 72 91 77 76 81 54 48 46 44 37 36 35 53 59 46 40 41
ANN 4.0 61 57 60 58 62 67 68 65 70 67 64 51 47 41 41 37 53 53 75 78 85 86 91 77 66 60 54 56 71 66 65 68 76 58 55 58 59 56 58 56 55 78 72 68 69
NetMHCpan 3.0 60 59 59 57 65 68 69 71 70 68 59 58 55 56 61 66 68 57 59 58 57 64 70 63 64 65 58 62 62 64 66 81 83 70 65 63 59 56 56 60 56 58 53 59 59
DeepSeqPa 60 61 57 58 35 21 21 40 79 61 61 45 37 51 40 40 60 60 53 73 27 21 21 21 8 - - - - - - - - - - - - - - - - - - - -
NetMHCpan 4.0 58 63 53 49 62 62 69 76 74 76 68 72 70 71 71 74 69 74 70 74 77 69 78 72 65 65 54 55 60 52 47 64 70 62 66 64 60 57 56 65 62 55 54 62 61
SMM 48 46 54 63 56 63 51 35 47 63 74 66 63 52 47 41 43 43 42 32 38 38 39 37 39 40 45 51 49 49 36 26 51 54 58 51 50 44 43 42 53 61 50 40 40
IEDB Consensus 40 49 66 74 66 68 54 43 57 62 68 59 53 43 44 36 45 45 65 48 40 48 51 44 45 51 51 59 62 56 36 24 55 46 48 48 47 37 39 37 48 52 48 59 58
PickPocket 36 25 26 20 28 36 26 43 53 51 53 55 51 66 56 60 59 53 20 25 12 21 23 35 35 47 50 44 57 52 47 47 53 33 36 32 33 27 24 29 25 64 62 46 42
ARB 24 42 73 74 60 62 36 38 67 51 38 51 46 53 53 56 - - - 0 50 38 26 26 19 9 10 10 30 25 25 33 55 31 34 29 30 28 29 30 18 52 52 24 21

Weekly ranking scores

Ranking scores based on data sets submitted to the IEDB within a single week.

Server 2021-06-11 2021-05-21 2021-04-16 2021-03-05 2021-02-19 2021-01-29 2021-01-15 2021-01-01 2020-12-25 2020-12-18 2020-12-11 2020-11-13 2020-10-02 2020-09-11 2020-07-10 2020-04-24 2020-03-13 2019-11-01 2019-10-04 2019-08-30 2019-06-21 2019-06-07 2019-05-24 2019-03-15 2019-02-01 2018-05-11 2018-01-26 2017-09-01 2017-03-23 2016-12-09 2016-05-03 2016-02-19 2015-08-07 2015-07-31 2015-07-17 2015-06-26 2015-06-19 2015-05-15 2015-02-06 2015-01-16 2014-10-31 2014-06-20 2014-05-23 2014-03-28 2014-03-21
mhcflurry 1.2.0 100 100 50 88 100 14 79 - - - 29 88 - - 0 0 - - - - - - - 71 90 68 - 50 89 10 59 - 95 87 86 - 89 93 92 95 55 91 95 91 85
NetMHCpan 2.8 91 100 58 50 78 31 54 82 56 10 50 88 59 32 46 73 - 70 70 0 100 64 72 63 22 45 75 74 70 60 56 - 95 60 77 50 89 46 54 52 43 64 86 82 60
NetMHCcons 91 73 92 33 69 83 54 73 68 50 71 88 32 45 64 80 - 60 70 75 100 86 78 79 49 57 72 89 74 65 88 - 95 60 41 50 89 69 70 63 71 18 82 82 63
ANN 3.4 73 73 100 25 64 98 83 55 78 55 75 96 64 0 73 66 - 70 - 85 - 89 67 63 43 61 75 92 59 85 100 - 95 46 55 - 95 64 66 62 59 32 36 36 48
SMMPMBEC 73 73 29 88 42 98 88 18 49 75 17 88 100 86 100 46 - 70 - 75 - 19 0 33 51 30 64 61 49 90 82 - 95 100 32 - 89 40 36 35 53 91 55 18 41
ANN 4.0 46 73 83 42 42 71 63 64 66 80 50 88 73 45 64 38 - 30 - 75 - 75 100 67 94 93 42 35 89 40 55 - 95 100 9 - 89 52 59 57 55 100 73 36 69
NetMHCpan 3.0 55 73 96 17 54 65 79 91 54 65 92 88 36 59 18 65 - 53 80 75 0 58 67 88 60 77 42 73 65 65 36 - 95 72 77 100 89 54 58 56 56 91 9 82 59
DeepSeqPa - 73 29 79 63 - 33 9 - - 71 88 9 77 - 33 - 80 30 100 - - - 46 8 - - - - - - - - - - - - - - - - - - - -
NetMHCpan 4.0 41 100 92 0 62 52 67 100 51 75 100 92 46 68 36 82 - 60 70 85 100 47 67 96 75 80 50 56 70 70 16 - 95 49 55 88 89 60 52 58 62 64 9 100 61
SMM 23 73 29 67 32 73 92 27 52 30 13 88 100 86 82 47 - 10 - 75 - 25 11 25 61 25 19 39 35 85 68 - 0 9 68 - 89 49 41 42 53 64 55 18 40
IEDB Consensus 27 - 33 58 54 83 79 46 33 60 33 100 82 36 91 44 - 0 - 90 - 53 28 17 63 61 14 36 61 85 73 - 9 27 36 - 100 36 44 33 48 36 36 0 58
PickPocket 23 73 42 8 2 42 0 36 44 5 88 88 18 55 18 64 - 100 10 75 25 0 44 8 32 7 81 27 47 85 24 - 95 34 82 0 70 27 31 24 25 0 77 82 42
ARB 0 0 0 96 57 100 8 0 - 100 13 88 0 55 36 56 - - - - - - - 0 50 2 - 0 26 0 0 - 100 0 0 - 89 25 20 37 18 9 36 82 21

Benchmark results get recalculated as new servers are added to the benchmarks. Numbers in light grey represent scores on datasets before the server was incorporated into the benchmarks. Previous versions of the benchmark results can be downloaded below