Evaluation

Named Entity Recognition Performance

The model file that comes with MITIE (ner_model_just_conll.dat) gets an F1 score of 88.10 on the CoNLL 2003 NER task. This is measured on the eng.testb test dataset. The model was trained on the train/val data from the CoNLL 2003 task.

For comparison, this is slightly better than the Stanford NER tool.

Named Entity Recognition Speed

The NER tool in MITIE runs at a reasonable speed. We compared it to the Stanford NER tool and MITIE is over three times faster. We measured this by running part of the English Gigaword corpus through both tools and measuring the total processing time. On the computer we used, MITIE ran at 53,600 words a second while the Stanford NER tool ran at 16,000 words a second.