-
Notifications
You must be signed in to change notification settings - Fork 23
Data
The TAC entity linking data is available to participants in the entity linking track of NIST's knowledge base population shared task. The data format is described briefly below. For more details, see the entity linking task definition.
In 2014, systems must provide two files: (1) an xml file containing entity mentions and (2) a tab file containing linking and nil clustering output.
The mention xml file includes a query element for each mention. This element must have an id attribute with a unique value as well as docid (document identifier), beg (start offset), end (end offset) elements:
<kbpentlink>
<query id="EDL14_ENG_TRAINING_0001">
<name>Xenophon</name>
<docid>bolt-eng-DF-170-181122-8792777</docid>
<beg>22103</beg>
<end>22110</end>
</query>
<query id="EDL14_ENG_TRAINING_0002">
<name>Richmond</name>
<docid>APW_ENG_20090826.0903</docid>
<beg>340</beg>
<end>347</end>
</query>
...
</kbpentlink>Note that offsets should be character offsets over the utf8-encoded sgml source files. The end offset should be the last character that is included in the span.
The tab-separated link ID file includes a line for each mention. Each line includes several fields: query_id (matching the id attribute on a query element in the corresponding mentions xml file), kb_or_nil_id (a knowledge base or nil cluster identifier), entity_type (the type is required for 2014 link evaluation), and score (a confidence value, optional):
EDL14_ENG_TRAINING_0001 NIL0001 PER 1.0
EDL14_ENG_TRAINING_0002 E0604067 GPE 1.0
Note that it is possible to provide more than one response for a given mention by adding extra lines. However, the current set of evaluation measures only consider one response per mention (the one with the highest score).
Before 2014, the mention xml was provided and systems only need to output a tab-separated link ID file containing query_id, kb_or_nil_id, and score fields. To evaluate on these data sets, first add a ne_type field as per the 2014 format. Then use the gold xml file when converting system output to evaluation format.
Note that when using 2011 data, the end offset is the first character that is not part of the span (rather than the last character that is included in the span).