Source code: Neural Architectures for Nested NER through Linearization

Jana Straková, Milan Straka and Jan Hajič https://aclweb.org/anthology/papers/P/P19/P19-1527/ {strakova,straka,hajic}@ufal.mff.cuni.cz

License

Copyright 2019 Institute of Formal and Applied Linguistics, Faculty of Mathematics and Physics, Charles University, Czech Republic.

This Source Code Form is subject to the terms of the Mozilla Public License, v. 2.0. If a copy of the MPL was not distributed with this file, You can obtain one at http://mozilla.org/MPL/2.0/.

Please cite as:

@inproceedings{strakova-etal-2019-neural, title = {{Neural Architectures for Nested {NER} through Linearization}}, author = {Jana Strakov{'a} and Milan Straka and Jan Haji\v{c}}, booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics}, month = jul, year = {2019}, address = {Florence, Italy}, publisher = {Association for Computational Linguistics}, url = {https://www.aclweb.org/anthology/P19-1527}, pages = {5326--5331}, }

How to run the tagger

1. Install requirements

pip install -r requirements.txt

2. Download the data

ACE-2004: https://catalog.ldc.upenn.edu/LDC2005T09 ACE-2005: https://catalog.ldc.upenn.edu/LDC2006T06 GENIA: http://www.geniaproject.org/

3. Create inputs

The input of the tagger is in the CoNLL-2003 BILOU format. CoNLL-2003 shared task data format is described here: https://www.clips.uantwerpen.be/conll2003/ner/ . BILOU format is described here (Ratinov and Roth, 2009): https://www.aclweb.org/anthology/W09-1119 .

The input format is a CoNLL format, with one token per line, sentences delimited by empty line. For each token, columns are separated by tabs. First column is the surface token, second column is lemma, third column is a POS tag and fourth column is the BILOU encoded NE label.

For flat corpora (e.g. CoNLL-2003 English and German), the fourth column bears exactly one NE label, e.g. (example from CoNLL-2003 English):

-DOCSTART- -docstart- NN O

EU EU NNP U-ORG rejects reject VBZ O German german JJ U-MISC call call NN O to to TO O boycott boycott VB O British british JJ U-MISC lamb lamb NN O . . . O

For nested NE corpora, the NE tags are linearized (flattened) according to rules described in the paper, e.g. (example from ACE-2004):

The the DT B-GPE Chinese chinese JJ I-GPE|U-GPE government government NN L-GPE and and CC O the the DT B-GPE Australian australian JJ I-GPE|U-GPE government government NN L-GPE signed sign VBD O an an DT O agreement agreement NN O today today NN O , , , O wherein wherein WRB O the the DT B-GPE Australian australian JJ I-GPE|U-GPE party party NN L-GPE would would MD O provide provide VB O China China NNP U-GPE with with IN O a a DT O preferential preferential JJ O financial financial JJ O loan loan NN O of of IN O 150 150 CD O million million CD O Australian australian JJ U-GPE dollars dollar NNS O . . . O

The lemmatization and POS tagging can be done with e.g. UDPipe (http://ufal.mff.cuni.cz/udpipe) or with MorphoDiTa (http://ufal.mff.cuni.cz/morphodita) or with any tool of your choice. If you don't have any POS tagger or lemmatizer, simply fill the respective columns with dummy (e.g. "_").

4. Get word embeddings

• word2vec,
• FastText,
• BERT,
• ELMo,
• Flair

from sources described in the paper. The input formats are:

• word2vec: The native word2vec text file.
• FastText: The native FastText binary.
• contextualized embeddings (BERT, ELMo, Flair): A text file with one token per line, first column is the token, all other columns are the vector real valued numbers; columns separated with space. The format is readable for human eyes, but quite large, sorry for the inconvenience. The per-token BERT contextualized word embeddings are created as an average of all token corresponding BERT subowords. The ELMo and Flair are generated using this code: https://github.com/zalandoresearch/flair.

You can also run the tagger without pretrained word embeddings just with end-to-end word embeddings and character-level embeddings (created inside the tagger), or with a subset of the above mentioned pretrained word embeddings.

5. Run the tagger

Usage example:

./tagger.py --corpus=CoNLL_en --train_data=conll_en/train_dev_bilou.conll --test_data=conll_en/test_bilou.conll --decoding=seq2seq --epochs=10:1e-3,8:1e-4 --form_wes_model=word_embeddings/conll_en_form.txt --lemma_wes_model=word_embeddings/conll_en_lemma.txt --bert_embeddings_train=bert_embeddings/conll_en_train_dev_bert_large_embeddings.txt --bert_embeddings_test=bert_embeddings/conll_en_test_bert_large_embeddings.txt --flair_train=flair_embeddings/conll_en_train_dev.txt --flair_test=flair_embeddings/conll_en_test.txt --elmo_train=elmo_embeddings/conll_en_train_dev.txt --elmo_test=elmo_embeddings/conll_en_test.txt --name=seq2seq+ELMo+BERT+Flair