pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference

Introduction

This repository contains the code for replicating results from

• pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference
• Mandar Joshi, Eunsol Choi, Omer Levy, Dan Weld, and Luke Zettlemoyer

Getting Started

• Install python3 requirements: pip install -r requirements.txt

Using pretrained pair2vec embeddings

• Download pretrained pair2vec: ./download_pair2vec.sh
  • If you want to reproduce results from the paper on QA/NLI, please use the following:
    • Download and extract the pretrained models tar file
    • Run evaluation:

  python -m allennlp.run evaluate [--output-file OUTPUT_FILE]
                               --cuda-device 0
                               --include-package endtasks
                               ARCHIVE_FILE INPUT_FILE
  

  • If you want to train your own QA/NLI model:

  python -m allennlp.run train <config_file> -s <serialization_dir> --include-package endtasks
  

See the experiments directory for relevant config files.

Training your own embeddings

• Download the preprocessed corpus if you want to train pair2vec from scratch: ./download_corpus.sh
• Training: This starts the training process which typically takes 7-10 days. It takes in a config file and a directory to save checkpoints.

python -m embeddings.train --config experiments/pair2vec_train.json --save_path <directory>

Miscellaneous

• If you use the code, please cite the following paper

@inproceedings{joshi-etal-2019-pair2vec,
    title = "pair2vec: Compositional Word-Pair Embeddings for Cross-Sentence Inference",
    author = "Joshi, Mandar  and
      Choi, Eunsol  and
      Levy, Omer  and
      Weld, Daniel  and
      Zettlemoyer, Luke",
    booktitle = "Proceedings of the 2019 Conference of the North {A}merican Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)",
    month = jun,
    year = "2019",
    address = "Minneapolis, Minnesota",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/N19-1362",
    pages = "3597--3608"
}