show-your-work
Code for the "Show Your Work" paper, EMNLP 2019.
Citation
If you use this repository for your research, please cite:
@inproceedings{showyourwork,
author = {Jesse Dodge and Suchin Gururangan and Dallas Card and Roy Schwartz and Noah A. Smith},
title = {Show Your Work: Improved Reporting of Experimental Results},
year = {2019},
booktitle = {Proceedings of EMNLP},
}
Installation
First, clone this repository.
Then, clone the allentune repository and install it on your system:
git clone https://github.com/allenai/allentune
cd allentune/
pip install --editable .
Run example
cd show-your-work/
allentune search --experiment-name cnn_search \
--search_space ./search_spaces/cnn_sst5.json \
--base-config ./training_config/cnn_classifier.jsonnet \
--cpus-per-trial 1 \
--num-gpus 0 \
--gpus-per-trial 0 \
--num-samples 1
--include-package show-your-work
If you have GPUs, set the --num-gpus flag and --gpus-per-trial flag appropriately.
Note: To run DGEM search on Scitail, you must use the implementation in https://github.com/allenai/scitail, which notably depends on a previous
version of allennlp. Clone that repository, copy the search_spaces/dgem.json and training_config/dgem.jsonnet files over, and run
allentune search --experiment-name dgem_search \
--search_space ./search_spaces/dgem.json \
--base-config ./training_config/dgem.jsonnet \
--cpus-per-trial 1 \
--num-gpus 0 \
--gpus-per-trial 0 \
--num-samples 1
--include-package scitail