Machine Translation Reading List

This is a machine translation reading list maintained by the Tsinghua Natural Language Processing Group.

The past three decades have witnessed the rapid development of machine translation, especially for data-driven approaches such as statistical machine translation (SMT) and neural machine translation (NMT). Due to the dominance of NMT at the present time, priority is given to collecting important, up-to-date NMT papers; the Edinburgh/JHU MT research survey wiki has good coverage of older papers and a brief description for each sub-topic of MT. Our list is still incomplete and the categorization might be inappropriate. We will keep adding papers and improving the list. Any suggestions are welcome!

• 10 Must Reads
• Tutorials and Surveys
• Statistical Machine Translation
  • Word-based Models
  • Phrase-based Models
  • Syntax-based Models
  • Discriminative Training
  • System Combination
  • Human-centered SMT
    • Interactive SMT
    • Adaptation
• Evaluation
• Neural Machine Translation
  • Model Architecture
  • Attention Mechanism
  • Open Vocabulary
  • Training Objectives and Frameworks
  • Decoding
  • Low-resource Language Translation
    • Semi-supervised Learning
    • Unsupervised Learning
    • Pivot-based Methods
    • Data Augmentation
    • Data Selection
    • Transfer Learning
    • Meta Learning
  • Multilingual Machine Translation
  • Prior Knowledge Integration
    • Word/Phrase Constraints
    • Syntactic/Semantic Constraints
    • Coverage Constraints
  • Document-level Translation
  • Robustness
  • Interpretability
  • Linguistic Interpretation
  • Fairness and Diversity
  • Efficiency
  • Non-Autoregressive Translation
  • Speech Translation and Simultaneous Translation
  • Multi-modality
  • Ensemble and Reranking
  • Pre-training
  • Domain Adaptation
  • Quality Estimation
  • Human-centered NMT
    • Interactive NMT
    • Automatic Post-Editing
  • Poetry Translation
  • Eco-friendly
  • Endangered Language Revitalization
• Word Translation (Bilingual Lexicon Induction)
• WMT Winners
  • WMT 2019
  • WMT 2018
  • WMT 2017
  • WMT 2016

10 Must Reads

• Peter E. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. The Mathematics of Statistical Machine Translation: Parameter Estimation. Computational Linguistics. (Citation: 5,218)
• Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a Method for Automatic Evaluation of Machine Translation. In Proceedings of ACL 2002. (Citation: 10,700)
• Philipp Koehn, Franz J. Och, and Daniel Marcu. 2003. Statistical Phrase-Based Translation. In Proceedings of NAACL 2003. (Citation: 3,713)
• Franz Josef Och. 2003. Minimum Error Rate Training in Statistical Machine Translation. In Proceedings of ACL 2003. (Citation: 3,115)
• David Chiang. 2007. Hierarchical Phrase-Based Translation. Computational Linguistics. (Citation: 1,235)
• Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to Sequence Learning with Neural Networks. In Proceedings of NIPS 2014. (Citation: 9,432)
• Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural Machine Translation by Jointly Learning to Align and Translate. In Proceedings of ICLR 2015. (Citation: 10,479)
• Diederik P. Kingma, Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In Proceedings of ICLR 2015. (Citation: 37,480)
• Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural Machine Translation of Rare Words with Subword Units. In Proceedings of ACL 2016. (Citation: 1,679)
• Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. In Proceedings of NIPS 2017. (Citation: 6,112)

Tutorials and Surveys

• Zhixing Tan, Shuo Wang, Zonghan Yang, Gang Chen, Xuancheng Huang, Maosong Sun, and Yang Liu. 2020. Neural Machine Translation: A Review of Methods, Resources, and Tools. AI Open.
• Felix Stahlberg. 2020. Neural Machine Translation: A Review and Survey. Journal of Artificial Intelligence Research.
• Philipp Koehn and Rebecca Knowles. 2017. Six Challenges for Neural Machine Translation. In Proceedings of the First Workshop on Neural Machine Translation.
• Philipp Koehn. 2017. Neural Machine Translation. arxiv:1709.07809.
• Oriol Vinyals and Navdeep Jaitly. 2017. Seq2Seq ICML Tutorial. ICML 2017 Tutorial.
• Raj Dabre, Chenhui Chu, and Anoop Kunchukuttan. 2020. A Survey of Multilingual Neural Machine Translation. ACM Computing Surveys. Surv. 53, 5, Article 99 (October 2020).
  • Tutorial on Multilingual Neural Machine Translation at COLING 2020.
• Graham Neubig. 2017. Neural Machine Translation and Sequence-to-sequence Models: A Tutorial. arXiv:1703.01619. (Citation: 45)
• Thang Luong, Kyunghyun Cho, and Christopher Manning. 2016. Neural Machine Translation. ACL 2016 Tutorial.
• Adam Lopez. 2008. Statistical Machine Translation. ACM Computing Surveys.
• Philipp Koehn. 2006. Statistical Machine Translation: the Basic, the Novel, and the Speculative. EACL 2006 Tutorial.

Statistical Machine Translation

Word-based Models

• Peter E. Brown, Stephen A. Della Pietra, Vincent J. Della Pietra, and Robert L. Mercer. 1993. The Mathematics of Statistical Machine Translation: Parameter Estimation. Computational Linguistics. (Citation: 4,965)
• Stephan Vogel, Hermann Ney, and Christoph Tillmann. 1996. HMM-Based Word Alignment in Statistical Translation. In Proceedings of COLING 1996. (Citation: 940)
• Franz Josef Och and Hermann Ney. 2003. A Systematic Comparison of Various Statistical Alignment Models. Computational Linguistics. (Citation: 3,980)
• Percy Liang, Ben Taskar, and Dan Klein. 2006. Alignment by Agreement. In Proceedings of NAACL 2006. (Citation: 452)
• Chris Dyer, Victor Chahuneau, and Noah A. Smith. 2013. A Simple, Fast, and Effective Reparameterization of IBM Model 2. In Proceedings of NAACL 2013. (Citation: 310)

Phrase-based Models

• Philipp Koehn, Franz J. Och, and Daniel Marcu. 2003. Statistical Phrase-Based Translation. In Proceedings of NAACL 2003. (Citation: 3,516)
• Michel Galley and Christopher D. Manning. 2008. A Simple and Effective Hierarchical Phrase Reordering Model. In Proceedings of EMNLP 2008. (Citation: 275)

Syntax-based Models

• Dekai Wu. 1997. Stochastic Inversion Transduction Grammars and Bilingual Parsing of Parallel Corpora. Computational Linguistics. (Citation: 1,009)
• Michel Galley, Jonathan Graehl, Kevin Knight, Daniel Marcu, Steve DeNeefe, Wei Wang, and Ignacio Thayer. 2006. Scalable Inference and Training of Context-Rich Syntactic Translation Models. In Proceedings of COLING/ACL 2006. (Citation: 475)
• Yang Liu, Qun Liu, and Shouxun Lin. 2006. Tree-to-String Alignment Template for Statistical Machine Translation. In Proceedings of COLING/ACL 2006. (Citation: 391)
• Deyi Xiong, Qun Liu, and Shouxun Lin. 2006. Maximum Entropy Based Phrase Reordering Model for Statistical Machine Translation. In Proceedings of COLING/ACL 2006. (Citation: 299)
• David Chiang. 2007. Hierarchical Phrase-Based Translation. Computational Linguistics. (Citation: 1,192)
• Liang Huang and David Chiang. 2007. Forest Rescoring: Faster Decoding with Integrated Language Models. In Proceedings of ACL 2007. (Citation: 280)
• Haitao Mi, Liang Huang, and Qun Liu. 2008. Forest-based Translation. In Proceedings of ACL 2008. (Citation: 239)
• Min Zhang, Hongfei Jiang, Aiti Aw, Haizhou Li, Chew Lim Tan, and Sheng Li. 2008. A Tree Sequence Alignment-based Tree-to-Tree Translation Model. In Proceedings of ACL 2008. (Citation: 124)
• Libin Shen, Jinxi Xu, and Ralph Weischedel. 2008. A New String-to-Dependency Machine Translation Algorithm with a Target Dependency Language Model. In Proceedings of ACL 2008. (Citation: 278)
• Haitao Mi and Liang Huang. 2008. Forest-based Translation Rule Extraction. In Proceedings of EMNLP 2008. (Citation: 239)
• Yang Liu, Yajuan Lü, and Qun Liu. 2009. Improving Tree-to-Tree Translation with Packed Forests. In Proceedings of ACL/IJNLP 2009. (Citation: 93)
• David Chiang. 2010. Learning to Translate with Source and Target Syntax. In Proceedings of ACL 2010. (Citation: 118)

Discriminative Training

• Franz Josef Och and Hermann Ney. 2002. Discriminative Training and Maximum Entropy Models for Statistical Machine Translation. In Proceedings of ACL 2002. (Citation: 1,258)
• Franz Josef Och. 2003. Minimum Error Rate Training in Statistical Machine Translation. In Proceedings of ACL 2003. (Citation: 2,984)
• Taro Watanabe, Jun Suzuki, Hajime Tsukada, and Hideki Isozaki. 2007. Online Large-Margin Training for Statistical Machine Translation. In Proceedings of EMNLP-CoNLL 2007. (Citation: 197)
• David Chiang, Kevin Knight, and Wei Wang. 2009. 11,001 New Features for Statistical Machine Translation. In Proceedings of NAACL 2009. (Citation: 251)

System Combination

• Antti-Veikko Rosti, Spyros Matsoukas, and Richard Schwartz. 2007. Improved Word-Level System Combination for Machine Translation. In Proceedings of ACL 2007. (Citation: 144)
• Xiaodong He, Mei Yang, Jianfeng Gao, Patrick Nguyen, and Robert Moore. 2008. Indirect-HMM-based Hypothesis Alignment for Combining Outputs from Machine Translation Systems. In Proceedings of EMNLP 2008. (Citation: 96)

Human-centered SMT

Interactive SMT

• George Foster, Pierre Isabelle and Pierre Plamondon. 1997. Target-text mediated interactive machine translation. Machine Translation. (Citation: 116)
• Philippe Langlais, Guy Lapalme and Marie Lorange. 2002. TransType: Development-Evaluation Cycles to Boost Translator’s Productivity. Machine Translation. (Citation: 74)
• Jesús Tomas and Francisco Casacuberta. 2006. Statistical phrase-based models for interactive computer-assisted translation. In Proceedings of COLING/ACL. (Citation: 31)
• Enrique Vidal, Francisco Casacuberta, Luis Rodríguez-Ruiz, Jorge Civera, Carlos D. Martínez-Hinarejos. 2006. Computer-Assisted Translation Using Speech Recognition. IEEE Transaction on Audio, Speech and Language Processing. (Citation: 62)
• Shahram Khadivi and Hermann Ney. 2008. Integration of Speech Recognition and Machine Translation in Computer-Assisted Translation. IEEE Transaction on Audio, Speech and Language Processing. (Citation: 30)
• Sergio Barrachina, Oliver Bender, Francisco Casacuberta, Jorge Civera, Elsa Cubel, Shahram Khadivi, Antonio L. Lagarda, Hermann Ney, Jesús Tomás and Enrique Vidal. 2009. Statistical approaches to computer-assisted translation. Computational Linguistics. (Citation: 207)
• Francisco Casacuberta, Jorge Civera, Elsa Cubel, Antonio L. Lagarda, Guy Lapalme, Elliott Macklovitch, Enrique Vidal. 2009. Human interaction for high quality machine translation. Communications of the ACM. (Citation: 49)
• Vicent Alabau, Alberto Sanchis and Francisco Casacuberta. 2014. Improving on-line handwritten recognition in interactive machine translation. Pattern Recognition. (Citation: 18)
• Shanbo Cheng, Shujian Huang, Huadong Chen, Xin-Yu Dai and Jiajun Chen. 2016. PRIMT: A Pick-Revise Framework for Interactive Machine Translation. In Proceedings of NAACL 2016. (Citation: 9)
• Miguel Domingo, Álvaro Peris and Francisco Casacuberta. 2018. Segment-based interactive-predictive machine translation. Machine Translation. (Citation: 2)

Adaptation

• Pascual Martínez-Gómez, Germán Sanchis-Trilles and Francisco Casacuberta. 2012. Online adaptation strategies for statistical machine translation in post-editing scenarios. Pattern Recognition. (Citation: 40)
• Jesús González-Rubio and Francisco Casacuberta. 2014. Cost-Sensitive Active Learning for Computer-Assisted Translation. Pattern Recognition Letters. (Citation: 11)
• Antonio L. Lagarda, Daniel Ortiz-Martínez, Vicent Alabau and Francisco Casacuberta. 2015. Translating without in-domain corpus: Machine translation post-editing with online learning techniques. Computer Speech & Language. (Citation: 10)
• Germán Sanchis-Trilles, Francisco Casacuberta. 2015. Improving translation quality stability using Bayesian predictive adaptation. Computer Speech & Language. (Citation: 1)
• Daniel Ortiz-Martínez. 2016. Online Learning for Statistical Machine Translation. Computational Linguistics. (Citation: 13)

Evaluation

• Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: a Method for Automatic Evaluation of Machine Translation. In Proceedings of ACL 2002. (Citation: 8,499)
• Philipp Koehn. 2004. Statistical Significance Tests for Machine Translation Evaluation. In Proceedings of EMNLP 2004. (Citation: 1,015)
• Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization. (Citation: 1,355)
• Matthew Snover and Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A Study of Translation Edit Rate with Targeted Human Annotation. In Proceedings of AMTA 2006. (Citation: 1,713)
• Maja Popovic. 2015. chrF: Character n-gram F-score for Automatic MT Evaluation. In Proceedings of WMT 2015. (Citation: 58)
• Xin Wang, Wenhu Chen, Yuan-Fang Wang, and William Yang Wang. 2018. No Metrics Are Perfect: Adversarial Reward Learning for Visual Storytelling. In Proceedings of ACL 2018. (Citation: 10)
• Arun Tejasvi Chaganty, Stephen Mussman, and Percy Liang. 2018. The price of debiasing automatic metrics in natural language evaluation. In Proceedings of ACL 2018.
• Graham Neubig, Zi-Yi Dou, Junjie Hu, Paul Michel, Danish Pruthi, and Xinyi Wang. 2019. compare-mt: A Tool for Holistic Comparison of Language Generation Systems. In Proceedings of NAACL 2019.
• Robert Schwarzenberg, David Harbecke, Vivien Macketanz, Eleftherios Avramidis, and Sebastian Möller. 2019. Train, Sort, Explain: Learning to Diagnose Translation Models. In Proceedings of NAACL 2019.
• Nitika Mathur, Timothy Baldwin, and Trevor Cohn. 2019. Putting Evaluation in Context: Contextual Embeddings Improve Machine Translation Evaluation. In Proceedings of ACL 2019.
• Prathyusha Jwalapuram, Shafiq Joty, Irina Temnikova, and Preslav Nakov. 2019. Evaluating Pronominal Anaphora in Machine Translation: An Evaluation Measure and a Test Suite. In Proceedings of ACL 2019.
• Sergey Edunov, Myle Ott, Marc’Aurelio Ranzato and Michael Auli. 2020. On The Evaluation of Machine Translation Systems Trained With Back-Translation. In Proceedings of ACL 2020.
• Wei Zhao, Goran Glavaš, Maxime Peyrard, Yang Gao, Robert West and Steffen Eger. 2020. On the Limitations of Cross-lingual Encoders as Exposed by Reference-Free Machine Translation Evaluation. In Proceedings of ACL 2020.
• Marina Fomicheva, Lucia Specia, and Francisco Guzmán. 2020. Multi-Hypothesis Machine Translation Evaluation. In Proceedings of ACL 2020.
• Kosuke Takahashi, Katsuhito Sudoh and Satoshi Nakamura. 2020. Automatic Machine Translation Evaluation using Source Language Inputs and Cross-lingual Language Model. In Proceedings of ACL 2020.
• Nitika Mathur, Timothy Baldwin and Trevor Cohn. 2020. Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics. In Proceedings of ACL 2020.
• Markus Freitag, David Grangier, Isaac Caswell. 2020. BLEU might be Guilty but References are not Innocent. In Proceedings of EMNLP 2020.
• Yvette Graham, Barry Haddow, Philipp Koehn. 2020. Statistical Power and Translationese in Machine Translation Evaluation. In Proceedings of EMNLP 2020.
• Brian Thompson, Matt Post. 2020. Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing. In Proceedings of EMNLP 2020.
• Ricardo Rei, Craig Stewart, Ana C Farinha, Alon Lavie. 2020. COMET: A Neural Framework for MT Evaluation. In Proceedings of EMNLP 2020.

Neural Machine Translation

Model Architecture

• Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent Continuous Translation Models. In Proceedings of EMNLP 2013. (Citation: 623)
• Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to Sequence Learning with Neural Networks. In Proceedings of NIPS 2014. (Citation: 5,452)
• Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural Machine Translation by Jointly Learning to Align and Translate. In Proceedings of ICLR 2015. (Citation: 5,596)
• Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Łukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2016. Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. In Proceedings of NIPS 2016. (Citation: 1,046)
• Jie Zhou, Ying Cao, Xuguang Wang, Peng Li, and Wei Xu. 2016. Deep Recurrent Models with Fast-Forward Connections for Neural Machine Translation. Transactions of the Association for Computational Linguistics. (Citation: 73)
• Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O.K. Li. 2016. Incorporating Copying Mechanism in Sequence-to-Sequence Learning. In Proceedings of ACL 2016. (Citation: 254)
• Biao Zhang, Deyi Xiong, Jinsong Su, Hong Duan, and Min Zhang. 2016. Variational Neural Machine Translation. In Proceedings of EMNLP 2016. (Citation: 38)
• Nal Kalchbrenner, Lasse Espeholt, Karen Simonyan, Aaron van den Oord, Alex Graves, and Koray Kavukcuoglu. 2016. Neural Machine Translation in Linear Time. arXiv:1610.10099. (Citation: 189)
• Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. 2017. Convolutional Sequence to Sequence Learning. In Proceedings of ICML 2017. (Citation: 453)
• Jonas Gehring, Michael Auli, David Grangier, and Yann Dauphin. 2017. A Convolutional Encoder Model for Neural Machine Translation. In Proceedings of ACL 2017. (Citation: 85)
• Mingxuan Wang, Zhengdong Lu, Jie Zhou, and Qun Liu. 2017. Deep Neural Machine Translation with Linear Associative Unit. In Proceedings of ACL 2017. (Citation: 21)
• Matthias Sperber, Graham Neubig, Jan Niehues, and Alex Waibel. 2017. Neural Lattice-to-Sequence Models for Uncertain Inputs. In Proceedings of EMNLP 2017. (Citation: 11)
• Denny Britz, Anna Goldie, Minh-Thang Luong, and Quoc Le. 2017. Massive Exploration of Neural Machine Translation Architectures. In Proceedings of EMNLP 2017. (Citation: 114)
• Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. In Proceedings of NIPS 2017. (Citation: 1,748)
• Yingce Xia, Fei Tian, Lijun Wu, Jianxin Lin, Tao Qin, Nenghai Yu, and Tie-Yan Liu. 2017. Deliberation Networks: Sequence Generation Beyond One-Pass Decoding. In Proceedings of NIPS 2017. (Citation: 38)
• Zhaopeng Tu, Yang Liu, Lifeng Shang, Xiaohua Liu, and Hang Li. 2017. Neural machine translation with reconstruction. In Proceedings of AAAI 2017. (Citation: 75)
• Lukasz Kaiser, Aidan N. Gomez, and Francois Chollet. 2018. Depthwise Separable Convolutions for Neural Machine Translation. In Proceedings of ICLR 2018. (Citation: 27)
• Yanyao Shen, Xu Tan, Di He, Tao Qin, and Tie-Yan Liu. 2018. Dense Information Flow for Neural Machine Translation. In Proceedings of NAACL 2018. (Citation: 3)
• Wenhu Chen, Guanlin Li, Shuo Ren, Shujie Liu, Zhirui Zhang, Mu Li, and Ming Zhou. 2018. Generative Bridging Network for Neural Sequence Prediction. In Proceedings of NAACL 2018. (Citation: 3)
• Mia Xu Chen, Orhan Firat, Ankur Bapna, Melvin Johnson, Wolfgang Macherey, George Foster, Llion Jones, Mike Schuster, Noam Shazeer, Niki Parmar, Ashish Vaswani, Jakob Uszkoreit, Lukasz Kaiser, Zhifeng Chen, Yonghui Wu, and Macduff Hughes. 2018. The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation. In Proceedings of ACL 2018. (Citation: 22)
• Weiyue Wang, Derui Zhu, Tamer Alkhouli, Zixuan Gan, and Hermann Ney. 2018. Neural Hidden Markov Model for Machine Translation. In Proceedings of ACL 2018. (Citation: 3)
• Jingjing Gong, Xipeng Qiu, Shaojing Wang, and Xuanjing Huang. 2018. Information Aggregation via Dynamic Routing for Sequence Encoding. In COLING 2018.
• Qiang Wang, Fuxue Li, Tong Xiao, Yanyang Li, Yinqiao Li, and Jingbo Zhu. 2018. Multi-layer Representation Fusion for Neural Machine Translation. In Proceedings of COLING 2018.
• Yachao Li, Junhui Li, and Min Zhang. 2018. Adaptive Weighting for Neural Machine Translation. In Proceedings of COLING 2018.
• Kaitao Song, Xu Tan, Di He, Jianfeng Lu, Tao Qin, and Tie-Yan Liu. 2018. Double Path Networks for Sequence to Sequence Learning. In Proceedings of COLING 2018.
• Zi-Yi Dou, Zhaopeng Tu, Xing Wang, Shuming Shi, and Tong Zhang. 2018. Exploiting Deep Representations for Neural Machine Translation. In Proceedings of EMNLP 2018. (Citation: 1)
• Biao Zhang, Deyi Xiong, Jinsong Su, Qian Lin, and Huiji Zhang. 2018. Simplifying Neural Machine Translation with Addition-Subtraction Twin-Gated Recurrent Networks. In Proceedings of EMNLP 2018.
• Gongbo Tang, Mathias Müller, Annette Rios, and Rico Sennrich. 2018. Why Self-Attention? A Targeted Evaluation of Neural Machine Translation Architectures. In Proceedings of EMNLP 2018. (Citation: 6)
• Ke Tran, Arianna Bisazza, and Christof Monz. 2018. The Importance of Being Recurrent for Modeling Hierarchical Structure. In Proceedings of EMNLP 2018. (Citation: 6)
• Parnia Bahar, Christopher Brix, and Hermann Ney. 2018. Towards Two-Dimensional Sequence to Sequence Model in Neural Machine Translation. In Proceedings of EMNLP 2018. (Citation: 1)
• Tianyu He, Xu Tan, Yingce Xia, Di He, Tao Qin, Zhibo Chen, and Tie-Yan Liu. 2018. Layer-Wise Coordination between Encoder and Decoder for Neural Machine Translation. In Proceedings of NeurIPS 2018. (Citation: 2)
• Harshil Shah and David Barber. 2018. Generative Neural Machine Translation. In Proceedings of NeurIPS 2018.
• Hany Hassan, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark, Christian Federmann, Xuedong Huang, Marcin Junczys-Dowmunt, William Lewis, Mu Li, Shujie Liu, Tie-Yan Liu, Renqian Luo, Arul Menezes, Tao Qin, Frank Seide, Xu Tan, Fei Tian, Lijun Wu, Shuangzhi Wu, Yingce Xia, Dongdong Zhang, Zhirui Zhang, and Ming Zhou. 2018. Achieving Human Parity on Automatic Chinese to English News Translation. Technical report. Microsoft AI & Research. (Citation: 41)
• Yikang Shen, Shawn Tan, Alessandro Sordoni, and Aaron Courville. 2019. Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks. In Proceedings of ICLR 2019.
• Felix Wu, Angela Fan, Alexei Baevski, Yann Dauphin, and Michael Auli. 2019. Pay Less Attention with Lightweight and Dynamic Convolutions. In Proceedings of ICLR 2019. (Citation: 1)
• Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, Lukasz Kaiser. 2019. Universal Transformers. In Proceedings of ICLR 2019. (Citation: 12)
• Zi-Yi Dou, Zhaopeng Tu, Xing Wang, Longyue Wang, Shuming Shi, and Tong Zhang. 2019. Dynamic Layer Aggregation for Neural Machine Translation with Routing-by-Agreement. In Proceedings of AAAI 2019.
• Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V. Le, and Ruslan Salakhutdinov. 2019. Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context. In Proceedings of ACL 2019. (Citation: 8)
• Qipeng Guo, Xipeng Qiu, Pengfei Liu, Yunfan Shao, Xiangyang Xue, and Zheng Zhang. 2019. Star-Transformer. In Proceedings of NAACL 2019.
• Sho Takase and Naoaki Okazaki. 2019. Positional Encoding to Control Output Sequence Length. In Proceedings of NAACL 2019.
• Jian Li, Baosong Yang, Zi-Yi Dou, Xing Wang, Michael R. Lyu, and Zhaopeng Tu. 2019. Information Aggregation for Multi-Head Attention with Routing-by-Agreement. In Proceedings of NAACL 2019.
• Baosong Yang, Longyue Wang, Derek Wong, Lidia S. Chao, and Zhaopeng Tu. 2019. Convolutional Self-Attention Networks. In Proceedings of NAACL 2019.
• Jie Hao, Xing Wang, Baosong Yang, Longyue Wang, Jinfeng Zhang, and Zhaopeng Tu. 2019. Modeling Recurrence for Transformer. In Proceedings of NAACL 2019.
• Nikolaos Pappas and James Henderson. 2019. Deep Residual Output Layers for Neural Language Generation. In Proceedings of ICML 2019.
• David R. So, Chen Liang, and Quoc V. Le. 2019. The Evolved Transformer. In Proceedings of ICML 2019.
• Ben Peters, Vlad Niculae, and André F.T. Martins. 2019. Sparse Sequence-to-Sequence Models. In Proceedings of ACL 2019.
• Roberto Dessì and Marco Baroni. 2019. CNNs found to jump around more skillfully than RNNs: Compositional generalization in seq2seq convolutional networks. In Proceedings of ACL 2019.
• Sainbayar Sukhbaatar, Edouard Grave, Piotr Bojanowski, and Armand Joulin. 2019. Adaptive Attention Span in Transformers. In Proceedings of ACL 2019.
• Yi Tay, Aston Zhang, Luu Anh Tuan, Jinfeng Rao, Shuai Zhang, Shuohang Wang, Jie Fu, and Siu Cheung Hui. 2019. Lightweight and Efficient Neural Natural Language Processing with Quaternion Networks. In Proceedings of ACL 2019.
• Qiang Wang, Bei Li, Tong Xiao, Jingbo Zhu, Changliang Li, Derek F. Wong, and Lidia S. Chao. 2019. Learning Deep Transformer Models for Machine Translation. In Proceedings of ACL 2019.
• Fengshun Xiao, Jiangtong Li, Hai Zhao, Rui Wang, and Kehai Chen. 2019. Lattice-Based Transformer Encoder for Neural Machine Translation. In Proceedings of ACL 2019.
• Matthias Sperber, Graham Neubig, Ngoc-Quan Pham, and Alex Waibel. 2019. Self-Attentional Models for Lattice Inputs. In Proceedings of ACL 2019.
• Xing Wang, Zhaopeng Tu, Longyue Wang, and Shuming Shi. 2019. Exploiting Sentential Context for Neural Machine Translation. In Proceedings of ACL 2019.
• Kris Korrel, Dieuwke Hupkes, Verna Dankers, and Elia Bruni. 2019. Transcoding compositionally: using attention to find more generalizable solutions. In Proceedings of ACL 2019.
• Lijun Wu, Yiren Wang, Yingce Xia, Fei Tian, Fei Gao, Tao Qin, Jianhuang Lai, and Tie-Yan Liu. 2019. Depth Growing for Neural Machine Translation. In Proceedings of ACL 2019.
• Shaojie Bai, J. Zico Kolter, and Vladlen Koltun. 2019. Deep Equilibrium Models. In Proceedings of NeurIPS 2019.
• Gonçalo M. Correia, Vlad Niculae, and André F.T. Martins. 2019. Adaptively Sparse Transformers. In Proceedings of EMNLP 2019.
• Yau-Shian Wang, Hung-Yi Lee, and Yun-Nung Chen. 2019. Tree Transformer: Integrating Tree Structures into Self-Attention. In Proceedings of EMNLP 2019.
• Mingxuan Wang, Jun xie, Zhixing Tan, Jinsong Su, Deyi Xiong and Lei Li. 2019. Towards Linear Time Neural Machine Translation with Capsule Networks. In Proceedings of EMNLP 2019.
• Biao Zhang, Ivan Titov and Rico Sennrich. 2019. Improving Deep Transformer with Depth-Scaled Initialization and Merged Attention. In Proceedings of EMNLP 2019.
• Jiatao Gu, Changhan Wang, Junbo Zhao. 2019. Levenshtein Transformer. In Proceedings of NeurIPS 2019.
• Xin Sheng, Linli Xu, Junliang Guo, Jingchang Liu, Ruoyu Zhao and Yinlong Xu. 2020. IntroVNMT: An Introspective Model for Variational Neural Machine Translation. In Proceedings of AAAI 2020.
• Jian Li, Xing Wang, Baosong Yang, Shuming Shi, Michael R. Lyu and Zhaopeng Tu. 2020. Neuron Interaction Based Representation Composition for Neural Machine Translation. In Proceedings of AAAI 2020.
• Benyou Wang, Donghao Zhao, Christina Lioma, Qiuchi Li, Peng Zhang, Jakob Grue Simonsen. 2020. Encoding word order in complex embeddings. In Proceedings of ICLR 2020.
• Nikita Kitaev, Lukasz Kaiser, Anselm Levskaya. 2020. Reformer: The Efficient Transformer. In Proceedings of ICLR 2020.
• Maha Elbayad, Jiatao Gu, Edouard Grave, Michael Auli. 2020. Depth-Adaptive Transformer. In Proceedings of ICLR 2020.
• Ofir Press, Noah A. Smith and Omer Levy. 2020. Improving Transformer Models by Reordering their Sublayers. In Proceedings of ACL 2020.
• Yekun Chai, Shuo Jin and Xinwen Hou. 2020. Highway Transformer: Self-Gating Enhanced Self-Attentive Networks. In Proceedings of ACL 2020.
• Xiangpeng Wei, Heng Yu, Yue Hu, Yue Zhang, Rongxiang Weng and Weihua Luo. 2020. Multiscale Collaborative Deep Models for Neural Machine Translation. In Proceedings of ACL 2020.
• Hendra Setiawan, Matthias Sperber, Udhyakumar Nallasamy and Matthias Paulik. 2020. Variational Neural Machine Translation with Normalizing Flows. In Proceedings of ACL 2020.
• Xuanqing Liu, Hsiang-Fu Yu, Inderjit Dhillon, and Cho-Jui Hsieh. 2020. Learning to Encode Position for Transformer with Continuous Dynamical Model. In Proceedings of ICML 2020.
• Ruibin Xiong, Yunchang Yang, Di He, Kai Zheng, Shuxin Zheng, Chen Xing, Huishuai Zhang, Yanyan Lan, Liwei Wang, and Tie-Yan Liu. 2020. On Layer Normalization in the Transformer Architecture. In Proceedings of ICML 2020.
• Thomas Bachlechner, Bodhisattwa Prasad Majumder, Huanru Henry Mao, Garrison W. Cottrell, and Julian McAuley. 2020. ReZero is All You Need: Fast Convergence at Large Depth. arXiv:2003.04887.
• Yongjing Yin, Fandong Meng, Jinsong Su, Chulun Zhou, Zhengyuan Yang, Jie Zhou and Jiebo Luo. 2020. A Novel Graph-based Multi-modal Fusion Encoder for Neural Machine Translation. In Proceedings of ACL 2020.
• Arya D. McCarthy, Xian Li, Jiatao Gu and Ning Dong. 2020. Addressing Posterior Collapse with Mutual Information for Improved Variational Neural Machine Translation. In Proceedings of ACL 2020.
• Yong Wang, Longyue Wang, Victor Li, Zhaopeng Tu. 2020. On the Sparsity of Neural Machine Translation Models. In Proceedings of EMNLP 2020.
• Bei Li, Ziyang Wang, Hui Liu, Yufan Jiang, Quan Du, Tong Xiao, Huizhen Wang, Jingbo Zhu. 2020. Shallow-to-Deep Training for Neural Machine Translation. In Proceedings of EMNLP 2020.
• Jianhao Yan, Fandong Meng, Jie Zhou. 2020. Multi-Unit Transformers for Neural Machine Translation. In Proceedings of EMNLP 2020.
• Xiangpeng Wei, Heng Yu, Yue Hu, Rongxiang Weng, Luxi Xing, Weihua Luo. 2020. Uncertainty-Aware Semantic Augmentation for Neural Machine Translation. In Proceedings of EMNLP 2020.
• Xian Li, Asa Cooper Stickland, Yuqing Tang, Xiang Kong. 2020. Deep Transformers with Latent Depth. In Proceedings of NeurIPS 2020.
• Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, Amr Ahmed. 2020. Big Bird: Transformers for Longer Sequences. In Proceedings of NeurIPS 2020.
• Shuhao Gu, Jinchao Zhang, Fandong Meng, Yang Feng, Wanying Xie, Jie Zhou, Dong Yu. 2020. Token-level Adaptive Training for Neural Machine Translation. In Proceedings of EMNLP 2020.
• Yufei Wang , Ian D. Wood , Stephen Wan , Mark Dras , Mark Johnson . 2021. Mention Flags (MF): Mention Flags (MF): Constraining Transformer-based Text Generators. In Proceedings of ACL 2021 .

Attention Mechanism

• Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural Machine Translation by Jointly Learning to Align and Translate. In Proceedings of ICLR 2015. (Citation: 5,596)
• Minh-Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective Approaches to Attention-based Neural Machine Translation. In Proceedings of EMNLP 2015. (Citation: 1,466)
• Shi Feng, Shujie Liu, Nan Yang, Mu Li, Ming Zhou, and Kenny Q. Zhu. 2016. Improving Attention Modeling with Implicit Distortion and Fertility for Machine Translation. In Proceedings of COLING 2016. (Citation: 18)
• Haitao Mi, Zhiguo Wang, and Abe Ittycheriah. 2016. Supervised Attentions for Neural Machine Translation. In Proceedings of EMNLP 2016. (Citation: 43)
• Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. 2017. A Structured Self-attentive Sentence Embedding. In Proceedings of ICLR 2017. (Citation: 216)
• Tao Shen, Tianyi Zhou, Guodong Long, Jing Jiang, Shirui Pan, and Chengqi Zhang. 2018. DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding. In Proceedings of AAAI 2018. (Citation: 60)
• Tao Shen, Tianyi Zhou, Guodong Long, Jing Jiang, and Chengqi Zhang. 2018. Bi-directional Block Self-attention for Fast and Memory-efficient Sequence Modeling. In Proceedings of ICLR 2018. (Citation: 13)
• Tao Shen, Tianyi Zhou, Guodong Long, Jing Jiang, Sen Wang, Chengqi Zhang. 2018. Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling. In Proceedings of IJCAI 2018. (Citation: 18)
• Peter Shaw, Jakob Uszkorei, and Ashish Vaswani. 2018. Self-Attention with Relative Position Representations. In Proceedings of NAACL 2018. (Citation: 24)
• Lesly Miculicich Werlen, Nikolaos Pappas, Dhananjay Ram, and Andrei Popescu-Belis. 2018. Self-Attentive Residual Decoder for Neural Machine Translation. In Proceedings of NAACL 2018. (Citation: 3)
• Xintong Li, Lemao Liu, Zhaopeng Tu, Shuming Shi, and Max Meng. 2018. Target Foresight Based Attention for Neural Machine Translation. In Proceedings of NAACL 2018.
• Biao Zhang, Deyi Xiong, and Jinsong Su. 2018. Accelerating Neural Transformer via an Average Attention Network. In Proceedings of ACL 2018. (Citation: 5)
• Tobias Domhan. 2018. How Much Attention Do You Need? A Granular Analysis of Neural Machine Translation Architectures. In Proceedings of ACL 2018. (Citation: 3)
• Shaohui Kuang, Junhui Li, António Branco, Weihua Luo, and Deyi Xiong. 2018. Attention Focusing for Neural Machine Translation by Bridging Source and Target Embeddings. In Proceedings of ACL 2018. (Citation: 1)
• Chaitanya Malaviya, Pedro Ferreira, and André F. T. Martins. 2018. Sparse and Constrained Attention for Neural Machine Translation. In Proceedings of ACL 2018. (Citation: 4)
• Jian Li, Zhaopeng Tu, Baosong Yang, Michael R. Lyu, and Tong Zhang. 2018. Multi-Head Attention with Disagreement Regularization. In Proceedings of EMNLP 2018. (Citation: 1)
• Wei Wu, Houfeng Wang, Tianyu Liu and Shuming Ma. 2018. Phrase-level Self-Attention Networks for Universal Sentence Encoding. In Proceedings of EMNLP 2018.
• Baosong Yang, Zhaopeng Tu, Derek F. Wong, Fandong Meng, Lidia S. Chao, and Tong Zhang. 2018. Modeling Localness for Self-Attention Networks. In Proceedings of EMNLP 2018. (Citation: 2)
• Junyang Lin, Xu Sun, Xuancheng Ren, Muyu Li, and Qi Su. 2018. Learning When to Concentrate or Divert Attention: Self-Adaptive Attention Temperature for Neural Machine Translation. In Proceedings of EMNLP 2018.
• Shiv Shankar, Siddhant Garg, and Sunita Sarawagi. 2018. Surprisingly Easy Hard-Attention for Sequence to Sequence Learning. In Proceedings of EMNLP 2018.
• Ankur Bapna, Mia Chen, Orhan Firat, Yuan Cao, and Yonghui Wu. 2018. Training Deeper Neural Machine Translation Models with Transparent Attention. In Proceedings of EMNLP 2018.
• Hareesh Bahuleyan, Lili Mou, Olga Vechtomova, and Pascal Poupart. 2018. Variational Attention for Sequence-to-Sequence Models. In Proceedings of COLING 2018. (Citation: 14)
• Maha Elbayad, Laurent Besacier, and Jakob Verbeek. 2018. Pervasive Attention: 2D Convolutional Neural Networks for Sequence-to-Sequence Prediction. In Proceedings of CoNLL 2018. (Citation: 4)
• Yuntian Deng, Yoon Kim, Justin Chiu, Demi Guo, and Alexander M. Rush. 2018 Latent Alignment and Variational Attention. In Proceedings of NeurIPS 2018. (Citation)
• Wenpeng Yin and Hinrich Schütze. 2019. Attentive Convolution: Equipping CNNs with RNN-style Attention Mechanisms. Transactions of the Association for Computational Linguistics.
• Shiv Shankar and Sunita Sarawagi. 2019. Posterior Attention Models for Sequence to Sequence Learning. In Proceedings of ICLR 2019.
• Baosong Yang, Jian Li, Derek Wong, Lidia S. Chao, Xing Wang, and Zhaopeng Tu. 2019. Context-Aware Self-Attention Networks. In Proceedings of AAAI 2019.
• Reza Ghaeini, Xiaoli Z. Fern, Hamed Shahbazi, and Prasad Tadepalli. 2019. Saliency Learning: Teaching the Model Where to Pay Attention. In Proceedings of NAACL 2019.
• Sameen Maruf, André F. T. Martins, and Gholamreza Haffari. 2019. Selective Attention for Context-aware Neural Machine Translation. In Proceedings of NAACL 2019.
• Sainbayar Sukhbaatar, Edouard Grave, Piotr Bojanowski, and Armand Joulin. 2019. Adaptive Attention Span in Transformers. In Proceedings of ACL 2019.
• Kris Korrel, Dieuwke Hupkes, Verna Dankers, and Elia Bruni. 2019. Transcoding compositionally: using attention to find more generalizable solutions. In Proceedings of ACL 2019.
• Jesse Vig. 2019. A Multiscale Visualization of Attention in the Transformer Model. In Proceedings of ACL 2019.
• Sathish Reddy Indurthi, Insoo Chung, and Sangha Kim. 2019. Look Harder: A Neural Machine Translation Model with Hard Attention. In Proceedings of ACL 2019.
• Mingzhou Xu, Derek F. Wong, Baosong Yang, Yue Zhang, and Lidia S. Chao. 2019. Leveraging Local and Global Patterns for Self-Attention Networks. In Proceedings of ACL 2019.
• Sarthak Jain and Byron C. Wallace. 2019. Attention is not Explanation. In Proceedings of NAACL 2019.
• Sarah Wiegreffe and Yuval Pinter. 2019. Attention is not not Explanation. In Proceedings of EMNLP 2019.
• Xing Wang, Zhaopeng Tu, Longyue Wang, and Shuming Shi. 2019. Self-Attention with Structural Position Representations. In Proceedings of EMNLP 2019.
• Yao-Hung Hubert Tsai, Shaojie Bai, Makoto Yamada, Louis-Philippe Morency, and Ruslan Salakhutdinov . 2019. Transformer Dissection: An Unified Understanding for Transformer's Attention via the Lens of Kernel. In Proceedings of EMNLP 2019.
• Kehai Chen, Rui Wang, Masao Utiyama and Eiichiro Sumita. 2019. Recurrent Position Embedding for Neural Machine Translation. In Proceedings of EMNLP 2019.
• Weiqiu You, Simeng Sun and Mohit Iyyer. 2020. Hard-Coded Gaussian Attention for Neural Machine Translation. In Proceedings of ACL 2020.
• Emanuele Bugliarello and Naoaki Okazaki. 2020. Enhancing Machine Translation with Dependency-Aware Self-Attention. In Proceedings of ACL 2020.
• Yann Dubois, Gautier Dagan, Dieuwke Hupkes, Elia Bruni. 2020. Location Attention for Extrapolation to Longer Sequences. In Proceedings of ACL 2020.
• Michael Hahn. 2020. Theoretical Limitations of Self-Attention in Neural Sequence Models. Transactions of the Association for Computational Linguistics.
• Apoorv Vyas, Angelos Katharopoulos, François Fleuret. 2020. Fast Transformers with Clustered Attention. In Proceedings of NeurIPS 2020.
• Chulhee Yun, Yin-Wen Chang, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank Reddi, Sanjiv Kumar. 2020. O(n) Connections are Expressive Enough: Universal Approximability of Sparse Transformers. In Proceedings of NeurIPS 2020.
• Yu Lu1 , Jiali Zeng , Jiajun Zhang , Shuangzhi Wu , Mu Li . 2021 . Attention Calibration for Transformer in Neural Machine Translation . In Proceedings of ACL 2021.

Open Vocabulary

• Felix Hill, Kyunghyun Cho, Sebastien Jean, Coline Devin, and Yoshua Bengio. 2015. Embedding Word Similarity with Neural Machine Translation. In Proceedings of ICLR 2015. (Citation: 24)
• Thang Luong, Ilya Sutskever, Quoc Le, Oriol Vinyals, and Wojciech Zaremba. 2015. Addressing the Rare Word Problem in Neural Machine Translation. In Proceedings of ACL 2015. (Citation: 367)
• Sébastien Jean, Kyunghyun Cho, Roland Memisevic, and Yoshua Bengio. 2015. On Using Very Large Target Vocabulary for Neural Machine Translation. In Proceedings of ACL 2015. (Citation: 455)
• Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural Machine Translation of Rare Words with Subword Units. In Proceedings of ACL 2016. (Citation: 795)
• Minh-Thang Luong and Christopher D. Manning. 2016. Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Models. In Proceedings of ACL 2016. (Citation: 173)
• Junyoung Chung, Kyunghyun Cho, and Yoshua Bengio. 2016. A Character-level Decoder without Explicit Segmentation for Neural Machine Translation. In Proceedings of ACL 2016. (Citation: 171)
• Jason Lee, Kyunghyun Cho, and Thomas Hofmann. 2017. Fully Character-Level Neural Machine Translation without Explicit Segmentation. Transactions of the Association for Computational Linguistics. (Citation: 116)
• Yang Feng, Shiyue Zhang, Andi Zhang, Dong Wang, and Andrew Abel. 2017. Memory-augmented Neural Machine Translation. In Proceedings of EMNLP 2017. (Citation: 9)
• Baosong Yang, Derek F. Wong, Tong Xiao, Lidia S. Chao, and Jingbo Zhu. 2017. Towards Bidirectional Hierarchical Representations for Attention-based Neural Machine Translation. In Proceedings of EMNLP 2017. (Citation: 5)
• Peyman Passban, Qun Liu, and Andy Way. 2018. Improving Character-Based Decoding Using Target-Side Morphological Information for Neural Machine Translation. In Proceedings of NAACL 2018. (Citation: 5)
• Huadong Chen, Shujian Huang, David Chiang, Xinyu Dai, and Jiajun Chen. 2018. Combining Character and Word Information in Neural Machine Translation Using a Multi-Level Attention. In Proceedings of NAACL 2018.
• Frederick Liu, Han Lu, and Graham Neubig. 2018. Handling Homographs in Neural Machine Translation. In Proceedings of NAACL 2018. (Citation: 8)
• Taku Kudo. 2018. Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates. In Proceedings of ACL 2018. (Citation: 17)
• Makoto Morishita, Jun Suzuki, and Masaaki Nagata. 2018. Improving Neural Machine Translation by Incorporating Hierarchical Subword Features. In Proceedings of COLING 2018.
• Yang Zhao, Jiajun Zhang, Zhongjun He, Chengqing Zong, and Hua Wu. 2018. Addressing Troublesome Words in Neural Machine Translation. In Proceedings of EMNLP 2018.
• Colin Cherry, George Foster, Ankur Bapna, Orhan Firat, and Wolfgang Macherey. 2018. Revisiting Character-Based Neural Machine Translation with Capacity and Compression. In Proceedings of EMNLP 2018. (Citation: 1)
• Rebecca Knowles and Philipp Koehn. 2018. Context and Copying in Neural Machine Translation. In Proceedings of EMNLP 2018.
• Matthias Huck, Viktor Hangya, and Alexander Fraser. 2019. Better OOV Translation with Bilingual Terminology Mining. In Proceedings of ACL 2019.
• Changhan Wang, Kyunghyun Cho, Jiatao Gu. 2020. Neural Machine Translation with Byte-Level Subwords. In Proceedings of AAAI 2020
• Duygu Ataman, Wilker Aziz, Alexandra Birch. 2020. A Latent Morphology Model for Open-Vocabulary Neural Machine Translation. In Proceedings of ICLR 2020.
• Xuanli He, Gholamreza Haffari and Mohammad Norouzi. 2020. Dynamic Programming Encoding for Subword Segmentation in Neural Machine Translation. In Proceedings of ACL 2020.
• Yingqiang Gao, Nikola I. Nikolov, Yuhuang Hu and Richard H.R. Hahnloser. 2020. Character-Level Translation with Self-attention. In Proceedings of ACL 2020.
• Ivan Provilkov, Dmitrii Emelianenko, Elena Voita. 2020. BPE-Dropout: Simple and Effective Subword Regularization. In Proceedings of ACL 2020.
• Jindřich Libovický, Alexander Fraser. 2020. Towards Reasonably-Sized Character-Level Transformer NMT by Finetuning Subword Systems. In Proceedings of EMNLP 2020.

Training Objectives and Frameworks

• Marc'Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2016. Sequence Level Training with Recurrent Neural Networks. In Proceedings of ICLR 2016. (Citation: 373)
• Minh-Thang Luong, Quoc V. Le, Ilya Sutskever, Oriol Vinyals, and Lukasz Kaiser. 2016. Multi-task Sequence to Sequence Learning. In Proceedings of ICLR 2016. (Citation: 282)
• Shiqi Shen, Yong Cheng, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. 2016. Minimum Risk Training for Neural Machine Translation. In Proceedings of ACL 2016. (Citation: 184)
• Sam Wiseman and Alexander M. Rush. 2016. Sequence-to-Sequence Learning as Beam-Search Optimization. In Proceedings of EMNLP 2016. (Citation: 141)
• Di He, Yingce Xia, Tao Qin, Liwei Wang, Nenghai Yu, Tie-Yan Liu, Wei-Ying Ma. 2016. Dual Learning for Machine Translation. In Proceedings of NIPS 2016. (Citation: 138)
• Dzmitry Bahdanau, Philemon Brakel, Kelvin Xu, Anirudh Goyal, Ryan Lowe, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2017. An Actor-Critic Algorithm for Sequence Prediction. In Proceedings of ICLR 2017. (Citation: 167)
• Julia Kreutzer, Artem Sokolov, Stefan Riezler. 2017. Bandit Structured Prediction for Neural Sequence-to-Sequence Learning. In Proceedings of ACL 2017. (Citation: 11)
• Yingce Xia, Tao Qin, Wei Chen, Jiang Bian, Nenghai Yu, and Tie-Yan Liu. 2017. Dual Supervised Learning. In Proceedings of ICML 2017. (Citation: 29)
• Yingce Xia, Jiang Bian, Tao Qin, Nenghai Yu, and Tie-Yan Liu. 2017. Dual Inference for Machine Learning. In Proceedings of IJCAI 2017. (Citation: 9)
• Di He, Hanqing Lu, Yingce Xia, Tao Qin, Liwei Wang, and Tieyan Liu. 2017. Decoding with Value Networks for Neural Machine Translation. In Proceedings of NIPS 2017. (Citation: 11)
• Sergey Edunov, Myle Ott, Michael Auli, David Grangier, and Marc’Aurelio Ranzato. 2018. Classical Structured Prediction Losses for Sequence to Sequence Learning. In Proceedings of NAACL 2018. (Citation: 20)
• Zihang Dai, Qizhe Xie, and Eduard Hovy. 2018. From Credit Assignment to Entropy Regularization: Two New Algorithms for Neural Sequence Prediction. In Proceedings of ACL 2018. (Citation: 1)
• Zhen Yang, Wei Chen, Feng Wang, and Bo Xu. 2018. Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets. In Proceedings of NAACL 2018. (Citation: 43)
• Kevin Clark, Minh-Thang Luong, Christopher D. Manning, and Quoc Le. 2018. Semi-Supervised Sequence Modeling with Cross-View Training. In Proceedings of EMNLP 2018.
• Lijun Wu, Fei Tian, Tao Qin, Jianhuang Lai, and Tie-Yan Liu. 2018. A Study of Reinforcement Learning for Neural Machine Translation. In Proceedings of EMNLP 2018. (Citation: 2)
• Semih Yavuz, Chung-Cheng Chiu, Patrick Nguyen, and Yonghui Wu. 2018. CaLcs: Continuously Approximating Longest Common Subsequence for Sequence Level Optimization. In Proceedings of EMNLP 2018.
• Lijun Wu, Fei Tian, Yingce Xia, Yang Fan, Tao Qin, Jianhuang Lai, and Tie-Yan Liu. 2018. Learning to Teach with Dynamic Loss Functions. In Proceedings of NeurIPS 2018.
• Yiren Wang, Yingce Xia, Tianyu He, Fei Tian, Tao Qin, ChengXiang Zhai, and Tie-Yan Liu. 2019. Multi-Agent Dual Learning. In Proceedings of ICLR 2019.
• Liqun Chen, Yizhe Zhang, Ruiyi Zhang, Chenyang Tao, Zhe Gan, Haichao Zhang, Bai Li, Dinghan Shen, Changyou Chen, and Lawrence Carin. 2019. Improving Sequence-to-Sequence Learning via Optimal Transport. In Proceedings of ICLR 2019.
• Sachin Kumar and Yulia Tsvetkov. 2019. Von Mises-Fisher Loss for Training Sequence to Sequence Models with Continuous Outputs. In Proceedings of ICLR 2019.
• Xing Niu, Weijia Xu, and Marine Carpuat. 2019. Bi-Directional Differentiable Input Reconstruction for Low-Resource Neural Machine Translation. In Proceedings of NAACL 2019.
• Weijia Xu, Xing Niu, and Marine Carpuat. 2019. Differentiable Sampling with Flexible Reference Word Order for Neural Machine Translation. In Proceedings of NAACL 2019.
• Inigo Jauregi Unanue, Ehsan Zare Borzeshi, Nazanin Esmaili, and Massimo Piccardi. ReWE: Regressing Word Embeddings for Regularization of Neural Machine Translation Systems. In Proceedings of NAACL 2019.
• Reuben Cohn-Gordon and Noah Goodman. 2019. Lost in Machine Translation: A Method to Reduce Meaning Loss. In Proceedings of NAACL 2019.
• Emmanouil Antonios Platanios, Otilia Stretcu, Graham Neubig, Barnabas Poczos, and Tom M. Mitchell. 2019. Competence-based Curriculum Learning for Neural Machine Translation. In Proceedings of NAACL 2019.
• Gaurav Kumar, George Foster, Colin Cherry, and Maxim Krikun. 2019. Reinforcement Learning based Curriculum Optimization for Neural Machine Translation. In Proceedings of NAACL 2019.
• Mitchell Stern, William Chan, Jamie Kiros, Jakob Uszkoreit. 2019. Insertion Transformer: Flexible Sequence Generation via Insertion Operations. In Proceedings of ICML 2019.
• Laura Jehl, Carolin Lawrence, and Stefan Riezler. 2019. Learning Neural Sequence-to-Sequence Models from Weak Feedback with Bipolar Ramp Loss. Transactions of the Association for Computational Linguistics.
• Motoki Sato, Jun Suzuki, and Shun Kiyono. 2019. Effective Adversarial Regularization for Neural Machine Translation. In Proceedings of ACL 2019.
• Kehai Chen, Rui Wang, Masao Utiyama, and Eiichiro Sumita. 2019. Neural Machine Translation with Reordering Embeddings. In Proceedings of ACL 2019.
• Bram Bulte and Arda Tezcan. 2019. Neural Fuzzy Repair: Integrating Fuzzy Matches into Neural Machine Translation. In Proceedings of ACL 2019.
• Mingming Yang, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita, Min Zhang, and Tiejun Zhao. 2019. Sentence-Level Agreement for Neural Machine Translation. In Proceedings of ACL 2019.
• Wen Zhang, Yang Feng, Fandong Meng, Di You, and Qun Liu. 2019. Bridging the Gap between Training and Inference for Neural Machine Translation. In Proceedings of ACL 2019.
• John Wieting, Taylor Berg-Kirkpatrick, Kevin Gimpel, and Graham Neubig. 2019. Beyond BLEU:Training Neural Machine Translation with Semantic Similarity. In Proceedings of ACL 2019.
• Zonghan Yang, Yong Cheng, Yang Liu, Maosong Sun. 2019. Reducing Word Omission Errors in Neural Machine Translation: A Contrastive Learning Approach. In Proceedings of ACL 2019.
• Kyra Yee, Nathan Ng, Yann N. Dauphin, and Michael Auli. 2019. Simple and Effective Noisy Channel Modeling for Neural Machine Translation. In Proceedings of EMNLP 2019.
• Sarthak Garg, Stephan Peitz, Udhyakumar Nallasamy, and Matthias Paulik. 2019. Jointly Learning to Align and Translate with Transformer Models. In Proceedings of EMNLP 2019.
• Tianchi Bi, Hao Xiong, Zhongjun He, Hua Wu and Haifeng Wang. 2019. Multi-agent Learning for Neural Machine Translation. In Proceedings of EMNLP 2019.
• Zaixiang Zheng, Shujian Huang, Zhaopeng Tu, Xin-Yu Dai, and Jiajun Chen. 2019. Dynamic Past and Future for Neural Machine Translation. In Proceedings of EMNLP 2019.
• Yiren Wang, Yingce Xia, Fei Tian, Fei Gao, Tao Qin, Cheng Xiang Zhai, Tie-Yan Liu. 2019. Neural Machine Translation with Soft Prototype. In Proceedings of NeurIPS 2019.
• Mingjun Zhao, Haijiang Wu, Di Niu and Xiaoli Wang. 2020. Reinforced Curriculum Learning on Pre-trained Neural Machine Translation Models. In Proceedings of AAAI 2020.
• Yang Feng, Wanying Xie, Shuhao Gu, Chenze Shao, Wen Zhang, Zhengxin Yang and Dong Yu. 2020. Modeling Fluency and Faithfulness for Diverse Neural Machine Translation. In Proceedings of AAAI 2020.
• Leshem Choshen, Lior Fox, Zohar Aizenbud, Omri Abend. 2020. On the Weaknesses of Reinforcement Learning for Neural Machine Translation. In Proceedings of ICLR 2020.
• Zaixiang Zheng, Hao Zhou, Shujian Huang, Lei Li, Xin-Yu Dai, Jiajun Chen. 2020. Mirror-Generative Neural Machine Translation. In Proceedings of ICLR 2020.
• Angela Fan, Edouard Grave, Armand Joulin. 2020. Reducing Transformer Depth on Demand with Structured Dropout. In Proceedings of ICLR 2020.
• Yikai Zhou, Baosong Yang, Derek F. Wong, Yu Wan and Lidia S. Chao. 2020. Uncertainty-Aware Curriculum Learning for Neural Machine Translation. In Proceedings of ACL 2020.
• Hongfei Xu, Josef van Genabith, Deyi Xiong and Qiuhui Liu. 2020. Dynamically Adjusting Transformer Batch Size by Monitoring Gradient Direction Change. In Proceedings of ACL 2020.
• Hongfei Xu, Qiuhui Liu, Josef van Genabith, Deyi Xiong and Jingyi Zhang. 2020. Lipschitz Constrained Parameter Initialization for Deep Transformers. In Proceedings of ACL 2020.
• Xintong Li, Lemao Liu, Rui Wang, Guoping Huang and Max Meng. 2020. Regularized Context Gates on Transformer for Machine Translation. In Proceedings of ACL 2020.
• Sheng Shen, Zhewei Yao, Amir Gholami, Michael Mahoney, and Kurt Keutzer. 2020. Rethinking Batch Normalization in Transformers. In Proceedings of ICML 2020.
• Xuebo Liu, Houtim Lai, Derek F. Wong, Lidia S. Chao. 2020. Norm-Based Curriculum Learning for Neural Machine Translation. In Proceedings of ACL 2020.
• Rongxiang Weng, Heng Yu, Xiangpeng Wei, Weihua Luo. 2020. Towards Enhancing Faithfulness for Neural Machine Translation. In Proceedings of EMNLP 2020.
• Yu Wan, Baosong Yang, Derek F. Wong, Yikai Zhou, Lidia S. Chao, Haibo Zhang, Boxing Chen. 2020. Self-Paced Learning for Neural Machine Translation. In Proceedings of EMNLP 2020.
• Wenxiang Jiao, Xing Wang, Shilin He, Irwin King, Michael Lyu, Zhaopeng Tu. 2020. Data Rejuvenation: Exploiting Inactive Training Examples for Neural Machine Translation. In Proceedings of EMNLP 2020.
• Shuhao Gu, Jinchao Zhang, Fandong Meng, Yang Feng, Wanying Xie, Jie Zhou, Dong Yu. 2020. Token-level Adaptive Training for Neural Machine Translation. In Proceedings of EMNLP 2020.
• Xiao Pan, Mingxuan Wang, Liwei Wu, Lei Li. 2021. Contrastive Learning for Many-to-many Multilingual Neural Machine Translation. In Proceedings of ACL 2021 .
• Zehui Lin , Liwei Wu , Mingxuan Wang, Lei Li . 2021. Learning Language Specific Sub-netswork for Multilingual Machine Translation . In Proceedings of ACL 2021 .

Decoding

• Mingxuan Wang, Zhengdong Lu, Hang Li, and Qun Liu. 2016. Memory-enhanced Decoder for Neural Machine Translation. In Proceedings of EMNLP 2016. (Citation: 30)
• Shonosuke Ishiwatari, Jingtao Yao, Shujie Liu, Mu Li, Ming Zhou, Naoki Yoshinaga, Masaru Kitsuregawa, and Weijia Jia. 2017. Chunk-based Decoder for Neural Machine Translation. In Proceedings of ACL 2017. (Citation: 4)
• Hao Zhou, Zhaopeng Tu, Shujian Huang, Xiaohua Liu, Hang Li, and Jiajun Chen. 2017. Chunk-Based Bi-Scale Decoder for Neural Machine Translation. In Proceedings of ACL 2017. (Citation: 6)
• Zichao Yang, Zhiting Hu, Yuntian Deng, Chris Dyer, and Alex Smola. 2017. Neural Machine Translation with Recurrent Attention Modeling. In Proceedings of EACL 2017. (Citation: 25)
• Markus Freitag and Yaser Al-Onaizan. 2017. Beam Search Strategies for Neural Machine Translation. In Proceedings of the First Workshop on Neural Machine Translation. (Citation: 14)
• Rajen Chatterjee, Matteo Negri, Marco Turchi, Marcello Federico, Lucia Specia, and Frédéric Blain. 2017. Guiding Neural Machine Translation Decoding with External Knowledge. In Proceedings of the Second Conference on Machine Translation. (Citation: 8)
• Cong Duy Vu Hoang, Gholamreza Haffari, and Trevor Cohn. 2017. Towards Decoding as Continuous Optimisation in Neural Machine Translation. In Proceedings of EMNLP 2017. (Citation: 4)
• Yin-Wen Chang and Michael Collins. 2017. Source-Side Left-to-Right or Target-Side Left-to-Right? An Empirical Comparison of Two Phrase-Based Decoding Algorithms. In Proceedings of EMNLP 2017.
• Jiatao Gu, Kyunghyun Cho, and Victor O.K. Li. 2017. Trainable Greedy Decoding for Neural Machine Translation. In Proceedings of EMNLP 2017. (Citation: 18)
• Huda Khayrallah, Gaurav Kumar, Kevin Duh, Matt Post, and Philipp Koehn. 2017. Neural Lattice Search for Domain Adaptation in Machine Translation. In Proceedings of IJCNLP 2017. (Citation: 4)
• Łukasz Kaiser, Aurko Roy, Ashish Vaswani, Niki Parmar, Samy Bengio, Jakob Uszkoreit, and Noam Shazeer. 2018. Fast Decoding in Sequence Models Using Discrete Latent Variables. In Proceedings of ICML 2018. (Citation: 3)
• Xiangwen Zhang, Jinsong Su, Yue Qin, Yang Liu, Rongrong Ji, and Hongji Wang. 2018. Asynchronous Bidirectional Decoding for Neural Machine Translation. In Proceedings of AAAI 2018. (Citation: 10)
• Jiatao Gu, Daniel Jiwoong Im, and Victor O.K. Li. 2018. Neural machine translation with gumbel-greedy decoding. In Proceedings of AAAI 2018. (Citation: 5)
• Philip Schulz, Wilker Aziz, and Trevor Cohn. 2018. A Stochastic Decoder for Neural Machine Translation. In Proceedings of ACL 2018. (Citation: 3)
• Raphael Shu and Hideki Nakayama. 2018. Improving Beam Search by Removing Monotonic Constraint for Neural Machine Translation. In Proceedings of ACL 2018.
• Junyang Lin, Xu Sun, Xuancheng Ren, Shuming Ma, Jinsong Su, and Qi Su. 2018. Deconvolution-Based Global Decoding for Neural Machine Translation. In Proceedings of COLING 2018. (Citation: 2)
• Chunqi Wang, Ji Zhang, and Haiqing Chen. 2018. Semi-Autoregressive Neural Machine Translation. In Proceedings of EMNLP 2018.
• Xinwei Geng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2018. Adaptive Multi-pass Decoder for Neural Machine Translation. In Proceedings of EMNLP 2018.
• Wen Zhang, Liang Huang, Yang Feng, Lei Shen, and Qun Liu. 2018. Speeding Up Neural Machine Translation Decoding by Cube Pruning. In Proceedings of EMNLP 2018.
• Xinyi Wang, Hieu Pham, Pengcheng Yin, and Graham Neubig. 2018. A Tree-based Decoder for Neural Machine Translation. In Proceedings of EMNLP 2018. (Citation: 1)
• Chenze Shao, Xilin Chen, and Yang Feng. 2018. Greedy Search with Probabilistic N-gram Matching for Neural Machine Translation. In Proceedings of EMNLP 2018.
• Zhisong Zhang, Rui Wang, Masao Utiyama, Eiichiro Sumita, and Hai Zhao. 2018. Exploring Recombination for Efficient Decoding of Neural Machine Translation. In Proceedings of EMNLP 2018.
• Jetic Gū, Hassan S. Shavarani, and Anoop Sarkar. 2018. Top-down Tree Structured Decoding with Syntactic Connections for Neural Machine Translation and Parsing. In Proceedings of EMNLP 2018.
• Yilin Yang, Liang Huang, and Mingbo Ma. 2018. Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translation. In Proceedings of EMNLP 2018. (Citation: 3)
• Yun Chen, Victor O.K. Li, Kyunghyun Cho, and Samuel R. Bowman. 2018. A Stable and Effective Learning Strategy for Trainable Greedy Decoding. In Proceedings of EMNLP 2018.
• Wouter Kool, Herke van Hoof, and Max Welling. 2019. Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement. In Proceedings of ICML 2019.
• Ashwin Kalyan, Peter Anderson, Stefan Lee, and Dhruv Batra. 2019. Trainable Decoding of Sets of Sequences for Neural Sequence Models. In Proceedings of ICML 2019.
• Eldan Cohen and Christopher Beck. 2019. Empirical Analysis of Beam Search Performance Degradation in Neural Sequence Models. In Proceedings of ICML 2019.
• Kartik Goyal, Chris Dyer, and Taylor Berg-Kirkpatrick. 2019. An Empirical Investigation of Global and Local Normalization for Recurrent Neural Sequence Models Using a Continuous Relaxation to Beam Search. In Proceedings of NAACL 2019.
• Mingbo Ma, Renjie Zheng, and Liang Huang. 2019. Learning to Stop in Structured Prediction for Neural Machine Translation. In Proceedings of NAACL 2019.
• Han Fu, Chenghao Liu, and Jianling Sun. 2019. Reference Network for Neural Machine Translation. In Proceedings of ACL 2019.
• Long Zhou, Jiajun Zhang, and Chengqing Zong. 2019. Synchronous Bidirectional Neural Machine Translation. Transactions of the Association for Computational Linguistics.
• Zhiqing Sun, Zhuohan Li, Haoqing Wang, Zi Lin, Di He, and Zhi-Hong Deng. 2019. Fast Structured Decoding for Sequence Models. In Proceedings of NeurIPS 2019.
• Lifu Tu, Richard Yuanzhe Pang, Sam Wiseman and Kevin Gimpel. 2020. ENGINE: Energy-Based Inference Networks for Non-Autoregressive Machine Translation. In Proceedings of ACL 2020.
• Pinzhen Chen, Nikolay Bogoychev, Kenneth Heafield, and Faheem Kirefu. 2020. Parallel Sentence Mining by Constrained Decoding. In Proceedings of ACL 2020.
• Julia Kreutzer, George Foster, Colin Cherry. 2020. Inference Strategies for Machine Translation with Conditional Masking. In Proceedings of EMNLP 2020.
• Yuntian Deng, Alexander Rush. 2020. Cascaded Text Generation with Markov Transformers. In Proceedings of NeurIPS 2020.
• Clara Meister, Ryan Cotterell, Tim Vieira. 2020. If beam search is the answer, what was the question?. In Proceedings of EMNLP 2020.
• Urvashi Khandelwal, Angela Fan, Dan Jurafsky, Luke Zettlemoyer, Mike Lewis. 2021. Nearest Neighbor Machine Translation. In Proceedings of ICLR 2021.
• Mathias Muller , Rico Sennrich. 2021. Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translation. In Proceedings of ACL 2021.
• Hongfei Xu, Qiuhui Liu , Josef van Genabith , Deyi Xiong , Meng Zhang . 2021. Multi-Head Highly Parallelized LSTM Decoder for Neural Machine Translation. In Proceedings of ACL 2021.
• Yang Feng , Shuhao Gu , Dengji Guo , Zhengxin Yang , Chenze Shao .2021. Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation . In Proceedings of ACL 2021.

Low-resource Language Translation

• Rico Sennrich and Biao Zhang. 2019. Revisiting Low-Resource Neural Machine Translation: A Case Study. In Proceedings of ACL 2019.
• Danni Liu , Jan Niehues , James Cross , Francisco Guzman , Xian Li . 2021. Improving Zero-Shot Translation by Disentangling Positional Information . In Proceedings of ACL 2021.

Semi-supervised Learning

• Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving Neural Machine Translation Models with Monolingual Data. In Proceedings of ACL 2016. (Citation: 220)
• Yong Cheng, Wei Xu, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. 2016. Semi-Supervised Learning for Neural Machine Translation. In Proceedings of ACL 2016. (Citation: 59)
• Tobias Domhan and Felix Hieber. 2017. Using Target-side Monolingual Data for Neural Machine Translation through Multi-task Learning. In Proceedings of EMNLP 2017. (Citation: 11)
• Anna Currey, Antonio Valerio Miceli Barone, and Kenneth Heafield. 2017. Copied Monolingual Data Improves Low-Resource Neural Machine Translation. In Proceedings of the Second Conference on Machine Translation. (Citation: 14)
• Shuo Wang, Yang Liu, Chao Wang, Huanbo Luan, and Maosong Sun. 2019. Improving Back-Translation with Uncertainty-based Confidence Estimation. In Proceedings of EMNLP 2019.
• Zaixiang Zheng, Hao Zhou, Shujian Huang, Lei Li, Xin-Yu Dai, Jiajun Chen. 2020. Mirror-Generative Neural Machine Translation. In Proceedings of ICLR 2020.

Unsupervised Learning

• Nima Pourdamghani and Kevin Knight. 2017. Deciphering Related Languages. In Proceedings of EMNLP 2017. (Citation: 5)
• Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. 2018. Unsupervised Neural Machine Translation. In Proceedings of ICLR 2018. (Citation: 78)
• Guillaume Lample, Alexis Conneau, Ludovic Denoyer, and Marc'Aurelio Ranzato. 2018. Unsupervised Machine Translation Using Monolingual Corpora Only. In Proceedings of ICLR 2018. (Citation: 78)
• Zhen Yang, Wei Chen, Feng Wang, and Bo Xu. 2018. Unsupervised Neural Machine Translation with Weight Sharing. In Proceedings of ACL 2018. (Citation: 6)
• Guillaume Lample, Myle Ott, Alexis Conneau, Ludovic Denoyer, and Marc'Aurelio Ranzato. 2018. Phrase-Based & Neural Unsupervised Machine Translation. In Proceedings of EMNLP 2018. (Citation: 24)
• Iftekhar Naim, Parker Riley, and Daniel Gildea. 2018. Feature-Based Decipherment for Machine Translation. Computational Linguistics. (Citation: 24)
• Jiawei Wu, Xin Wang, and William Yang Wang. 2019. Extract and Edit: An Alternative to Back-Translation for Unsupervised Neural Machine Translation. In Proceedings of NAACL 2019.
• Nima Pourdamghani, Nada Aldarrab, Marjan Ghazvininejad, Kevin Knight, and Jonathan May. 2019. Translating Translationese: A Two-Step Approach to Unsupervised Machine Translation. In Proceedings of ACL 2019.
• Jiaming Luo, Yuan Cao, and Regina Barzilay. 2019. Neural Decipherment via Minimum-Cost Flow: from Ugaritic to Linear B. In Proceedings of ACL 2019.
• Yichong Leng, Xu Tan, Tao Qin, Xiang-Yang Li, and Tie-Yan Liu. 2019. Unsupervised Pivot Translation for Distant Languages. In Proceedings of ACL 2019.
• Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2019. An Effective Approach to Unsupervised Machine Translation. In Proceedings of ACL 2019.
• Viktor Hangya and Alexander Fraser. 2019. Unsupervised Parallel Sentence Extraction with Parallel Segment Detection Helps Machine Translation. In Proceedings of ACL 2019.
• Haipeng Sun, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita, and Tiejun Zhao. 2019. Unsupervised Bilingual Word Embedding Agreement for Unsupervised Neural Machine Translation. In Proceedings of ACL 2019.
• Jiatao Gu, Yong Wang, Kyunghyun Cho, and Victor O.K. Li. 2019. Improved Zero-shot Neural Machine Translation via Ignoring Spurious Correlations. In Proceedings of ACL 2019.
• Sukanta Sen, Kamal Kumar Gupta, Asif Ekbal, and Pushpak Bhattacharyya. 2019. Multilingual Unsupervised NMT using Shared Encoder and Language-Specific Decoders. In Proceedings of ACL 2019.
• Shuo Ren, Yu Wu, Shujie Liu, Ming Zhou and Shuai Ma. 2019. Explicit Cross-lingual Pre-training for Unsupervised Machine Translation. In Proceedings of EMNLP 2019.
• Haipeng Sun, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita and Tiejun Zhao. 2020. Knowledge Distillation for Multilingual Unsupervised Neural Machine Translation. In Proceedings of ACL 2020.
• Xiangyu Duan, Baijun Ji, Hao Jia, Min Tan, Min Zhang, Boxing Chen, Weihua Luo and Yue Zhang. 2020. Bilingual Dictionary Based Neural Machine Translation without Using Parallel Sentences. In Proceedings of ACL 2020.
• Shuo Ren, Yu Wu, Shujie Liu, Ming Zhou and Shuai Ma. 2020. A Retrieve-and-Rewrite Initialization Method for Unsupervised Machine Translation. In Proceedings of ACL 2020.
• Alexandra Chronopoulou, Dario Stojanovski, Alexander Fraser. 2020. Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT. In Proceedings of EMNLP 2020.
• Jerin Philip, Alexandre Berard, Matthias Gallé, Laurent Besacier. 2020. Monolingual Adapters for Zero-Shot Neural Machine Translation. In Proceedings of EMNLP 2020.
• Dana Ruiter, Josef van Genabith, Cristina España-Bonet. 2020. Self-Induced Curriculum Learning in Self-Supervised Neural Machine Translation. In Proceedings of EMNLP 2020.
• Wei-Jen Ko , Ahmed El-Kishky , Adithya Renduchintala , Vishrav Chaudhary , Naman Goyal , Francisco Guzman , Pascale Fung , Philipp Koehn , Mona Diab . 2021 . Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Data . In Proceedings of ACL 2021.

Pivot-based Methods

• Orhan Firat, Baskaran Sankaran, Yaser Al-Onaizan, Fatos T. Yarman Vural, and Kyunghyun Cho. 2016. Zero-Resource Translation with Multi-Lingual Neural Machine Translation. In Proceedings of EMNLP 2016. (Citation: 50)
• Hao Zheng, Yong Cheng, and Yang Liu. 2017. Maximum Expected Likelihood Estimation for Zero-resource Neural Machine Translation. In Proceedings of IJCAI 2017. (Citation: 9)
• Yong Cheng, Qian Yang, Yang Liu, Maosong Sun, and Wei Xu. 2017. Joint Training for Pivot-based Neural Machine Translation. In Proceedings of IJCAI 2017. (Citation: 11)
• Yun Chen, Yang Liu, Yong Cheng and Victor O.K. Li. 2017. A Teacher-Student Framework for Zero-resource Neural Machine Translation. In Proceedings of ACL 2017. (Citation: 15)
• Yun Chen, Yang Liu, and Victor O. K. Li. 2018. Zero-Resource Neural Machine Translation with Multi-Agent Communication Game. In Proceedings of AAAI 2018. (Citation: 6)
• Shuo Ren, Wenhu Chen, Shujie Liu, Mu Li, Ming Zhou, and Shuai Ma. 2018. Triangular Architecture for Rare Language Translation. In Proceedings of ACL 2018. (Citation: 1)
• Yunsu Kim, Petre Petrov, Pavel Petrushkov, Shahram Khadivi, and Hermann Ney. 2019. Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languages. In Proceedings of EMNLP 2019.

Data Augmentation Methods

• Marzieh Fadaee, Arianna Bisazza, and Christof Monz. 2017. Data Augmentation for Low-Resource Neural Machine Translation. In Proceedings of ACL 2017. (Citation: 26)
• Marzieh Fadaee and Christof Monz. 2018. Back-Translation Sampling by Targeting Difficult Words in Neural Machine Translation. In Proceedings of EMNLP 2018.
• Sergey Edunov, Myle Ott, Michael Auli, and David Grangier. 2018. Understanding Back-Translation at Scale. In Proceedings of EMNLP 2018. (Citation: 6)
• Xinyi Wang, Hieu Pham, Zihang Dai, and Graham Neubig. 2018. SwitchOut: an Efficient Data Augmentation Algorithm for Neural Machine Translation. In Proceedings of EMNLP 2018. (Citation: 4)
• Mengzhou Xia, Xiang Kong, Antonios Anastasopoulos, and Graham Neubig. 2019. Generalized Data Augmentation for Low-Resource Translation. In Proceedings of ACL 2019.
• Jinhua Zhu, Fei Gao, Lijun Wu, Yingce Xia, Tao Qin, Wengang Zhou, Xueqi Cheng, and Tie-Yan Liu. 2019. Soft Contextual Data Augmentation for Neural Machine Translation. In Proceedings of ACL 2019.
• Chunting Zhou, Xuezhe Ma, Junjie Hu, and Graham Neubig. 2019. Handling Syntactic Divergence in Low-resource Machine Translation. In Proceedings of EMNLP 2019.
• Yuanpeng Li, Liang Zhao, Jianyu Wang, and Joel Hestness. 2019. Compositional Generalization for Primitive Substitutions. In Proceedings of EMNLP 2019.
• Guanlin Li, Lemao Liu, Guoping Huang, Conghui Zhu and Tiejun Zhao. 2019. Understanding Data Augmentation in Neural Machine Translation: Two Perspectives towards Generalization. In Proceedings of EMNLP 2019.
• Sergey Edunov, Myle Ott, Marc’Aurelio Ranzato and Michael Auli. 2020. On The Evaluation of Machine Translation Systems Trained With Back-Translation. In Proceedings of ACL 2020.
• Aditya Siddhant, Ankur Bapna, Yuan Cao, Orhan Firat, Mia Chen, Sneha Kudugunta, Naveen Arivazhagan and Yonghui Wu. 2020. Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation. In Proceedings of ACL 2020.
• Jitao XU, Josep Crego and Jean Senellart. 2020. Boosting Neural Machine Translation with Similar Translations. In Proceedings of ACL 2020.
• Yong Cheng, Lu Jiang, Wolfgang Macherey and Jacob Eisenstein. 2020. AdvAug: Robust Adversarial Augmentation for Neural Machine Translation. In Proceedings of ACL 2020.
• Benjamin Marie, Raphael Rubino and Atsushi Fujita. 2020. Tagged Back-translation Revisited: Why Does It Really Work?. In Proceedings of ACL 2020.
• Huda Khayrallah, Brian Thompson, Matt Post, Philipp Koehn. 2020. Simulated multiple reference training improves low-resource machine translation. In Proceedings of EMNLP 2020.
• Hao-Ran Wei, Zhirui Zhang, Boxing Chen, Weihua Luo. 2020. Iterative Domain-Repaired Back-Translation. In Proceedings of EMNLP 2020.
• Xuan-Phi Nguyen, Shafiq Joty, Kui Wu, Ai Ti Aw. 2020. Data Diversification: A Simple Strategy For Neural Machine Translation. In Proceedings of NeurIPS 2020.
• Christos Baziotis, Barry Haddow, Alexandra Birch. 2020. Language Model Prior for Low-Resource Neural Machine Translation. In Proceedings of EMNLP 2020.
• Hieu Pham, Xinyi Wang, Yiming Yang, Graham Neubig. 2021. Meta Back-Translation. In Proceedings of ICLR 2021.
• M Saiful Bari , Tasnim Mohiuddin , and Shafiq Joty . 2021 . UXLA: A Robust Unsupervised Data Augmentation Framework for Zero-Resource Cross-Lingual NLP . In Proceedings of ACL 2021.

Data Selection Methods

• Marlies van der Wees, Arianna Bisazza and Christof Monz. 2017. Dynamic Data Selection for Neural Machine Translation. In Proceedings of EMNLP 2017. (Citation: 16)
• Wei Wang, Taro Watanabe, Macduff Hughes, Tetsuji Nakagawa, and Ciprian Chelba. 2018. Denoising Neural Machine Translation Training with Trusted Data and Online Data Selection. In Proceedings of the Third Conference on Machine Translation.
• Minh Quang Pham, Josep Crego, Jean Senellart, and François Yvon. 2018. Fixing Translation Divergences in Parallel Corpora for Neural MT. In Proceedings of EMNLP 2018.
• Xinyi Wang and Graham Neubig. 2019. Target Conditioned Sampling: Optimizing Data Selection for Multilingual Neural Machine Translation. In Proceedings of ACL 2019.
• Wei Wang, Isaac Caswell, and Ciprian Chelba. 2019. Dynamically Composing Domain-Data Selection with Clean-Data Selection by “Co-Curricular Learning” for Neural Machine Translation. In Proceedings of ACL 2019.
• Dana Ruiter, Cristina España-Bonet, and Josef van Genabith. 2019. Self-Supervised Neural Machine Translation. In Proceedings of ACL 2019.
• Xabier Soto, Dimitar Shterionov, Alberto Poncelas and Andy Way. 2020. Selecting Backtranslated Data from Multiple Sources for Improved Neural Machine Translation. In Proceedings of ACL 2020.
• Jiawei Zhou and Phillip Keung. 2020. Improving Non-autoregressive Neural Machine Translation with Monolingual Data. In Proceedings of ACL 2020.
• Boliang Zhang, Ajay Nagesh and Kevin Knight. 2020. Parallel Corpus Filtering via Pre-trained Language Models. In Proceedings of ACL 2020.
• Zi-Yi Dou, Antonios Anastasopoulos, Graham Neubig. 2020. Dynamic Data Selection and Weighting for Iterative Back-Translation. In Proceedings of EMNLP 2020.
• Wenxiang Jiao , Xing Wang , Zhaopeng Tu , Shuming Shi , Michael R. Lyu , Irwin King . 2021 . Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translation

Transfer Learning

• Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. 2016. Transfer Learning for Low-Resource Neural Machine Translation. In Proceedings of EMNLP 2016. (Citation: 104)
• Jiatao Gu, Hany Hassan, Jacob Devlin, and Victor O.K. Li. 2018. Universal Neural Machine Translation for Extremely Low Resource Languages. In Proceedings of NAACL 2018. (Citation: 17)
• Tom Kocmi and Ondřej Bojar. 2018. Trivial Transfer Learning for Low-Resource Neural Machine Translation. In Proceedings of the Third Conference on Machine Translation: Research Papers.
• Boyuan Pan, Yazheng Yang, Hao Li, Zhou Zhao, Yueting Zhuang, Deng Cai, and Xiaofei He. 2018. MacNet: Transferring Knowledge from Machine Comprehension to Sequence-to-Sequence Models. In Proceedings of NeurIPS 2018.
• Yunsu Kim, Yingbo Gao, and Hermann Ney. 2019. Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabularies. In Proceedings of ACL 2019.
• Baijun Ji, Zhirui Zhang, Xiangyu Duan, Min Zhang, Boxing Chen and Weihua Luo. 2020. Cross-lingual Pre-training Based Transfer for Zero-shot Neural Machine Translation. In Proceedings of AAAI 2020.
• Alham Fikri Aji, Nikolay Bogoychev, Kenneth Heafield and Rico Sennrich. 2020. In Neural Machine Translation, What Does Transfer Learning Transfer?. In Proceedings of ACL 2020.
• Mikel Artetxe, Gorka Labaka, Eneko Agirre. 2020. Translation Artifacts in Cross-lingual Transfer Learning. In Proceedings of EMNLP 2020.

Meta Learning

• Jiatao Gu, Yong Wang, Yun Chen, Kyunghyun Cho, and Victor O.K. Li. 2018. Meta-Learning for Low-Resource Neural Machine Translation. In Proceedings of EMNLP 2018. (Citation: 3)
• Rumeng Li, Xun Wang and Hong Yu. 2020. MetaMT,a MetaLearning Method Leveraging Multiple Domain Data for Low Resource Machine Translation. In Proceedings of AAAI 2020.

Multilingual Machine Translation

• Daxiang Dong, Hua Wu, Wei He, Dianhai Yu, and Haifeng Wang. 2015. Multi-Task Learning for Multiple Language Translation. In Proceedings of ACL 2015. (Citation: 126)
• Orhan Firat, Kyunghyun Cho and Yoshua Bengio. 2016. Multi-way, Multilingual Neural Machine Translation with a Shared Attention Mechanism. In Proceedings of NAACL 2016. (Citation: 146)
• Barret Zoph and Kevin Knight. 2016. Multi-Source Neural Translation. In Proceedings of NAACL 2016. (Citation: 87)
• Orhan Firat, Baskaran SanKaran, Yaser Al-Onaizan, Fatos T.Yarman Vural, Kyunghyun Cho. 2016. Zero-Resource Translation with Multi-Lingual Neural Machine Translation. In Proceedings of EMNLP 2016. (Citation: 50)
• Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2017. Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation. Transactions of the Association for Computational Linguistics. (Citation: 297)
• Poorya Zaremoodi and Gholamreza Haffari. 2018. Neural Machine Translation for Bilingually Scarce Scenarios: a Deep Multi-Task Learning Approach. In Proceedings of NAACL 2018. (Citation: 1)
• Poorya Zaremoodi, Wray Buntine, and Gholamreza Haffari. 2018. Adaptive Knowledge Sharing in Multi-Task Learning: Improving Low-Resource Neural Machine Translation. In Proceedings of ACL 2018.
• Surafel Melaku Lakew, Mauro Cettolo, and Marcello Federico. 2018. A Comparison of Transformer and Recurrent Neural Networks on Multilingual Neural Machine Translation. In Proceedings of COLING 2018. (Citation: 2)
• Graeme Blackwood, Miguel Ballesteros, and Todd Ward. 2018. Multilingual Neural Machine Translation with Task-Specific Attention. In Proceedings of COLING 2018. (Citation: 1)
• Devendra Singh Sachan and Graham Neubig. 2018. Parameter Sharing Methods for Multilingual Self-Attentional Translation Models. In Proceedings of the Third Conference on Machine Translation: Research Papers.
• Emmanouil Antonios Platanios, Mrinmaya Sachan, Graham Neubig, and Tom Mitchell. 2018. Contextual Parameter Generation for Universal Neural Machine Translation. In Proceedings of EMNLP 2018.
• Yining Wang, Jiajun Zhang, Feifei Zhai, Jingfang Xu, and Chengqing Zong. 2018. Three Strategies to Improve One-to-Many Multilingual Translation. In Proceedings of EMNLP 2018.
• Xu Tan, Yi Ren, Di He, Tao Qin, Zhou Zhao, and Tie-Yan Liu. 2019. Multilingual Neural Machine Translation with Knowledge Distillation. In Proceedings of ICLR 2019.
• Xinyi Wang, Hieu Pham, Philip Arthur, and Graham Neubig. 2019. Multilingual Neural Machine Translation With Soft Decoupled Encoding. In Proceedings of ICLR 2019.
• Maruan Al-Shedivat and Ankur P. Parikh. 2019. Consistency by Agreement in Zero-shot Neural Machine Translation. In Proceedings of NAACL 2019.
• Roee Aharoni, Melvin Johnson, and Orhan Firat. 2019. Massively Multilingual Neural Machine Translation. In Proceedings of NAACL 2019.
• Yunsu Kim, Yingbo Gao, and Hermann Ney. 2019. Effective Cross-lingual Transfer of Neural Machine Translation Models without Shared Vocabularies. In Proceedings of ACL 2019.
• Carlos Escolano, Marta R. Costa-Jussà, and José A. R. Fonollosa. 2019. From Bilingual to Multilingual Neural Machine Translation by Incremental Training. In Proceedings of ACL 2019.
• Yining Wang, Long Zhou, Jiajun Zhang, Feifei Zhai, Jingfang Xu, and Chengqing Zong. 2019. A Compact and Language-Sensitive Multilingual Translation Method. In Proceedings of ACL 2019.
• Sukanta Sen, Kamal Kumar Gupta, Asif Ekbal, and Pushpak Bhattacharyya. 2019. Multilingual Unsupervised NMT using Shared Encoder and Language-Specific Decoders. In Proceedings of ACL 2019.
• Xu Tan, Jiale Chen, Di He, Yingce Xia, Tao Qin, and Tie-Yan Liu. 2019. Multilingual Neural Machine Translation with Language Clustering. In Proceedings of EMNLP 2019.
• Sneha Reddy Kudugunta, Ankur Bapna, Isaac Caswell, Naveen Arivazhagan, and Orhan Firat. 2019. Investigating Multilingual NMT Representations at Scale. In Proceedings of EMNLP 2019.
• Ankur Bapna, Naveen Arivazhagan, and Orhan Firat. 2019. Simple, Scalable Adaptation for Neural Machine Translation. In Proceedings of EMNLP 2019.
• Raj Dabre, Atsushi Fujita and Chenhui Chu. 2019. Low-Resource Neural Machine Translation by Exploiting Multilingualism through Multi-Step Fine-Tuning Using N-way Parallel Corpora. In Proceedings of EMNLP 2019.
• Xinyi Wang, Yulia Tsvetkov and Graham Neubig. 2020. Balancing Training for Multilingual Neural Machine Translation. In Proceedings of ACL 2020.
• Biao Zhang, Philip Williams, Ivan Titov and Rico Sennrich. 2020. Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translation. In Proceedings of ACL 2020.
• Haipeng Sun, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita and Tiejun Zhao. 2020. Knowledge Distillation for Multilingual Unsupervised Neural Machine Translation. In Proceedings of ACL 2020.
• Aditya Siddhant, Ankur Bapna, Yuan Cao, Orhan Firat, Mia Chen, Sneha Kudugunta, Naveen Arivazhagan and Yonghui Wu. 2020. Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation. In Proceedings of ACL 2020.
• Changfeng Zhu, Heng Yu, Shanbo Cheng and Weihua Luo. 2020. Language-aware Interlingua for Multilingual Neural Machine Translation. In Proceedings of ACL 2020.
• Zehui Lin, Xiao Pan, Mingxuan Wang, Xipeng Qiu, Jiangtao Feng, Hao Zhou, Lei Li. 2020. Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information. In Proceedings of EMNLP 2020.
• Sungwon Lyu, Bokyung Son, Kichang Yang, Jaekyoung Bae. 2020. Revisiting Modularized Multilingual NMT to Meet Industrial Demands. In Proceedings of EMNLP 2020.
• Arturo Oncevay, Barry Haddow, Alexandra Birch. 2020. Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations. In Proceedings of EMNLP 2020.
• Yiren Wang, ChengXiang Zhai, Hany Hassan. 2020. Multi-task Learning for Multilingual Neural Machine Translation. In Proceedings of EMNLP 2020.
• Raj Dabre, Chenhui Chu, Anoop Kunchukuttan. 2020. A Comprehensive Survey of Multilingual Neural Machine Translation. arXiv:2001.01115.
• Biao Zhang, Ankur Bapna, Rico Sennrich, Orhan Firat. 2021. Share or Not? Learning to Schedule Language-Specific Capacity for Multilingual Translation. In Proceedings of ICLR 2021.

Prior Knowledge Integration

Word/Phrase Constraints

• Wei He, Zhongjun He, Hua Wu, and Haifeng Wang. 2016. Improved nerual machine translation with SMT features. In Proceedings of AAAI 2016. (Citation: 46)
• Haitao Mi, Zhiguo Wang, and Abe Ittycheriah. 2016. Vocabulary Manipulation for Neural Machine Translation. In Proceedings of ACL 2016. (Citation: 36)
• Philip Arthur, Graham Neubig, and Satoshi Nakamura. 2016. Incorporating Discrete Translation Lexicons into Neural Machine Translation. In Proceedings of EMNLP 2016. (Citation: 55)
• Xing Wang, Zhengdong Lu, Zhaopeng Tu, Hang Li, Deyi Xiong, Min Zhang. 2017. Neural Machine Translation Advised by Statistical Machine Translation. In Proceedings of AAAI 2016. (Citation: 34)
• Jiacheng Zhang, Yang Liu, Huanbo Luan, Jingfang Xu and Maosong Sun. 2017. Prior Knowledge Integration for Neural Machine Translation using Posterior Regularization. In Proceedings of ACL 2017. (Citation: 13)
• Chris Hokamp and Qun Liu. 2017. Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search. In Proceedings of ACL 2017. (Citation: 19)
• Zichao Yang, Zhiting Hu, Yuntian Deng, Chris Dyer, and Alex Smola. 2017. Neural Machine Translation with Recurrent Attention Modeling. In Proceedings of EACL 2017. (Citation: 25)
• Ofir Press and Lior Wolf. 2017. Using the Output Embedding to Improve Language Models. In Proceedings of EACL 2017. (Citation: 127)
• Rajen Chatterjee, Matteo Negri, Marco Turchi, Marcello Federico, Lucia Specia, and Frédéric Blain. 2017. Guiding Neural Machine Translation Decoding with External Knowledge. In Proceedings of the Second Conference on Machine Translation. (Citation: 8)
• Rongxiang Weng, Shujian Huang, Zaixiang Zheng, Xinyu Dai, and Jiajun Chen. 2017. Neural Machine Translation with Word Predictions. In Proceedings of EMNLP 2017. (Citation: 8)
• Yang Feng, Shiyue Zhang, Andi Zhang, Dong Wang, and Andrew Abel. 2017. Memory-augmented Neural Machine Translation. In Proceedings of EMNLP 2017. (Citation: 9)
• Leonard Dahlmann, Evgeny Matusov, Pavel Petrushkov, and Shahram Khadivi. 2017. Neural Machine Translation Leveraging Phrase-based Models in A Hybrid Search. In Proceedings of EMNLP 2017. (Citation: 11)
• Xing Wang, Zhaopeng Tu, Deyi Xiong, and Min Zhang. 2017. Translating Phrases in Neural Machine Translation. In Proceedings of EMNLP 2017. (Citation: 15)
• Baosong Yang, Derek F. Wong, Tong Xiao, Lidia S. Chao, and Jingbo Zhu. 2017. Towards Bidirectional Hierarchical Representations for Attention-based Neural Machine Translation. In Proceedings of EMNLP 2017. (Citation: 5)
• Po-Sen Huang, Chong Wang, Sitao Huang, Dengyong Zhou, and Li Deng. 2018. Towards Neural Phrase-based Machine Translation. In Proceedings of ICLR 2018. (Citation: 15)
• Toan Nguyen and David Chiang. 2018. Improving Lexical Choice in Neural Machine Translation. In Proceedings of NAACL 2018. (Citation: 8)
• Huadong Chen, Shujian Huang, David Chiang, Xinyu Dai, and Jiajun Chen. 2018. Combining Character and Word Information in Neural Machine Translation Using a Multi-Level Attention. In Proceedings of NAACL 2018.
• Matt Post and David Vilar. 2018. Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation. In Proceedings of NAACL 2018. (Citation: 6)
• Jingyi Zhang, Masao Utiyama, Eiichro Sumita, Graham Neubig, and Satoshi Nakamura. 2018. Guiding Neural Machine Translation with Retrieved Translation Pieces. In Proceedings of NAACL 2018. (Citation: 2)
• Eva Hasler, Adrià de Gispert, Gonzalo Iglesias, and Bill Byrne. 2018. Neural Machine Translation Decoding with Terminology Constraints. In Proceedings of NAACL 2018. (Citation: 2)
• Nima Pourdamghani, Marjan Ghazvininejad, and Kevin Knight. 2018. Using Word Vectors to Improve Word Alignments for Low Resource Machine Translation. In Proceedings of NAACL 2018. (Citation: 2)
• Shuming Ma, Xu SUN, Yizhong Wang, and Junyang Lin. 2018. Bag-of-Words as Target for Neural Machine Translation. In Proceedings of ACL 2018. (Citation: 10)
• Mingxuan Wang, Jun Xie, Zhixing Tan, Jinsong Su, Deyi Xiong, and Chao Bian. 2018. Neural Machine Translation with Decoding-History Enhanced Attention. In Proceedings of COLING 2018.
• Arata Ugawa, Akihiro Tamura, Takashi Ninomiya, Hiroya Takamura, and Manabu Okumura. 2018. Neural Machine Translation Incorporating Named Entity. In Proceedings of COLING 2018.
• Longyue Wang, Zhaopeng Tu, Andy Way, and Qun Liu. 2018. Learning to Jointly Translate and Predict Dropped Pronouns with a Shared Reconstruction Mechanism. In Proceedings of EMNLP 2018. (Citation: 1)
• Qian Cao and Deyi Xiong. 2018. Encoding Gated Translation Memory into Neural Machine Translation. In Proceedings of EMNLP 2018.
• Chengyue Gong, Di He, Xu Tan, Tao Qin, Liwei Wang, and Tie-Yan Liu. 2018. FRAGE: Frequency-Agnostic Word Representation. In Proceedings of NeurIPS 2018. (Citation: 2)
• Inigo Jauregi Unanue, Ehsan Zare Borzeshi, Nazanin Esmaili, and Massimo Piccardi. ReWE: Regressing Word Embeddings for Regularization of Neural Machine Translation Systems. In Proceedings of NAACL 2019.
• Kai Song, Yue Zhang, Heng Yu, Weihua Luo, Kun Wang, and Min Zhang. 2019. Code-Switching for Enhancing NMT with Pre-Specified Translation. In Proceedings of NAACL 2019.
• Xuebo Liu, Derek F. Wong, Yang Liu, Lidia S. Chao, Tong Xiao, and Jingbo Zhu. 2019. Shared-Private Bilingual Word Embeddings for Neural Machine Translation. In Proceedings of ACL 2019.
• Georgiana Dinu, Prashant Mathur, Marcello Federico, and Yaser Al-Onaizan. 2019. Training Neural Machine Translation to Apply Terminology Constraints. In Proceedings of ACL 2019.
• Longyue Wang, Zhaopeng Tu, Xing Wang and Shuming Shi. 2019. One Model to Learn Both: Zero Pronoun Prediction and Translation. In Proceedings of EMNLP 2019
• Jinhua Zhu, Yingce Xia, Lijun Wu, Di He, Tao Qin, Wengang Zhou, Houqiang Li, Tieyan Liu. 2020. Incorporating BERT into Neural Machine Translation. In Proceedings of ICLR 2020.
• Hongfei Xu, Josef van Genabith, Deyi Xiong, Qiuhui Liu and Jingyi Zhang. 2020. Learning Source Phrase Representations for Neural Machine Translation. In Proceedings of ACL 2020.
• Raymond Hendy Susanto, Shamil Chollampatt and Liling Tan. 2020. Lexically Constrained Neural Machine Translation with Levenshtein Transformer. In Proceedings of ACL 2020.
• Thomas Zenkel, Joern Wuebker and John DeNero. 2020. End-to-End Neural Word Alignment Outperforms GIZA++. In Proceedings of ACL 2020.
• Liang Ding, Longyue Wang and Dacheng Tao. 2020. Self-Attention with Cross-Lingual Position Representation. In Proceedings of ACL 2020.
• Marion Weller-Di Marco and Alexander Fraser. 2020. Modeling Word Formation in English–German Neural Machine Translation. In Proceedings of ACL 2020.
• Yun Chen, Yang Liu, Guanhua Chen, Xin Jiang, Qun Liu. 2020. Accurate Word Alignment Induction from Neural Machine Translation. In Proceedings of EMNLP 2020.
• Prathyusha Jwalapuram, Shafiq Joty, Youlin Shen. 2020. Pronoun-Targeted Fine-tuning for NMT with Hybrid Losses. In Proceedings of EMNLP 2020.
• Jingyi Zhang , Josef van Genabith . A Bidirectional Transformer Based Alignment Model for Unsupervised Word Alignment . In Proceedings of ACL 2021.

Syntactic/Semantic Constraints

• Trevor Cohn, Cong Duy Vu Hoang, Ekaterina Vymolova, Kaisheng Yao, Chris Dyer, and Gholamreza Haffari. 2016. Incorporating Structural Alignment Biases into an Attentional Neural Translation Model. In Proceedings of NAACL 2016. (Citation: 80)
• Yong Cheng, Shiqi Shen, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. 2016. Agreement-based Joint Training for Bidirectional Attention-based Neural Machine Translation. In Proceedings of IJCAI 2016. (Citation: 26)
• Akiko Eriguchi, Kazuma Hashimoto, and Yoshimasa Tsuruoka. 2016. Tree-to-Sequence Attentional Neural Machine Translation. In Proceedings of ACL 2016. (Citation: 79)
• Felix Stahlberg, Eva Hasler, Aurelien Waite, and Bill Byrne. 2016. Syntactically Guided Neural Machine Translation. In Proceedings of ACL 2016. (Citation: 32)
• Xing Shi, Inkit Padhi, and Kevin Knight. 2016. Does string-based neural MT learn source syntax?. In Proceedings of the EMNLP 2016. (Citation: 57)
• Junhui Li, Deyi Xiong, Zhaopeng Tu, Muhua Zhu, Min Zhang, and Guodong Zhou. 2017. Modeling Source Syntax for Neural Machine Translation. In Proceedings of ACL 2017. (Citation: 30)
• Shuangzhi Wu, Dongdong Zhang, Nan Yang, Mu Li, and Ming Zhou. 2017. Sequence-to-Dependency Neural Machine Translation. In Proceedings of ACL 2017. (Citation: 19)
• Jinchao Zhang, Mingxuan Wang, Qun Liu, and Jie Zhou. 2017. Incorporating Word Reordering Knowledge into Attention-based Neural Machine Translation. In Proceedings of ACL 2017. (Citation: 8)
• Huadong Chen, Shujian Huang, David Chiang, and Jiajun Chen. 2017. Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder. In Proceedings of ACL 2017. (Citation: 32)
• Akiko Eriguchi, Yoshimasa Tsuruoka, and Kyunghyun Cho. 2017. Learning to Parse and Translate Improves Neural Machine Translation. In Proceedings of ACL 2017. (Citation: 29)
• Roee Aharoni and Yoav Goldberg. 2017. Towards String-To-Tree Neural Machine Translation. In Proceedings of ACL 2017. (Citation: 45)
• Kazuma Hashimoto and Yoshimasa Tsuruoka. 2017. Neural Machine Translation with Source-Side Latent Graph Parsing. In Proceedings of EMNLP 2017. (Citation: 9)
• Joost Bastings, Ivan Titov, Wilker Aziz, Diego Marcheggiani, and Khalil Simaan. 2017. Graph Convolutional Encoders for Syntax-aware Neural Machine Translation. In Proceedings of EMNLP 2017. (Citation: 31)
• Kehai Chen, Rui Wang, Masao Utiyama, Lemao Liu, Akihiro Tamura, Eiichiro Sumita, and Tiejun Zhao. 2017. Neural Machine Translation with Source Dependency Representation. In Proceedings of EMNLP 2017. (Citation: 7)
• Peyman Passban, Qun Liu, and Andy Way. 2018. Improving Character-Based Decoding Using Target-Side Morphological Information for Neural Machine Translation. In Proceedings of NAACL 2018. (Citation: 5)
• Diego Marcheggiani, Joost Bastings, and Ivan Titov. 2018. Exploiting Semantics in Neural Machine Translation with Graph Convolutional Networks. In Proceedings of NAACL 2018. (Citation: 7)
• Chunpeng Ma, Akihiro Tamura, Masao Utiyama, Tiejun Zhao, and Eiichiro Sumita. 2018. Forest-Based Neural Machine Translation. In Proceedings of ACL 2018. (Citation: 1)
• Shaohui Kuang, Junhui Li, António Branco, Weihua Luo, and Deyi Xiong. 2018. Attention Focusing for Neural Machine Translation by Bridging Source and Target Embeddings. In Proceedings of ACL 2018. (Citation: 1)
• Duygu Ataman and Marcello Federico. 2018. Compositional Representation of Morphologically-Rich Input for Neural Machine Translation. In Proceedings of ACL 2018. (Citation: 4)
• Daniel Beck, Gholamreza Haffari, and Trevor Cohn. 2018. Graph-to-Sequence Learning using Gated Graph Neural Networks. In Proceedings of ACL 2018. (Citation:3)
• Danielle Saunders, Felix Stahlberg, Adrià de Gispert, and Bill Byrne. 2018. Multi-representation Ensembles and Delayed SGD Updates Improve Syntax-based NMT. In Proceedings of ACL 2018.
• Wen Zhang, Jiawei Hu, Yang Feng, and Qun Liu. 2018. Refining Source Representations with Relation Networks for Neural Machine Translation. In Proceedings of COLING 2018.
• Poorya Zaremoodi and Gholamreza Haffari. 2018. Incorporating Syntactic Uncertainty in Neural Machine Translation with a Forest-to-Sequence Model. In Proceedings of COLING 2018.
• Hao Zhang, Axel Ng, and Richard Sproat. 2018. Fast and Accurate Reordering with ITG Transition RNN. In Proceedings of COLING 2018.
• Jetic Gū, Hassan S. Shavarani, and Anoop Sarkar. 2018. Top-down Tree Structured Decoding with Syntactic Connections for Neural Machine Translation and Parsing. In Proceedings of EMNLP 2018.
• Anna Currey and Kenneth Heafield. 2018. Multi-Source Syntactic Neural Machine Translation. In Proceedings of EMNLP 2018.
• Xinyi Wang, Hieu Pham, Pengcheng Yin, and Graham Neubig. 2018. A Tree-based Decoder for Neural Machine Translation. In Proceedings of EMNLP 2018. (Citation: 1)
• Eliyahu Kiperwasser and Miguel Ballesteros. 2018. Scheduled Multi-Task Learning: From Syntax to Translation. Transactions of the Association for Computational Linguistics. (Citation: 4)
• Xiao Pu, Nikolaos Pappas, James Henderson, and Andrei Popescu-Belis. 2018. Integrating Weakly Supervised Word Sense Disambiguation into Neural Machine Translation. Transactions of the Association for Computational Linguistics.
• Kai Song, Yue Zhang, Min Zhang, and Weihua Luo. 2018. Improved English to Russian Translation by Neural Suffix Prediction. In Proceedings of AAAI 2018.
• Rudra Murthy V, Anoop Kunchukuttan, and Pushpak Bhattacharyya. 2019. Addressing word-order Divergence in Multilingual Neural Machine Translation for extremely Low Resource Languages. In Proceedings of NAACL 2019.
• Meishan Zhang, Zhenghua Li, Guohong Fu, and Min Zhang. 2019. Syntax-Enhanced Neural Machine Translation with Syntax-Aware Word Representations. In Proceedings of NAACL 2019.
• Linfeng Song, Daniel Gildea, Yue Zhang, Zhiguo Wang, and Jinsong Su. 2019. Semantic Neural Machine Translation Using AMR. Transactions of the Association for Computational Linguistics.
• Nader Akoury, Kalpesh Krishna, and Mohit Iyyer. 2019. Syntactically Supervised Transformers for Faster Neural Machine Translation. In Proceedings of ACL 2019.
• Zhijiang Guo, Yan Zhang, Zhiyang Teng, and Wei Lu. 2019. Densely Connected Graph Convolutional Networks for Graph-to-Sequence Learning. Transactions of the Association for Computational Linguistics.
• Xuewen Yang, Yingru Liu, Dongliang Xie, Xin Wang, and Niranjan Balasubramanian. 2019. Latent Part-of-Speech Sequences for Neural Machine Translation. In Proceedings of EMNLP 2019.
• Jie Hao, Xing Wang, Shuming Shi, Jinfeng Zhang, and Zhaopeng Tu. 2019. Multi-Granularity Self-Attention for Neural Machine Translation. In Proceedings of EMNLP 2019.
• Jie Hao, Xing Wang, Shuming Shi, Jinfeng Zhang, and Zhaopeng Tu. 2019. Towards Better Modeling Hierarchical Structure for Self-Attention with Ordered Neurons. In Proceedings of EMNLP 2019.
• KayYen Wong, Sameen Maruf and Gholamreza Haffari. 2020. Contextual Neural Machine Translation Improves Translation of Cataphoric Pronouns. In Proceedings of ACL 2020.
• Emanuele Bugliarello and Naoaki Okazaki. 2020. Enhancing Machine Translation with Dependency-Aware Self-Attention. In Proceedings of ACL 2020.
• Kehai Chen, Rui Wang, Masao Utiyama, and Eiichiro Sumita. 2020. Content Word Aware Neural Machine Translation. In Proceedings of ACL 2020.
• Jian Yang, Shuming Ma, Dongdong Zhang, Zhoujun Li and Ming Zhou. 2020. Improving Neural Machine Translation with Soft Template Prediction. In Proceedings of ACL 2020.
• R. Thomas McCoy, Robert Frank and Tal Linzen. 2020. Does Syntax Need to Grow on Trees? Sources of Hierarchical Inductive Bias in Sequence-to-Sequence Networks. Transactions of the Association for Computational Linguistics.

Coverage Constraints

• Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. 2016. Modeling Coverage for Neural Machine Translation. In Proceedings of ACL 2016. (Citation: 236)
• Haitao Mi, Baskaran Sankaran, Zhiguo Wang, and Abe Ittycheriah. 2016. Coverage Embedding Models for Neural Machine Translation. In Proceedings of EMNLP 2016. (Citation: 59)
• Zhaopeng Tu, Yang Liu, Zhengdong Lu, Xiaohua Liu, and Hang Li. 2017. Context Gates for Neural Machine Translation. Transactions of the Association for Computational Linguistics. (Citation: 36)
• Yanyang Li, Tong Xiao, Yinqiao Li, Qiang Wang, Changming Xu, and Jingbo Zhu. 2018. A Simple and Effective Approach to Coverage-Aware Neural Machine Translation. In Proceedings of ACL 2018. (Citation: 1)
• Zaixiang Zheng, Hao Zhou, Shujian Huang, Lili Mou, Xinyu Dai, Jiajun Chen, and Zhaopeng Tu. 2018. Modeling Past and Future for Neural Machine Translation. Transactions of the Association for Computational Linguistics. (Citation: 10)
• Xiang Kong, Zhaopeng Tu, Shuming Shi, Eduard Hovy, and Tong Zhang. Neural Machine Translation with Adequacy-Oriented Learning. In Proceedings of AAAI 2019.
• Zaixiang Zheng, Shujian Huang, Zhaopeng Tu, Xin-Yu Dai, and Jiajun Chen. 2019. Dynamic Past and Future for Neural Machine Translation. In Proceedings of EMNLP 2019.

Document-level Translation

• Longyue Wang, Zhaopeng Tu, Andy Way, and Qun Liu. 2017. Exploiting Cross-Sentence Context for Neural Machine Translation. In Proceedings of EMNLP 2017. (Citation: 19)
• Jörg Tiedemann, and Yves Scherrer. 2017. Neural Machine Translation with Extended Context. In Proceedings of the Third Workshop on Discourse in Machine Translation. (Citation: 12)
• Rachel Bawden, Rico Sennrich, Alexandra Birch, and Barry Haddow. 2018. Evaluating Discourse Phenomena in Neural Machine Translation. In Proceedings of NAACL 2018. (Citation: 11)
• Elena Voita, Pavel Serdyukov, Rico Sennrich, and Ivan Titov. 2018. Context-Aware Neural Machine Translation Learns Anaphora Resolution. In Proceedings of ACL 2018. (Citation: 7)
• Sameen Maruf and Gholamreza Haffari. 2018. Document Context Neural Machine Translation with Memory Networks. In Proceedings of ACL 2018. (Citation: 5)
• Shaohui Kuang, Deyi Xiong, Weihua Luo, Guodong Zhou. 2018. Modeling Coherence for Neural Machine Translation with Dynamic and Topic Caches. In Proceedings of COLING 2018. (Citation: 1)
• Shaohui Kuang and Deyi Xiong. 2018. Fusing Recency into Neural Machine Translation with an Inter-Sentence Gate Model. In Proceedings of COLING 2018.
• Jiacheng Zhang, Huanbo Luan, Maosong Sun, Feifei Zhai, Jingfang Xu, Min Zhang and Yang Liu. 2018. Improving the Transformer Translation Model with Document-Level Context. In Proceedings of EMNLP 2018.
• Samuel Läubli, Rico Sennrich, and Martin Volk. 2018. Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation. In Proceedings of EMNLP 2018. (Citation: 1)
• Lesly Miculicich, Dhananjay Ram, Nikolaos Pappas, and James Henderson. 2018. Document-Level Neural Machine Translation with Hierarchical Attention Networks. In Proceedings of EMNLP 2018.
• Zhaopeng Tu, Yang Liu, Shuming Shi, and Tong Zhang. 2018. Learning to Remember Translation History with a Continuous Cache. Transactions of the Association for Computational Linguistics. (Citation: 9)
• Elena Voita, Rico Sennrich, and Ivan Titov. 2019. When a Good Translation is Wrong in Context: Context-Aware Machine Translation Improves on Deixis, Ellipsis, and Lexical Cohesion. In Proceedings of ACL 2019.
• Elena Voita, Rico Sennrich, and Ivan Titov. 2019. Context-Aware Monolingual Repair for Neural Machine Translation. In Proceedings of EMNLP 2019.
• Zhengxin Yang, Jinchao Zhang, Fandong Meng, Shuhao Gu, Yang Feng, and Jie Zhou. 2019. Enhancing Context Modeling with a Query-Guided Capsule Network for Document-level Translation. In Proceedings of EMNLP 2019.
• Xin Tan, Longyin Zhang, Deyi Xiong, Guodong Zhou. 2019. Hierarchical Modeling of Global Context for Document-Level Neural Machine Translation. In Proceedings of EMNLP 2019.
• Yunsu Kim, Duc Thanh Tran, Hermann Ney. 2019. When and Why is Document-level Context Useful in Neural Machine Translation? In Proceedings of DiscoMT@EMNLP 2019.
• Zuchao Li, Rui Wang, Kehai Chen, Masao Utiyama, Eiichiro Sumita, Zhuosheng Zhang and Hai Zhao. 2020. Explicit Sentence Compression for Neural Machine Translation. In Proceedings of AAAI 2020.
• Bei Li, Hui Liu, Ziyang Wang, Yufan Jiang, Tong Xiao, Jingbo Zhu, Tongran Liu and Changliang Li. 2020. Does Multi-Encoder Help? A Case Study on Context-Aware Neural Machine Translation. In Proceedings of ACL 2020.
• Xintong Li, Lemao Liu, Rui Wang, Guoping Huang and Max Meng. 2020. Regularized Context Gates on Transformer for Machine Translation. In Proceedings of ACL 2020.
• Danielle Saunders, Felix Stahlberg and Bill Byrne. 2020. Using Context in Neural Machine Translation Training Objectives. In Proceedings of ACL 2020.
• Shuming Ma, Dongdong Zhang and Ming Zhou. 2020. A Simple and Effective Unified Encoder for Document-Level Machine Translation. In Proceedings of ACL 2020.
• Zaixiang Zheng, Xiang Yue, Shujian Huang, Jiajun Chen, Alexandra Birch. 2020. Towards Making the Most of Context in Neural Machine Translation. In Proceedings of IJCAI 2020.
• Lei Yu, Laurent Sartran, Wojciech Stokowiec, Wang Ling, Lingpeng Kong, Phil Blunsom and Chris Dyer. 2020. Better Document-Level Machine Translation with Bayes' Rule. Transactions of the Association for Computational Linguistics.
• Xiaomian Kang, Yang Zhao, Jiajun Zhang, Chengqing Zong. 2020. Dynamic Context Selection for Document-level Neural Machine Translation via Reinforcement Learning. In Proceedings of EMNLP 2020.
• Pei Zhang, Boxing Chen, Niyu Ge, Kai Fan. 2020. Long-Short Term Masking Transformer: A Simple but Effective Baseline for Document-level Neural Machine Translation. In Proceedings of EMNLP 2020.
• Domenic Donato, Lei Yu, Chris Dyer . 2021 . Diverse Pretrained Context Encodings Improve Document Translation. In Proceedings of ACL 2021.

Robustness

• Yonatan Belinkov and Yonatan Bisk. 2018. Synthetic and Natural Noise Both Break Neural Machine Translation. In Proceedings of ICLR 2018. (Citation: 33)
• Zhengli Zhao, Dheeru Dua, and Sameer Singh. 2018. Generating Natural Adversarial Examples. In Proceedings of ICLR 2018. (Citation: 45)
• Yong Cheng, Zhaopeng Tu, Fandong Meng, Junjie Zhai, and Yang Liu. 2018. Towards Robust Neural Machine Translation. In Proceedings of ACL 2018. (Citation: 5)
• Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Semantically Equivalent Adversarial Rules for Debugging NLP models. In Proceedings of ACL 2018. (Citation: 12)
• Javid Ebrahimi, Daniel Lowd, and Dejing Dou. 2018. On Adversarial Examples for Character-Level Neural Machine Translation. In Proceedings of COLING 2018.
• Paul Michel and Graham Neubig. 2018. MTNT: A Testbed for Machine Translation of Noisy Text. In Proceedings of EMNLP 2018.
• Antonios Anastasopoulos, Alison Lui, Toan Nguyen, and David Chiang. 2019. Neural Machine Translation of Text from Non-Native Speakers. In Proceedings of NAACL 2019.
• Paul Michel, Xian Li, Graham Neubig, and Juan Miguel Pino. 2019. On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models. In Proceedings of NAACL 2019.
• Vaibhav Vaibhav, Sumeet Singh, Craig Stewart, and Graham Neubig. 2019. Improving Robustness of Machine Translation with Synthetic Noise. In Proceedings of NAACL 2019.
• Yong Cheng, Lu Jiang, and Wolfgang Macherey. 2019. Robust Neural Machine Translation with Doubly Adversarial Inputs. In Proceedings of ACL 2019.
• Hairong Liu, Mingbo Ma, Liang Huang, Hao Xiong, and Zhongjun He. 2019. Robust Neural Machine Translation with Joint Textual and Phonetic Embedding. In Proceedings of ACL 2019.
• Zhouxing Shi, Huan Zhang, Kai-Wei Chang, Minlie Huang, Cho-Jui Hsieh. 2020. Robustness Verification for Transformers. In Proceedings of ICLR 2020.
• Wei Zou, Shujian Huang, Jun Xie, Xinyu Dai and Jiajun Chen. 2020. A Reinforced Generation of Adversarial Examples for Neural Machine Translation. In Proceedings of ACL 2020.
• Xing Niu, Prashant Mathur, Georgiana Dinu and Yaser Al-Onaizan. 2020. Evaluating Robustness to Input Perturbations for Neural Machine Translation. In Proceedings of ACL 2020.
• Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, Adam Dziedzic, Rishabh Krishnan and Dawn Song. 2020. Pretrained Transformers Improve Out-of-Distribution Robustness. In Proceedings of ACL 2020.
• Yong Cheng, Lu Jiang, Wolfgang Macherey and Jacob Eisenstein. 2020. AdvAug: Robust Adversarial Augmentation for Neural Machine Translation. In Proceedings of ACL 2020.
• Eric Wallace, Mitchell Stern, Dawn Song. 2020. Imitation Attacks and Defenses for Black-box Machine Translation Systems. In Proceedings of EMNLP 2020.
• Denis Emelin, Ivan Titov, Rico Sennrich. 2020. Detecting Word Sense Disambiguation Biases in Machine Translation for Model-Agnostic Adversarial Attacks. In Proceedings of EMNLP 2020.
• Xinze Zhang , Junzhe Zhang , Zhenhua Chen , Kun He . 2021. Crafting Adversarial Examples for Neural Machine Translation . In Proceedings of ACL 2021.

Interpretability

• Yanzhuo Ding, Yang Liu, Huanbo Luan and Maosong Sun. 2017. Visualizing and Understanding Neural Machine Translation. In Proceedings of ACL 2017. (Citation: 22)
• Hendrik Strobelt, Sebastian Gehrmann, Michael Behrisch, Adam Perer, Hanspeter Pfister, and Alexander M. Rush. 2018. Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models. In Proceedings of VAST 2018 and Proceedings of EMNLP-BlackBox 2018. (Citation: 6)
• Alessandro Raganato and Jorg Tiedemann. 2018. An Analysis of Encoder Representations in Transformer-Based Machine Translation. In Proceedings of EMNLP-BlackBox 2018.
• Felix Stahlberg, Danielle Saunders, and Bill Byrne. 2018. An Operation Sequence Model for Explainable Neural Machine Translation. In Proceedings of EMNLP-BlackBox 2018.
• Fahim Dalvi, Nadir Durrani, Hassan Sajjad, Yonatan Belinkov, D. Anthony Bau, and James Glass. 2019. What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Models. In Proceedings of AAAI 2019. (Citation: 2)
• Anthony Bau, Yonatan Belinkov, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and James Glass. 2019. Identifying and Controlling Important Neurons in Neural Machine Translation. In Proceedings of ICLR 2019. (Citation: 2)
• Yonatan Belinkov, and James Glass. 2019. Analysis Methods in Neural Language Processing: A Survey. Transactions of the Association for Computational Linguistics.
• Sofia Serrano and Noah A. Smith. 2019. Is Attention Interpretable?. In Proceedings of ACL 2019.
• Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. 2019. Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned. In Proceedings of ACL 2019.
• Joris Baan, Jana Leible, Mitja Nikolaus, David Rau, Dennis Ulmer, Tim Baumgärtner, Dieuwke Hupkes, and Elia Bruni. 2019. On the Realization of Compositionality in Neural Networks. In Proceedings of ACL 2019.
• Jesse Vig and Yonatan Belinkov. 2019. Analyzing the Structure of Attention in a Transformer Language Model. In Proceedings of ACL 2019.
• Baosong Yang, Longyue Wang, Derek F. Wong, Lidia S. Chao, and Zhaopeng Tu. 2019. Assessing the Ability of Self-Attention Networks to Learn Word Order. In Proceedings of ACL 2019.
• Xintong Li, Guanlin Li, Lemao Liu, Max Meng, and Shuming Shi. 2019. On the Word Alignment from Neural Machine Translation. In Proceedings of ACL 2019.
• Elena Voita, Rico Sennrich, and Ivan Titov. 2019. The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives. In Proceedings of EMNLP 2019.
• Shilin He, Zhaopeng Tu, Xing Wang, Longyue Wang, Michael R. Lyu, and Shuming Shi. 2019. Towards Understanding Neural Machine Translation with Word Importance. In Proceedings of EMNLP 2019.
• Felix Stahlberg and Bill Byrne. 2019. On NMT Search Errors and Model Errors: Cat Got Your Tongue?. In Proceedings of EMNLP 2019.
• Gino Brunner, Yang Liu, Damian Pascual, Oliver Richter, Massimiliano Ciaramita, Roger Wattenhofer. 2020. On Identifiability in Transformers. In Proceedings of ICLR 2020.
• Chulhee Yun, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank Reddi, Sanjiv Kumar. 2020. Are Transformers universal approximators of sequence-to-sequence functions?. In Proceedings of ICLR 2020.
• Akash Kumar Mohankumar, Preksha Nema, Sharan Narasimhan, Mitesh M. Khapra, Balaji Vasan Srinivasan and Balaraman Ravindran. 2020. Towards Transparent and Explainable Attention Models. In Proceedings of ACL 2020.
• Samira Abnar and Willem Zuidema. 2020. Quantifying Attention Flow in Transformers. In Proceedings of ACL 2020.
• Jierui Li, Lemao Liu, Huayang Li, Guanlin Li, Guoping Huang and Shuming Shi. 2020. Evaluating Explanation Methods for Neural Machine Translation. In Proceedings of ACL 2020.
• Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, Kentaro Inui. 2020. Attention is Not Only a Weight: Analyzing Transformers with Vector Norms. In Proceedings of EMNLP 2020.
• Liyuan Liu, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Jiawei Han. 2020. Understanding the Difficulty of Training Transformers. In Proceedings of EMNLP 2020.
• Wenxuan Wang, Zhaopeng Tu. 2020. Rethinking the Value of Transformer Components. In Proceedings of COLING 2020.
• Elena Voita , Rico Sennrich , Ivan Titov . 2021 . Analyzing the Source and Target Contributions to Predictions in Neural Machine Translation . In Proceedings of ACL 2021.
• Weicheng Ma, Kai Zhang , Renze Lou , Lili Wang . 2021 . Contributions of Transformer Attention Heads in Multi- and Cross-lingual Tasks . In Proceedings of ACL 2021.

Linguistic Interpretation

• Felix Hill, Kyunghyun Cho, Sebastien Jean, Coline Devin, and Yoshua Bengio. 2015. Embedding Word Similarity with Neural Machine Translation. In Proceedings of ICLR 2015. (Citation: 24)
• Xing Shi, Inkit Padhi, and Kevin Knight. 2016. Does String-based Neural MT Learn Source Syntax?. In Proceedings of the EMNLP 2016. (Citation: 57)
• Yonatan Belinkov, Nadir Durrani, Fahim Dalvi, Hassan Sajjad, and James Glass. 2017. What do Neural Machine Translation Models Learn about Morphology?. In Proceedings of ACL 2017. (Citation: 50)
• Ella Rabinovich, Noam Ordan, and Shuly Wintner. 2017. Found in Translation: Reconstructing Phylogenetic Language Trees from Translations. In Proceedings of ACL 2017. (Citation: 6)
• Rico Sennrich. 2017. How Grammatical is Character-level Neural Machine Translation? Assessing MT Quality with Contrastive Translation Pairs. In Proceedings of EACL 2017. (Citation: 25)
• Adam Poliak, Yonatan Belinkov, James Glass, and Benjamin Van Durme. 2018. On the Evaluation of Semantic Phenomena in Neural Machine Translation Using Natural Language Inference. In Proceedings of NAACL 2018. (Citation: 5)
• Arianna Bisazza and Clara Tump. 2018. The Lazy Encoder: A Fine-Grained Analysis of the Role of Morphology in Neural Machine Translation. In Proceedings of EMNLP 2018.
• Lijun Wu, Xu Tan, Di He, Fei Tian, Tao Qin, Jianhuang Lai, and Tie-Yan Liu. 2018. Beyond Error Propagation in Neural Machine Translation: Characteristics of Language Also Matter. In Proceedings of EMNLP 2018. (Citation: 4)
• Gongbo Tang, Rico Sennrich, and Joakim Nivre. 2019. Encoders Help You Disambiguate Word Senses in Neural Machine Translation. In Proceedings of EMNLP 2019.
• Parker Riley, Isaac Caswell, Markus Freitag and David Grangier. 2020. Translationese as a Language in “Multilingual” NMT. In Proceedings of ACL 2020.
• Emanuele Bugliarello, Sabrina J. Mielke, Antonios Anastasopoulos, Ryan Cotterell and Naoaki Okazaki. 2020. It’s Easier to Translate out of English than into it: Measuring Neural Translation Difficulty by Cross-Mutual Information. In Proceedings of ACL 2020.

Fairness and Diversity

• Hayahide Yamagishi, Shin Kanouchi, Takayuki Sato, and Mamoru Komachi. 2016. Controlling the Voice of a Sentence in Japanese-to-English Neural Machine Translation. In Proceedings of the 3rd Workshop on Asian Translation. (Citation: 11)
• Rico Sennrich, Barry Haddow and Alexandra Birch. 2016. Controlling Politeness in Neural Machine Translation via Side Constraints. In Proceedings of NAACL 2016. (Citation: 49)
• Xing Niu, Marianna Martindale, and Marine Carpuat. 2017. A Study of Style in Machine Translation: Controlling the Formality of Machine Translation Output. In Proceedings of EMNLP 2016. (Citation: 8)
• Ella Rabinovich, Raj Nath Patel, Shachar Mirkin, Lucia Specia, and Shuly Wintner. 2017. Personalized Machine Translation: Preserving Original Author Traits. In Proceedings of EACL 2017. (Citation: 10)
• Myle Ott, Michael Auli, David Grangier, and Marc'Aurelio Ranzato. 2018. Analyzing Uncertainty in Neural Machine Translation. In Proceedings of ICML 2018. (Citation: 11)
• Paul Michel and Graham Neubig. 2018. Extreme Adaptation for Personalized Neural Machine Translation. In Proceedings of ACL 2018. (Citation: 6)
• Eva Vanmassenhove, Christian Hardmeier, and Andy Way. 2018. Getting Gender Right in Neural Machine Translation. In Proceedings of EMNLP 2018.
• Ashwin Kalyan, Peter Anderson, Stefan Lee, and Dhruv Batra. 2019. Trainable Decoding of Sets of Sequences for Neural Sequence Models. In Proceedings of ICML 2019.
• Tianxiao Shen, Myle Ott, Michael Auli, and Marc’Aurelio Ranzato. 2019. Mixture Models for Diverse Machine Translation: Tricks of the Trade. In Proceedings of ICML 2019.
• Wouter Kool, Herke van Hoof, and Max Welling. 2019. Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacement. In Proceedings of ICML 2019.
• Won Ik Cho, Ji Won Kim, Seok Min Kim, and Nam Soo Kim. 2019. On Measuring Gender Bias in Translation of Gender-neutral Pronouns. In Proceedings of ACL 2019.
• Gabriel Stanovsky, Noah A. Smith, and Luke Zettlemoyer. 2019. Evaluating Gender Bias in Machine Translation. In Proceedings of ACL 2019.
• Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019. Measuring Bias in Contextualized Word Representations. In Proceedings of ACL 2019.
• Raphael Shu, Hideki Nakayama, and Kyunghyun Cho. 2019. Generating Diverse Translations with Sentence Codes. In Proceedings of ACL 2019.
• Daphne Ippolito, Reno Kriz, Joao Sedoc, Maria Kustikova, and Chris Callison-Burch. 2019. Comparison of Diverse Decoding Methods from Conditional Language Models. In Proceedings of ACL 2019.
• Xing Niu and Marine Carpuat. 2020. Controlling Neural Machine Translation Formality with Synthetic Supervision. In Proceedings of AAAI 2020.
• Zewei Sun, Shujian Huang, Hao-Ran Wei, Xin-yu Dai, and Jiajun Chen. 2020. Generating Diverse Translation by Manipulating Multi-Head Attention. In Proceedings of AAAI 2020.
• Shuo Wang, Zhaopeng Tu, Shuming Shi and Yang Liu. 2020. On the Inference Calibration of Neural Machine Translation. In Proceedings of ACL 2020.
• Danielle Saunders and Bill Byrne. 2020. Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem. In Proceedings of ACL 2020.
• Dirk Hovy, Federico Bianchi and Tommaso Fornaciari. 2020. “You Sound Just Like Your Father” Commercial Machine Translation Systems Include Stylistic Biases. In Proceedings of ACL 2020.
• Luisa Bentivogli, Beatrice Savoldi, Matteo Negri, Mattia A. Di Gangi, Roldano Cattoni and Marco Turchi. 2020. Gender in Danger? Evaluating Speech Translation Technology on the MuST-SHE Corpus. In Proceedings of ACL 2020.
• Sorami Hisamoto, Matt Post and Kevin Duh. 2020. Membership Inference Attacks on Sequence-to-Sequence Models: Is My Data In Your Machine Translation System?. Transactions of the Association for Computational Linguistics.
• Huda Khayrallah, Brian Thompson, Matt Post, Philipp Koehn. 2020. Simulated multiple reference training improves low-resource machine translation. In Proceedings of EMNLP 2020.
• Xuanfu Wu, Yang Feng, Chenze Shao. 2020. Generating Diverse Translation from Model Distribution with Dropout. In Proceedings of EMNLP 2020.

Efficiency

• Abigail See, Minh-Thang Luong, and Christopher D. Manning. 2016. Compression of Neural Machine Translation Models via Pruning. In Proceedings of CoNLL 2016. (Citation: 33)
• Yusuke Oda, Philip Arthur, Graham Neubig, Koichiro Yoshino, and Satoshi Nakamura. 2017. Neural Machine Translation via Binary Code Prediction. In Proceedings of ACL 2017. (Citation: 3)
• Xing Shi and Kevin Knight. 2017. Speeding Up Neural Machine Translation Decoding by Shrinking Run-time Vocabulary. In Proceedings of ACL 2017. (Citation: 5)
• Ofir Press and Lior Wolf. 2017. Using the Output Embedding to Improve Language Models. In Proceedings of EACL 2017. (Citation: 126)
• Xiaowei Zhang, Wei Chen, Feng Wang, Shuang Xu, and Bo Xu. 2017. Towards Compact and Fast Neural Machine Translation Using a Combined Method. In Proceedings of EMNLP 2017. (Citation: 5)
• Felix Stahlberg and Bill Byrne. 2017. Unfolding and Shrinking Neural Machine Translation Ensembles. In Proceedings of EMNLP 2017. (Citation: 5)
• Jacob Devlin. 2017. Sharp Models on Dull Hardware: Fast and Accurate Neural Machine Translation Decoding on the CPU. In Proceedings of EMNLP 2017. (Citation: 8)
• Dakun Zhang, Jungi Kim, Josep Crego, and Jean Senellart. 2017. Boosting Neural Machine Translation. In Proceedings of IJCNLP 2017. (Citation: 3)
• Łukasz Kaiser, Aurko Roy, Ashish Vaswani, Niki Parmar, Samy Bengio, Jakob Uszkoreit, and Noam Shazeer. 2018. Fast Decoding in Sequence Models Using Discrete Latent Variables. In Proceedings of ICML 2018. (Citation: 3)
• Gonzalo Iglesias, William Tambellini, Adrià de Gispert, Eva Hasler, and Bill Byrne. 2018. Accelerating NMT Batched Beam Decoding with LMBR Posteriors for Deployment. In Proceedings of NAACL 2018.
• Jerry Quinn and Miguel Ballesteros. 2018. Pieces of Eight: 8-bit Neural Machine Translation. In Proceedings of NAACL 2018.
• Matt Post and David Vilar. 2018. Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation. In Proceedings of NAACL 2018. (Citation: 6)
• Biao Zhang, Deyi Xiong, and Jinsong Su. 2018. Accelerating Neural Transformer via an Average Attention Network. In Proceedings of ACL 2018. (Citation: 5)
• Rui Wang, Masao Utiyama, and Eiichiro Sumita. 2018. Dynamic Sentence Sampling for Efficient Training of Neural Machine Translation. In Proceedings of ACL 2018. (Citation: 2)
• Myle Ott, Sergey Edunov, David Grangier, and Michael Auli. 2018. Scaling Neural Machine Translation. In Proceedings of the Third Conference on Machine Translation: Research Papers.
• Joern Wuebker, Patrick Simianer, and John DeNero. 2018. Compact Personalized Models for Neural Machine Translation. In Proceedings of EMNLP 2018.
• Wen Zhang, Liang Huang, Yang Feng, Lei Shen, and Qun Liu. 2018. Speeding Up Neural Machine Translation Decoding by Cube Pruning. In Proceedings of EMNLP 2018.
• Zhisong Zhang, Rui Wang, Masao Utiyama, Eiichiro Sumita, and Hai Zhao. 2018. Exploring Recombination for Efficient Decoding of Neural Machine Translation. In Proceedings of EMNLP 2018.
• Nikolay Bogoychev, Kenneth Heafield, Alham Fikri Aji, and Marcin Junczys-Dowmunt. 2018. Accelerating Asynchronous Stochastic Gradient Descent for Neural Machine Translation. In Proceedings of EMNLP 2018. (Citation: 2)
• Mitchell Stern, Noam Shazeer, and Jakob Uszkoreit. 2018. Blockwise Parallel Decoding for Deep Autoregressive Models. In Proceedings of NeurIPS 2018.
• Zhanghao Wu, Zhijian Liu, Ji Lin, Yujun Lin, Song Han. 2020. Efficient Transformer for Mobile Applications. In Proceedings of ICLR 2020.
• Christopher Brix, Parnia Bahar and Hermann Ney. 2020. Successfully Applying the Stabilized Lottery Ticket Hypothesis to the Transformer Architecture. In Proceedings of ACL 2020.
• Hanrui Wang, Zhanghao Wu, Zhijian Liu, Han Cai, Ligeng Zhu, Chuang Gan, and Song Han. 2020. HAT: Hardware-Aware Transformers for Efficient Natural Language Processing. In Proceedings of ACL 2020.
• Zhuohan Li, Eric Wallace, Sheng Shen, Kevin Lin, Kurt Keutzer, Dan Klein, Joseph E. Gonzalez. 2020. Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers. In Proceedings of ICML 2020.
• Maximiliana Behnke, Kenneth Heafield. 2020. Losing Heads in the Lottery: Pruning Transformer Attention in Neural Machine Translation. In Proceedings of EMNLP 2020.
• Minjia Zhang, Yuxiong He. 2020. Accelerating Training of Transformer-Based Language Models with Progressive Layer Dropping. In Proceedings of NeurIPS 2020.
• Yimeng Wu, Peyman Passban, Mehdi Rezagholizadeh, Qun Liu. 2020. Why Skip If You Can Combine: A Simple Knowledge Distillation Technique for Intermediate Layers. In Proceedings of EMNLP 2020.

Pre-Training

• Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. 2017. Learned in Translation: Contextualized Word Vectors. In Proceedings of NIPS 2017. (Citation: 136)
• Ye Qi, Devendra Sachan, Matthieu Felix, Sarguna Padmanabhan, and Graham Neubig. 2018. When and Why Are Pre-Trained Word Embeddings Useful for Neural Machine Translation?. In Proceedings of NAACL 2018. (Citation: 19)
• Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep Contextualized Word Representations. In Proceedings of NAACL 2018. (Citation: 519)
• Jeremy Howard and Sebastian Ruder. 2018. Universal Language Model Fine-tuning for Text Classification. In Proceedings of ACL 2018. (Citation: 114)
• Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. XNLI: Evaluating Cross-lingual Sentence Representations. In Proceedings of EMNLP 2018. (Citation: 9)
• Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving Language Understanding by Generative Pre-Training. Technical Report, OpenAI. (Citation: 94)
• Guillaume Lample and Alexis Conneau. 2019. Cross-lingual Language Model Pretraining. arXiv:1901.07291. (Citation: 3)
• Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of NAACL 2019. (Citation: 292)
• Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners. Technical Report, OpenAI. (Citation: 9)
• Sergey Edunov, Alexei Baevski, and Michael Auli. 2019. Pre-trained Language Model Representations for Language Generation. In Proceedings of NAACL 2019. (Citation: 1)
• Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2019. MASS: Masked Sequence to Sequence Pre-training for Language Generation. In Proceedings of ICML 2019.
• Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019. XLNet: Generalized Autoregressive Pretraining for Language Understanding. arXiv:1906.08237.
• Jiacheng Yang, Mingxuan Wang, Hao Zhou, Chengqi Zhao, Yong Yu, Weinan Zhang and Lei Li. 2020. Towards Making the Most of BERT in Neural Machine Translation. In Proceedings of AAAI 2020.
• Rongxiang Weng, Heng Yu, Shujian Huang, Shanbo Cheng and Weihua Luo. 2020. Acquiring Knowledge from Pre-trained Model to Neural Machine Translation. In Proceedings of AAAI 2020.
• Jinhua Zhu, Yingce Xia, Lijun Wu, Di He, Tao Qin, Wengang Zhou, Houqiang Li, Tieyan Liu. 2020. Incorporating BERT into Neural Machine Translation. In Proceedings of ICLR 2020.
• Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov and Luke Zettlemoyer. 2020. BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension. In Proceedings of ACL 2020.
• Thibault Sellam, Dipanjan Das and Ankur Parikh. 2020. BLEURT: Learning Robust Metrics for Text Generation. In Proceedings of ACL 2020.
• Sascha Rothe, Shashi Narayan and Aliaksei Severyn. 2020. Leveraging Pre-trained Checkpoints for Sequence Generation Tasks. Transactions of the Association for Computational Linguistics.
• Zhen Yang, Bojie Hu, Ambyera Han, Shen Huang, Qi Ju. 2020. CSP:Code-Switching Pre-training for Neural Machine Translation. In Proceedings of EMNLP 2020.
• Zehui Lin, Xiao Pan, Mingxuan Wang, Xipeng Qiu, Jiangtao Feng, Hao Zhou, Lei Li. 2020. Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information. In Proceedings of EMNLP 2020.
• Junliang Guo, Zhirui Zhang, Linli Xu, Hao-Ran Wei, Boxing Chen, Enhong Chen. 2020. Incorporating BERT into Parallel Sequence Decoding with Adapters. In Proceedings of NeurIPS 2020.
• Linqing Chen , Junhui Li , Zhengxian Gong , Boxing Chen , Weihua Luo , Min Zhang , Guodong Zhou . 2021. Breaking the Corpus Bottleneck for Context-Aware Neural Machine Translation with Cross-Task Pre-training . In Proceedings of ACL 2021.

Non-Autoregressive Translation

• Jiatao Gu, James Bradbury, Caiming Xiong, Victor O.K. Li, and Richard Socher. 2018. Non-Autoregressive Neural Machine Translation. In Proceedings of ICLR 2018. (Citation: 93)
• Chunqi Wang, Ji Zhang, and Haiqing Chen. 2018. Semi-Autoregressive Neural Machine Translation. In Proceedings of EMNLP 2018.
• Jason Lee, Elman Mansimov, and Kyunghyun Cho. 2018. Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement. In Proceedings of EMNLP 2018.
• Jindřich Libovický and Jindřich Helcl. 2018. End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classification. In Proceedings of EMNLP 2018.
• Xuezhe Ma, Chunting Zhou, Xian Li, Graham Neubig, and Eduard Hovy. 2019. FlowSeq: Non-Autoregressive Conditional Sequence Generation with Generative Flow. In Proceedings of EMNLP 2019.
• Zhuohan Li, Zi Lin, Di He, Fei Tian, Tao Qin, Liwei Wang, and Tie-Yan Liu. 2019. Hint-Based Training for Non-Autoregressive Machine Translation. In Proceedings of EMNLP 2019.
• Marjan Ghazvininejad, Omer Levy, Yinhan Liu, Luke Zettlemoyer. 2019. Mask-Predict: Parallel Decoding of Conditional Masked Language Models. In Proceedings of EMNLP 2019.
• Sean Welleck, Kianté Brantley, Hal Daumé III, and Kyunghyun Cho. 2019. Non-Monotonic Sequential Text Generation. In Proceedings of ICML 2019.
• Mitchell Stern, William Chan, Jamie Kiros, Jakob Uszkoreit. 2019. Insertion Transformer: Flexible Sequence Generation via Insertion Operations. In Proceedings of ICML 2019.
• Chenze Shao, Yang Feng, Jinchao Zhang, Fandong Meng, Xilin Chen, and Jie Zhou. 2019. Retrieving Sequential Information for Non-Autoregressive Neural Machine Translation. In Proceedings of ACL 2019.
• Bingzhen Wei, Mingxuan Wang, Hao Zhou, Junyang Lin, and Xu Sun. 2019. Imitation Learning for Non-Autoregressive Neural Machine Translation. In Proceedings of ACL 2019.
• Jiatao Gu, Changhan Wang, Junbo Zhao. 2019. Levenshtein Transformer. In Proceedings of NeurIPS 2019.
• Junliang Guo, Xu Tan, Di He, Tao Qin, Linli Xu, and Tie-Yan Liu. 2019. Non-Autoregressive Neural Machine Translation with Enhanced Decoder Input. In Proceedings of AAAI 2019.
• Yiren Wang, Fei Tian, Di He, Tao Qin, ChengXiang Zhai, and Tie-Yan Liu. 2019. Non-Autoregressive Machine Translation with Auxiliary Regularization. In Proceedings of AAAI 2019.
• Junliang Guo, Xu Tan, Linli Xu, Tao Qin, Enhong Chen, Tie-Yan Liu. 2020. Fine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine Translation. In Proceedings of AAAI 2020.
• Chenze Shao, Jinchao Zhang, Yang Feng, Fandong Meng and Jie Zhou. 2020. Minimizing the Bag-of-Ngrams Difference for Non-Autoregressive Neural Machine Translation. In Proceedings of AAAI 2020
• Raphael Shu, Jason Lee, Hideki Nakayama and Kyunghyun Cho. 2020. Latent-Variable Non-Autoregressive Neural Machine Translation with Deterministic Inference Using a Delta Posterior. In Proceedings of AAAI 2020.
• Jiawei Zhou and Phillip Keung. 2020. Improving Non-autoregressive Neural Machine Translation with Monolingual Data. In Proceedings of ACL 2020.
• Marjan Ghazvininejad, Vladimir Karpukhin, Luke Zettlemoyer, and Omer Levy. 2020. Aligned Cross Entropy for Non-Autoregressive Machine Translation. In Proceedings of ICML 2020.
• Jungo Kasai, James Cross, Marjan Ghazvininejad, and Jiatao Gu. 2020. Parallel Machine Translation with Disentangled Context Transformer. In Proceedings of ICML 2020.
• Junliang Guo, Linli Xu and Enhong Chen. 2020. Jointly Masked Sequence-to-Sequence Model for Non-Autoregressive Neural Machine Translation. In Proceedings of ACL 2020.
• Qiu Ran, Yankai Lin, Peng Li and Jie Zhou. 2020. Learning to Recover from Multi-Modality Errors for Non-Autoregressive Neural Machine Translation. In Proceedings of ACL 2020.
• William Chan, Mitchell Stern, Jamie Kiros, Jakob Uszkoreit. 2020. An Empirical Study of Generation Order for Machine Translation. In Proceedings of EMNLP 2020.
• Jason Lee, Raphael Shu, Kyunghyun Cho. 2020. Iterative Refinement in the Continuous Space for Non-Autoregressive Neural Machine Translation. In Proceedings of EMNLP 2020.
• Xiang Kong, Zhisong Zhang, Eduard Hovy. 2020. Incorporating a Local Translation Mechanism into Non-autoregressive Translation. In Proceedings of EMNLP 2020.
• Chitwan Saharia, William Chan, Saurabh Saxena, Mohammad Norouzi. 2020. Non-Autoregressive Machine Translation with Latent Alignments. In Proceedings of EMNLP 2020.
• Liang Ding, Longyue Wang, Di Wu, Dacheng Tao, Zhaopeng Tu. 2020. Context-Aware Cross-Attention for Non-Autoregressive Translation. In Proceedings of COLING 2020.
• Jungo Kasai, Nikolaos Pappas, Hao Peng, James Cross, Noah Smith. 2021. Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translation. In Proceedings of ICLR 2021.
• Liang Ding, Longyue Wang, Xuebo Liu, Derek F. Wong, Dacheng Tao, Zhaopeng Tu. 2021. Understanding and Improving Lexical Choice in Non-Autoregressive Translation. In Proceedings of ICLR 2021.
• Qiu Ran, Yankai Lin, Peng Li, Jie Zhou. 2021. Guiding Non-Autoregressive Neural Machine Translation Decoding with Reordering Information. In Proceedings of AAAI 2021.
• Yongchang Hao, Shilin He, Wenxiang Jiao, Zhaopeng Tu, Michael R. Lyu, Xing Wang. 2021. Multi-Task Learning with Shared Encoder for Non-Autoregressive Machine Translation. In Proceedings of NAACL 2021.
• Yu Bao, Shujian Huang, Tong Xiao, Dongqi Wang, Xinyu Dai, Jiajun Chen. 2021. Non-Autoregressive Translation by Learning Target Categorical Codes. In Proceedings of NAACL 2021.
• Liang Ding, Longyue Wang, Xuebo Liu, Derek F. Wong, Dacheng Tao, Zhaopeng Tu. 2021. Progressive Multi-Granularity Training for Non-Autoregressive Translation. In Proceedings of ACL 2021.
• Liang Ding, Longyue Wang, Xuebo Liu, Derek F. Wong, Dacheng Tao, Zhaopeng Tu. 2021. Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translation. In Proceedings of ACL 2021.
• Cunxiao Du, Zhaopeng Tu, Jing Jiang. 2021. Order-Agnostic Cross Entropy for Non-Autoregressive Machine Translation. In Proceedings of ICML 2021.
• Lihua Qian, Hao Zhou , Yu Bao , Mingxuan Wang , Lin Qiu , Weinan Zhang , Yong Yu , Lei Li . 2021. Glancing Transformer for Non-Autoregressive Neural Machine Translation . In Proceedings of ACL 2021.

Speech Translation and Simultaneous Translation

• Matt Post, Gaurav Kumar, Adam Lopez, Damianos Karakos, Chris Callison-Burch and Sanjeev Khudanpur. 2013. Improved Speech-to-Text Translation with the Fisher and Callhome Spanish–English Speech Translation Corpus. In Proceedings of IWSLT 2013. (Citation: 24)
• Gaurav Kumar, Matt Post, Daniel Povey and Sanjeev Khudanpur. 2014. Some insights from translating conversational telephone speech In Proceedings of ICASSP 2014. (Citation: 9)
• Long Duong, Antonios Anastasopoulos, David Chiang, Steven Bird, and Trevor Cohn. 2016. An Attentional Model for Speech Translation without Transcription. In Proceedings of NAACL 2016. (Citation: 37)
• Antonios Anastasopoulos, David Chiang, and Long Duong. 2016. An Unsupervised Probability Model for Speech-to-translation Alignment of Low-resource Languages. In Proceedings of EMNLP 2016. (Citation: 9)
• Ron J. Weiss, Jan Chorowski, Navdeep Jaitly, Yonghui Wu and Zhifeng Chen. 2017. Sequence-to-sequence Models can Directly Translate Foreign Speech. In Proceedings of Interspeech 2017. (Citation: 41)
• Jiatao Gu, Graham Neubig, Kyunghyun Cho, and Victor O.K. Li. 2017. Learning to Translate in Real-time with Neural Machine Translation. In Proceedings of EACL 2017. (Citation: 17)
• Sameer Bansal, Herman Kamper, Adam Lopez, and Sharon Goldwater. 2017. Towards Speech-to-text Translation without Speech Recognition. In Proceedings of EACL 2017. (Citation: 13)
• Antonios Anastasopoulos and David Chiang. 2018. Tied Multitask Learning for Neural Speech Translation. In Proceedings of NAACL 2018. (Citation: 10)
• Fahim Dalvi, Nadir Durrani, Hassan Sajjad, and Stephan Vogel. 2018. Incremental Decoding and Training Methods for Simultaneous Translation in Neural Machine Translation. In Proceedings of NAACL 2018.
• Craig Stewart, Nikolai Vogler, Junjie Hu, Jordan Boyd-Graber, and Graham Neubig. 2018. Automatic Estimation of Simultaneous Interpreter Performance. In Proceedings of ACL 2018. (Citation: 1)
• Florian Dessloch, Thanh-Le Ha, Markus Müller, Jan Niehues, Thai Son Nguyen, Ngoc-Quan Pham, Elizabeth Salesky, Matthias Sperber, Sebastian Stüker, Thomas Zenkel, and Alexander Waibel. 2018. KIT Lecture Translator: Multilingual Speech Translation with One-Shot Learning. In Proceedings of COLING 2018.
• Ashkan Alinejad, Maryam Siahbani, and Anoop Sarkar. 2018. Prediction Improves Simultaneous Neural Machine Translation. In Proceedings of EMNLP 2018.
• Sameer Bansal, Herman Kamper, Karen Livescu, Adam Lopez, and Sharon Goldwater. 2019. Pre-training on high-resource speech recognition improves low-resource speech-to-text translation. In Proceedings of NAACL 2019.
• Nikolai Vogler, Craig Stewart, and Graham Neubig. 2019. Lost in Interpretation: Predicting Untranslated Terminology in Simultaneous Interpretation. In Proceedings of NAACL 2019.
• Elizabeth Salesky, Matthias Sperber, and Alex Waibel. 2019. Fluent Translations from Disfluent Speech in End-to-End Speech Translation. In Proceedings of NAACL 2019.
• Naveen Arivazhagan, Colin Cherry, Wolfgang Macherey, Chung-Cheng Chiu, Semih Yavuz, Ruoming Pang, Wei Li, and Colin Raffel. 2019. Monotonic Infinite Lookback Attention for Simultaneous Machine Translation. In Proceedings of ACL 2019.
• Matthias Sperber, Graham Neubig, Ngoc-Quan Pham, and Alex Waibel. 2019. Self-Attentional Models for Lattice Inputs. In Proceedings of ACL 2019.
• Pei Zhang, Boxing Chen, Niyu Ge, and Kai Fan. 2019. Lattice Transformer for Speech Translation. In Proceedings of ACL 2019.
• Naveen Arivazhagan, Colin Cherry, Wolfgang Macherey, Chung-Cheng Chiu, Semih Yavuz, Ruoming Pang, Wei Li, and Colin Raffel. 2019. Monotonic Infinite Lookback Attention for Simultaneous Machine Translation. In Proceedings of ACL 2019.
• Elizabeth Salesky, Matthias Sperber, and Alan W Black. 2019. Exploring Phoneme-Level Speech Representations for End-to-End Speech Translation. In Proceedings of ACL 2019.
• Mingbo Ma, Liang Huang, Hao Xiong, Renjie Zheng, Kaibo Liu, Baigong Zheng, Chuanqiang Zhang, Zhongjun He, Hairong Liu, Xing Li, Hua Wu, and Haifeng Wang. 2019. STACL: Simultaneous Translation with Implicit Anticipation and Controllable Latency using Prefix-to-Prefix Framework. In Proceedings of ACL 2019.
• Baigong Zheng, Renjie Zheng, Mingbo Ma, and Liang Huang. 2019. Simultaneous Translation with Flexible Policy via Restricted Imitation Learning. In Proceedings of ACL 2019.
• Matthias Sperber, Graham Neubig, Jan Niehues, Alex Waibel. 2019. Attention-Passing Models for Robust and Data-Efficient End-to-End Speech Translation. Transactions of the Association for Computational Linguistics.
• Baigong Zheng, Renjie Zheng, Mingbo Ma, and Liang Huang. 2019. Simpler and Faster Learning of Adaptive Policies for Simultaneous Translation. In Proceedings of EMNLP 2019.
• Renjie Zheng, Mingbo Ma, Baigong Zheng, and Liang Huang. 2019. Speculative Beam Search for Simultaneous Translation. In Proceedings of EMNLP 2019.
• Jiatao Gu, Changhan Wang, Junbo Zhao. 2019. Levenshtein Transformer. In Proceedings of NeurIPS 2019.
• Chengyi Wang, Yu Wu, Shujie Liu, Ming Zhou and Zhenglu Yang. 2020. Curriculum Pre-training for End-to-End Speech Translation. In Proceedings of ACL 2020.
• Elizabeth Salesky and Alan W Black. 2020. Phone Features Improve Speech Translation. In Proceedings of ACL 2020.
• Matthias Sperber and Matthias Paulik. 2020. Speech Translation and the End-to-End Promise: Taking Stock of Where We Are. In Proceedings of ACL 2020.
• Renjie Zheng, Mingbo Ma, Baigong Zheng, Kaibo Liu and Liang Huang. 2020. Opportunistic Decoding with Timely Correction for Simultaneous Translation. In Proceedings of ACL 2020.
• Baigong Zheng, Kaibo Liu, Renjie Zheng, Mingbo Ma, Hairong Liu and Liang Huang. 2020. Simultaneous Translation Policies: From Fixed to Adaptive. In Proceedings of ACL 2020.
• Shun-Po Chuang, Tzu-Wei Sung, Alexander H. Liu and Hung-yi Lee. 2020. Worse WER, but Better BLEU? Leveraging Word Embedding as Intermediate in Multitask End-to-End Speech Translation. In Proceedings of ACL 2020.
• Yi Ren, Jinglin Liu, Xu Tan, Chen Zhang, Tao QIN, Zhou Zhao and Tie-Yan Liu. 2020. SimulSpeech: End-to-End Simultaneous Speech to Text Translation. In Proceedings of ACL 2020.
• Ashkan Alinejad, Anoop Sarkar. 2020. Effectively pretraining a speech translation decoder with Machine Translation data. In Proceedings of EMNLP 2020.
• Ruiqing Zhang, Chuanqiang Zhang, Zhongjun He, Hua Wu, Haifeng Wang. 2020. Learning Adaptive Segmentation Policy for Simultaneous Translation. In Proceedings of EMNLP 2020.
• Ozan Caglayan, Julia Ive, Veneta Haralampieva, Pranava Madhyastha, Loïc Barrault, Lucia Specia. 2020. Simultaneous Machine Translation with Visual Context. In Proceedings of EMNLP 2020.
• Javier Iranzo-Sánchez, Adrià Giménez Pastor, Joan Albert Silvestre-Cerdà, Pau Baquero-Arnal, Jorge Civera Saiz, Alfons Juan. 2020. Direct Segmentation Models for Streaming Speech Translation. In Proceedings of EMNLP 2020.

Multi-modality

• Julian Hitschler, Shigehiko Schamoni, Stefan Riezler. 2016. Multimodal Pivots for Image Caption Translation. In Proceedings of ACL 2016. (Citation: 34)
• Lucia Specia, Stella Frank, Khalil Sima'an, and Desmond Elliott. 2016. A Shared Task on Multimodal Machine Translation and Crosslingual Image Description. In Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers. (Citation: 47)
• Sergio Rodríguez Guasch, Marta R. Costa-jussà. 2016. WMT 2016 Multimodal Translation System Description based on Bidirectional Recurrent Neural Networks with Double-Embeddings. In Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers. (Citation: 2)
• Po-Yao Huang, Frederick Liu, Sz-Rung Shiang, Jean Oh, and Chris Dyer. 2016. Attention-based Multimodal Neural Machine Translation. In Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers. (Citation: 34)
• Iacer Calixto, Desmond Elliott, and Stella Frank. 2016. DCU-UvA Multimodal MT System report. In Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers. (Citation: 12)
• Kashif Shah, Josiah Wang, and Lucia Specia. 2016. SHEF-Multimodal: Grounding Machine Translation on Images. In Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers. (Citation: 17)
• Desmond Elliott, Stella Frank, Loïc Barrault, Fethi Bougares, and Lucia Specia. 2017. Findings of the Second Shared Task on Multimodal Machine Translation and Multilingual Image Description. In Proceedings of the Second Conference on Machine Translation. (Citation: 24)
• Iacer Calixto, Qun Liu, and Nick Campbell. 2017. Doubly-Attentive Decoder for Multi-modal Neural Machine Translation. In Proceedings of ACL 2017. (Citation: 31)
• Jean-Benoit Delbrouck and Stéphane Dupont. 2017. An empirical study on the effectiveness of images in Multimodal Neural Machine Translation. In Proceedings of EMNLP 2017. (Citation: 2)
• Iacer Calixto and Qun Liu. 2017. Incorporating Global Visual Features into Attention-based Neural Machine Translation. In Proceedings of EMNLP 2017. (Citation: 14)
• Jason Lee, Kyunghyun Cho, Jason Weston, and Douwe Kiela. 2018. Emergent Translation in Multi-Agent Communication. In Proceedings of ICLR 2018. (Citation: 8)
• Yun Chen, Yang Liu, and Victor O. K. Li. 2018. Zero-Resource Neural Machine Translation with Multi-Agent Communication Game. In Proceedings of AAAI 2018. (Citation: 6)
• Loïc Barrault, Fethi Bougares, Lucia Specia, Chiraag Lala, Desmond Elliott, and Stella Frank. 2018. Findings of the Third Shared Task on Multimodal Machine Translation. In Proceedings of the Third Conference on Machine Translation: Shared Task Papers. (Citation: 1)
• John Hewitt, Daphne Ippolito, Brendan Callahan, Reno Kriz, Derry Tanti Wijaya, and Chris Callison-Burch. 2018. Learning Translations via Images with a Massively Multilingual Image Dataset. In Proceedings of ACL 2018. (Citation: 1)
• Mingyang Zhou, Runxiang Cheng, Yong Jae Lee, and Zhou Yu. 2018. A Visual Attention Grounding Neural Model for Multimodal Machine Translation. In Proceedings of EMNLP 2018.
• Desmond Elliott. 2018. Adversarial Evaluation of Multimodal Machine Translation. In Proceedings of EMNLP 2018.
• Ozan Caglayan, Pranava Madhyastha, Lucia Specia, and Loïc Barrault. 2019. Probing the Need for Visual Context in Multimodal Machine Translation. In Proceedings of NAACL 2019.
• Iacer Calixto, Miguel Rios, and Wilker Aziz. 2019. Latent Variable Model for Multi-modal Translation. In Proceedings of ACL 2019.
• Julia Ive, Pranava Madhyastha, and Lucia Specia. 2019. Distilling Translations with Visual Awareness. In Proceedings of ACL 2019.
• Zhuosheng Zhang, Kehai Chen, Rui Wang, Masao Utiyama, Eiichiro Sumita, Zuchao Li, Hai Zhao. 2020. Neural Machine Translation with Universal Visual Representation. In Proceedings of ICLR 2020.
• Po-Yao Huang, Junjie Hu, Xiaojun Chang and Alexander Hauptmann. 2020. Unsupervised Multimodal Neural Machine Translation with Pseudo Visual Pivoting. In Proceedings of ACL 2020.
• Shu Okabe, Frédéric Blain, and Lucia Specia. 2020. Multimodal Quality Estimation for Machine Translation. In Proceedings of ACL 2020.
• Shaowei Yao and Xiaojun Wan. 2020. Multimodal Transformer for Multimodal Machine Translation. In Proceedings of ACL 2020.
• Shuo Sun, Francisco Guzmán and Lucia Specia. 2020. Are we Estimating or Guesstimating Translation Quality?. In Proceedings of ACL 2020.
• Ozan Caglayan, Julia Ive, Veneta Haralampieva, Pranava Madhyastha, Loïc Barrault, Lucia Specia. 2020. Simultaneous Machine Translation with Visual Context. In Proceedings of EMNLP 2020.

Ensemble and Reranking

• Ekaterina Garmash, and Christof Monz. 2016. Ensemble Learning for Multi-Source Neural Machine Translation. In Proceedings of COLING 2016. (Citation: 18)
• Long Zhou, Wenpeng Hu, Jiajun Zhang, and Chengqing Zong. 2017. Neural System Combination for Machine Translation. In Proceedings of ACL 2017. (Citation: 21)
• Jiaji Huang, Yi Li, Wei Ping, and Liang Huang. 2018. Large Margin Neural Language Model. In Proceedings of EMNLP 2018.
• Tianxiao Shen, Myle Ott, Michael Auli, and Marc’Aurelio Ranzato. 2019. Mixture Models for Diverse Machine Translation: Tricks of the Trade. In Proceedings of ICML 2019.
• Yiren Wang, Lijun Wu, Yingce Xia, Tao Qin, ChengXiang Zhai and Tie-Yan Liu. 2020. Transductive Ensemble Learning for Neural Machine Translation. In Proceedings of AAAI 2020.

Domain Adaptation

• Chenhui Chu, Raj Dabre, and Sadao Kurohashi. 2017. An Empirical Comparison of Domain Adaptation Methods for Neural Machine Translation. In Proceedings of ACL 2017. (Citation: 40)
• Rui Wang, Andrew Finch, Masao Utiyama, and Eiichiro Sumita. 2017. Sentence Embedding for Neural Machine Translation Domain Adaptation. In Proceedings of ACL 2017. (Citation: 8)
• Boxing Chen, Colin Cherry, George Foster, and Samuel Larkin. 2017. Cost Weighting for Neural Machine Translation Domain Adaptation. In Proceedings of the First Workshop on Neural Machine Translation. (Citation: 10)
• Reid Pryzant and Denny Britz. 2017. Effective Domain Mixing for Neural Machine Translation. In Proceedings of the Second Conference on Machine Translation. (Citation: 6)
• Mara Chinea-Rios, Álvaro Peris and Francisco Casacuberta. 2017. Adapting Neural Machine Translation with Parallel Synthetic Data. In Proceedings of the Second Conference on Machine Translation. (Citation: 3)
• Rui Wang, Masao Utiyama, Lemao Liu, Kehai Chen, and Eiichiro Sumita. 2017. Instance Weighting for Neural Machine Translation Domain Adaptation. In Proceedings of EMNLP 2017. (Citation: 13)
• Antonio Valerio Miceli Barone, Barry Haddow, Ulrich Germann, and Rico Sennrich. 2017. Regularization techniques for fine-tuning in neural machine translation. In Proceedings of EMNLP 2017. (Citation: 6)
• David Vilar. 2018. Learning Hidden Unit Contribution for Adapting Neural Machine Translation Models. In Proceedings of NAACL 2018. (Citation: 2)
• Paul Michel and Graham Neubig. 2018. Extreme Adaptation for Personalized Neural Machine Translation. In Proceedings for ACL 2018. (Citation: 6)
• Shiqi Zhang and Deyi Xiong. 2018. Sentence Weighting for Neural Machine Translation Domain Adaptation. In Proceedings of COLING 2018.
• Chenhui Chu and Rui Wang. 2018. A Survey of Domain Adaptation for Neural Machine Translation. In Proceedings of COLING 2018. (Citation: 7)
• Jiali Zeng, Jinsong Su, Huating Wen, Yang Liu, Jun Xie, Yongjing Yin, and Jianqiang Zhao. 2018. Multi-Domain Neural Machine Translation with Word-Level Domain Context Discrimination. In Proceedings of EMNLP 2018.
• Graham Neubig and Junjie Hu. 2018. Rapid Adaptation of Neural Machine Translation to New Languages. In Proceedings of EMNLP 2018. (Citation: 4)
• Shuhao Gu, Yang Feng, and Qun Liu. 2019. Improving Domain Adaptation Translation with Domain Invariant and Specific Information. In Proceedings of NAACL 2019.
• Ankur Bapna and Orhan Firat. 2019. Non-Parametric Adaptation for Neural Machine Translation. In Proceedings of NAACL 2019.
• Junjie Hu, Mengzhou Xia, Graham Neubig, and Jaime Carbonell. 2019. Domain Adaptation of Neural Machine Translation by Lexicon Induction. In Proceedings of ACL 2019.
• Danielle Saunders, Felix Stahlberg, Adria de Gispert, and Bill Byrne. 2019. Domain Adaptive Inference for Neural Machine Translation. In Proceedings of ACL 2019.
• Zi-Yi Dou, Junjie Hu, Antonios Anastasopoulos, and Graham Neubig. 2019. Unsupervised Domain Adaptation for Neural Machine Translation with Domain-Aware Feature Embeddings. In Proceedings of EMNLP 2019.
• Ankur Bapna, Naveen Arivazhagan, and Orhan Firat. 2019. Simple, Scalable Adaptation for Neural Machine Translation. In Proceedings of EMNLP 2019.
• Jiali Zeng, Yang Liu, jinsong su, yubing Ge, Yaojie Lu, Yongjing Yin and jiebo luo. 2019. Iterative Dual Domain Adaptation for Neural Machine Translation. In Proceedings of EMNLP 2019.
• Wei Wang, Ye Tian, Jiquan Ngiam, Yinfei Yang, Isaac Caswell and Zarana Parekh. 2020. Learning a Multi-Domain Curriculum for Neural Machine Translation. In Proceedings of ACL 2020.
• Chaojun Wang and Rico Sennrich. 2020. On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation. In Proceedings of ACL 2020.
• Haoming Jiang, Chen Liang, Chong Wang and Tuo Zhao. 2020. Multi-Domain Neural Machine Translation with Word-Level Adaptive Layer-wise Domain Mixing. In Proceedings of ACL 2020.
• Anna Currey, Prashant Mathur, Georgiana Dinu. 2020. Distilling Multiple Domains for Neural Machine Translation. In Proceedings of EMNLP 2020.
• Haoyue Shi , Luke Zettlemoyer , Sida I. Wang . 2021. Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignment . In Proceedings of ACL 2021.

Quality Estimation

• Julia Kreutzer, Shigehiko Schamoni, Stefan Riezler. 2015. Quality Estimation from Scratch (QUETCH): Deep Learning for Word-Level Translation Quality Estimation. In Proceedings of the Tenth Workshop on Statistical Machine Translation. (Citation: 24)
• Hyun Kim and Jong-Hyeok Lee. 2016. A Recurrent Neural Networks Approach for Estimating the Quality of Machine Translation Output. In Proceedings of NAACL 2016. (Citation: 11)
• Hyun Kim and Jong-Hyeok Lee, Seung-Hoon Na. 2017. Predictor-Estimator using Multilevel Task Learning with Stack Propagation for Neural Quality Estimation. In Proceedings of WMT 2017. (Citation: 10)
• Osman Baskaya, Eray Yildiz, Doruk Tunaoglu, Mustafa Tolga Eren, and A. Seza Doğruöz. 2017. Integrating Meaning into Quality Evaluation of Machine Translation. In Proceedings of EACL 2017.
• Yvette Graham, Qingsong Ma, Timothy Baldwin, Qun Liu, Carla Parra, and Carolina Scarton. 2017. Improving Evaluation of Document-level Machine Translation Quality Estimation. In Proceedings of EACL 2017. (Citation: 1)
• Rico Sennrich. 2017. How Grammatical is Character-level Neural Machine Translation? Assessing MT Quality with Contrastive Translation Pairs. In Proceedings of EACL 2017. (Citation: 25)
• Pierre Isabelle, Colin Cherry, and George Foster. 2017. A Challenge Set Approach to Evaluating Machine Translation. In Proceedings of EMNLP 2017. (Citation: 26)
• André F.T. Martins, Marcin Junczys-Dowmunt, Fabio N. Kepler, Ramón Astudillo, Chris Hokamp, and Roman Grundkiewicz. 2017. Pushing the Limits of Translation Quality Estimation. Transactions of the Association for Computational Linguistics. (Citation: 13)
• Maoxi Li, Qingyu Xiang, Zhiming Chen, and Mingwen Wang. 2018. A Unified Neural Network for Quality Estimation of Machine Translation. IEICE Transactions on Information and Systems. (Citation: 13)
• Lucia Specia, Frédéric Blain, Varvara Logacheva, Ramón F. Astudillo, and André Martins. 2018. Findings of the WMT 2018 Shared Task on Quality Estimation. In Proceedings of the Third Conference on Machine Translation. (Citation: 2)
• Craig Stewart, Nikolai Vogler, Junjie Hu, Jordan Boyd-Graber, and Graham Neubig. 2018. Automatic Estimation of Simultaneous Interpreter Performance. In Proceedings of ACL 2018. (Citation: 1)
• Holger Schwenk. 2018. Filtering and Mining Parallel Data in a Joint Multilingual Space. In Proceedings of ACL 2018. (Citation: 4)
• Julia Ive, Frédéric Blain, and Lucia Specia. 2018. deepQuest: A Framework for Neural-based Quality Estimation. In Proceedings of COLING 2018. (Citation: 1)
• Kai Fan, Jiayi Wang, Bo Li, Fengming Zhou, Boxing Chen, and Luo Si. 2019. "Bilingual Expert" Can Find Translation Errors. In Proceedings of AAAI 2019.
• Aditya Siddhant, Melvin Johnson, Henry Tsai, Naveen Arivazhagan, Jason Riesa, Ankur Bapna, Orhan Firat and Karthik Raman. 2020. Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation. In Proceedings of AAAI 2020.
• Shu Okabe, Frédéric Blain, and Lucia Specia. 2020. Multimodal Quality Estimation for Machine Translation. In Proceedings of ACL 2020.
• Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Francisco Guzmán, Mark Fishel, Nikolaos Aletras, Vishrav Chaudhary, and Lucia Specia. 2020. Unsupervised Quality Estimation for Neural Machine Translation. Transactions of the Association for Computational Linguistics.
• Jingyi Zhang, Josef van Genabith. 2020. Translation Quality Estimation by Jointly Learning to Score and Rank. In Proceedings of EMNLP 2020.
• Vania Mendonca , Ricardo Rei , Lu´ısa Coheur , Alberto Sardinha , Ana Lucia Santos . 2021 . Online Learning Meets Machine Translation Evaluation: Finding the Best Systems with the Least Human Effort . In Proceedings of ACL 2021.

Human-centered NMT

Interactive NMT

• Joern Wuebker, Spence Green, John DeNero, Saša Hasan and Minh-Thang Luong. 2016. Models and Inference for Prefix-Constrained Machine Translation. In Proceedings of ACL 2016. (Citation: 14)
• Rebecca Knowles and Philipp Koehn. 2017. Neural Interactive Translation Prediction. In Proceedings of AMTA 2016. (Citation: 24)
• Álvaro Peris, Miguel Domingo and Francisco Casacuberta. 2017. Interactive neural machine translation. In Computer Speech and Language. (Citation: 21)
• Khanh Nguyen, Hal Daumé III, and Jordan Boyd-Graber. 2017. Reinforcement Learning for Bandit Neural Machine Translation with Simulated Human Feedback. In Proceedings of EMNLP 2017. (Citation: 11)
• Álvaro Peris and Francisco Casacuberta. 2018. Active Learning for Interactive Neural Machine Translation of Data Streams. In Proceedings of CoNLL 2018. (Citation: 1)
• Tsz Kin Lam, Julia Kreutzer, and Stefan Riezler. 2018. A Reinforcement Learning Approach to Interactive-Predictive Neural Machine Translation. In Proceedings of EAMT 2018.
• Julia Kreutzer, Shahram Khadivi, Evgeny Matusov, Stefan Riezler. 2018. Can Neural Machine Translation be Improved with User Feedback?. In Proceedings of NAACL 2018. (Citation: 3).
• Pavel Petrushkov, Shahram Khadivi and Evgeny Matusov. 2018. Learning from Chunk-based Feedback in Neural Machine Translation. In Proceedings of ACL 2018. (Citation: 1)
• Julia Kreutzer, Joshua Uyheng, and Stefan Riezler. 2018. Reliability and Learnability of Human Bandit Feedback for Sequence-to-Sequence Reinforcement Learning. In Proceedings of ACL 2018. (Citation: 2)
• Álvaro Peris and Francisco Casacuberta. 2019. A Neural, Interactive-predictive System for Multimodal Sequence to Sequence Tasks. In Proceedings of ACL 2019.
• Miguel Domingo, Mercedes García-Martínez, Amando Estela, Laurent Bié, Alexandre Helle, Álvaro Peris, Francisco Casacuberta, and Manuerl Herranz. 2019. Demonstration of a Neural Machine Translation System with Online Learning for Translators. In Proceedings of ACL 2019.
• Julia Kreutzer and Stefan Riezler. 2019. Self-Regulated Interactive Sequence-to-Sequence Learning. In Proceedings of ACL 2019.

Automatic Post-Editing

• Santanu Pal, Sudip Kumar Naskar, Mihaela Vela, and Josef van Genabith. 2016. A neural network based approach to automatic post-editing. In Proceedings of ACL 2016. (Citation: 14)
• Marcin Junczys-Dowmunt and Roman Grundkiewicz. 2016. Log-linear Combinations of Monolingual and Bilingual Neural Machine Translation Models for Automatic Post-Editing. In Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers. (Citation: 27)
• Santanu Pal, Sudip Kumar Naskar, Mihaela Vela, Qun Liu, and Josef van Genabith. 2017. Neural Automatic Post-Editing Using Prior Alignment and Reranking. In Proceedings of EACL 2017. (Citation: 11)
• Rajen Chatterjee, Gebremedhen Gebremelak, Matteo Negri, and Marco Turchi. 2017. Online Automatic Post-editing for MT in a Multi-Domain Translation Environment. In Proceedings of EACL 2017. (Citation: 1)
• Marcin Junczys-Dowmunt, Roman Grundkiewicz. 2017. An Exploration of Neural Sequence-to-Sequence Architectures for Automatic Post-Editing. In Proceedings of IJCNLP 2017.
• David Grangier and Michael Auli. 2018. QuickEdit: Editing Text & Translations by Crossing Words Out. In Proceedings of NAACL 2018. (Citation: 1)
• Thuy-Trang Vu and Gholamreza Haffari. 2018. Automatic Post-Editing of Machine Translation: A Neural Programmer-Interpreter Approach. In Proceedings of EMNLP 2018.
• Marcin Junczys-Dowmunt, Roman Grundkiewicz. 2018. MS-UEdin Submission to the WMT2018 APE Shared Task: Dual-Source Transformer for Automatic Post-Editing. In Proceedings of WMT 2018.
• Gonçalo M. Correia and André F. T. Martins. 2019. A Simple and Effective Approach to Automatic Post-Editing with Transfer Learning. In Proceedings of ACL 2019.
• Xuancheng Huang, Yang Liu, Huanbo Luan, Jingfang Xu, Maosong Sun. 2019. Learning to Copy for Automatic Post-Editing. In Proceedings of EMNLP 2019.
• Nico Herbig, Tim Düwel, Santanu Pal, Kalliopi Meladaki, Mahsa Monshizadeh, Antonio Krüger and Josef van Genabith. 2020. MMPE: A Multi-Modal Interface for Post-Editing Machine Translation. In Proceedings of ACL 2020.
• Shamil Chollampatt, Raymond Hendy Susanto, Liling Tan, Ewa Szymanska. 2020. Can Automatic Post-Editing Improve NMT?. In Proceedings of EMNLP 2020.

Poetry Translation

• Marjan Ghazvininejad, Yejin Choi, and Kevin Knight. 2018. Neural Poetry Translation. In Proceedings of NAACL 2018. (Citation: 1)

Eco-friendly

• Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2019. Energy and Policy Considerations for Deep Learning in NLP. In Proceedings of ACL 2019.

Endangered Language Revitalization

• Shiyue Zhang, Benjamin Frey, Mohit Bansal. 2020. ChrEn: Cherokee-English Machine Translation for Endangered Language Revitalization. In Proceedings of EMNLP 2020.
• Tahmid Hasan, Abhik Bhattacharjee, Kazi Samin, Masum Hasan, Madhusudan Basak, M. Sohel Rahman, Rifat Shahriyar. 2020. Not Low-Resource Anymore: Aligner Ensembling, Batch Filtering, and New Datasets for Bengali-English Machine Translation. In Proceedings of EMNLP 2020.

Word Translation

• Tomas Mikolov, Quoc V. Le, and Ilya Sutskever. 2013. Exploiting Similarities among Languages for Machine Translation. arxiv:1309.4168. (Citation: 581)
• Chao Xing, Dong Wang, Chao Liu, and Yiye Lin. 2015. Normalized Word Embedding and Orthogonal Transform for Bilingual Word Translation. In Proceedings of NAACL 2015. (Citation: 89)
• Georgiana Dinu, Angeliki Lazaridou, and Marco Baroni. 2015. Improving Zero-shot Learning by Mitigating the Hubness Problem. In Proceedings of ICLR 2015. (Citation: 110)
• Meng Zhang, Yang Liu, Huanbo Luan, Maosong Sun, Tatsuya Izuha, and Jie Hao. 2016. Building Earth Mover's Distance on Bilingual Word Embeddings for Machine Translation. In Proceedings of AAAI 2016. (Citation: 11)
• Meng Zhang, Yang Liu, Huanbo Luan, Yiqun Liu, and Maosong Sun. 2016. Inducing Bilingual Lexica From Non-Parallel Data With Earth Mover's Distance Regularization. In Proceedings of COLING 2016. (Citation: 4)
• Ivan Vulić and Anna Korhonen. On the Role of Seed Lexicons in Learning Bilingual Word Embeddings. In Proceedings of ACL 2016. (Citation: 39)
• Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2016. Learning principled bilingual mappings of word embeddings while preserving monolingual invariance. In Proceedings of EMNLP 2016. (Citation: 73)
• Meng Zhang, Haoruo Peng, Yang Liu, Huanbo Luan, and Maosong Sun. Bilingual Lexicon Induction from Non-Parallel Data with Minimal Supervision. In Proceedings of AAAI 2017. (Citation: 11)
• Ann Irvine and Chris Callison-Burch. 2017. A Comprehensive Analysis of Bilingual Lexicon Induction. Computational Linguistics. (Citation: 12)
• Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2017. Learning Bilingual Word Embeddings with (Almost) No Bilingual Data. In Proceedings of ACL 2017. (Citation: 62)
• Meng Zhang, Yang Liu, Huanbo Luan, and Maosong Sun. 2017. Adversarial Training for Unsupervised Bilingual Lexicon Induction. In Proceedings of ACL 2017. (Citation: 41)
• Geert Heyman, Ivan Vulić, and Marie-Francine Moens. 2017. Bilingual Lexicon Induction by Learning to Combine Word-Level and Character-Level Representations. In Proceedings of EACL 2017. (Citation: 9)
• Bradley Hauer, Garrett Nicolai, and Grzegorz Kondrak. 2017. Bootstrapping Unsupervised Bilingual Lexicon Induction. In Proceedings of EACL 2017. (Citation: 5)
• Yunsu Kim, Julian Schamper, and Hermann Ney. 2017. Unsupervised Training for Large Vocabulary Translation Using Sparse Lexicon and Word Classes. In Proceedings of EACL 2017. (Citation: 1)
• Derry Tanti Wijaya, Brendan Callahan, John Hewitt, Jie Gao, Xiao Ling, Marianna Apidianaki, and Chris Callison-Burch. 2017. Learning Translations via Matrix Completion. In Proceedings of EMNLP 2017. (Citation: 3)
• Meng Zhang, Yang Liu, Huanbo Luan, and Maosong Sun. 2017. Earth Mover's Distance Minimization for Unsupervised Bilingual Lexicon Induction. In Proceedings of EMNLP 2017. (Citation: 26)
• Ndapandula Nakashole and Raphael Flauger. 2017. Knowledge Distillation for Bilingual Dictionary Induction. In Proceedings of EMNLP 2017. (Citation: 5)
• Hanan Aldarmaki, Mahesh Mohan, and Mona Diab. 2018. Unsupervised Word Mapping Using Structural Similarities in Monolingual Embeddings. Transactions of the Association for Computational Linguistics. (Citation: 5)
• Guillaume Lample, Alexis Conneau, Marc'Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. 2018. Word Translation without Parallel Data. In Proceedings of ICLR 2018. (Citation: 11)
• Fabienne Braune, Viktor Hangya, Tobias Eder, and Alexander Fraser. 2018. Evaluating Bilingual Word Embeddings on the Long Tail. In Proceedings of NAACL 2018. (Citation: 1)
• Ndapa Nakashole and Raphael Flauger. 2018. Characterizing Departures from Linearity in Word Translation. In Proceedings of ACL 2018. (Citation: 3)
• Anders Søgaard, Sebastian Ruder, and Ivan Vulić. 2018. On the Limitations of Unsupervised Bilingual Dictionary Induction. In Proceedings of ACL 2018. (Citation: 17)
• Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2018. A Robust Self-learning Method for Fully Unsupervised Cross-lingual Mappings of Word Embeddings. In Proceedings of ACL 2018. (Citation: 17)
• Parker Riley and Daniel Gildea. 2018. Orthographic Features for Bilingual Lexicon Induction. In Proceedings of ACL 2018.
• Amir Hazem and Emmanuel Morin. 2018. Leveraging Meta-Embeddings for Bilingual Lexicon Extraction from Specialized Comparable Corpora. In Proceedings of COLING 2018.
• Lifu Huang, Kyunghyun Cho, Boliang Zhang, Heng Ji, and Kevin Knight. 2018. Multi-lingual Common Semantic Space Construction via Cluster-consistent Word Embedding. In Proceedings of EMNLP 2018.
• Xilun Chen and Claire Cardie. 2018. Unsupervised Multilingual Word Embeddings. In Proceedings of EMNLP 2018. (Citation: 4)
• Ta Chung Chi and Yun-Nung Chen. 2018. CLUSE: Cross-Lingual Unsupervised Sense Embeddings. In Proceedings of EMNLP 2018. (Citation: 1)
• Yerai Doval, Jose Camacho-Collados, Luis Espinosa Anke, and Steven Schockaert. 2018. Improving Cross-Lingual Word Embeddings by Meeting in the Middle. In Proceedings of EMNLP 2018.
• Sebastian Ruder, Ryan Cotterell, Yova Kementchedjhieva, and Anders Søgaard. 2018. A Discriminative Latent-Variable Model for Bilingual Lexicon Induction. In Proceedings of EMNLP 2018.
• Yedid Hoshen and Lior Wolf. 2018. Non-Adversarial Unsupervised Word Translation. In Proceedings of EMNLP 2018.
• Ndapa Nakashole. 2018. NORMA: Neighborhood Sensitive Maps for Multilingual Word Embeddings. In Proceedings of EMNLP 2018.
• Mareike Hartmann, Yova Kementchedjhieva, and Anders Søgaard. 2018. Why is unsupervised alignment of English embeddings from different algorithms so hard?. In Proceedings of EMNLP 2018.
• Zi-Yi Dou, Zhi-Hao Zhou, and Shujian Huang. 2018. Unsupervised Bilingual Lexicon Induction via Latent Variable Models. In Proceedings of EMNLP 2018.
• Tanmoy Mukherjee, Makoto Yamada, and Timothy Hospedales. 2018. Learning Unsupervised Word Translations Without Adversaries. In Proceedings of EMNLP 2018.
• David Alvarez-Melis and Tommi Jaakkola. 2018. Gromov-Wasserstein Alignment of Word Embedding Spaces. In Proceedings of EMNLP 2018.
• Ruochen Xu, Yiming Yang, Naoki Otani, and Yuexin Wu. 2018. Unsupervised Cross-lingual Transfer of Word Embedding Spaces. In Proceedings of EMNLP 2018. (Citation: 2)
• Armand Joulin, Piotr Bojanowski, Tomas Mikolov, Hervé Jégou, and Edouard Grave. 2018. Loss in Translation: Learning Bilingual Word Mapping with a Retrieval Criterion. In Proceedings of EMNLP 2018. (Citation: 2)
• Sebastian Ruder, Ivan Vulić, and Anders Søgaard. 2019. A Survey Of Cross-lingual Word Embedding Models. Journal of Artificial Intelligence Research. (Citation: 22)
• Pratik Jawanpuria, Arjun Balgovind, Anoop Kunchukuttan, and Bamdev Mishra. 2019. Learning Multilingual Word Embeddings in Latent Metric Space: A Geometric Approach. Transactions of the Association for Computational Linguistics. (Citation: 3)
• Tasnim Mohiuddin and Shafiq Joty. 2019. Revisiting Adversarial Autoencoder for Unsupervised Word Translation with Cycle Consistency and Improved Training. In Proceedings of NAACL 2019.
• Chunting Zhou, Xuezhe Ma, Di Wang, and Graham Neubig. 2019. Density Matching for Bilingual Word Embedding. In Proceedings of NAACL 2019.
• Noa Yehezkel Lubin, Jacob Goldberger, and Yoav Goldberg. 2019. Aligning Vector-spaces with Noisy Supervised Lexicons. In Proceedings of NAACL 2019.
• Tal Schuster, Ori Ram, Regina Barzilay, and Amir Globerson. 2019. Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing. In Proceedings of NAACL 2019.
• Hanan Aldarmaki and Mona Diab. 2019. Context-Aware Cross-Lingual Mapping. In Proceedings of NAACL 2019.
• Yoshinari Fujinuma, Jordan Boyd-Graber, and Michael J. Paul. 2019. A Resource-Free Evaluation Metric for Cross-Lingual Word Embeddings Based on Graph Modularity. In Proceedings of ACL 2019.
• Mozhi Zhang, Keyulu Xu, Ken-ichi Kawarabayashi, Stefanie Jegelka, and Jordan Boyd-Graber. 2019. Are Girls Neko or Shōjo? Cross-Lingual Alignment of Non-Isomorphic Embeddings with Iterative Normalization. In Proceedings of ACL 2019.
• Aitor Ormazabal, Mikel Artetxe, Gorka Labaka, Aitor Soroa, and Eneko Agirre. 2019. Analyzing the Limitations of Cross-lingual Word Embedding Mappings. In Proceedings of ACL 2019.
• Takashi Wada, Tomoharu Iwata, and Yuji Matsumoto. 2019. Unsupervised Multilingual Word Embedding with Limited Resources using Neural Language Models. In Proceedings of ACL 2019.
• Pengcheng Yang, Fuli Luo, Peng Chen, Tianyu Liu, and Xu Sun. 2019. MAAM: A Morphology-Aware Alignment Model for Unsupervised Bilingual Lexicon Induction. In Proceedings of ACL 2019.
• Benjamin Marie and Atsushi Fujita. 2019. Unsupervised Joint Training of Bilingual Word Embeddings. In Proceedings of ACL 2019.
• Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2019. Bilingual Lexicon Induction through Unsupervised Machine Translation. In Proceedings of ACL 2019.
• Elias Stengel-Eskin, Tzu-Ray Su, Matt Post, and Benjamin Van Durme. 2019. A Discriminative Neural Model for Cross-Lingual Word Alignment. In Proceedings of EMNLP 2019.
• Ivan Vulić, Goran Glavaš, Roi Reichart, and Anna Korhonen. 2019. Do We Really Need Fully Unsupervised Cross-Lingual Embeddings?. In Proceedings of EMNLP 2019.
• Paula Czarnowska, Sebastian Ruder, Edouard Grave, Ryan Cotterell, and Ann Copestake. 2019. Don't Forget the Long Tail! A Comprehensive Analysis of Morphological Generalization in Bilingual Lexicon Induction. In Proceedings of EMNLP 2019.
• Yova Kementchedjhieva, Mareike Hartmann, and Anders Søgaard. 2019. Lost in Evaluation: Misleading Benchmarks for Bilingual Dictionary Induction. In Proceedings of EMNLP 2019.
• Mareike Hartmann, Yova Kementchedjhieva, Anders Søgaard. 2019. Comparing Unsupervised Word Translation Methods Step by Step. In Proceedings of NeurIPS 2019.
• Steven Cao, Nikita Kitaev, Dan Klein. 2020. Multilingual Alignment of Contextual Word Representations. In Proceedings of ICLR 2020.
• Gábor Berend. 2020. Massively Multilingual Sparse Word Representations. In Proceedings of ICLR 2020.
• Xu Zhao, Zihao Wang, Yong Zhang and Hao Wu. 2020. A Relaxed Matching Procedure for Unsupervised BLI. In Proceedings of ACL 2020.
• Mladen Karan, Ivan Vulić, Anna Korhonen and Goran Glavaš. 2020. Classification-Based Self-Learning for Weakly Supervised Bilingual Lexicon Induction. In Proceedings of ACL 2020.
• Shuo Ren, Shujie Liu, Ming Zhou and Shuai Ma. 2020. A Graph-based Coarse-to-fine Method for Unsupervised Bilingual Lexicon Induction. In Proceedings of ACL 2020.
• Pratik Jawanpuria, Mayank Meghwanshi and Bamdev Mishra. 2020. Geometry-aware domain adaptation for unsupervised alignment of word embeddings. In Proceedings of ACL 2020.
• Tasnim Mohiuddin, M Saiful Bari, Shafiq Joty. 2020. LNMap: Departures from Isomorphic Assumption in Bilingual Lexicon Induction Through Non-Linear Mapping in Latent Space. In Proceedings of EMNLP 2020.

WMT Winners

WMT is the most important annual international competition on machine translation. We collect the competition results on the news translation task since WMT 2016 (the First Conference of Machine Translation) and summarize the techniques used in the systems with the top performance. Currently, we focus on four directions: ZH-EN, EN-ZH, DE-EN, and EN-DE. The summarized algorithms might be incomplete; your suggestions are welcome!

WMT 2019

• The winner of ZH-EN, DE-EN and EN-DE: Microsoft

  • System report: Yingce Xia, Xu Tan, Fei Tian, Fei Gao, Di He, Weicong Chen, Yang Fan, Linyuan Gong, Yichong Leng, Renqian Luo, Yiren Wang, Lijun Wu, Jinhua Zhu, Tao Qin and Tie-Yan Liu. 2019. Microsoft Research Asia’s Systems for WMT19. In Proceedings of the Fourth Conference on Machine Translation: Shared Task Papers.
  • News: Microsoft Research Asia (MSRA) leads in 2019 WMT international machine translation competition
  • Techniques:
    • Yiren Wang, Yingce Xia, Tianyu He, Fei Tian, Tao Qin, ChengXiang Zhai, and Tie-Yan Liu. 2019. Multi-Agent Dual Learning. In Proceedings of ICLR 2019.
    • Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2019. MASS: Masked Sequence to Sequence Pre-training for Language Generation. In Proceedings of ICML 2019.
    • Renqian Luo, Fei Tian, Tao Qin, Enhong Chen, and Tie-Yan Liu. 2018. Neural Architecture Optimization. In Proceedings of NeurIPS 2018.
    • Jinhua Zhu, Fei Gao, Lijun Wu, Yingce Xia, Tao Qin, Wengang Zhou, Xueqi Cheng, Tie-Yan Liu. 2019. Soft Contextual Data Augmentation for Neural Machine Translation. In Proceedings of ACL 2019.

• The winner of EN-ZH: PATECH

  • System report: Coming soon...
  • Techniques: Transformer + Back-Translation + Reranking + Ensemble

WMT 2018

• The winner of ZH-EN: Tencent

  • System report: Mingxuan Wang, Li Gong, Wenhuan Zhu, Jun Xie, and Chao Bian. 2018. Tencent Neural Machine Translation Systems for WMT18. In Proceedings of the Third Conference on Machine Translation: Shared Task Papers.
  • Techniques: RNMT + Transformer + BPE + Rerank ensemble outputs with 48 features (including t2t R2l, t2t L2R, rnn L2R, rnn R2L etc.) + Back Translation + Joint Train with English to Chinese systems + Fine-tuning with selected data + Knowledge distillation

• The winner of EN-ZH: GTCOM

  • System report: Chao Bei, Hao Zong, Yiming Wang, Baoyong Fan, Shiqi Li, and Conghu Yuan. 2018. An Empirical Study of Machine Translation for the Shared Task of WMT18. In Proceedings of the Third Conference on Machine Translation: Shared Task Papers.
  • Techniques: Transformer + Back-Translation + Data Filtering by rules, language models and translation models + BPE + Greedy Ensemble Decoding + Fine-Tuning with newstest2017 back translation

• The winner of DE-EN: RWTH Aachen University

  • System report: Julian Schamper, Jan Rosendahl, Parnia Bahar, Yunsu Kim, Arne Nix, and Hermann Ney. 2018. The RWTH Aachen University Supervised Machine Translation Systems for WMT 2018. In Proceedings of the Third Conference on Machine Translation: Shared Task Papers.
  • Techniques: Ensemble of 3-strongest Transformer models + Data Selection + BPE + Fine-Tuning + Important Hyperparameters (batch size and model dimension)

• The winner of EN-DE: Microsoft

  • System report: Marcin Junczys-Dowmunt. 2018. Microsoft’s Submission to the WMT2018 News Translation Task: How I Learned to Stop Worrying and Love the Data. In Proceedings of the Third Conference on Machine Translation: Shared Task Papers.
  • Techniques: Marian + Transformer-big + BPE + Ensemble + Data Filtering + Domain-Weighted {ParaCrawl, original data} + Decoder-time ensemble with in-domain Transformer-style language model + Reranking with Right-to-left Transformer-big models

WMT 2017

• The winner of ZH-EN: Sogou

  • System report: Yuguang Wang, Shanbo Cheng, Liyang Jiang, Jiajun Yang, Wei Chen, Muze Li, Lin Shi, Yanfeng Wang, and Hongtao Yang. 2017. Sogou Neural Machine Translation Systems for WMT17. In Proceedings of the Second Conference on Machine Translation: Shared Task Papers.
  • Techniques: Encoder-Decoder with Attention + BPE + Reranking (R2L, T2S, N-gram language models) + Tagging Model + Name Entity Translation + Ensemble

• The winner of EN-ZH, DE-EN and EN-DE: University of Edinburgh

  • System report: Rico Sennrich, Alexandra Birch, Anna Currey, Ulrich Germann, Barry Haddow, Kenneth Heafield, Antonio Valerio Miceli Barone, and Philip Williams. 2017. The University of Edinburgh’s Neural MT Systems for WMT17. In Proceedings of the Second Conference on Machine Translation: Shared Task Papers.
  • Techniques: Encoder-Decoder with Attention + Deep Model + Layer Normalization + Weight Tying + Back-Translation + BPE + Reranking(L2R, R2L) + Ensemble

WMT 2016

• The winner of DE-EN: University of Regensburg

  • System report: Failed to find it
  • Techniques: Failed to find it

• The winner of EN-DE: University of Edinburgh

  • System report: Edinburgh Neural Machine Translation Systems for WMT 16. In Proceedings of the First Conference on Machine Translation: Shared Task Papers.
  • Techniques: Encoder-Decoder with Attention + Back-Translation + BPE + Reranking(R2L) + Ensemble