Much of recent progress in NLU was shown to be due to models' learning dataset-specific heuristics. We conduct a case study of generalization in NLI (from MNLI to the adversarially constructed HANS dataset) in a range of BERT-based architectures (adapters, Siamese Transformers, HEX debiasing), as well as with subsampling the data and increasing the model size. We report 2 successful and 3 unsuccessful strategies, all providing insights into how Transformer-based models learn to generalize.
Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics
Several strategies, including adapter and debiasing techniques, were tested for their effectiveness in improving the generalization of BERT-based models across different datasets in natural language inference tasks.
- Year
- 2021
- Venue
- EMNLP (insights) 2021 11
- Authors
- 3
- Hosting
- Abstract onlyARXIV-DEFAULT
Cite
Notes
Only stored in your browser.
Attribution
- Abstract & full text
- arxiv.org/abs/2110.01518ARXIV-DEFAULT
- TL;DR
- Semantic Scholar