Relation classification models are conventionally evaluated using only a single measure, e.g., micro-F1, macro-F1 or AUC. In this work, we analyze weighting schemes, such as micro and macro, for imbalanced datasets. We introduce a framework for weighting schemes, where existing schemes are extremes, and two new intermediate schemes. We show that reporting results of different weighting schemes better highlights strengths and weaknesses of a model.
Why only Micro-F1? Class Weighting of Measures for Relation Classification
A new framework for weighting schemes in relation classification highlights model strengths and weaknesses by reporting results across different schemes, particularly useful for imbalanced datasets.
- Year
- 2022
- Venue
- why-only-micro-f1-class-weighting-of-measures
- Authors
- 4
- Hosting
- Abstract onlyARXIV-DEFAULT
Cite
Notes
Only stored in your browser.
Attribution
- Abstract & full text
- arxiv.org/abs/2205.09460ARXIV-DEFAULT
- TL;DR
- Semantic Scholar