The Transformers for Text Classification Baseline (TTCB) System to a Shared Task on Implicit and Underspecified Language @ACL-2021. We won the competition by sampling more models based on the validation scores. (There were only 2 teams by the way.)
The best result based on XLNet (68.84% dev accuracy) can be viewed as a colab file at https://colab.research.google.com/drive/1TQ5vy07WKCbF2T-DY7lEHQVdZtXO43n8?usp=sharing.
The bertviz visualization of the XLNet model is at https://colab.research.google.com/drive/1W4DqKlrnZN-xAhoUZ9_OAqJIL1i09-y7?usp=sharing.
(After the end of the challenge, we tried MPNet and the result improves a little bit by ~0.2% (69.06% dev accuracy). The colab file for MPNet is at https://colab.research.google.com/drive/1NTUChSN2t1a968fWll4ScnokM2ZZOBGT?usp=sharing)
A link to the paper.
Wiriyathammabhum, P. (2021, August). TTCB System description to a shared task on implicit and underspecified language 2021. In Proceedings of the 1st Workshop on Understanding Implicit and Underspecified Language (pp. 64-70).
@inproceedings{wiriyathammabhum2021ttcb,
title={TTCB System description to a shared task on implicit and underspecified language 2021},
author={Wiriyathammabhum, Peratham},
booktitle={Proceedings of the 1st Workshop on Understanding Implicit and Underspecified Language},
pages={64--70},
year={2021}
}