LUS midterm project

Midterm project for the Language Understanding System course 2019/2020 @ UniTN.

Concept tagging for the movie domain. The models were trained and performed using the following dataset: NL2SparQL4NLU.

Results - SCLM

Baseline	Remove of OoS	Use of POS tags	Normalization	F1-score	Accuracy	Precision	Recall
Output symbol priors				0.0036	0.0255	0.0036	0.0036
Random path				0.0268	0.4982	0.0218	0.0348
None				0.7269	0.9165	0.7600	0.6966
MLE				0.7223	0.9127	0.7197	0.7250
MLE	☑️			0.7466	0.9055	0.6883	0.8157
MLE	☑️	☑️		0.8119	0.9421	0.8046	0.8194
MLE	☑️	☑️	☑️	0.8021	0.9380	0.7960	0.8084

Results - HMM

Estimator	Remove of OoS	Use of POS tags	Normalization	F1-score	Accuracy	Precision	Recall
Laplace				0.6326	0.8871	0.7237	0.5618
ELE				0.6948	0.9050	0.7406	0.6544
Lindstone 0.05				0.7078	0.9074	0.7062	0.7094
Lindstone 0.15				0.7093	0.9099	0.7129	0.7057
Witten Bell				0.7159	0.9124	0.7512	0.6837
MLE				0.7173	0.9086	0.7719	0.6700
Lindstone 0.1				0.7213	0.9130	0.7213	0.7213
Witten Bell + Witten Bell	☑️			0.7862	0.9326	0.7821	0.7873
Witten Bell + Witten Bell + Witten Bell	☑️	☑️		0.7576	0.9277	0.7618	0.7534
Witten Bell + Witten Bell + Witten Bell + Witten Bell	☑️	☑️	☑️	0.7311	0.9178	0.7318	0.7305