HEBREW NLP TF-IDF

TF-IDF and vod2vec on 3 types of files in Hebrew.

• Type 1: Files with whole words (without punctuation).

• Type 2: Files with the roots of the words.

• Type 3: Files with words broken to their roots ("שבמצח" => "שב מצח")

Below are the results for the query חמאס מלחמה עזה טיל טילים פלסטינים

The contents of the files can be viewed by clicking on them

TF-IDF top 10 files for each category:

Clean punctuation	similarity	PrefSufWords	similarity	RootWord	similarity
2544137.txt	0.6328771355794626	2763273prefsuf.txt	0.2369911739213888	2516532root.txt	0.25789237861633374
1763006.txt	0.6028766297608992	3224668prefsuf.txt	0.2360844896168185	1763006root.txt	0.25522720092802675
3200525.txt	0.5932105336176698	1682206prefsuf.txt	0.23207942157980932	2763273root.txt	0.24829179106946508
3224668.txt	0.5529987921150823	1716814prefsuf.txt	0.21486838009209142	2764728root.txt	0.24366686574589502
2763273.txt	0.5493357642260276	3200181prefsuf.txt	0.2141802599463245	3224668root.txt	0.23987286401430108
3227173.txt	0.5298776018606244	3227173prefsuf.txt	0.21406999200178278	2589060root.txt	0.23731395822517332
2767441.txt	0.5237825246168819	2762821prefsuf.txt	0.2089475380770338	3044679root.txt	0.2358677035792669
1863195.txt	0.5081887327650091	2544137prefsuf.txt	0.20859923531448055	3234654root.txt	0.23536472642344508
3234654.txt	0.5000208050141923	2546411prefsuf.txt	0.207481033009436	3200181root.txt	0.23139911815232295
2767820.txt	0.49866617638441957	1716669prefsuf.txt	0.20698391367803537	1716814root.txt	0.22784591716399072

doc2vec top 10 files for each category:

Clean punctuation	similarity	PrefSufWords	similarity	RootWord	similarity
2815711.txt	0.8379393219947815	3215204prefsuf.txt	0.8451696634292603	2654951root.txt	0.8426624536514282
3184780.txt	0.8259774446487427	1811559prefsuf.txt	0.8209106922149658	3204941root.txt	0.8411526679992676
1608580.txt	0.8181542754173279	1811562prefsuf.txt	0.8071666955947876	1811562root.txt	0.814696192741394
1743096.txt	0.8105580806732178	3225440prefsuf.txt	0.7961132526397705	1853284root.txt	0.8077710866928101
3244484.txt	0.8082942366600037	3218931prefsuf.txt	0.7953389286994934	2649467root.txt	0.8067579865455627
3206106.txt	0.8035317659378052	2759660prefsuf.txt	0.7943722605705261	3164207root.txt	0.79481041431427
3173668.txt	0.8018320798873901	2771551prefsuf.txt	0.7932097315788269	3125850root.txt	0.7935506105422974
3026879.txt	0.794529378414154	3215394prefsuf.txt	0.7927966713905334	3173668root.txt	0.7855309844017029
3036123.txt	0.7933730483055115	2931445prefsuf.txt	0.7896900773048401	1811559root.txt	0.7811357975006104
2683497.txt	0.7872658371925354	1544126prefsuf.txt	0.7872987389564514	3209043root.txt	0.776106595993042

---

Groups A, B clean punctuation TF-IDF

---

Groups A, B clean punctuation doc2vec

---

Groups A, B clean punctuation BERT

---

Groups A, B prefSufWord TF-IDF

---

Groups A, B prefSufWord doc2vec

---

Groups A, B prefSufWord BERT

---

Groups A, B rootWord TF-IDF

---

Groups A, B rootWord doc2vec

---

Groups A, B rootWord BERT

---

Groups A, C clean punctuation TF-IDF

---

Groups A, C clean punctuation doc2vec

---

Groups A, C clean punctuation BERT

---

Groups A, C prefSufWord TF-IDF

---

Groups A, C prefSufWord doc2vec

---

Groups A, C prefSufWord BERT

---

Groups A, C rootWord TF-IDF

---

Groups A, C rootWord doc2vec

---

Groups A, C rootWord BERT

---

Groups C, B clean punctuation TF-IDF

---

Groups C, B clean punctuation doc2vec

---

Groups C, B clean punctuation BERT

---

Groups C, B prefSufWord TF-IDF

---

Groups C, B prefSufWord doc2vec

---

Groups C, B prefSufWord BERT

---

Groups C, B rootWord TF-IDF

---

Groups C, B rootWord doc2vec

---

Groups C, B rootWord BERT

---