Mistake label initial push

.gitignore

@@ -0,0 +1 @@
+**/.DS_Store

README.md

@@ -1,6 +1,65 @@
 # mistake-detection
 
-# License
+## Annotation
+Each csv file corresponds to an assembly sequence within each csv file are the annotations. 
+- There are in total 328 sequences. 
+- Each annotation contains the 
+  - **start** and **end** timestamps, followed by the 
+  - **verb** (attach, detach) and the 
+  - two working objects (**this, that**), the 
+  - mistake **label** and the 
+  - **remark** are then appened.
+
+### Examplar
+
+For example:
+**nusar-2021_action_both_9033-c02a_9033_user_id_2021-02-04_140532.csv**
+
+- **9033** is the actor while **c02d** is the toy id.
+
+| start      | end      | verb     | this          | that          | label      | remark                  |
+|------------|----------|----------|---------------|---------------|------------|-------------------------|
+| 7257       | 7817     | attach   | interior      | chassis       | correct    |                         |
+| 8419       | 8736     | detach   | interior      | chassis       | mistake    | shouldn't have happened |
+| 8736       | 9138     | attach   | interior      | chassis       | correct    |                         |
+| 10039      | 10355    | attach   | body          | chassis       | mistake    | wrong order             |
+| 10749      | 11184    | attach   | cabin         | interior      | mistake    | previous one is mistake |
+| 11184      | 11601    | interior | cabin         | interior      | correction |                         |
+| 11601      | 11806    | detach   | body          | chassis       | correction |                         |
+| 11806      | 11864    | attach   | cabin         | interior      | correct    |                         |
+| 11864      | 12105    | attach   | body          | chassis       | correct    |                         |
+| 12615      | 13443    | attach   | roof          | cabin         | correct    |                         |
+| 13443      | 14098    | attach   | arm           | bucket        | correct    |                         |
+| 14098      | 14147    | attach   | arm           | boom          | correct    |                         |
+| 14658      | 15729    | attach   | arm connector | chassis       | mistake    | wrong order             |
+| 16490      | 16848    | detach   | arm connector | chassis       | correction |                         |
+| 16848      | 18532    | attach   | arm connector | boom          | mistake    | wrong position          |
+| 18532      | 19342    | attach   | arm connector | chassis       | correct    |                         |
+| 19342      | 19509    | detach   | arm connector | chassis       | mistake    | shouldn't have happened |
+| 19509      | 19835    | detach   | arm           | boom          | correction |                         |
+| 19835      | 20641    | attach   | arm connector | boom          | correct    |                         |
+| 20641      | 21476    | attach   | arm connector | chassis       | correct    |                         |
+| 21476      | 21059    | detach   | arm connector | chassis       | mistake    | shouldn't have happened |
+| 21509      | 21527    | detach   | boom          | arm connector | mistake    | shouldn't have happened |
+| 21527      | 21795    | attach   | arm connector | chassis       | correct    |                         |
+| 21795      | 21830    | detach   | arm connector | chassis       | mistake    | shouldn't have happened |
+| 21830      | 22234    | attach   | arm connector | boom          | correct    |                         |
+| 22234      | 23444    | attach   | arm connector | chassis       | correct    |                         |
+| 23444      | 24401    | attach   | arm           | boom          | correct    |                         |
+| 24401      | 25738    | attach   | bumper        | body          | correct    |                         |
+| 25738      | 27434    | attach   | wheel         | chassis       | correct    |                         |
+
+- **wrong order**: meaning the action is a mistake in ordering
+- **previous one is mistake**: meaning this action is also a ordering mistake but caused by preceding ordering mistakes in context
+- **shouldn't have happened**: meaning this action is uncessary detach of parts
+- **wrong position**: meaning the two parts are not attached in the right position
+
+
+There is no *correction* for the *shouldn't have happened* type of mistake.
+
+
+
+## License
 Assembly101 is licensed by us under the Creative Commons Attribution-NonCommerial 4.0 International License, found [here](https://creativecommons.org/licenses/by-nc/4.0/). The terms are :
 - **Attribution** : You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
 - **NonCommercial** : You may not use the material for commercial purposes.

annots/nusar-2021_action_both_9011-b06b_9011_user_id_2021-02-01_154253.csv

@@ -0,0 +1,6 @@
+2833,3156,attach,interior,chassis,correct,
+3156,4607,attach,wheel,chassis,correct,
+4607,5151,attach,arm,turntable top,correct,
+5151,5613,attach,hook,arm,correct,
+5613,6110,attach,turntable top,chassis,correct,
+6110,6394,attach,cabin,interior,correct,

annots/nusar-2021_action_both_9011-b08c_9011_user_id_2021-02-01_154736.csv

@@ -0,0 +1,9 @@
+4777,5009,attach,base,chassis,correct,
+5352,6316,attach,wheel,chassis,correct,
+6316,6564,attach,interior,chassis,correct,
+6564,8952,attach,bucket,arm,correct,
+8952,9392,attach,nut,base,correct,
+9392,9560,attach,arm connector,nut,correct,
+9560,10158,attach,arm,arm connector,correct,
+10334,10397,attach,cabin,interior,correct,
+10397,10680,attach,roof,cabin,correct,

annots/nusar-2021_action_both_9011-c01c_9011_user_id_2021-02-01_155620.csv

@@ -0,0 +1,13 @@
+4380,5284,attach,wheel,chassis,correct,
+5284,5635,attach,interior,chassis,correct,
+5635,5841,attach,cabin,interior,correct,
+5841,6392,attach,bumper,cabin,correct,
+6392,6602,attach,base,chassis,correct,
+6970,7254,attach,turntable base,base,mistake,wrong position
+7254,7289,detach,turntable base,base,correction,
+7289,7547,attach,cabin window,turntable base,correct,
+7547,7836,attach,cabin window,base,correct,
+7836,8459,attach,hook,arm,correct,
+8459,8708,attach,arm,boom,correct,
+8708,9196,attach,boom,turntable base,correct,
+9196,9617,attach,cabin,roof,correct,

annots/nusar-2021_action_both_9011-c03f_9011_user_id_2021-02-01_160239.csv

@@ -0,0 +1,8 @@
+2916,3205,attach,chassis,chassis,correct,
+3494,4564,attach,wheel,chassis,correct,
+4564,4912,attach,roller,arm,correct,
+4912,5245,attach,arm connector,arm,correct,
+5245,5549,attach,arm connector,chassis,correct,
+5549,5936,attach,body,chassis,correct,
+5936,6090,attach,cabin window,cabin,correct,
+6090,6370,attach,cabin window,chassis,correct,

annots/nusar-2021_action_both_9011-c13b_9011_user_id_2021-02-01_160915.csv

@@ -0,0 +1,17 @@
+3680,5021,attach,wheel,chassis,correct,
+5202,5600,attach,step,chassis,mistake,wrong position
+5600,5940,attach,cabin,chassis,mistake,wrong order
+5940,6076,attach,light,cabin,correct,
+6076,6301,attach,body,chassis,mistake,wrong position
+6301,6397,detach,body,chassis,correction,
+6397,6460,detach,cabin,chassis,correction,
+6460,6507,attach,cabin,chassis,mistake,wrong order
+6507,6872,attach,body,chassis,correct,
+6965,6984,detach,cabin,chassis,correction,
+6984,7104,detach,step,chassis,correction,
+7104,7209,attach,step,chassis,correct,
+7236,7329,attach,cabin,chassis,correct,
+7435,7496,attach,strap,chassis,correct,
+8319,8445,attach,boom,cabin,correct,
+8729,9020,attach,arm,boom,correct,
+9509,9984,attach,jackhammer,arm,correct,

annots/nusar-2021_action_both_9012-a16_9012_user_id_2021-02-01_162904.csv

@@ -0,0 +1,13 @@
+5222,6420,attach,wheel,chassis,correct,
+6420,6600,attach,interior,cabin,correct,
+6600,6660,attach,interior,chassis,correct,
+6976,7230,attach,base,chassis,correct,
+7770,8250,attach,body,base,correct,
+8250,8814,attach,turntable top,body,correct,
+8814,9531,attach,ladder,turntable top,correct,
+9531,9709,detach,ladder,turntable top,mistake,shouldn't have happened
+9867,10198,detach,basket,ladder,mistake,shouldn't have happened
+10361,11201,attach,ladder,turntable top,correct,
+11201,12034,attach,basket,ladder,correct,
+12034,12362,attach,roof,cabin,correct,
+12362,12717,attach,bumper,cabin,correct,

annots/nusar-2021_action_both_9012-a17_9012_user_id_2021-02-01_162209.csv

@@ -0,0 +1,13 @@
+4013,5393,attach,wheel,chassis,correct,
+5393,5634,attach,base,chassis,mistake,wrong order
+5634,5879,attach,cabin,chassis,mistake,wrong order
+6371,6376,detach,cabin,chassis,correction,
+6376,6390,detach,base,chassis,correction,
+6390,6435,attach,interior,chassis,correct,
+6435,6754,attach,cabin,interior,correct,
+6754,6929,attach,base,chassis,correct,
+7965,8659,attach,water tank,base,correct,
+8659,9119,attach,turntable top,water tank,correct,
+9119,9445,attach,fire extinguisher,turntable top,correct,
+9445,9824,attach,roof,cabin,correct,
+9824,10300,attach,bumper,cabin,correct,

annots/nusar-2021_action_both_9012-b06d_9012_user_id_2021-02-01_163713.csv

@@ -0,0 +1,6 @@
+3853,4495,attach,interior,chassis,correct,
+4495,5082,attach,tilter,interior,correct,
+5082,5765,attach,mixer stand,tilter,correct,
+5765,6350,attach,mixer,mixer stand,correct,
+6350,6719,attach,cabin,interior,correct,
+6719,8016,attach,wheel,chassis,correct,

annots/nusar-2021_action_both_9012-c06d_9012_user_id_2021-02-18_121034.csv

@@ -0,0 +1,12 @@
+3650,4961,attach,wheel,chassis,correct,
+5139,5548,attach,interior,chassis,correct,
+5548,5942,attach,cabin,interior,correct,
+5942,6452,attach,roof,cabin,mistake,wrong order
+6452,6710,attach,water tank,chassis,mistake,wrong position
+6710,6732,detach,water tank,chassis,correction,
+6732,6814,attach,base,chassis,correct,
+6814,6884,attach,water tank,base,correct,
+7368,7803,attach,water tank,water tank,correct,
+7803,8065,detach,roof,cabin,correction,
+8065,8201,attach,sound module,cabin,correct,
+8201,8630,attach,roof,cabin,correct,

annots/nusar-2021_action_both_9012-c07c_9012_user_id_2021-02-01_164345.csv

@@ -0,0 +1,6 @@
+3832,5364,attach,wheel,chassis,correct,
+5364,5460,attach,interior,chassis,correct,
+5460,5563,attach,cabin,interior,correct,
+5854,6242,attach,bumper,cabin,correct,
+6242,6437,attach,container,chassis,correct,
+7504,7995,attach,roof,cabin,correct,

annots/nusar-2021_action_both_9012-c14b_9012_user_id_2021-02-01_164944.csv

@@ -0,0 +1,14 @@
+4780,5079,attach,step,chassis,correct,
+5079,5307,attach,bumper,chassis,correct,
+5307,5558,attach,base,chassis,correct,
+5830,5959,attach,body,base,correct,
+6395,6453,detach,body,base,mistake,shouldn't have happened
+6453,6544,attach,body,base,correct,
+6544,7293,attach,boom,body,correct,
+7293,8248,attach,arm,boom,correct,
+8248,8466,detach,body,base,mistake,shouldn't have happened
+8466,8670,attach,body,base,correct,
+8670,9567,attach,rear body,chassis,correct,
+9567,9651,detach,body,base,mistake,shouldn't have happened
+10477,12253,attach,wheel,chassis,correct,
+13505,13648,attach,body,base,correct,

annots/nusar-2021_action_both_9013-a02_9013_user_id_2021-02-02_130807.csv

@@ -0,0 +1,15 @@
+3966,4371,attach,cabin,chassis,correct,
+4979,5133,attach,body,chassis,correct,
+6835,6880,detach,cabin,chassis,mistake,shouldn't have happened
+6880,6965,detach,body,chassis,mistake,shouldn't have happened
+7277,7318,attach,body,chassis,mistake,wrong position
+7318,7389,attach,cabin,chassis,mistake,wrong position
+7389,7459,detach,cabin,chassis,correction,
+7835,7921,detach,body,chassis,correction,
+8338,8433,attach,body,chassis,correct,
+8433,8736,attach,cabin,chassis,correct,
+9461,9477,detach,body,chassis,mistake,shouldn't have happened
+10540,10624,attach,body,chassis,correct,
+12524,14264,attach,wheel,chassis,correct,
+16573,16734,position,figurine,cabin,correct,
+16734,17044,attach,bulldozer arm,body,correct,

annots/nusar-2021_action_both_9013-a28_9013_user_id_2021-02-02_134923.csv

@@ -0,0 +1,10 @@
+2748,2778,attach,interior,chassis,correct,
+3054,4120,attach,wheel,chassis,correct,
+4120,4290,attach,body,interior,correct,
+4290,4968,attach,wheel,chassis,correct,
+5150,5214,detach,body,interior,mistake,shouldn't have happened
+5214,5282,attach,body,interior,mistake,wrong position
+5282,5455,detach,body,interior,correction,
+5455,5623,attach,body,interior,correct,
+5950,6415,attach,roof,body,correct,
+6415,6710,attach,engine cover,body,correct,

annots/nusar-2021_action_both_9013-b01a_9013_user_id_2021-02-02_135446.csv

@@ -0,0 +1,10 @@
+4517,4629,attach,interior,chassis,correct,
+5271,6829,attach,wheel,chassis,correct,
+6829,7333,attach,arm connector,interior,correct,
+7333,7824,attach,arm,nut,correct,
+7824,9506,attach,bucket,arm,correct,
+9506,9557,detach,bucket,arm,mistake,shouldn't have happened
+9557,9875,attach,bucket,arm,correct,
+9875,10787,attach,arm,nut,correct,
+10787,11316,attach,cabin,interior,correct,
+11316,11465,attach,roof,cabin,correct,

annots/nusar-2021_action_both_9013-c03b_9013_user_id_2021-02-24_113410.csv

@@ -0,0 +1,6 @@
+3886,4276,attach,body,chassis,correct,
+4276,7032,attach,wheel,chassis,correct,
+7032,7329,attach,bulldozer arm,chassis,correct,
+7329,7700,attach,arm connector,bulldozer arm,correct,
+7700,7859,attach,cabin window,chassis,correct,
+7859,8534,attach,cabin,cabin window,correct,

annots/nusar-2021_action_both_9013-c09c_9013_user_id_2021-02-24_113951.csv

@@ -0,0 +1,31 @@
+5687,5834,attach,sound module,chassis,correct,
+5834,7161,attach,wheel,chassis,correct,
+7161,7822,attach,base,chassis,mistake,wrong order
+7822,8023,attach,interior,cabin,correct,
+8023,8091,detach,base,chassis,correction,
+8091,8163,attach,interior,chassis,correct,
+8163,8333,attach,base,chassis,correct,
+8881,9226,attach,bumper,cabin,correct,
+9578,9951,attach,roof,cabin,correct,
+9951,10465,attach,transport cabin,base,mistake,wrong position
+10832,10945,attach,rear roof,transport cabin,mistake,wrong order
+11065,11109,detach,transport cabin,base,correction,
+11109,11220,detach,rear roof,transport cabin,correction,
+11220,11460,attach,door,transport cabin,correct,
+11460,11542,attach,rear roof,transport cabin,correct,
+11542,11590,attach,transport cabin,base,mistake,wrong position
+11590,11610,detach,transport cabin,base,correction,
+11610,11655,detach,door,transport cabin,mistake,shouldn't have happened
+11655,11760,attach,transport cabin,base,correct,
+11900,11940,detach,rear roof,transport cabin,mistake,shouldn't have happened
+11940,11967,attach,door,transport cabin,correct,
+11967,12015,attach,rear roof,transport cabin,correct,
+12332,12440,detach,transport cabin,base,mistake,shouldn't have happened
+12440,12643,attach,transport cabin,base,correct,
+12850,13213,detach,transport cabin,base,mistake,shouldn't have happened
+13213,13501,detach,rear roof,transport cabin,mistake,shouldn't have happened
+13501,13544,attach,transport cabin,base,correct,
+13544,13581,detach,door,transport cabin,mistake,shouldn't have happened
+13581,13598,detach,rear roof,transport cabin,mistake,shouldn't have happened
+13598,13825,attach,door,transport cabin,correct,
+14290,15044,attach,rear roof,transport cabin,correct,

annots/nusar-2021_action_both_9014-a12_9014_user_id_2021-02-02_141945.csv

@@ -0,0 +1,3 @@
+6204,8385,attach,dump bed,chassis,correct,
+8385,9304,attach,bumper,chassis,correct,
+9304,11936,attach,wheel,chassis,correct,

annots/nusar-2021_action_both_9014-a23_9014_user_id_2021-02-02_142800.csv

@@ -0,0 +1,8 @@
+7048,8638,attach,wheel,chassis,correct,
+8638,9041,attach,engine,chassis,correct,
+9226,9592,attach,body,chassis,correct,
+9818,10682,attach,rocker panel,body,correct,
+11189,11592,attach,spoiler,body,correct,
+11592,12096,attach,bumper,body,correct,
+12214,12554,attach,roof,body,correct,
+12554,12915,attach,engine cover,body,correct,

annots/nusar-2021_action_both_9014-b02a_9014_user_id_2021-02-02_143628.csv

@@ -0,0 +1,4 @@
+6065,6356,attach,dump bed,chassis,correct,
+7517,7838,attach,roof,cabin,correct,
+7838,8046,attach,cabin,chassis,correct,
+8508,9675,attach,wheel,chassis,correct,

annots/nusar-2021_action_both_9014-b05a_9014_user_id_2021-02-02_144248.csv

@@ -0,0 +1,20 @@
+10259,10929,attach,interior,chassis,correct,
+10929,11621,attach,rear bumper,chassis,correct,
+11919,11960,attach,turntable base,interior,correct,
+11960,11997,detach,turntable base,interior,mistake,shouldn't have happened
+12915,13138,attach,turntable top,turntable base,correct,
+13138,13271,detach,turntable top,turntable base,mistake,shouldn't have happened
+13271,13889,attach,turntable base,interior,mistake,shouldn't have happened
+13889,15280,attach,turntable base,interior,correct,
+15280,16639,attach,turntable top,turntable base,correct,
+16914,17038,attach,cabin,interior,correct,
+17508,19293,attach,boom,boom,correct,
+19293,19691,attach,arm,boom,correct,
+19691,19730,detach,arm,boom,mistake,shouldn't have happened
+20357,20408,detach,bucket,arm,mistake,shouldn't have happened
+20408,20975,attach,arm,arm,correct,
+20975,23048,attach,bucket,arm,correct,
+24039,24534,detach,rear bumper,chassis,mistake,shouldn't have happened
+24534,27068,attach,wheel,chassis,correct,
+27068,27798,attach,rear bumper,chassis,correct,
+27798,28435,attach,bumper,cabin,correct,

annots/nusar-2021_action_both_9014-c01b_9014_user_id_2021-02-02_150612.csv

@@ -0,0 +1,7 @@
+5671,6851,attach,wheel,chassis,correct,
+6851,7290,attach,interior,chassis,correct,
+7290,7416,attach,cabin,interior,correct,
+9273,9421,attach,base,chassis,correct,
+10878,12619,attach,dump bed,base,correct,
+12619,13247,attach,roof,cabin,correct,
+14368,14875,attach,bumper,cabin,correct,

annots/nusar-2021_action_both_9014-c13e_9014_user_id_2021-02-02_151834.csv

@@ -0,0 +1,6 @@
+6244,6710,attach,step,chassis,correct,
+6710,7810,attach,cabin,chassis,correct,
+7810,9882,attach,base,chassis,correct,
+9882,10391,attach,dump bed,base,correct,
+10391,12615,attach,wheel,chassis,correct,
+12615,13844,attach,bumper,chassis,correct,

annots/nusar-2021_action_both_9015-a08_9015_user_id_2021-02-02_155549.csv

@@ -0,0 +1,5 @@
+5659,5928,attach,grill,cabin,correct,
+5928,6463,attach,bumper,grill,correct,
+6604,8198,attach,cabin,chassis,correct,
+9486,9996,attach,crane arm,chassis,correct,
+10439,12704,attach,wheel,chassis,correct,

annots/nusar-2021_action_both_9015-a09_9015_user_id_2021-02-02_160807.csv

@@ -0,0 +1,7 @@
+7088,7651,attach,interior,chassis,correct,
+7651,8016,attach,cabin,interior,correct,
+8016,9071,attach,body,chassis,correct,
+9597,9652,detach,clamp,arm,mistake,shouldn't have happened
+9652,10417,attach,arm,boom,correct,
+10417,11094,attach,clamp,arm,correct,
+11296,12765,attach,wheel,chassis,correct,

annots/nusar-2021_action_both_9015-b05b_9015_user_id_2021-02-02_161800.csv

@@ -0,0 +1,10 @@
+7554,8253,attach,turntable base,interior,correct,
+8253,9281,attach,rear bumper,interior,correct,
+9281,10737,attach,wheel,chassis,correct,
+10848,11977,attach,turntable top,turntable base,correct,
+13848,14506,attach,bumper,chassis,mistake,wrong order
+14506,15271,attach,cabin,chassis,mistake,wrong order
+15271,15808,detach,cabin,chassis,correction,
+15808,16079,detach,bumper,chassis,correction,
+16079,16451,attach,cabin,interior,correct,
+16451,16944,attach,bumper,cabin,correct,

annots/nusar-2021_action_both_9015-c03a_9015_user_id_2021-02-02_163503.csv

@@ -0,0 +1,14 @@
+5337,5416,attach,excavator arm,chassis,mistake,wrong position
+5416,5989,attach,arm connector,excavator arm,mistake,previous one is a mistake
+5989,6451,detach,excavator arm,chassis,correction,
+6451,6509,detach,arm connector,excavator arm,correction,
+6509,6561,attach,excavator arm,chassis,correct,
+6561,7109,attach,arm connector,excavator arm,correct,
+7779,8226,attach,body,chassis,correct,
+8226,8294,attach,cabin window,chassis,correct,
+8294,8802,attach,cabin,cabin window,correct,
+8802,8847,detach,cabin,cabin window,mistake,shouldn't have happened
+8847,8911,detach,cabin window,chassis,mistake,shouldn't have happened
+8911,8969,attach,cabin window,chassis,correct,
+8969,9484,attach,cabin,cabin window,correct,
+9484,11515,attach,wheel,chassis,correct,

annots/nusar-2021_action_both_9015-c04a_9015_user_id_2021-02-02_164257.csv

@@ -0,0 +1,23 @@
+5144,5528,attach,bucket,arm,correct,
+5528,5951,attach,arm,boom,correct,
+5951,6233,attach,arm connector,chassis,correct,
+6233,6548,attach,boom,arm connector,correct,
+6548,6597,detach,boom,arm connector,mistake,shouldn't have happened
+6602,6693,detach,arm connector,chassis,mistake,shouldn't have happened
+6693,6764,attach,arm connector,chassis,correct,
+6907,6940,detach,arm connector,chassis,mistake,shouldn't have happened
+7086,7424,attach,cabin,chassis,mistake,wrong order
+7424,7457,detach,cabin,chassis,correction,
+7457,7663,attach,interior,chassis,mistake,wrong position
+7720,7738,detach,interior,chassis,correction,
+7738,7763,attach,interior,chassis,correct,
+7763,8328,attach,cabin,interior,correct,
+8328,8756,attach,arm connector,chassis,correct,
+8756,9365,attach,boom,arm connector,correct,
+9365,9590,detach,boom,arm connector,mistake,shouldn't have happened
+9773,10528,attach,roof,cabin,correct,
+11304,11977,attach,bumper,cabin,correct,
+12880,14481,attach,wheel,chassis,correct,
+14534,14614,detach,bumper,cabin,mistake,shouldn't have happened
+16140,18407,attach,bumper,cabin,correct,
+18407,18903,attach,boom,arm connector,correct,