Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create supplemental events for IOPAN legacy protists #62

Open
cnrdh opened this issue Nov 10, 2021 · 11 comments
Open

Create supplemental events for IOPAN legacy protists #62

cnrdh opened this issue Nov 10, 2021 · 11 comments
Assignees
@cnrdh
Milestone
Original IOPAN pr...

Comments

@cnrdh
Copy link
Member

cnrdh commented Nov 10, 2021

Some occurrences fails to match with events via fieldNumber and expedition.
Sometimes due to ABC suffixing.
@cnrdh
Copy link
Member Author

cnrdh commented Nov 10, 2021 •
edited
Loading 

"bottomDepth"	"decimalLatitude"	"decimalLongitude"	"eventDate"	"eventRemarks"	"expedition"	"fieldNumber"	"locationID"	"maximumDepthInMeters"	"minimumDepthInMeters"	"modified"	"type"	"year"	"gear"	"sampletype"
187	80.605	22.1166	"2010-08-19T06:15:00Z"	"ICE10-16-7; Chl a max"	"ICE2010"	"ICE10-158"	"R4"	38	38	"2020-06-11T07:25:16.805Z"	"MaterialSample"	2010		
187	80.605	22.1166	"2010-08-19T06:15:00Z"	"ICE10-16-7"	"ICE2010"	"ICE10-155"	"R4"	25	25	"2020-06-11T07:25:16.803Z"	"MaterialSample"	2010		
187	80.605	22.1166	"2010-08-19T06:15:00Z"	"ICE10-16-7"	"ICE2010"	"ICE10-156"	"R4"	50	50	"2020-06-11T07:25:16.804Z"	"MaterialSample"	2010		
187	80.605	22.1166	"2010-08-19T06:15:00Z"	"ICE10-16-7"	"ICE2010"	"ICE10-157"	"R4"	100	100	"2020-06-11T07:25:16.804Z"	"MaterialSample"	2010		
187	80.605	22.1166	"2010-08-19T06:15:00Z"	"ICE10-16-7"	"ICE2010"	"ICE10-157"	"R4"	100	100	"2020-06-11T07:25:16.805Z"	"MaterialSample"	2010		
187	80.605	22.1166	"2010-08-19T06:15:00Z"	"ICE10-16-7"	"ICE2010"	"ICE10-159"	"R4"	0	0	"2020-06-11T07:25:16.805Z"	"MaterialSample"	2010		
644	81.4673	22.5182	"2010-08-22T07:34:00Z"	"ICE10-16-13"	"ICE2010"	"ICE10-384"	"ICE10-16-13"	100	100	"2020-06-11T07:25:16.842Z"	"MaterialSample"	2010		
644	81.4673	22.5182	"2010-08-22T07:34:00Z"	"ICE10-16-13"	"ICE2010"	"ICE10-383"	"ICE10-16-13"	50	50	"2020-06-11T07:25:16.842Z"	"MaterialSample"	2010		
644	81.4673	22.5182	"2010-08-22T07:34:00Z"	"ICE10-16-13; Chla max"	"ICE2010"	"ICE10-382"	"ICE10-16-13"	25	25	"2020-06-11T07:25:16.842Z"	"MaterialSample"	2010		
644	81.4673	22.5182	"2010-08-22T07:34:00Z"	"ICE10-16-13"	"ICE2010"	"ICE10-381"	"ICE10-16-13"	10	10	"2020-06-11T07:25:16.841Z"	"MaterialSample"	2010		
644	81.4673	22.5182	"2010-08-22T07:34:00Z"	"ICE10-16-13"	"ICE2010"	"ICE10-380"	"ICE10-16-13"	5	5	"2020-06-11T07:25:16.841Z"	"MaterialSample"	2010		
644	81.4673	22.5182	"2010-08-22T07:34:00Z"	"ICE10-16-13"	"ICE2010"	"ICE10-379"	"ICE10-16-13"	0	0	"2020-06-11T07:25:16.840Z"	"MaterialSample"	2010		
65	77.8317	16.4865	"2016-05-18T04:00:00Z"		"ARCEx2016"	"PHT-HN1"	"vMf1"	20	0		"MaterialSample"	2016	"Handnet"	"Phytoplankton taxonomy"
	78.9733	12.4623	"2017-07-28T10:30:00Z"		"GlacierFront2017"	"HAN-Cc1"	"Cc1"	20				2017	"Handnet"	
	78.9773	12.4423	"2017-07-28T09:25:00Z"		"GlacierFront2017"	"HAN-Cc2"	"Cc2"	20				2017	"Handnet"	
	78.9809	12.4532	"2017-07-28T07:13:00Z"		"GlacierFront2017"	"HAN-Cc3"	"Cc3"	20				2017	"Handnet"	
	78.9899	12.4999	"2017-07-28T13:45:00Z"		"GlacierFront2017"	"HAN-CpN3"	"CpN3"	20				2017	"Handnet"	
	78.9785	12.489	"2017-07-28T17:40:00Z"		"GlacierFront2017"	"HAN-CpS1"	"CpS1"	20				2017	"Handnet"	
	78.931	12.382	"2017-07-29T00:45:00Z"		"GlacierFront2017"	"HAN-Kb6"	"Kb6"	20				2017	"Handnet"	
	78.8815	12.4987	"2017-07-26T16:50:00Z"		"GlacierFront2017"	"HAN-KpM3"	"KpM3"	20				2017	"Handnet"	
	78.8778	12.5297	"2017-07-26T09:00:00Z"		"GlacierFront2017"	"HAN-KpM4"	"KpM4"	20				2017	"Handnet"	
	78.8737	12.5518	"2017-07-25T15:20:00Z"		"GlacierFront2017"	"HAN-KpM5"	"KpM5"	20				2017	"Handnet"	
	78.894	12.4652	"2017-07-27T08:00:00Z"		"GlacierFront2017"	"HAN-KpN2"	"KpN2"	20				2017	"Handnet"	
	78.8912	12.5137	"2017-07-26T19:00:00Z"		"GlacierFront2017"	"HAN-KpN3"	"KpN3"	20				2017	"Handnet"	
	78.8863	12.5656	"2017-07-25T20:40:00Z"		"GlacierFront2017"	"HAN-KpN5"	"KpN5"	20				2017	"Handnet"	
	78.8758	12.4994	"2017-07-26T13:20:00Z"		"GlacierFront2017"	"HAN-KpS3"	"KpS3"	20				2017	"Handnet"	
	78.8703	12.5178	"2017-07-26T12:00:00Z"		"GlacierFront2017"	"HAN-KpS4"	"KpS4"	20				2017	"Handnet"	
	78.8655	12.5542	"2017-07-25T08:53:00Z"		"GlacierFront2017"	"HAN-KpS5"	"KpS5"	20				2017	"Handnet"	
					"GlacierFront2017"	"__HAN-KC4_Kb5__"	"KC4_Kb5"						"Handnet"

@cnrdh cnrdh self-assigned this Nov 10, 2021
@cnrdh
Copy link
Member Author

cnrdh commented Nov 10, 2021

List of non-matching samples:

["ICE2010","ICE10-152","R4",0,null]
["ICE2010","ICE10-253","R6b",5,5]
["ICE2010","ICE10-undefined","16-13",20,20]
["ICE2010","ICE10-undefined","ICE-22",20,20]
["ICE2010","ICE10-undefined","R10b",20,20]
["ICE2010","ICE10-undefined","R7b",20,20]
["ICE2010","ICE10-undefined","R8b",20,20]
["ICE2010","ICE10-undefined","R9b",20,20]
["ICE2012","Agneta","Underice",2,2]
["ICE2012","Core2.1","1mbelowice",null,null]
["ICE2012","Core2.1","undericehole",null,null]
["ICE2012","Divehole","5mdepth",null,null]
["ICE2012","ICE12-760","Floe1",47,47]
["ICE2012","ICE12-822","Floe1",32,32]
["ICE2012","ICE12-996a","Floe1",36,36]
["ICE2012","Pond","handnet20um",null,null]
["ICE2012","Ridge","handnet20um",null,null]
["PolarNight2012","Rijpfjorden_0m_2012-01-12",null,0,null]
["PolarNight2012","Rijpfjorden_15m_2012-01-12",null,15,15]
["PolarNight2012","Rijpfjorden_35m_2012-01-12",null,35,35]
["PolarNight2012","Rijpfjorden_75m_2012-01-12",null,75,75]
["PolarNight2012","Rijpfjorden_m_2012-01-12",null,null,null]
["PolarNight2012","Sofiadypet_0m_2012-01-12",null,0,null]
["PolarNight2012","Sofiadypet_150m_2012-01-14",null,150,150]
["PolarNight2012","Sofiadypet_15m_2012-01-14",null,15,15]
["PolarNight2012","Sofiadypet_5m_2012-01-14",null,5,5]
["PolarNight2012","Sofiadypet_75m_2012-01-14",null,75,75]
["MOSJ2015","PHT-0?",null,null,null]
["N-ICE2015","PHT-144","inlogdoesnotexist",null,null]
["N-ICE2015","PHT-202","Floe3","in log does not exist",null]
["N-ICE2015","IAT-018",null,null,null]
["N-ICE2015","IAT-021",null,null,null]
["N-ICE2015","IAT-022",null,null,null]
["N-ICE2015","IAT-023",null,null,null]
["N-ICE2015","IAT-024",null,null,null]
["N-ICE2015","IAT-025",null,null,null]
["N-ICE2015","IAT-026",null,null,null]
["N-ICE2015","IAT-027",null,null,null]
["N-ICE2015","IAT-028",null,null,null]
["N-ICE2015","IAT-029",null,null,null]
["N-ICE2015","IAT-030",null,null,null]
["N-ICE2015","IAT-041",null,null,null]
["N-ICE2015","IAT-072",null,null,null]
["N-ICE2015","IAT-073",null,null,null]
["N-ICE2015","IAT-099",null,null,null]
["N-ICE2015","IAT-100",null,null,null]
["N-ICE2015","IAT-280",null,null,null]
["MOSJ2016","MIT-015","KB1_KB2",25,25]
["MOSJ2016","HAN-018","R6",20,20]
["GlacierFront2017","PHT-031","CpS1",1,1]
["GlacierFront2017","HAN-KC4_Kb5","KC4_Kb5",20,20]

This was referenced Nov 10, 2021
Open
Open
Closed
@AnetteWold
Copy link
Collaborator

Info about missing sample log records for ICE 2010:
["ICE2010","ICE10-152","R4",0,null] could be ICE10-159 if this doesn't already have data attached to it
["ICE2010","ICE10-253","R6b",5,5] add samplelog info: [ICE2010, ICE10-253, R6b, 81.1379, 22.7625, 215, 20.08.2010, 02:30, Niskin bottle, 5.5, 5.5, Phytoplankton taxonomy, Philipp Assmy]
["ICE2010","ICE10-undefined","16-13",20,20] add samplelog info: [ICE2010, blancID, ICE10-16-13, 81.4673, 22.5182, 644, 22.08.2010, 07:34, Niskin bottle, 20, 20, Phytoplankton taxonomy, Philipp Assmy]
["ICE2010","ICE10-undefined","ICE-22",20,20] add samplelog info: [ICE2010, blanc, Ice22, 81.5115, 30.1349 860, 29.08.2010, 08:25, Niskin bottle, 20, 20, Phytoplankton taxonomy, Philipp Assmy]
["ICE2010","ICE10-undefined","R10b",20,20] add samplelog info: [ICE2010, blanc, R10b, 81.7168, 23.1800, 2798, 23.08.2010, 13:00, Niskin bottle, 20, 20, Phytoplankton taxonomy, Philipp Assmy]
["ICE2010","ICE10-undefined","R7b",20,20] add samplelog info: [ICE2010, blanc, R7b, 81.3044, 21.9615, 368, 21.08.2010, 02:40, Niskin bottle, 20, 20, Phytoplankton taxonomy, Philipp Assmy]
["ICE2010","ICE10-undefined","R8b",20,20] add samplelog info: [ICE2010, blanc, R8b, 81.3892, 22.3506, 508, 21.08.2010, 14:15, Niskin bottle, 20, 20, Phytoplankton taxonomy, Philipp Assmy] add samplelog info: [ICE2010, blanc, R9b, 81.5532, 22.6899, 889, 22.08.2010 16:10, Niskin bottle, 20, 20, Phytoplankton taxonomy, Philipp Assmy]

Si ifra hvis du ønsker tilbakemelding på annet format. Jeg sender også oppdattert samplelog2010 der missing records er lagt til i rad [882-888] på epost

@cnrdh cnrdh mentioned this issue Nov 10, 2021
Closed
@cnrdh
Copy link
Member Author

cnrdh commented Nov 10, 2021 •
edited
Loading

Ok, fint. Det er ingen data med ICE10-159, se også #51
Antall linjer som går tapt per nu:

$ cat data/input/iopan/protist-biodiversity/log/error-expeditions-occurrences-not-found.txt 

1183
    461 [2010,"ICE2010"]
    219 [2012,"ICE2012"]
     40 [2012,"PolarNight2012"]
      3 [2015,"MOSJ2015"]
    330 [2015,"N-ICE2015"]
     20 [2016,"ARCEx2016"]
     39 [2016,"MOSJ2016"]
     71 [2017,"GlacierFront2017"]

@AnetteWold
Copy link
Collaborator

Info missing records ICE 2012:
["ICE2012","Core2.1","1mbelowice",null,null]: No records of station or date. No record of Core2.1 in samplelog
["ICE2012","Core2.1","undericehole",null,null]: No records of station or date. No record of Core2.1 in samplelog
["ICE2012","Divehole","5mdepth",null,null]: No records of station or date. One Station called Dive Site 1 [ Dive site 1, 82.3573, 20.6642, 3751, 28.07.2012, 11:30]
["ICE2012","ICE12-760","Floe1",47,47]: record found [ICE2012, ICE12-760, Floe1, 82.3483, 21.2582, 3746, 28.07.2012, 06:54, Ship CTD, 47, Chl a max, Phytoplankton taxonomy, Philipp Assmy]
["ICE2012","ICE12-822","Floe1",32,32]: record found [ICE2012, ICE12-822, Floe1, 82.3627, 21.5692, 3786, 30.07.2012, 08:00, Niskin bottle, 32, Chl a max, Phytoplankton taxonomy, Philipp Assmy]
["ICE2012","ICE12-996a","Floe1",36,36] record found [ICE2012, ICE12-996a, Floe1, 82.0522, 21.9165, 3603, 02.08.2012, 06:30, Niskin bottle, 36, Phytoplankton taxonomy,Philipp Assmy]
["ICE2012","Pond","handnet20um",null,null] No records of station or date. Could assume that it is from Floe1 (check w/ Philipp)
["ICE2012","Ridge","handnet20um",null,null] Could assume that it is from Floe1 (check w/ Philipp)

@AnetteWold
Copy link
Collaborator

Info missing records Polar Night Cruise 2021:
["PolarNight2012","Rijpfjorden_0m_2012-01-12",null,0,null]
["PolarNight2012","Rijpfjorden_15m_2012-01-12",null,15,15]
["PolarNight2012","Rijpfjorden_35m_2012-01-12",null,35,35]
["PolarNight2012","Rijpfjorden_75m_2012-01-12",null,75,75]
Only one station in Rjipjorden and one CTD with water samples so all samples can have the following info with the different sampling depths given (0, 15, 35, 75)
[PolarNight2012, R3, 80.3098, 22.2599, 274, 12.01.2012, 07:10, CTD, 0, 0, Phytoplankton taxonomy]

["PolarNight2012","Rijpfjorden_m_2012-01-12",null,null,null]: Not sure what to do with this

["PolarNight2012","Sofiadypet_0m_2012-01-12",null,0,null]
["PolarNight2012","Sofiadypet_150m_2012-01-14",null,150,150]
["PolarNight2012","Sofiadypet_15m_2012-01-14",null,15,15]
["PolarNight2012","Sofiadypet_5m_2012-01-14",null,5,5]
["PolarNight2012","Sofiadypet_75m_2012-01-14",null,75,75]
Four CTD taken from Sofiadypet / Ice station. If we choose the first CTD all samples could have the following info with different depths
[PolarNight2012, Ice station, 81.6799, 14.3100, 2395, 14.01.2012, 12:23, 0, 0, Phytoplankton taxonomy]

I will also send the updated samplelog where the added records are in row 93-102

@AnetteWold
Copy link
Collaborator

Info missing samples N-ICE2015:
["N-ICE2015","PHT-144","inlogdoesnotexist",null,null]: Should be related to On-ice CTD-056 with depth 50m: [N-ICE2015, PHT-144, On-ice CTD-056, 56, 81.3710, 9.6187, 1412, 12.05.2015 07:33, On-ice CTD, 50]

["N-ICE2015","PHT-202","Floe3","in log does not exist",null]: Sampleset before (PHT196-201) and after (PHT203-208) looks complete. Does it exist data for all these IDs?

N-ICE2015,IAT-018 to IAT-030: We found some written notes from leg 2. Missing records sent by email (excell)

["N-ICE2015","IAT-041",null,null,null] Samplesets before (IAT33-40) & after (IAT42-49) looks complete. Does it exist data for all these IDs?

["N-ICE2015","IAT-072",null,null,null] Sampleset before (IAT66-71) & after (IAT74-76) looks complete. Does it exist data for all these IDs?
["N-ICE2015","IAT-073",null,null,null] same as above

["N-ICE2015","IAT-099",null,null,null]. Refrozen lead transect, could be one more point in the transect, but we do not have position, ice conditions etc for this point.
["N-ICE2015","IAT-100",null,null,null] same as above

["N-ICE2015","IAT-280",null,null,null] Could be part of Ridge thin ice transect, but we don't have any records of

@cnrdh
Copy link
Member Author

cnrdh commented Nov 11, 2021

Anette, I think we should move your comments into the relevant issue (for each expedition). OK?

@AnetteWold
Copy link
Collaborator

Ok

@AnetteWold
Copy link
Collaborator

AnetteWold commented Nov 11, 2021 •
edited
Loading

MOSJ 2016:
["MOSJ2016","MIT-015","KB1_KB2",25,25]: There are two samples MIT-015a (Kb2) & MIT-015b (Kb1)

["MOSJ2016","HAN-018","R6",20,20]: According to the log the Handnet sample from R6 20-0m = HAN-17. Does it exist data for HAN-17? If it exist data for HAN-17 then add a record for HAN-18 for R7 [MOSJ2016, HAN-018, R7, 81.5115, 21.8475, 1046, 30.07.2016, 22:49, Handnet 20 μm, 20, 0, Handnet 20 μm, Philipp Assmy]

@AnetteWold
Copy link
Collaborator

GlacierFront2017
["GlacierFront2017","PHT-031","CpS1",1,1]: Should be PHT-033
["GlacierFront2017","HAN-KC4_Kb5","KC4_Kb5",20,20] Antar dette er Hand net samples som ikke er logget. KC4 ??? Kan legge til følgende
[GlacierFront2017, HAN-KC4, Kb5, 78.8961, 12.4424, 80, 27.07.2017, 22:15, Niskin 20, 0, Phytoplankton taxonomy]
Prøve fra Kb4 tilhører MOSJ og er registrert = MOSJ2017 HAN-001

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cnrdh cnrdh

Labels
None yet
Projects
None yet
Milestone
Original IOPAN protist data 2009-2017...
Development

No branches or pull requests
2 participants
@cnrdh @AnetteWold