Flow calc and initial site parser code mods #24
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ingest_csvs v2, adds users ability
ingest: fixes <dataFrame>.append deprecation
b1de362 Merge pull request #3 from sauuyer/notebook-cleaning 5318d90 flow rate calc updated. site indicator updated for underway file. ee64317 Rename IFCB_Log-Moored_IFCB_Data_Processing.ipynb to IFCB_Log-Underway_IFCB_Data_Processing.ipynb 44c3b34 Rename IFCB_Log-Underway_IFCB_Data_Processing.ipynb to IFCB_Log-Moored_IFCB_Data_Processing.ipynb d90457b added notes about fast factorsettings in md a6aa809 Merge pull request #2 from sauuyer/pioneer-22 1680167 Delete IFCB_Log_Notebooks/IFCB_Dashboard_Metadata_CSV_shipboard_AR87.ipynb c3aa3be Delete IFCB_Log_Notebooks/IFCB_Dashboard_Metadata_CSV_shipboard_AR82.ipynb ab2f2fb updated log notebooks 8cdffda moored and shipboard notebooks updated 0f9ae6c cleaned up variable updates 14e107a Merge pull request #1 from sauuyer/notebook-cleaning 9a7faba Updated code in jupyter notebooks that create dashboard metadata git-subtree-dir: ifcb-parsing-processing git-subtree-split: b1de362d6e2ca08d69a040177d4eae78273d5133
…-parsers Flow rate code changes added through subtree of personal repo
cooleyky
previously approved these changes
Feb 3, 2026
cooleyky
left a comment
cooleyky
left a comment
There was a problem hiding this comment.
Updates summarized in the PR look good. A future PR should include adding the .ipynb_checkpoints directory to .gitignore
There was a problem hiding this comment.
Updates to site tags and volume analyzed calculation worked as expected.
There was a problem hiding this comment.
Updates to volume analyzed calculation worked as expected.
sbeaulieu
previously approved these changes
Feb 3, 2026
| "\n", | ||
| "# Create a column for volumeAnalyzed\n", | ||
| "discrete_IFCB_hdr_output['volumeAnalyzed'] = (discrete_IFCB_hdr_output['RunFastFactor'] * discrete_IFCB_hdr_output['runSampleFast_Int']) * discrete_IFCB_hdr_output['flowRate_mins'] * (discrete_IFCB_hdr_output['lookTime']/60)/5\n", | ||
| "#discrete_IFCB_hdr_output['volumeAnalyzed'] = (discrete_IFCB_hdr_output['RunFastFactor'] * discrete_IFCB_hdr_output['runSampleFast_Int']) * discrete_IFCB_hdr_output['flowRate_mins'] * (discrete_IFCB_hdr_output['lookTime']/60)/5\n", |
There was a problem hiding this comment.
confirmed with Sawyer in Slack that we no longer need the runSampleFast_Int variable
| "## SyringeSampleVolume (this is usually 5 ml unless the sample time is cut short by the operator (usually for testing reasons)\n", | ||
| "base_flow_rate = 0.25\n", | ||
| "\n", | ||
| "discrete_IFCB_hdr_output['flowRate_mins'] = np.where(\n", |
| "## SyringeSampleVolume (this is usually 5 ml unless the sample time is cut short by the operator (usually for testing reasons)\n", | ||
| "base_flow_rate = 0.25\n", | ||
| "\n", | ||
| "underway_IFCB_hdr_output['flowRate_mins'] = np.where(\n", |
| " 'lat': [35.95, 36.175, 35.725, 35.95, 35.95, 35.95, 36.175, 35.725, 36.0536, 35.8514],\n", | ||
| " 'lon': [-75.125, -74.8267, -74.853, -75.125, -75.3333, -74.8457, -74.8267, -74.853, \n", | ||
| " -74.7776, -74.8482]\n", | ||
| " 'site': ['CP10CNSM;CP12CNSW', 'CP11NOSM;CP13NOPM', 'CP11SOSM;CP13SOPM', 'CP12WESW', 'CP13EAPM', \n", |
… from notebooks, updated LAT/LON file
cooleyky
approved these changes
Feb 3, 2026
| *.py,cover | ||
| .hypothesis/ | ||
| .pytest_cache/ | ||
| .ipynb_checkpoints |
There was a problem hiding this comment.
updates to remove runSampleFast_Int look good.
There was a problem hiding this comment.
Looks good with runSampleFast_Int removed
sbeaulieu
approved these changes
Feb 4, 2026
| "## syringeSamplingSpeed (usually 20 mins), SyringeSampleVolume (usually 5 ml)\n", | ||
| "underway_IFCB_output_with_calcs['flowRate_mins'] = underway_IFCB_output_with_calcs['SyringeSampleVolume'] / underway_IFCB_output_with_calcs['syringeSamplingSpeed']\n", | ||
| "\n", | ||
| "underway_IFCB_hdr_output_with_calcs['flowRate_mins'] = np.where(\n", |
| { | ||
| "cells": [ | ||
| { | ||
| "cell_type": "code", |
There was a problem hiding this comment.
for next version perhaps consider adding markdown cell at top to describe that this one notebook combines the two notebooks "IFCB_Log-Underway_IFCB_Data_Processing.ipynb" and "Ship_Underway_Lat_and_Lon_Processing.ipynb"
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Files to Review
Underway and Discrete Log Notebook Updates Summaries
Reasoning: syringeSamplingSpeed (McLane sets this to 20 minutes, i.e., this is a fixed 20 in the header file, which is not always accurate); 20 minutes is accurate in the cases where the sampling volume is 5 mL, but becomes inaccurate when the sampling times are altered. The more operationally precise way to view this is to focus on the default flow rate being 0.25 mL/min (which is true, and is usually accurately represented by the defaults (5 mL SyringeSampleVolume / syringeSamplingSpeed). 0.25 mL/min is only NOT accurate in cases where runSampleFast_Int is changed from 1 (the default) AND runSampleFast == TRUE.
SyringeSampleVolume (this is usually 5 ml unless the sample time is cut short by the operator (usually for testing reasons)
base_flow_rate = 0.25
underway_IFCB_hdr_output['flowRate_mins'] = np.where(
underway_IFCB_hdr_output['runSampleFast'],
base_flow_rate * underway_IFCB_hdr_output["RunFastFactor"],
base_flow_rate
)
New volume analyzed calculation:
underway_IFCB_hdr_output['volumeAnalyzed'] = underway_IFCB_hdr_output['flowRate_mins'] * (underway_IFCB_hdr_output['lookTime']/60)
Underway Notebook Change Summary
";" sep added between sites in cases where there are two sites (e.g., CP10CNSM;CP12CNSW)