Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
af3ad65
Initial uv pyproject.toml and uv.lock files.
MBARIMike Jun 4, 2025
fd84669
Pass lower() case 'S' to Pandas in anticipation of future warning.
MBARIMike Jun 4, 2025
1c7e65e
Bypass numpy.core._exceptions not being found until we can figure out…
MBARIMike Jun 4, 2025
c9b9de1
Removed exception handling for UFuncTypeError - processed Dorado 2008…
MBARIMike Jun 4, 2025
439cc50
Test Dorado 2008.010.010.
MBARIMike Jun 4, 2025
8b0274c
Add pysolar.
MBARIMike Jun 4, 2025
0deacf9
Add back the ruff settings.
MBARIMike Jun 5, 2025
d52fcb9
Use uv and python3.13 in ci and in production docker.
MBARIMike Jun 5, 2025
ef5b24a
Re-install pre-commit using uv.
MBARIMike Jun 5, 2025
419baaf
Modify pre-commit to install from pyproject.toml.
MBARIMike Jun 5, 2025
c9c2b58
Reformatted using new pre-commit setup.
MBARIMike Jun 5, 2025
fbfb93b
Change all freq to freq.lower() for new pandas.
MBARIMike Jun 5, 2025
ac696e2
Change [dependency-groups] to [project.optional-dependencies].
MBARIMike Jun 5, 2025
badf59b
Frustratringly trying to sync pre-commit and vscode ruff behavior.
MBARIMike Jun 6, 2025
505952a
Following a 'uv sync'.
MBARIMike Jun 6, 2025
93dbb33
Change [project.optional-dependencies] back to [dependency-groups].
MBARIMike Jun 9, 2025
1a88687
Updated with Python 3.12.
MBARIMike Jun 9, 2025
b26a4f0
No longer needed with uv.
MBARIMike Jun 9, 2025
45c2ef2
Update EXPECTED_SIZE_LOCALs.
MBARIMike Jun 9, 2025
9532dbb
Make the linter happier.
MBARIMike Jun 9, 2025
23b5b21
Use separate EXPECTED_SIZE_GITHUB and EXPECTED_SIZE_ACT values.
MBARIMike Jun 9, 2025
dfa8abd
Downgrade to Python 3.12.
MBARIMike Jun 9, 2025
6fbaefb
Use official GitHub pre-commit/action.
MBARIMike Jun 9, 2025
1e04c9b
Correct typo in pre-commit/action version.
MBARIMike Jun 9, 2025
20f5df7
Remove lint job, rename test to pytest.
MBARIMike Jun 9, 2025
76e4651
Rename pre-commit to lint.
MBARIMike Jun 9, 2025
54eeefb
Use just 'extra_args' to avoid warning in CI.
MBARIMike Jun 9, 2025
9d2fd9c
Add python version, remove comma.
MBARIMike Jun 9, 2025
6fadb9b
Remove ruff-format.
MBARIMike Jun 9, 2025
1a15d1a
Add args to ruff.
MBARIMike Jun 9, 2025
77c801b
Correct the ruff args.
MBARIMike Jun 9, 2025
de428d0
Okay, don't use ruff arg, use --all-files.
MBARIMike Jun 9, 2025
3d7df23
Use "1s" literal for resample()ing.
MBARIMike Jun 9, 2025
fa34218
Remove RuntimeWarnings.
MBARIMike Jun 9, 2025
8943b8b
Suppress RuntimeWarnings instead of altering values of Tr.
MBARIMike Jun 9, 2025
0b7bf42
Remove top level files no longer used.
MBARIMike Jun 10, 2025
5ecd9f9
Add jupyter.
MBARIMike Jun 10, 2025
b2f24eb
Add MD5 checksum test.
MBARIMike Jun 10, 2025
661e2fb
Test processing with new uv-update code.
MBARIMike Jun 10, 2025
4ff0681
Changed for using python3.12 in uv-update.
MBARIMike Jun 10, 2025
5f48ae7
Update following replacing poetry with uv.
MBARIMike Jun 10, 2025
151c8f3
Removed as contributing instructions are now in README.md.
MBARIMike Jun 11, 2025
f967ec9
Remove unused files created by cookiecutter-data-science.
MBARIMike Jun 11, 2025
1f88b4b
Update Copyright date and change to MBARI.
MBARIMike Jun 11, 2025
518d6f0
Makefile is not used.
MBARIMike Jun 11, 2025
e99cc67
A few revisions.
MBARIMike Jun 11, 2025
c30d73b
Update to python3.12.
MBARIMike Jun 11, 2025
1f542a1
Updating this Notebook required 'uv add ipympl'.
MBARIMike Jun 11, 2025
6579787
Try a new value for EXPECTED_MD5_GITHUB.
MBARIMike Jun 11, 2025
d48765d
Updates for using python3.12.
MBARIMike Jun 11, 2025
e4c4c16
Add pooch and pyarrow so that the 5.0 Notebook works.
MBARIMike Jun 11, 2025
43fce60
Updates for using python3.12.
MBARIMike Jun 11, 2025
a286941
Add rolling so that the 5.2 Notebook works.
MBARIMike Jun 11, 2025
9f4fb00
Add rolling so that the 5.2 Notebook works.
MBARIMike Jun 11, 2025
0d77a02
Add --noinput for more automated processing.
MBARIMike Jun 11, 2025
2c851f5
Add statsmodels so that Notebook 5.3 works.
MBARIMike Jun 11, 2025
a0beddc
Make work with python3.12.
MBARIMike Jun 11, 2025
9734c1e
Add mission 2025.161.00.
MBARIMike Jun 11, 2025
d765b55
Make work with python3.12.
MBARIMike Jun 12, 2025
fc8c50d
Update for python3.12 and use new known_hash.
MBARIMike Jun 12, 2025
8154d4d
Scripts in src/data are executable so remove 'python'.
MBARIMike Jun 12, 2025
bd257d6
Update for use with python3.12.
MBARIMike Jun 12, 2025
28adb65
Update for python3.12. Interactive cells have same problem as the 5.9…
MBARIMike Jun 12, 2025
55b63ae
UIpdated for use with pyhon3.12.
MBARIMike Jun 12, 2025
c8c2fc3
Update for python3.12 and add "bbp" plots.
MBARIMike Jun 12, 2025
924808f
Remove old execution instructions and update known_hash.
MBARIMike Jun 12, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 9 additions & 32 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ on:
pull_request:

jobs:
lint:
pytest:
runs-on: ubuntu-latest
steps:
- name: Checkout code
Expand All @@ -16,37 +16,15 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: '3.10'
python-version: '3.12'

- name: Install Poetry
run: |
curl -sSL https://install.python-poetry.org | python3 -
echo "$HOME/.local/bin" >> $GITHUB_PATH

- name: Install dependencies
run: poetry install

- name: Run pre-commit
run: poetry run pre-commit run --all-files

test:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
python-version: '3.10'

- name: Install Poetry
run: |
curl -sSL https://install.python-poetry.org | python3 -
echo "$HOME/.local/bin" >> $GITHUB_PATH
version: "0.7.11"

- name: Install dependencies
run: poetry install
run: uv pip install --system .

- name: Set environment variables
run: |
Expand All @@ -56,16 +34,15 @@ jobs:
- name: Download and unzip required data files
run: |
mkdir -p "$HOME/auv_data"
# Dorado389.tar.gz contains original log files of short dorado mission for testing: 2011.256.02
wget -q --no-check-certificate -O "$HOME/auv_data/Dorado389.tar.gz" https://stoqs.mbari.org/auv_data/Dorado389.tar.gz
tar -xvf "$HOME/auv_data/Dorado389.tar.gz" -C "$HOME/auv_data" 2> /dev/null
# i2MAP.tar.gz contains original log files of i2MAP mission for testing: 2018.34.01
wget -q --no-check-certificate -O "$HOME/auv_data/i2map.tar.gz" https://stoqs.mbari.org/auv_data/i2map.tar.gz
tar -xf "$HOME/auv_data/i2map.tar.gz" -C "$HOME/auv_data" 2> /dev/null
# i2map processing needs access to calibration files on an on-premisis server, unpack a subset of it here
wget -q --no-check-certificate -O "$HOME/auv_data/SBE-43.tar.gz" https://stoqs.mbari.org/auv_data/SBE-43.tar.gz
tar -xf "$HOME/auv_data/SBE-43.tar.gz" -C "$HOME/auv_data" 2> /dev/null

- name: Install pytest
run: uv pip install --system pytest

- name: Run tests
run: poetry run pytest
run: pytest
19 changes: 19 additions & 0 deletions .github/workflows/pre-commit.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
name: pre-commit

on:
pull_request:
push:
branches: [main]

jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v3
with:
python-version: '3.12'
- uses: pre-commit/action@v3.0.1
with:
extra_args: --all-files

39 changes: 3 additions & 36 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -1,40 +1,7 @@
exclude: '^notebooks/'
default_stages: [pre-commit]
minimum_pre_commit_version: "3.2.0"

default_language_version:
python: python3.10

# .pre-commit-config.yaml
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-json
exclude: ^.vscode/launch.json$
- id: check-toml
- id: check-xml
- id: check-yaml
- id: debug-statements
- id: check-builtin-literals
- id: check-case-conflict
- id: check-docstring-first
- id: detect-private-key
exclude: ^compose/production/nginx/localhost.key$

# Run the Ruff linter.
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.9.2
rev: v0.4.4
hooks:
# Linter
- id: ruff
args: [--fix, --exit-non-zero-on-fix]
# Formatter
- id: ruff-format

# sets up .pre-commit-ci.yaml to ensure pre-commit dependencies stay up to date
ci:
autoupdate_schedule: weekly
skip: []
submodules: false
- id: ruff-format
5 changes: 3 additions & 2 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
Expand Up @@ -262,12 +262,13 @@
//"args": ["-v", "1", "--noinput", "--start_year", "2016", "--end_year", "2016", "--start_yd", "270", "--end_yd", "270", "--num_cores", "1", "--create_products", "--archive", "--archive_only_products"]
//"args": ["-v", "1", "--noinput", "--num_cores", "1", "--mission", "2023.285.01"]
//"args": ["-v", "1", "--noinput", "--no_cleanup", "--mission", "2018.079.00"]
//"args": ["-v", "1", "--noinput", "--no_cleanup", "--mission", "2024.317.01"]
"args": ["-v", "1", "--noinput", "--no_cleanup", "--mission", "2024.317.01", "--clobber"]
//"args": ["-v", "1", "--noinput", "--no_cleanup", "--mission", "2016.090.01", "--resample", "--archive", "--flash_threshold", "3E10"]
//"args": ["-v", "1", "--noinput", "--num_cores", "8"]
//"args": ["-v", "1", "--noinput", "--no_cleanup", "--mission", "2011.256.02", "--clobber"]
//"args": ["-v", "1", "--noinput", "--no_cleanup", "--mission", "2010.341.00", "--download_process", "--local"]
"args": ["-v", "1", "--noinput", "--no_cleanup", "--mission", "2020.337.00", "--clobber"]
//"args": ["-v", "1", "--noinput", "--no_cleanup", "--mission", "2020.337.00", "--clobber"]
//"args": ["-v", "1", "--noinput", "--no_cleanup", "--mission", "2008.010.10"]

},
]
Expand Down
75 changes: 0 additions & 75 deletions CONTRIBUTING.md

This file was deleted.

57 changes: 0 additions & 57 deletions LAYOUT.md

This file was deleted.

2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@

Copyright (c) 2021, Mike McCann
Copyright (c) 2025, Monterey Bay Aquarium Research Institute
All rights reserved.

Redistribution and use in source and binary forms, with or without modification,
Expand Down
Loading