Skip to content

Colabdesign parallelism - multiple sub jobs and checkpoints #1011

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 128 commits into from
Aug 5, 2024
Merged

Colabdesign parallelism - multiple sub jobs and checkpoints #1011

merged 128 commits into from
Aug 5, 2024

Conversation

supraja-968
Copy link
Collaborator

@supraja-968 supraja-968 commented Aug 2, 2024 •
edited
Loading

What type of PR is this?

  • 🎮 Feature

Description

  • handling multiple outputs for a single colabdesign input, to support the changes made in this PR - https://github.com/convexitylabs/convexity/pull/179
  • adding a new 'file_processed' state to the inference events to identify already processed checkpoints vs newly posted checkpoints
  • some text fixes for stripe UI

to do for dev: checkpoints component auto refresh while the job is running (raw files are getting updated, but checkpoints only after a manual refresh at the moment)

image image

supraja-968 added 30 commits June 18, 2024 17:57
@supraja-968
flow to experiment rename
2b13851
@supraja-968
Merge branch 'main' of https://github.com/labdao/plex into LAB-1484-r…
f279039 
…ename-DB-tables
@supraja-968
tool to model rename WIP
aa0e3a0
@supraja-968
tool to model WIP
cef8191
@supraja-968
tool to model remaining changes
20cc775
@supraja-968
db script for tool and flow renames
9c79e7e
@supraja-968
db script update
ccab0d2
@supraja-968
tool manifest update
b0b0c7b
@supraja-968
Merge branch 'main' of https://github.com/labdao/plex into LAB-1484-r…
32d8cf0 
…ename-DB-tables
@supraja-968
checkpoint updates
a91c8ea
@supraja-968
removing old tools from CI
b3f6b46
@supraja-968
removing unused tools from CI
9afdf9d
@supraja-968
removed unused checkpoint handlers
6d5eff2
@supraja-968
datafiles to files, flow to experiment fixes after main merge
3e88319
@supraja-968
datafile to file
9f02989
@supraja-968
update DB scripts
39d5465
@supraja-968
db script update
c3dcfa4
@supraja-968
run to job
fd6e669
@supraja-968
db error logs silent
62bf79f
@supraja-968
db script updates
1c2b667
@supraja-968
migration script and models wip
98cc001
@supraja-968
initial DB schema update and script WIP
1c1a01f
@supraja-968
input and output files join table fix
f1ebdf0
@supraja-968
user files join table fix
9192348
@supraja-968
file tags join table fixes
582f98b
@supraja-968
add default no_org
241eb60
@supraja-968
datafiles handler fixes
ea5cdd5
@supraja-968
datafiles -> files rename
3afb57a
@supraja-968
model handlers
42deaa5
@supraja-968
experiments, jobs handlers fix WIP
ad3c1af
@supraja-968 supraja-968 requested a review from acashmoney August 2, 2024 10:12
@supraja-968 supraja-968 merged commit 064d609 into main Aug 5, 2024
17 checks passed
@supraja-968 supraja-968 deleted the multi-checkpoint branch August 5, 2024 11:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@acashmoney acashmoney Awaiting requested review from acashmoney

Assignees
No one assigned
Labels
deployed to test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@supraja-968 @acashmoney