-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement offline DB update #25
Implement offline DB update #25
Commits on Jun 15, 2022
-
Configuration menu - View commit details
-
Copy full SHA for f8cac74 - Browse repository at this point
Copy the full SHA f8cac74View commit details -
Update Jenkins pipeline to submit job updating the database. The subm…
…ission script now uses the environment variables set by pipeline to set the database name and path.
Configuration menu - View commit details
-
Copy full SHA for 82172c0 - Browse repository at this point
Copy the full SHA 82172c0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2dc9790 - Browse repository at this point
Copy the full SHA 2dc9790View commit details -
Remove the check that the file we are trying to create during the upd…
…ate does not exist yet. If non concurrent Jenkins runs are allowed, then it should be safe to restart an indexing job from an unfinished one.
Configuration menu - View commit details
-
Copy full SHA for e898053 - Browse repository at this point
Copy the full SHA e898053View commit details
Commits on Jun 16, 2022
-
Reorganize the different stages of the Jenkins pipeline and move seve…
…ral steps to run at the end of the pipeline, depending on the result of the update. Change the name of the PBS logs, so that we can easily archive then with Jenkins.
Configuration menu - View commit details
-
Copy full SHA for eca3c41 - Browse repository at this point
Copy the full SHA eca3c41View commit details
Commits on Jun 20, 2022
-
Refactor to separate bash and python scripts.
Use parallel to allow simultaneous indexing and shuffling inputs. Set file chunks to 1000.
Configuration menu - View commit details
-
Copy full SHA for bafb5c7 - Browse repository at this point
Copy the full SHA bafb5c7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 47f4ebf - Browse repository at this point
Copy the full SHA 47f4ebfView commit details -
Update update script to use an older version of the cookbook. Allow t…
…o pass the database to be updated through the command line.
Configuration menu - View commit details
-
Copy full SHA for 838ebbe - Browse repository at this point
Copy the full SHA 838ebbeView commit details -
Update Jenkinsfile to run then update script interactively, i.e., wit…
…hout using PBS. We are still setting a time limit for the update, so that updates that run for longer than a reasonable amount of time don't go unnoticed. An update that fails to complete because of the time limit will mark the build as unstable.
Configuration menu - View commit details
-
Copy full SHA for 16d0697 - Browse repository at this point
Copy the full SHA 16d0697View commit details -
Better handling of some situations (plus some comments):
- Do not allow to update the database twice during the same day. - Remove the new database file if an update fails. - Remove the new database file if the run is aborted by a user.
Configuration menu - View commit details
-
Copy full SHA for 34a7895 - Browse repository at this point
Copy the full SHA 34a7895View commit details -
Configuration menu - View commit details
-
Copy full SHA for bec388f - Browse repository at this point
Copy the full SHA bec388fView commit details
Commits on Jun 22, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 3722473 - Browse repository at this point
Copy the full SHA 3722473View commit details
Commits on Jun 30, 2022
-
Only prune old files if current update was successful. Otherwise we m…
…ight end up deleting all databases if the update fails over a long period of time.
Configuration menu - View commit details
-
Copy full SHA for ed744fc - Browse repository at this point
Copy the full SHA ed744fcView commit details -
Configuration menu - View commit details
-
Copy full SHA for a3ca9a8 - Browse repository at this point
Copy the full SHA a3ca9a8View commit details
Commits on Jul 12, 2022
-
Go back to using a PBS job to update the database, as the interactive…
… job was being killed after some time.
Configuration menu - View commit details
-
Copy full SHA for 61a02bc - Browse repository at this point
Copy the full SHA 61a02bcView commit details
Commits on Jul 13, 2022
-
Configuration menu - View commit details
-
Copy full SHA for ae401c2 - Browse repository at this point
Copy the full SHA ae401c2View commit details
Commits on Jul 18, 2022
-
Configuration menu - View commit details
-
Copy full SHA for abe7177 - Browse repository at this point
Copy the full SHA abe7177View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0fe7db5 - Browse repository at this point
Copy the full SHA 0fe7db5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 699c060 - Browse repository at this point
Copy the full SHA 699c060View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5d011d6 - Browse repository at this point
Copy the full SHA 5d011d6View commit details -
Configuration menu - View commit details
-
Copy full SHA for f015ab9 - Browse repository at this point
Copy the full SHA f015ab9View commit details
Commits on Jul 19, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 092b606 - Browse repository at this point
Copy the full SHA 092b606View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1557726 - Browse repository at this point
Copy the full SHA 1557726View commit details
Commits on Jul 20, 2022
-
Configuration menu - View commit details
-
Copy full SHA for ba4c72e - Browse repository at this point
Copy the full SHA ba4c72eView commit details