Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Run migrations in initContainer and use galaxy-ng image #17

Closed
wants to merge 78 commits into from
Closed

Run migrations in initContainer and use galaxy-ng image #17

wants to merge 78 commits into from

Conversation

rooftopcellist
Copy link
Member

@rooftopcellist rooftopcellist commented Jan 19, 2024 •
edited by aknochow
Loading

SUMMARY

Goals:

  • Run migrations in an initContainer like we do for the eda-server-operator
  • Move to galaxy-ng and galaxy-ui images (which no longer run migrations in the init-script
ADDITIONAL INFORMATION

Previously, when using galaxy-minimal and galaxy-web, we were using these container scripts. We could vendor these as needed and configmap them in... But ideally we use what the galaxy team uses where possible for consistency.

Here are what I think are the new configuration scripts used by the galaxy team's images:

@rooftopcellist rooftopcellist force-pushed the initcontainer-migrations branch 2 times, most recently from a9cc5e9 to 26aa232 Compare January 19, 2024 16:52
@dsavineau
Copy link
Contributor

Move to galaxy-ng and galaxy-ui images (which no longer run migrations in the init-script

As far I can see, we're still using the pulp images in the code [1].

Am I missing something ?

I think changing the container images from pulp to galaxy will require more work.

This will at least remove feature like configuring the admin password and the collection/container signing feature (because those aren't currently handled in the pulp-api entrypoint) [2]

But we'll also need to update or add other configuration like PULP_API_WORKERS or PULP_CONTENT_WORKERS env var that are renamed GUNICORN_WORKERS.
Also the default api port value from the galaxy api entrypoint is the same and would need to be overrided. [3]

[1] https://github.com/ansible/galaxy-operator/blob/main/playbooks/pulp.yml#L31-L34
[2] https://github.com/pulp/pulp-oci-images/blob/3.23/images/assets/pulp-api#L18-L31
[3] https://github.com/ansible/galaxy_ng/blob/master/docker/bin/start-api#L14

@rooftopcellist rooftopcellist marked this pull request as draft January 19, 2024 18:19
@rooftopcellist
Copy link
Member Author

@dsavineau Correct on all points! I meant to make this a draft PR, @aknochow and I are iterating on this to make all of the changes needed to use the galaxy-ng and galaxy-ui images.

I just added a commit to switch all of the default references and doc examples to the new images.

Thanks for calling out the admin password creation and signing scripts. I'll look at those. I am hoping to use the galaxy-ng startup scripts where possible, then configmap in anything else that may be needed, but we'll see how that turns out in practice :)

@rooftopcellist rooftopcellist added debug-3ci Keep 3ci environment around for CI enhancement New feature or request labels Jan 19, 2024
@rooftopcellist rooftopcellist force-pushed the initcontainer-migrations branch 5 times, most recently from bfc7040 to fbd2311 Compare January 23, 2024 16:19
@aknochow aknochow force-pushed the initcontainer-migrations branch from db100a7 to 288dd91 Compare January 24, 2024 02:23
@dsavineau
Copy link
Contributor

is there a reason to change the default api backend port from 24817 to 8000 rather than using GUNICORN_PORT=24817 environment variable in the api deployment template ?

1f4b9d3
https://github.com/ansible/galaxy_ng/blob/master/docker/bin/start-api#L14

@rooftopcellist rooftopcellist force-pushed the initcontainer-migrations branch 4 times, most recently from 90e1e01 to 8878a14 Compare January 26, 2024 19:17
@aknochow aknochow closed this Feb 1, 2024
@aknochow aknochow reopened this Feb 1, 2024
@rooftopcellist rooftopcellist marked this pull request as ready for review February 1, 2024 16:02
@aknochow aknochow closed this Feb 1, 2024
@aknochow aknochow reopened this Feb 1, 2024
aknochow and others added 10 commits February 6, 2024 21:48
@aknochow
updating worker deployment
8ea91cc
@aknochow
undoing home from /var/lib/pulp
a87e49e
@aknochow
trying /app for home
43d3f05
@aknochow
undoing home from /var/lib/pulp
ad161f2
@aknochow
removing references to signing_service.asc
56f2fc6
@rooftopcellist
Add back $HOME=/var/lib/pulp for worker deployment to fix gpg error
9a3aef9 
- Resolve this error: gpg: Fatal: can't create directory /app/.gnupg: No such file or directory
@rooftopcellist
Make add-signing-service command idempotent
604000b
@rooftopcellist
Pre-create /var/lib/pulp/media directory for file-storage scenario
b5556fd 
- without this, we see this error in the api pod: [Errno 2] No such file or directory: /var/lib/pulp/media
@rooftopcellist
Reduce initialDelaySeconds for web -iness checks
62039e2 
- the web service was waiting unnecessarily while the api was reachable

Debug: add tempory sleep to debug worker initContainer
@rooftopcellist
Use galaxy-ng image for gpg-importer initContainer in worker deployment
53a0662 
- without this we see errors when running gpg --import

gpg: WARNING: unsafe ownership on homedir /var/lib/pulp/.gnupg
gpg: failed to create temporary file /var/lib/pulp/.gnupg/.#lk0x0000562d20c3e270.galaxy-4632a12-worker-7fbb9b9cf9-t98wl.15: Permission denied
@rooftopcellist rooftopcellist force-pushed the initcontainer-migrations branch 2 times, most recently from d44619a to edc14be Compare February 7, 2024 20:58
@rooftopcellist rooftopcellist changed the title Run migrations in initContainer and user galaxy-ng image Run migrations in initContainer and use galaxy-ng image Feb 7, 2024
@rooftopcellist rooftopcellist force-pushed the initcontainer-migrations branch 5 times, most recently from d5e6d0f to ccf1719 Compare February 8, 2024 22:49
@rooftopcellist
Add debugging statement to CI to determine status
3ffa8c4 
* Temporarily use galaxy-ng:add-galaxy-operator-support image
* Explicitly set operator imagePullPolicy to IfNotPresent for CI
@rooftopcellist rooftopcellist force-pushed the initcontainer-migrations branch from ccf1719 to 3ffa8c4 Compare February 8, 2024 23:08
@rooftopcellist rooftopcellist mentioned this pull request Feb 13, 2024
Merged
@rooftopcellist
Copy link
Member Author

Closing in favor of

@aknochow aknochow deleted the initcontainer-migrations branch February 16, 2024 02:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
debug-3ci Keep 3ci environment around for CI enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@rooftopcellist @dsavineau @aknochow