Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rake kithe:create_derivatives gets some options for creating Background Activejob #166

Merged
merged 6 commits into from
Aug 2, 2023

Conversation

jrochkind
Copy link
Contributor

@jrochkind jrochkind commented Aug 2, 2023

We already had a kithe:create_derivatives task with many options for different things you'd want to do for creating derivatives -- but it did it all in the foreground, serially, in a single thread.

I had need to instead create many small background jobs (which we can then easily handle on heroku by scaling up workers, to get it done quickly!). And realized small changes to the existing CreateDerivativesJob and the rake task would make that possible.

So do so here.

For the thing we want to do in our app, for instance, we can run (on heroku even)

heroku run "rake kithe:create_derivatives -- --derivatives=graphiconly_pdf --lazy --bg=special_jobs"

That will create one BG job per Asset (for all assets!), in the ActiveJob queue special_jobs. The job will be queued up for every Asset, but since we said "lazy", it will only actually create the graphiconly_pdf derivative if one doesn't already exist, otherwise the job will just exit.

@jrochkind
Copy link
Contributor Author

No tests here, this stuff doesn't have GREAT ways to test and lacked existing tests.

I did some tests manually.

Eh, maybe I'll think about adding at least a bit of tests....

@jrochkind
Copy link
Contributor Author

Nah, I'm not going to do tests right now.

@jrochkind
Copy link
Contributor Author

OK I do need tests, cause I had a bug ha!

@jrochkind jrochkind merged commit dd46fa5 into master Aug 2, 2023
5 checks passed
@jrochkind jrochkind deleted the create_derivative_job_arguments branch August 2, 2023 19:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant