Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support ZIMing local Kolibri data created by a dry-run of ricecooker #84

Open
benoit74 opened this issue Jan 29, 2024 · 2 comments
Open
Labels
question Further information is requested

Comments

@benoit74
Copy link
Collaborator

Many (most ?) Kolibri channels are populated by ricecooker which creates everything needed locally and then uploads this to a Kolibri Studio Channel

The scraper then retrieves this Studio Channel and creates a ZIM.

While convenient when the Studio Channel is well maintained, it proves to be a burden when we struggle to get Studio Channel updated (like we face with openzim/zim-requests#729).

We might want to consider to:

  • run ricecooker in "dry-run" mode, where files are created locally but not uploaded to a Studio Channel
  • directly process these files with the scraper to create a ZIM

This change might be quite straightforward since what we download from the Studio Channel is maybe very close to what we would get locally from rice-cooker. And we would continue to benefit from the rest of the scraper enhancements (ZIM UI, ...).

@benoit74 benoit74 added the question Further information is requested label Jan 29, 2024
@rgaudin
Copy link
Member

rgaudin commented Jan 29, 2024

I believe the collection is just a SQLite DB with files named based on IDs in the DB. Why not adding a script to either:

  • build a collection similar to that of studio from that dry-run folder. Might already exists.
  • serve (using python's bundled http server) that dry-run folder in a way that mimics the studio

This would eliminate the need for scraper change. Running it with just a env change for the studio URL

@benoit74
Copy link
Collaborator Author

Yup, to be investigated, good idea

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants