Doc with required packages

[MarcAntoine-Arnaud]

In our crates, we have some dependencies to native packages (OpenSSL, FFMpeg).

Builds of the documentation are broken.

What is the best way to install packages ?

• recreate a complete Dockerfile ?
• have a way to connect a worker with custom dependencies ?

[woutersl]

Hi @MarcAntoine-Arnaud, we have the same issue with openssl, for a very small number of our packages. The simplest solution for the time being is to extend the pre-built docker image with the native libs installed (or create a completely new one if this is not appropriate).
A public docker image with all native libs pre-installed would probably not be useful for anyone.

Docs.rs has the same problem. I think they use huge images with many pre-installed native libs ; but I'll check again how they handle this.

Having a system with specialized workers is a possibility but it complicates the overall architecture (need to think about the interface between the core server and the workers, etc.) I'd like to avoid that if possible.

Side note : The same problem arises for uncommon documentation targets. Installing a required target with rustup on the fly is easy but sometimes additional packages on the system are required and we are back to the same problem.

[woutersl]

Hi @MarcAntoine-Arnaud , on the master branch it is not possible to use cratery in a master-worker configuration with multiple workers running on different hosts. Documentation generation jobs can then be offloaded to the workers. It is also possible to configure crates to require certain capabilities from worker nodes. Nodes themselves need to explicitly declare those.

I think that would fit your use-case of having specific workers with openssl installed and exposing them with an appropriate capability name. Then the relevant crates can be configured (in the admin tab) to require that capability.

[MarcAntoine-Arnaud]

Hi @woutersl ,
Yes it sounds awesome if it's possible to have that kind of configuration.

As we deploy with Docker, it's for me not relevant to think on the SSL between worker/master as a private network is enough.
But to be more generic it can make sense.

I just dit not understand, you mentioned it is not possible, but it seems you have pushed some commits to handle that right ?
It's not yet finished ?

[woutersl]

Oops, I meant it is now possible ;)