[ feature request ] Add support for alternative sources of package collections

[buzden]

I ran into a problem that some upstream package (which is not yet present in the pack collection) requires some additional or alternative versions of some packages, and thus downstream package needs to copy all the settings of upstream's pack.toml to their local pack.toml. I was thinking of how this can be better, and realised that alternative sources of collections can help. As it turned out, I'm not the only one who is struggling with this problem.

So, what I suggest is to make able people to create their own pack collections easily and be able to use them easily. For that, I think of the following steps that we can achieve this:

• Support for setting alternative source of a pack collection

  This can be done either by having an additional property in the pack.toml, say

  collection = "my-fancy-collection"
  collection-source = "http://github.com/username/my-fancy-project/.pack-config"
  

  or by allowing a URL in the collection property:

  collection = "http://github.com/username/my-fancy-project/.pack-config/my-fancy-collection.toml"
  

  I personally prefer the latter for simplicity

• (optional) Support of collections of known, possibly popular, collection sources, allowing easier use and future-proofing for potential migrations. Say, we have a special alternative-sources in the pack-db project, which contains, say, that frex project has a list of its alternative collections in its repo. This would allow us to use it in out pack.toml in the following way:

  collection = "frex:my-fancy-frex-based-collection-20230000"
  

• Support for inheritance or joining of collections

  This is for making creation of new collections easier. The way I see it can work is the following: we add an ability to add a property to a collection description which collection/collections it inherits, i.e. uses all the definitions as a base. After that the author of this inherited collection has add, redefine or remove some packages. This allows us to use alternative versions of existing libraries (and remove incompatible users), and add something not yet published in the main pack collection (or not publishable due to tricky dependencies, say dependencies on sufficiently older versions). Addition and redefinition is easy, we can use normal way of packages definition, we need to invent a way of removing packages from the inheritance.

• As the final thing, we probably should provide a way for users to check their custom collections on the consistency, as not it is done in the pack-db repository. This can be done my making this check a reusable workflow (one, two). This would make it much easier to launch checks of custom collection consistency on a CI.

[stefan-hoeck]

Sorry for the late response. This would surely be a nice thing to have, but it will require some planning - so of which you have already done above - and I'm afraid I won't be able to tackle this before the summer (earliest).

[Russoul]

I second that feature request! Would like to be able to at least tell pack to use a custom local collection file.