Documentation updates for Lektor PR lektor/lektor#1065

[dairiki]

Documentation updates for PR lektor/lektor#1065.

[nixjdm]

This was the case before, but I think it would be good to note some specifics about how the plugins system works internally. It may not be obvious what's going on, and what implications those internals has, and it's all potentially useful for plugin developers. For example

• All of the plugins have their shared, nested virtual env, not each their own venv. Thus, plugin specified packages may conflict and cause the project to fail to build.
• Q) What happens if a plugin specifies a dependency that conflicts with the primary lektor env? E.g. lektor's env uses mistune==0.8.4 but a plugin requires mistune>=2 A) The plugin environment currently will fetch it's required dependency, but when the plugin's code is actually ran, the primary virtual environment's version of the dependency will be used (0.8.4 in this example).

I'm sure there are plenty of possible points of confusion.

[dairiki]

I'm sure there are plenty of possible points of confusion.

You are not kidding there! I'll see what I can do. (I am not a great writer.)

(PRs accepted 😁. I think GitHub will allow you to push directly to my branch for this PR.)

---

As an aside, I really think Lektor should move away altogether from managing the installation of its plugins.

The general approach I'm thinking of is to use a single virtual environment per Lektor project. Into that virtualenv gets installed (by means other than running Lektor):

• Whatever version of Lektor and its dependencies is used to build the site
• Any Lektor plugins used by the project. These could be installed from PyPI (or some other package repository) and/or from project-local source code.

Poetry, pipenv, or plain old pip with a requirements.txt are obvious candidates for how such a virtualenv could be managed. PR #327 by @yagebu does just this for lektor-website (using pipenv) and seems to work fine.

To get to that point, all Lektor, as a project, really needs to do is:

1. Document the new way of doing things. (Easier said than done.)
2. Deprecate the configuration of plugins in the .lektorproject file.
3. Deprecate putting plugin source into the project's packages subdirectory. (Any project-local plugins would be moved elsewhere — e.g. into a src subdirectory ` — so that Lektor doesn't try to install it. It would then be installed — possibly/probably in editable mode — by Poetry, Pipenv, Setuptools or whatever other packaging tool the author wants to use.)

and possibly

4. In order to allow using a single virtualenv to build multiple projects (which may use different sets of plugins) add a way to disable (and/or explicitly enable) particular plugins that are installed in the virtual environment.

5. Loosen up the quasi-arbitrary checks that Lektor currently does regarding the relation between plugin names and distribution names. This would allow, e.g., multiple plugins to be placed in a single distribution, which could be particularly useful for writing project-local plugins (i.e. what's currently in packages).

Running in a single virtual environment is just so much cleaner. And I don't really see why Lektor should be in the distribution installation business at all. (If it were possible to make plugin use dead-simple, then sure, I see the point; but, as you say, there are a bunch of subtle gotchas that arise from Lektor's attempts to make things so simple, so, in the end, it's not so simple anyway...)