Resolve conflict: overwrite, skip, keep both or diff

[ppo]

Use case: I've a template file (.env) containing auto-generated passwords (using make_secret()), and it is ignored in Git (obviously).

When I update the project, it is of course every time different because of the auto-generated passwords… but it can also be different because of changes in the rest of the content. And the current file must keep the current passwords.

So, it would be great to have more choices (than "overwrite? y/n") to resolve the conflict.

I'm thinking about creating a new file containing either a diff or the new version (to be safe and not modifying the current file). That file would have the same name with a suffix ".diff" or ".new".

[yajo]

There are a few tools in your belt that you can use:

1. Add .env to skip_if_exists
2. Generate 2 identical files (.env.new.tmpl can just contain [% include ".env" -%] and jinja will replicate it).
3. Use a pre-commit hook that forbids the inclusion of that file in your git history, just like you'd do with .rej files.
4. Use tasks to program a more specific workflow.

I think that a smart combination of some of those should be enough for your use case.

[ppo]

Why make it easy when you can do it manually in just 4 steps with an extra useless file and a custom task workflow, right? 😉😁

Maybe a _protected_files could be useful to never overwrite them but create a .new file.
I guess that all templates using make_secret() would need that.

So sorry to insist but I think it makes sense to have that built in Copier.

[yajo]

Use cases might vary a lot. Yours, for instance, is not very common.

If you're adding secrets, the common use case is to add .env to skip_if_exists, because you can safely assume that the user already provided the secrets he wants.

Otherwise your app has to consider 2 files: .env and .env.new, which isn't very common and usually requires modifying the app to adapt to the template, while the common thing is to do it the other way around.

Besides, you don't need all 4 steps. These are just options to let you get wherever you want, but possibly just steps 1 or 1 and 2 are enough. 3 is just a commodity and 4 only for really advanced cases.

And I hope you're not asking me to replace pre-commit with copier, because that'd be like replacing git with croissants... 😄

[ppo]

Ignoring the file is not the problem here (ignoring with skip_if_exists, .gitignore, pre-commit, whatever).

The use case is any file that:

• contains auto-generated values that must not change once created
• is git ignored
• maybe be updated in the template

Or put simply, any file that has some parts that must never be updated, while the rest of the content must.

For example, my .env contains environment variables (weird, right).
There could be new ones later or some may disappear.
Or there could be updates in the comments.

OK, it may concern only a single file but any code handling sensitive data should have that one file.
BTW, if you provide a make_secret(), shouldn't you also provide a way to handle properly files containing those values? 😁

PS: This is my last comment, I don't want to debate about that. I think it would be better to handle that properly out of the box. If you don't want to include that, it's fine by me too… but I would be really disappointed 😟🥺

[pawamoy]

Wouldn't it be hard (or impossible) to properly update parts of a file if it is ignored by git, and therefore we don't have any modification history for this file? Copier's update mechanism relies on git, so without tracking a file it's not possible to update parts of it I guess.

[ppo]

Generate 2 identical files (.env.new.tmpl can just contain [% include ".env" -%] and jinja will replicate it).

That would work… but it's so ugly!

.
The others can't solve my problem 😕

Add .env to skip_if_exists

I don't need to simply skip it as it may contain real changes.

Use a pre-commit hook that forbids the inclusion of that file in your git history, [just like you'd do with .rej files]
(https://copier.readthedocs.io/en/stable/updating/).

OK but not solving my problem.

Use tasks to program a more specific workflow.

Not solving my problem as executed after the update, so after the file was changed.
A pre-update task could allow me to rename the file… but then every time there's no changes, you'll have to delete the new file and rename that one.

[ppo]

Idea: Have a _protected setting; similar to _exclude.

During creation, nothing special.

During updates, these files are not overwritten.
They are generated in a temporary location and a diff is created (with a .diff suffix).

When reviewing the update, we can easily see if it is just the auto-generated values that are different.
If there are other changes, we can edit the .diff to remove the auto-generated lines before patching.

We could also create a task to remove these lines automatically. And delete the .diff file if empty.

Or cherry on the case, Copier could provide a smarter config.
_protected would define a dictionary with the key being the file pattern and the value a regex matching the lines to ignore.
Copier would then automatically perform the task explained above.

This still doesn't solve my problem with values like DB_URL=engine://user:password@host/dbname where password must be ignore but not the rest of the line (in case other options change). 🤔

[pawamoy]

Can't you just split your env file in two in the template, and track the one that needs update, and ignore the one that has secrets that shouldn't be updated?

[yajo]

That's also a simple and elegant solution.

[ppo]

Yes, thanks, but…

Sorry, I was probably not clear enough.
I'm not looking for "dirty fixes" to solve a problem.
I'm looking to find an elegant solution to make Copier even better (elegant for the user and the template) 😉

[yajo]

I think I finally got what would be the solution for you: #343