Disallowing crawling/indexing

[terhorst]

The current robots.txt allows the whole site to be crawled:

https://github.com/hay-kot/mealie/blob/7f03b1bea8effba5df61d57af378298591bc9805/frontend/public/robots.txt#L1-L2

I would like to be able to change this (without having to fork, rebuild the Docker image, etc.) to prevent our family recipe site from ending up on Google.

[hay-kot]

You can use a docker volume mount to mount a replacement robot.txt with one that you create with the settings of your choosing.

[tekchip]

Moderated traffic control seems like it would be the ideal default, given that this is typically self-hosted. Allowing folks to then grant more access to their instance, not having to clear a hurdle to go the other way around seems ideal. My two cents. Thanks for all the work you do!

[hay-kot]

Seems reasonable, I'm open to a PR on this if you or anyone else wants to jump on this one.