Add lazymc (on-demand) functionality to nix-minecraft

[Yeshey]

Solves #142
This adds the options:

        lazymc = {
          enable = true;
          package = pkgs.lazymc;
          config = { ... };
        };

Under each instance. This makes servers go to sleep when there are no players and wake up when one player tries to join lazymc

servers.<name>.lazymc

Integrates lazymc, putting server to sleep when idle and waking it upon player connection.

• enable: boolean, default false

• package: package, default pkgs.lazymc
  The lazymc package to use. You might have to change lazymc version according to your minecraft server version, for example lazymc v0.2.10 supports Minecraft Java Edition 1.7.2+, while for Minecraft Java Edition 1.20.3+ you'll need lazymc v0.2.11

• config: attribute set, default {}
  Allows defining and overriding settings in the lazymc.toml, for all options see here. The auto generated config options are:

  • server.command: Automatically set to use the server package and jvmOpts;
  • server.directory;
  • server.address: Automatically set to 127.0.0.1:<serverProperties.server-port or 25565>";

  Firewall

  • When lazymc.enable = true, the openFirewall option for this server instance will open the port specified in lazymc.config.public.address (or its default), not the internal Minecraft serverProperties.server-port.

  Example:

  { pkgs, ... }: {
    services.minecraft-servers = {
      eula = true;
      servers.myLazyServer = {
        enable = true;
        package = pkgs.paperServers.paper-1_18_2;
        serverProperties = {
          "server-port" = 25566;
          "max-tick-time" = -1; # Recommended with lazymc
        };
  
        lazymc = {
          enable = true;
          package = let
          # you can use https://lazamar.co.uk/nix-versions/
            pkgs-with-lazymc_0_2_10 = import (builtins.fetchTarball {
                url = "https://github.com/NixOS/nixpkgs/archive/336eda0d07dc5e2be1f923990ad9fdb6bc8e28e3.tar.gz";
                sha256 = "sha256:0v8vnmgw7cifsp5irib1wkc0bpxzqcarlv8mdybk6dck5m7p10lr";
            }) { inherit (pkgs) system; };
          in pkgs-with-lazymc_0_2_10.lazymc;
          config = {
            public.address = "0.0.0.0:25565";
          };
        };
      };
    };
  }

[Yeshey]

Maybe it would have been better to not use the server-port - 1 and just let the user do all the config themeselves, there is quite a bit of logic bc of that, but there would always be some abstraction because openFirewall would have to automatically open the port in lazymc instead of in serverproperties if it was active. And I do think that having a one click solution like this is more user friendly as long as we can make it work well

And if a user sets two mc instances in adjacent ports with lazymc, there is an assertion for that

[Infinidoge]

Please avoid closing and opening pull requests. You can force push to your branch to edit commits.
Additionally, please avoid doing PRs from a master branch, as it leads to CI commits that shouldn't be in the PR (as evidenced).

You can change the branch a PR is targeting in the PR edit menu on GitHub.

[Infinidoge]

With regards to the actual content of the PR:

• TOML writing can be handled through the files option of the server, as opposed to reimplementing it yourself. See here for an example
• The lazymc config overwrites the management system config, which it should never do. Based on how you wrote it, it seems that lazymc should instead be a management system in and of itself, as right now it incorrectly overwrites load-bearing parts of the existing management systems, breaking them.
• Overall the implementation feels... very verbose and overcomplicated.

[Yeshey]

Thanks for reviewing, I didn't realize that deleting and renaming my repo would also close my PRs, I'll try to keep further changes in this one.
Yeah, the code isn't ideal, when I did it i was probably thinking about my own needs more than doing something clean to try to merge, and might have used an LLM to help a bit 😅.
Sorry if it wasted your time, soon as I get a bit of time I'll try to rewrite it. On a rewrite I think I'd drop the "server-port" - 1 logic, openFirewall would open the lazymc port instead of the mc server one if lazymc is enabled; and just let the user manage the ports themeselves, idk if you want to weigh in on that

[Titaniumtown]

Dont use a llm for anything. opens the project up to legal issues.

[CalmSeoul]

Will this eventually be merged with the main branch? This would be an awesome addition cause I'm having a tough time trying to get lazymc working with nix-minecraft manually, but having it built in would be incredible

[Yeshey]

@CalmSeoul this is a clunky implementation, as it stands it should be rewriten to get merged. I will try to look at it in the summer but i cant promise. I'm not the most proficiant nixer as well.
If u want to use it rn, you can import the flake from my fork and check the readme for how its set up.

[Yeshey]

Rewrote the implementation, sorry for the fuss earlier, no more vibe code now, hopefully it can get merged

[Titaniumtown]

I tested this and it works! Good job.

Only comment is that, shouldn't lazymc's public port be 25565 and the internal port be 25566? It should be transparent to the end user and not be another port imo.

[Yeshey]

I thought shifting the port up or down would be kinda arbitrary and code heavy, so I decided to leave the assertion to force the user to always pick their lazymc public port instead. Maybe you're right that that should suggest port 25565, and now that I'm thinking about that, there should probably be another assertion for when the internal server port and the lazymc public port are the same

[Yeshey]

Or maybe, to do it this way, the right method would be to force the user to set the serverProperties.server-port instead when lazymc is enabled

[Yeshey]

There you go, I think this is better

now if a user goes

        servers.testerido = {
          enable = true;
          lazymc.enable = true;
        };

they'll get:

       error: Server 'testerido' has the same port set for serverProperties.server-port and lazymc.config.public.address
       Please set for example: `testerido.serverProperties.server-port = 25566;`, lazymc's internal server.address will automatically point to it
       Lazymc's public.address is "0.0.0.0:25565" by default

[Titaniumtown]

Just a few comments, mostly good!

[rharish101]

Hi, are there any blockers left preventing this PR being merged? I'm interested in running lazymc with this flake.

[Yeshey]

Hi, are there any blockers left preventing this PR being merged? I'm interested in running lazymc with this flake.

@Titaniumtown @Infinidoge let me know if there are any more blockers, I can try to work on them to get this done! The automatic github action to update package lock files has also been failing recently, not sure why.

[rharish101]

The automatic github action to update package lock files has also been failing recently, not sure why.

That was recently fixed by #176, which got merged a few hours ago.

Also, if you need someone to test it, I can do so, once you've rebased your flake. I'm running a single Paper server. Fair warning, I've never used lazymc before 😛.

[Yeshey]

The automatic github action to update package lock files has also been failing recently, not sure why.

That was recently fixed by #176, which got merged a few hours ago.

Also, if you need someone to test it, I can do so, once you've rebased your flake. I'm running a single Paper server. Fair warning, I've never used lazymc before 😛.

That'd be delightful :), sorry I didn't see your response earlier!
Let me see if I can rebase my flake to fix this! Lazymc is pretty straightforward, I added a section to the readme file that should help you set it up if you're willing to test it, if that's enough for you to find your way around it it would already be a testament to my assertions and bit of documentation hahaah

[rharish101]

I just tested it, and it works! My setup is just a single Paper server with lazymc enabled for it, with Velocity as a proxy. I saw a slow connection time the first time I connected to the server (after a while), but every other time, it was fast enough. I think this means that lazymc is working, right?

[Yeshey]

I just tested it, and it works! My setup is just a single Paper server with lazymc enabled for it, with Velocity as a proxy. I saw a slow connection time the first time I connected to the server (after a while), but every other time, it was fast enough. I think this means that lazymc is working, right?

if you have time, check if it works withmanagementSystem.systemd-socket.enable = true; as well. Last time I checked it was working properly
I guess we wait for someone with write access to give us a heads up! ahah

[Infinidoge]

If I am honest, I don't really want this functionality to be baked into the module. This is something the user can theoretically manage for themself. However considering the complexity of that, I am not (yet) rejecting this outright.

With that said, in addition to the comments on files, please rebase out all automated lockfile commits, and change your branch to something other than master. Automated commits pollute the PR. (Note to self, add that to CONTRIBUTING.md)

[Infinidoge]

This will create the lazymc.toml file even if not configured. Please add a mkIf guard.

The other values don't have a mkIf guard as their values will be an empty attribute set (or otherwise empty value) when not configured.

[Yeshey]

Couldn't chuck a mkIf in there, ended up solving it like this:

            // (optionalAttrs conf.lazymc.enable {
              "lazymc.toml".value = lib.recursiveUpdate {
                server = {
                  command = "${getExe conf.package} ${conf.jvmOpts}";
                  directory = ".";
                  address = "127.0.0.1:${toString conf.serverProperties.server-port or 25565}";
                };
              } conf.lazymc.config;
            })

[Infinidoge]

Strange, I don't see why a mkIf wouldn't work.

[Infinidoge]

The server must stop gracefully if running.
Stopping while the server is running is an expected situation,

[Yeshey]

This is probably the main problem. I've tried a couple ways but there doesn't seem to be any good way talk with lazymc. Or a cross minecraft version way to check if the server is running (that should be lazymcs job).

If the server is running we can send the stop command through stdin and it will stop. But the service won't quit because when it stops lazymc will keep going... The best way to solve this would probably be for this to get merged to lazymc: timvisee/lazymc#81.

Right now I can try to check if maybe we can see if something still running on the port but I've looked at this a bunch and didn't find any good way to solve this.
Paper and Velocity servers do close properly but that's besides the point

[Yeshey]

Okay, apparently I can use something like this ps -o state= -p $(pgrep -f java) to check if the server is sleeping or not, I'll try to make use of that to implement graceful exits

[Infinidoge]

Mmm I wouldn't rely on ps and pgrep, especially based on name. ps based on PID would be fine.

[Yeshey]

Okay, after looking a lot more into this I think I can safely say that lazymc does take care of the shutdown process gracefully and we don't need to be messing with it

Apparantly, vanilla servers don't show anything in the log when shut down with a SIGTERM or SIGINT, but it saves everything and exits safely. It's only in windows where the stop command being issued either manually or through rcon is requiered. I thought flavours like papper were working because they don't disable the logger and display that they're exiting normally.

Furthermore I was spooked by exceptions like these that would be thrown if the server was on:

[12:04:04] [Server console handler/ERROR]: Exception handling console input java.io.IOException: Input/output error at java.base/java.io.FileInputStream.readBytes(Native Method) ~[?:?] at java.base/java.io.FileInputStream.read(FileInputStream.java:287) ~[?:?] at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:345) ~[?:?] at java.base/java.io.BufferedInputStream.implRead(BufferedInputStream.java:420) ~[?:?] at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:399) ~[?:?] at java.base/sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:350) ~[?:?] at java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:393) ~[?:?] at java.base/sun.nio.cs.StreamDecoder.lockedRead(StreamDecoder.java:217) ~[?:?] at java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:171) ~[?:?] at java.base/java.io.InputStreamReader.read(InputStreamReader.java:188) ~[?:?] at java.base/java.io.BufferedReader.fill(BufferedReader.java:160) ~[?:?] at java.base/java.io.BufferedReader.implReadLine(BufferedReader.java:370) ~[?:?] at java.base/java.io.BufferedReader.readLine(BufferedReader.java:347) ~[?:?] at java.base/java.io.BufferedReader.readLine(BufferedReader.java:436) ~[?:?] at aro$1.run(SourceFile:189) [server-1.21.10.jar:?]

And even more like that appear on servers like paper because they seem to have more safeguards, but that happens because we loose the connection to tmux and is harmless as far as I can see

I also noticed that only in unix lazymc suspends the process, and tmux is also suspended, but that's expected behavior, it keeps working as soon as someone enters.

So the logic for exiting when using tmux should be even simpler as lazymc takes care of it. I'll push an update when I can, but the functionality is the same as it's already implemented

[Yeshey]

I should have guessed that lazymc would have had this into consideration, but if this is the case, the sending stop commands etc. functionality of this module can also probably be simplified

[Yeshey]

simplified a bit, mostly left your stop code untouched, sending the two CTRL+C to tmux was redundant so I removed that. But I believe everything is working well now, the exception on lost input device might still happen on the server bc tmux dies first but that's harmless as far as I can see and doesn't affect the shutdown, and for us to prevent that it would be a lot of complicated bash logic to detect server states and PIDs and stuff to know when to close tmux, I was going down that path and trust me it's not pretty. I think how it is now is as good an implementation as we'll get, or at least as I can make

[Yeshey]

Cool thanks! I rebased, but instead of renaming the branch (that would close the PR and we would loose the discussion) I disabled github actions on my fork, hopefully that prevents the clutter, I'll go through the file comments soon as I can

[rharish101]

Is there anything else left to do? I'd like to use this without manually using @Yeshey's fork.

[Yeshey]

Is there anything else left to do? I'd like to use this without manually using @Yeshey's fork.

I'll try to explain the situation as it is right now
The way this nixOS module stops the Minecraft servers is by directly sending the letters "stop" (or any other keyword to stop the server according to the type of server) into the stdin of tmux so the server stops itself, and when the server stops, only then tmux will also exit and the systemd service will exit as well.
But this is not how lazymc works, lazymc sends a SIGINT (or SIGTERM?) signal to the server and hopes that the server stops correctly independently of the type of server it is. And it doesn't expose any API so we can't easily know if the server has already shut down or not, so what enabling lazymc now does is it kills tmux (witch might bring up non fatal exception on the minecraft server because the terminal suddenly dissapears) that in turn kills lazymc witch takes care of killing the server as well with signals.
This to say that lazymc handles server shutdowns fundamentally differently than this module, and accepting that would mean that all the logic for stopping servers by sending stop words is kinda irrelevant at least when lazymc is enabled.
We could theoretically check if the server process is suspended or running to see if it is sleeping or not initialized and work around this, but this is complex and I think this is a pretty bad idea as lazymc is already expecting to handle server shutdown with signals.

I see why there's friction, but if @Infinidoge sees a path for this to get merged I'd also love to be able to finally get this done

[Infinidoge]

Sorry for the lack of responses, currently midterm season at university so I haven't been able to be active on GitHub.
As such it will likely be a while longer before this will be merged.

[Yeshey]

Sorry for the lack of responses, currently midterm season at university so I haven't been able to be active on GitHub. As such it will likely be a while longer before this will be merged.

Don't worry I know the feeling, I'm looking into this now instead of sleeping as well

[Yeshey]

Rebased to add missing commits from upstream

[samiser]

are there any other blockers on this? i also would be interested in using this functionality. is the shutdown signals/stop words issue resolved?

[Yeshey]

are there any other blockers on this? i also would be interested in using this functionality. is the shutdown signals/stop words issue resolved?

I don't think the issue can be resolved cleanly, so weather this is merged is up to the repo maintainers, I'm still available to look into other potential blockers.
In the meantime I'm using this myself, so I'm rebasing my fork with upstream once per month, if you're okay with a potential one month delay on mc updates, you can use my fork for now.

[Infinidoge]

I think among the PRs we've gotten on this repository, this one has been the one I've been most conflicted about.

It's clearly a feature people want and care about, but it also feels weirdly outside of the feature set of the module. So we don't feel right closing it, nor merging it.

I think the best way would be for lazymc to be a separate module, as opposed to directly integrated into the module. That way lazymc isn't making changes within the module that obstruct other features.

Further, looking towards #150, it is likely that this will need to be reorganized down the line anyhow.

[samiser]

@Infinidoge just to make sure i understand, by a "separate module" do you mean a distinct NixOS module still within nix-minecraft (like how #150 is adding a separate home-manager module)? or something else?

[Infinidoge]

Yes. The user would have 2 separate imports.

[samiser]

i've been working on a fork that implements lazymc as a separate module, but in testing i've found an issue where lazymc fails to detect the server starting (see timvisee/lazymc#89). i've found that a workaround is to use an older version (0.2.10) but i'm confused how @Yeshey your fork works, did you test it using papermc? maybe that's my issue

in any case this is a bit awkward since i've had to provide an overlay for the older package. unsure if there's some other workaround for this that would avoid having to pin and overlay this version, i spent some time digging into it but didn't find anything.

regardless of that, is my implementation more in line with what you were thinking @Infinidoge? i added an example of usage in examples/ in my fork, but you can also see my real usage of it in my personal configs

side note: there's some shared code between my module and the base module that could be refactored out of both (mainly the hardening stuff and option helpers) but for now i wanted to leave the base module untouched.

[Yeshey]

@samiser hmm, I remember that Paper was actually one of the few types of servers that would emit server shutdown logs even when it was being shutdown with lazymc and signals. I don't remember if I tested with the latest version at the time. I've been using vanilla servers and the latest versions have been working fine for me, but I can check when I have some time if I can reproduce.
I couldn't be bothered to make lazymc overlays 😅 actually I added in the README an example of how a user can set a specific lazymc version themselves, because different lazymc versions work for different minecraft versions so it would probably be a bit of a chore to make an overlay for every lazymc version for all the times mojang changed the way their servers work

[samiser]

yea you're right, i figured since folks would likely be using servers other than vanilla it would be useful to add the override but it just clutters the module. i already had an option to override the package in the module (copying from you lol) and i think it's cleaner to have the user manage overriding the version if needed.

removed the overlay from my fork (it was just using the pre-existing overlay for server packages) and now it's just the module (and example + test + readme changes). i agree that #150 would interfere with the organisation of the modules, so i might wait for that to be merged before opening a pull request, but hopefully my module is in line with what @Infinidoge was thinking.

also fwiw i basically just used your code @Yeshey but refactored it out into its own module so thanks for the work in implementing this!