Add support for C# compiled by .NET SDK

[gollux]

I tried adding support for C# compiled by .NET SDK 7.0. I have a very preliminary version running in the Czech national olympiad, but before I submit a pull request, I would like to discuss some problems I stumbled upon and agree on how they should be solved properly.

(1) Permission problems. The dotnet compiler creates a whole forest of intermediate directories, often with restrictive permissions. This hits the issue #1242. I worked around this by creating a temporary directory within /box and symlinking it as obj in the current directory.

(2) User lookup problem. The compiler expects that the current UID has an entry in /etc/passwd. With a typical installation of Isolate (as installed by CMS installation scripts), the sandbox users don't have their entries. As CMS bind-mounts /etc in compilation sandboxes, this can be worked around by adding all sandbox users to /etc/passwd of the host system. I wonder if this has a nicer solution...

(3) To use dotnet build, one must create a *.cproj project file first. I figured out how to do it, but not how to integrate it nicely with the CMS language modules: they allow generating compilation commands, but I would rather need to call a Python function which would generate the file and put it in the sandbox directory. I worked around it by calling a helper script, but then we have to install the helper script to a location which is available inside the sandbox (e.g., /usr/local/bin), which is different from where we install the CMS itself (in my case, it's a virtual environment).

(4) I need to modify some parameters of the sandbox: increase the maximum number of open files, pass some extra environment variables, possibly bind-mount more directories. I did it by adding these options to all languages, but it would be really nice to have them per-language. I handled environment variables in the helper script for the time being.

I would appreciate any ideas on how to solve these problems in an upstreamable way.

[andreyv]

(2) User lookup problem. The compiler expects that the current UID has an entry in /etc/passwd. [...] I wonder if this has a nicer solution

Does the compiler really read /etc/passwd directly, or does it use getpwuid()?

In the latter case it should be possible to write a glibc NSS plugin for isolate that would resolve isolate uids dynamically. systemd's DynamicUser= feature is resolved this way via nss-systemd(8).

Alternatively, it may be possible to define the user records in /etc/userdb or /run/userdb. Then they would get resolved through systemd-userdbd(8). But the sandboxed program needs to have the ability to call host D-Bus for this, which likely has security implications.

EDIT: Another possibility is to bind-mount a temporary /etc/passwd and /etc/group in the sandbox.

[gollux]

Does the compiler really read /etc/passwd directly, or does it use getpwuid()?

I don't know yet. I suspect it does the latter, but I didn't want to dig deeper before I find any solution where using getpwuid() would actually help :)

Adding a NSS plugin would work, but I don't think it's easier than just adding all the sandbox UIDs to /etc/passwd :-)

Running D-Bus inside the sandbox is even more complicated. (Also, I'm not sure that Isolate should depend on systemd being available.)

Bind-mounting temporary /etc/passwd could solve the problem, but so far, Isolate didn't assume anything about /etc being always mounted.

Generally, adding the entries to host /etc/passwd looks as the easiest solution.

I am more worried with the other problems.

[prandla]

the permission problems are solvable i think; my fix for #1242 seems to work well enough.

Regarding users: maybe an user namespace could help? This way, the inside of the sandbox could always (pretend to) run as a specific user ID. But that user ID presumably still needs to be defined somewhere then, so it's not a full solution, and it might add too much complexity to isolate. (Actually, this user ID could then be just the ID of cmsuser, which should already be defined.)

regarding the csproj, I think this has some UI design questions: the csproj file needs to be specified somewhere. If it's stuffed into the commandline arguments to dotnet (echo -e "<XmlBlahBlahBlah/>" >project.csproj), then the "compilation commands" view in CWS gets a bit ugly. But if it's in a separate file somewhere, then contestants don't have direct access to view it (which can be annoying for debugging when a submission doesn't run as expected on the server). I guess implementation-wise we could add the generated csproj file as an additional source file, and in CWS, we can add a link somewhere near the compilation commands that shows the generated source file.

the extra parameters to the sandbox could maybe be extra return values of get_compilation_commands, but then we'd need to invent a better return type for that function.

[prandla]

I implemented this here: prandla@87372f2 (I based it on #1530 and #1545), using https://gitlab.kam.mff.cuni.cz/mo-p/cms/-/commit/2f976f19a2734a6cb84f6cc6200e0c2a3ed8b0ca and https://gitlab.kam.mff.cuni.cz/mo-p/cms/-/commit/2f1eda2cac19b398a824271339d7e566e4dfbfa8 as inspiration. Some notes:

• The user lookup problem seems to be resolved in a newer version of dotnet: it seems to work fine when the user doesn't have a name. However, it still requires /etc/passwd to exist, otherwise it crashes instead of returning an empty name. Ideally we'd put a dummy /etc/passwd in the sandbox, for now I just bind-mounted /etc from the host.

• dotnet really loves the internet. In order to get it to work at all, I had to create a config file for nuget to get it to not fetch some files from its package index. Even so, this seems to only work for dotnet 8.0: for 7.0, it then errors about some missing references, even though from what I can tell said references are present locally. I'm a bit worried because I'm not confident that this is something that was actually fixed in dotnet 8.0, as opposed to just accidentally working on 8.0 and might break again on 9.0.

  • There is a "more proper" way to do this, which is to first do a dotnet restore outside the sandbox (can be done once when setting up the server), and then make the resulting nuget cache available inside the box. But this requires a lot more annoying setup on the server admin's side.

• the contestant UI looks like this (well, not quite, I needed to hack the css a little):

  

  I think this is alright because it makes it very obvious what configuration the dotnet project is built with, which might help debug why some solution might be behaving differently on the server.