Packaging question/request for feedback

[capr]

Hi Justin,

Today I wanted to add ljsyscall to luapower (luapower is a module distro for luajit) and I stumbled upon a problem which I want to discuss with you if you can spare some time.

So I realized that luapower makes a big assumption about the modules in a package. Basically, it assumes that all the Lua files in package.path are modules that a user can require() anytime, in other words, there's no notion of private module vs user-loadable module. I had no problem with this assumption till now. But with ljsyscall, the only user-loadable module is syscall, all other being private submodules, not intended to be required by the user directly, which is perfectly reasonable, though I think it can be useful to be able to require() those modules individually too, so that you can track and list their dependencies for instance (this is how luapower tracks dependencies for the entire distro so that you don't have to worry about manually declaring them).

Eg. if you take a look at [1], you'll see that it can show dependencies and supported platforms for each module of the package and it does that automatically, while for ljsyscall it currently shows this[2], because 1) it doesn't load the modules in any defined order and 2) loading a module for the wrong platform doesn't error out with a clean error like "invalid platform" or whatever and so the errors are reported as being genuine problems.

So given how ljsyscall is constructed, does it make sense to make it friendly to luapower? i.e. 1) error out with "platform is not XXX" if the user tries to load a module for a diff. platform, and 2) have each submodule require all its dependencies so that it doesn't break when required by the user directly.

[1] https://luapower.com/nw/info
[1] https://luapower.com/syscall/info

[justincormack]

Ok well thats several things:

1. each file requiring its own dependencies, rather than doing it in reverse like some of them do eg syscall/openbsd/ffi ends with require "syscall.ffifunctions", require "syscall.bsd.ffi" - this makes sense, should be possible to do sanely with some thought.
2. Stopping you loading modules for wrong platform, at least ones that set global state (ie use ffi.cdef). Already have platform detection code, just need to enforce in quite a few places. Some issues with rump, but can detect if you have rump installed, not sure how this will work.

It is mainly around the ffi parts, as the non ffi parts just return tables and mostly work fine, will take a look.

(I guess in the short term if you tried the modules top down you would at least get to load syscall).

[capr]

Hi, thanks for responding.

It is mainly around the ffi parts, as the non ffi parts just return tables and mostly work fine, will take a look.

Thanks. If you do that, please make it so that the error contains something like "platform not XXX" or "invalid platform" or something "standard" like that. It's what allows me to know the platforms of each module without having to parse rockspecs and thus require the module author to _make_ rockspecs (I know that you're already making those, just saying).

(I guess in the short term if you tried the modules top down you would at least get to load syscall).

Ok, fewer errors now (but a workaround as you said - you can say that it's hiding dependency bugs, depending on how you look at it).

[justincormack]

Ok, first stage is done - modules should all include their dependencies, and so they should all be able to individually be included (from any OS) without errors.

Multiple includes are currently still going to fail if they load ffi stuff, as those will have multiple definitions.