Mention of performance in comparison with Dhall?

[arobertn]

Is your feature request related to a problem? Please describe.

Nickel is pitched as a configuration language and compares itself to Dhall in its main README and Rationale documents. The comparison focuses on the need for type annotations and the strength of typing in Dhall, however Dhall also suffers from performance problems impacting scaling to larger configurations when working with, for example, Kubernetes:

• Performance advice for dhall-kubernetes based package – dhall-haskell Issue #1890
• Production usage – dhall-kubernetes Issue #157
• Slow (or even hanging) dhall-kubernetes based generation – dhall-haskell Issue #1960
• Roadmap for improved Kubernetes support – Dhall discussion
• Using env in import causing a huge performance hit – dhall-haskell Issue #1878
• "dhall resolve" does too much – dhall-haskell Issue #2116
• RFC: Integrity check modes · dhall-lang Issue #1185

These problems accrue because the Dhall interpreter resolves every definition down to base primitives before executing. This causes exponential resource growth interpreting code where progressively functionality is built hierarchically on progressively complex / specialized utility functions, because definitions can be repeated at low levels and are then repeated multiplicatively at higher levels if functions that call them are themselves called multiple times. Similar expansion also occurs over all branches of if-else statements, compounding the issue.

In a private project I work on, we will need to either abandon Dhall or rearchitect substantially due to the high memory and compile-time requirements the above brings. It would be interesting to know whether Nickel will improve this area or not.

Describe the solution you'd like

If scalability is something that will be improved in Nickel, then the comparison to Dhall in the RATIONALE (and maybe the main README) should mention this point.

Describe alternatives you've considered

Remaining silent on the issue is another alternative. If Nickel does not, in fact, scale, projects may discover this the hard way after having committed to the technology.

Additional context

–

[yannham]

I think Nickel has the potential to scale well, and scalability is in some form part of the design goals, e.g. we've been taking parallelizability into account when thinking about language features, about incremental evaluation, and so on. What's more, there is no reason we would do like Dhall does, especially now in hindsight.

That being said, the efforts have been focused mainly on core language design and implementation until now. Once #426 lands (and maybe another PR on closurization), we should hopefully not do anything obviously overly naive and stupid anymore with respect to performances. But we can't say much effort have been put into optimizing the interpreter further right now. There's still a lot of room for improvement.

So, the answer is something like "yes, most probably, eventually" situation, but it may be a bit hard to advertise and substantiate right now.

[arobertn]

👍 It sounds good that performance is in mind and, though optimization is not a focus yet, the design won't be rendering it impossible. Maybe this issue can be left as a placeholder / reminder.