I make a lot of mistakes related to the error rail, especially when I'm not using Dark regularly

[pbiggar]

A user said:

I've used/tried dark for 4 different projects:

1. Web analytics backend for a personal website: at core two routes: POST accepting information for a single pageview and GET which computes summary statistics (how many visits in the last week, etc). This worked really well, was easy to set up, and hasn't encountered any issues that I know of in the several months since it's been running ( though I rarely look at it).
2. web service for running multi-armed bandit experiments - accepts experiment results and then tells you which treatment to deliver next. Tried to build this a few months ago but couldn't due to lack of a math library, haven't revisited since that was added
3. routing logic and storage for a new Twilio Autopilot flow. Dark was perfect for this - spinning up endpoints quickly and getting to see live values made debugging the twilio side much easier, and let me work around limitations on their side as well. We decided to move it off Dark and reimplement in AWS (Chalice->Lambda, DynamoDB), because some of the data collected could have been sensitive and we wanted to have better control over retention/access. (this was a small project, now dropped)
4. backend for a small webapp using elm on the frontend (working on this currently). This worked well to get started - again, spinning up new endpoints quickly and not having to worry about an additional system element for data storage was really nice at the beginning. However I'm encountering a lot of small frustrations that are making me consider moving this project off Dark as well, will go into more detail below.

Overall thoughts:
-Structured editing is cool, I like seeing holes and knowing what needs to be filled in. It took me a little while to get used to the command palette but it works pretty well once you know what you're doing. However, compared to text-based editing, it gives up a lot of functionality that I take for granted and find myself missing - search, find/replace, diffs, version control, copy/paste (which works ~okay in Dark but often doesn't do what I want or expect). I don't really know how to bring the best of both worlds into it.
-Live program traces are great, I've never had them in any tool before and it's so helpful. Definitely wish this were more ubiquitous!
-I make a lot of mistakes related to the error rail, especially when I'm not using Dark regularly. The mental model is not that "sticky" for me. I think you recently added something to the popup documentation that says whether a function is on the rail or not, and explains what's going on - that's helpful.
-I haven't used Cron or event handlers at all - right now doing everything synchronously.
You're building a great community on the Slack, getting such detailed and quick support from the team is amazing.
-Minor nit: key handling on datastores. I'm more used to a relational DB where the identifier is a column as well, gets returned in results and passed around, etc. I often end up adding an "id" column into my datastores and just duplicating the key there. Is there a better pattern?
-Authorization on HTTP handlers seems to require a lot of repetitive boilerplate.
-I don't remember how I came across the full list of functions but I find it helpful and refer to it often, and I don't think it's really linked in the documentation (or if it is I can never find it there).
-I find that working with functions quite frustrating - especially when programming in a functional style I like to break things into many small precise functions. Dark makes this difficult for a few reasons:
-functions can be impure (side effects) - would be helpful to be able to mark them as pure.
-functions don't have (or expose) return types, and the tooling I think just checks that values are the right type - would like more static type checking!
-the type system is really limited, record types in particular would be helpful. And you can do things that don't make sense in other languages - for example you could have record types generated automatically from datastores.
-the canvas design just doesn't really work for functions at present - I usually want to see several functions together, be able to see them next to the http handlers where they're called, etc.
-I find refactoring extremely tedious - changing a function signature, or splitting a function into two - the UI just doesn't really support this.
-Because working with functions is difficult, my HTTP handlers are often long messes where I leave lots of functions on the error rail because I just don't want the canvas box to get too big or indented - partially because it gets hard to deal with, but also partly because refactoring deeply nested code in Dark is really difficult/error-prone. This is not ideal.
-Being able to "checkpoint" versions would be great - in project (4) where i'm working on FE/BE in tandem, I often get to a working state, commit my FE changes, and want to freeze/checkpoint the BE as well - when I go on to the next feature there's no way to get back to a working state if my exploratory changes don't work or make a mess of things. I haven't tried the feature flags yet but that seems more suited to a working system where a small change is being explored - rather than one that is fundamentally going through lots of change.

Overall my sense of dark is:
-really great for tiny things or prototypes- eg projects (1), (3)
-if you had something stable and working, would be straightforward to maintain in dark, though adding new features might be tough
-Finding it becoming unworkable for even a relatively small simple project (4), and thinking about moving off it - the additional complexity of a few managed AWS services might just be worth it, because I'm now spending quite a bit of time fighting with Dark.