current brainstorming status

[mflatt]

Welcome again to the Rhombus-brainstorming discussions area!

First, an easy answer to a rhetorical question: Why didn't we enable discussions in the first place? It's because GitHub's discussions feature didn't exist when we created the repo. So, we started with posts on the Racket mailing list and individual issues. At this point, though, the discussions feature seems like the right mechanism for general Rhombus discussion.

We're still in the brainstorming phase of the original plan. Yes, that plan says “months”, which is still technically correct. :) For various reasons — obviously including the fact that Racket CS stabilized — things have picked up a lot here in the last few weeks.

Here's how brainstorming has played out so far:

• There are lots of issues, where each issue is a brainstorming idea with some associated discussion. The list of issues has evolved to be mostly a wish list, which is fine. For my own part, I've found the wish list useful to nudge some aspect of a proposal in one direction or another. Some issues serve more as discussion prompts, and likely this discussion area will be better for more of those.

• There are a small number of pull requests (PRs). The PRs are not just wishes, because they should come with specific implementation work behind them. But they're also not conclusions. Even the most worked-out PR is not “Rhombus”, but merely one candidate/prototype/experiment toward Rhombus.

For what it's worth, I expected to see more PRs by now on surface syntax. If anyone is interested enough in the issue to do something, now is probably the time.

Here's a guide to the current implementation PRs:

PRs #114, #119, and #122 are surface-syntax proposals. The last one has received the most effort, in part because it was based on the other two.

PR #162 builds on #122, and it mostly ports Honu to the surface syntax of #122. The PR describes neither surface syntax nor an actual language; as intermediate infrastructure, it's probably difficult to appreciate unless you've already spent time combining macros with infix syntax.

PR #163 builds on #162 and #122. It's the most concrete in the sense that it assembles pieces into a candidate #lang rhombus that you can install and run. At the same time, the PR evolved to be two separable proposals that are currently bundled into one PR: (1) a set of rules for implementing and using static information, which is closely related to types and contracts, and (2) specific choices for some definition forms, function forms, etc. The first part could be shifted to a surface syntax other than #122, or it could be dropped completely in favor of different rules or not trying to handle static information.

At this point, I think we need a couple of things to proceed:

• a sense of whether there will be more surface-syntax proposals, or whether we should keep investing in the direction of Shrubbery notation #122, and

• a sense of whether the static-information part of initial candidate experimental prototype Rhombus #163 is on a good track or should be reconsidered.

Those bullets are not meant to constrain discussion or further brainstorming, but given the balance of brainstorming discussion versus specific experiments so far, those two specific discussion areas may be the most productive to move things along. Of course, you should also feel free to continue commenting on details of specific proposals (where the PR-specific discussions may still be the best place for that), creating new PRs, or creating new issues.

Another good step will working out a set of datatypes to include in a base language. I know that a new proposal (certainly better than the sketchy List, Array, and Map of #163) is in the works, so watch for that.

[samth]

I made three more-specific comments here and here and here, which helped me think through some broader issues. So this is a very-revised comment.

I think the biggest question is whether the general direction in #163 is the best way to keep making progress right now. I think the answer is obviously yes, both because of its own merits and because we don't have any alternative proposals.

Surface syntax

The surface syntax seems like it's at least on its way to a local optimum, where we combine both the generality of s-expressions in that shrubberies denote a tree structure rather than a program, supporting language extensibility in a similar way to Racket, with the ability to write more traditional syntax. This works quite well already -- it's easy to define new operators, simple programs look "normal", macros are extremely flexible.

My sense is that building on shrubbery notation is the right way to go if we want to aim for that kind of syntax, and that even the two issues I raised, which @mflatt seemed to worry suggested moving to a different syntax, can be addressed in that context, perhaps by adding extra special cases or making the macro system even more flexible.

The alternative approach, I think, would be to go for less flexibility and thus limiting the ability to customize things in favor of picking particular syntaxes for particular parts of the language, and ending up in a place more like Elixir or Rust or Scala, with real macros but without the flexibility to build arbitrary new language forms do everything the core language can do. I still think the Racket community could do a better and more flexible job in that design space, but it would be limiting some things in favor of getting to make particular syntactic choices.

I don't think we want to go in the second direction (and I'm not planning to implement a full proposal in that direction even if I did), so I support sticking with shrubbery notation.

With that said, I think the best way to decide if it's the right direction is to pick some things that we want to be able to do with the syntax and see how they work. So far, the two things I tried (pipeline operators that span lines, annotation/decorator syntax) don't work quite yet but seem doable within the shrubbery approach.

The rest of the language design

The opening post doesn't ask for comments specifically on this, but I think it's worth mentioning. Overall, I think it works together very nicely. val, def, and let work nicely together. Having . notation is fantastic -- the single thing I'm most excited about for Rhombus. Pattern matching everywhere is great, as one might expect me to think. I have various thoughts about data structures, which I'll save for when that proposal comes out, but in general it definitely seems to be the right direction.

Static information

The proposal emphasizes the efficiency benefits of static information for .. This suggests that if dynamic lookup of fields in structs was as efficient as using posn-x we wouldn't need static information, I expect that @mflatt has some other uses in mind, but they aren't really described yet.

Right now, the rhombus prototype is about 10x faster accessing p.x when p has static information, which seems to definitely justify the static info. Probably compiler optimization could reduce that a lot, but even a 5x improvement would still not be competitive with current idiomatic Racket code. Which suggests that we continue in the static information direction, neither giving up . nor taking a major step backwards in performance seems like a good idea.

With that said, I worry that the pervasiveness of static information makes it hard to re-use core Racket libraries in Rhombus. For example, right now the Rhombus prototype has its own small implementation of pattern matching, which cooperates with the static information system. That implementation could be expanded to be comparable to racket/match, but that seems like a major additional workload to add to the burden of building Rhombus, especially if we imagine similar efforts for racket/class and other core libraries.

Of course, there's also the question about how the static information system would work with a hypothetical Typed Rhombus, but I think that's a further-out concern that should be deferred at the moment.

[rooney]

Can we also re-allow "?" in variable names? Ruby (a "normie" language, which is what the goal of Rhombus seems to be) has it and I don't see their newcomers find it confusing/hard to learn. In fact, some find it "beautiful". Remember that Naming Things is one of the only two hard problems in CS, so I'd say allowing names to be more expressive would be a great help!

[mflatt]

On match: The approach in #163 is that racket/match functionality is mostly subsumed by binding forms, and we would be creating a new set of binding forms anyway — so, a new implementation of most of match. The match form provides optimizations that the match in #163 doesn't, and probably #163 can't optimize as much due to the way that binding forms in different clauses expand independently. Maybe some of that optimization could be recovered by lower-level optimizations (e.g., recognizing tests that look for a common prefix). You may have in mind other aspects of racket/match that I haven't considered, though.

On reusing Racket libraries: I don't think it's so common that a library provides a structure type that is meant to be used in a match style. For example, although pict and scribble do expose structure types, instances of the structure types are mostly created by abstractions, and I think it would work just as well to expose the construction, predicate, access operations as non-structure functions. If it's useful, there could be a binding operator in Rhombus that uses Racket-shaped compile-time structure-type information.

On reimplementing parts of Racket: For the most part, I lean toward creating new Rhombus versions of things and creating bridges as needed. For example, racket/class seems like a good candidate for starting again with some different goals and constraints, instead of using racket/class wholesale. Ditto for racket/contract. Still, there are many things that I'd advocate sharing: numbers, lists, vectors/arrays, syntax objects, functions (including keyword and optional arguments), and the run-time component structure types (but not the compile-time component). I think syntax/parse can be reused, and my experience using it to implement the predefined forms in #163 has been good; it may need more hooks for error reporting, or maybe I just need to learn more about what's already there. I don't see static information as being a big obstacle or question on this front, but I see that it may depend on how transparently Racket and Rhombus to work together.

[michaelballantyne]

I have a surface syntax idea I’ve been kicking around that’s a simplification of shrubberies. I can’t promise an implementation, but I’ll open an issue with a description later this week to see if anyone finds it appealing.

[Lazerbeak12345]

I'm interested, if only to get more ideas out there. I still like the look of sweet expressions a bit better than shrubberies, as it's simple. Perhaps something in-between them would be nice to play around with a bit. 😉

[mflatt]

Thanks @michaelballantyne for that issue! In case anyone missed it, it's #172. I found the ideas and discussion useful for improving #122 and #163.

[mflatt]

Just a follow-up on the datatypes question, so that everyone is aware: @jackfirth is actively working on a datatypes proposal.

With that piece, maybe it will be time to move out of the brainstorming phase and into the iterative-design phase.

[zamora]

Does shrubbery notation support an array notation for referencing vectors and other array-like structures? For example, replacing (string-ref name 0) with name[0], etc. would be of great value (same for vector-ref, list-ref, or even hash-ref). It looks like brackets are using for grouping and it may not be possible/practical to overload them this way. Is there a proposed improved syntax for arrays?

[jackfirth]

@zamora Yes, see the section on Lists in Matthew's Rhombus prototype.

[AlexKnauth]

I have another surface syntax idea that will be a hybrid between Shrubbery notation (#122) and Wraith (https://github.com/cwebber/racket2-rfcs/tree/wraith/wraith).