Although I like the conceptual simplicity of the current construct-a-tree/render-the-tree loop in Flitter, it obviously has performance limitations. In particular, the compiler is generally able to simplify large amounts of a program to literals, but there's no action we can take to turn those literals into render information until the render-the-tree phase.
Consider the simple program:
!window size=1920;1080
!canvas3d
!light color=1 direction=0;0;-1
!transform rotate=time/30
for i in ..10
!sphere size=20 position=(beta(:pos, i)[..3] - 0.5) * 1000
Almost everything about what needs to be rendered here is known at compile time except for the rotation matrix. In fact, the simplifier will create a literal !window size=1920;1080 node, a literal !canvas3d << !light color=1 direction=0;0;-1, and a literal !transform, with 10 !sphere size=20 position=... nodes already appended. All that's left to do at runtime in the VM is the time/30 calculation, a set-attribute on the !transform node and then append that to the !canvas3d and append that to the !window.
If I could replace the generic nodes with specialised versions, I could then speed up the render phase. For instance, in this example I could create/lookup the sphere mesh model, pre-calculate the position/size transform matrices, create the light parameters, initialise the render group data structures for the 3D canvas and maybe some window context information. Lots of stuff that is currently cached and looked-up on the fly at render time could instead be directly referenced from the nodes.
The issue I have here is that the evaluation model works bottom up and has no understanding of what the different node kinds or attributes mean. We could have some kind of central register of node kinds and the renderers that they belong to, but some node kinds repeat in multiple places with different meanings – e.g., !transform could be a window shader node, a 2D canvas transform, a 3D canvas model/light transform or a CSG internal transform.
To make this work without breaking the core paradigm, I'd need to push contextual information top-down, i.e., I'd need to work from knowing that the !window node is going to be at the top-level, the !canvas3d node will end up being appended to it, etc., in order to correctly determine that I should be making specialised window and 3D canvas nodes.
What if Node was a protocol instead of a data structure and node creation was done via a contextual factory? So within the top-level sequence, we have some root factory that knows the renderers. When we hit the !window node, we call .createNode("window") on that factory. When we start evaluating the append sequence within !window, we make that specialised node the current factory. So then when we hit the !canvas3d we call .createNode("canvas3d") on the window node. This allows us to make a specialised node that understands it is going to be rendered as a 3D canvas.
Similarly, the !light node will be created by the !canvas3d node. A specialised lighting node can interpret its attributes according to their purpose and build the actual lighting data structure. When we subsequently append that to the !canvas3d node we can consider this node to be "locked-in" and immediately incorporate that lighting information into the render group, throwing the specialised light node away. Crucially, these steps can be done at compile time.
At run time, when the !transform node's rotate attribute it set, it can immediately compute the transform matrix and then, when this is appended to the !canvas3d, we can multiply this into all of the pre-computed model matrices and build the render dispatch array. The final step would just be to iterate over all of the top-level specialised render nodes calling their .render() method.
In cases where we can't determine a node creation factory – e.g., inside a dynamic function – we could fall back on just creating the normal generic nodes. Whenever a generic node is appended to a specialised one, we can walk the generic nodes specialising them using the parent as the factory. This is, to a degree, what happens inside each renderer anyway.
I think the language logic would just be that the LHS of an append becomes the context for the RHS. That context pushes through if expressions and for loops. The generic node factory is used inside let bindings, function bodies, call arguments, other unary and binary expressions. The simplifier could subsequently specialise literal generic nodes if they end up on the RHS of an append with a specialised LHS – for instance, if a function body is inlined.
It's going to be important that specialised nodes are cheaply-clonable, as literal nodes are always cloned into the render tree and this may happen multiple times over, for instance if we have a literal node inside a dynamic loop. It will also be important that setting further attributes on a specialised node does The Right Thing.
Although I like the conceptual simplicity of the current construct-a-tree/render-the-tree loop in Flitter, it obviously has performance limitations. In particular, the compiler is generally able to simplify large amounts of a program to literals, but there's no action we can take to turn those literals into render information until the render-the-tree phase.
Consider the simple program:
Almost everything about what needs to be rendered here is known at compile time except for the rotation matrix. In fact, the simplifier will create a literal
!window size=1920;1080node, a literal!canvas3d << !light color=1 direction=0;0;-1, and a literal!transform, with 10!sphere size=20 position=...nodes already appended. All that's left to do at runtime in the VM is thetime/30calculation, a set-attribute on the!transformnode and then append that to the!canvas3dand append that to the!window.If I could replace the generic nodes with specialised versions, I could then speed up the render phase. For instance, in this example I could create/lookup the sphere mesh model, pre-calculate the position/size transform matrices, create the light parameters, initialise the render group data structures for the 3D canvas and maybe some window context information. Lots of stuff that is currently cached and looked-up on the fly at render time could instead be directly referenced from the nodes.
The issue I have here is that the evaluation model works bottom up and has no understanding of what the different node kinds or attributes mean. We could have some kind of central register of node kinds and the renderers that they belong to, but some node kinds repeat in multiple places with different meanings – e.g.,
!transformcould be a window shader node, a 2D canvas transform, a 3D canvas model/light transform or a CSG internal transform.To make this work without breaking the core paradigm, I'd need to push contextual information top-down, i.e., I'd need to work from knowing that the
!windownode is going to be at the top-level, the!canvas3dnode will end up being appended to it, etc., in order to correctly determine that I should be making specialised window and 3D canvas nodes.What if
Nodewas a protocol instead of a data structure and node creation was done via a contextual factory? So within the top-level sequence, we have some root factory that knows the renderers. When we hit the!windownode, we call.createNode("window")on that factory. When we start evaluating the append sequence within!window, we make that specialised node the current factory. So then when we hit the!canvas3dwe call.createNode("canvas3d")on the window node. This allows us to make a specialised node that understands it is going to be rendered as a 3D canvas.Similarly, the
!lightnode will be created by the!canvas3dnode. A specialised lighting node can interpret its attributes according to their purpose and build the actual lighting data structure. When we subsequently append that to the!canvas3dnode we can consider this node to be "locked-in" and immediately incorporate that lighting information into the render group, throwing the specialised light node away. Crucially, these steps can be done at compile time.At run time, when the
!transformnode'srotateattribute it set, it can immediately compute the transform matrix and then, when this is appended to the!canvas3d, we can multiply this into all of the pre-computed model matrices and build the render dispatch array. The final step would just be to iterate over all of the top-level specialised render nodes calling their.render()method.In cases where we can't determine a node creation factory – e.g., inside a dynamic function – we could fall back on just creating the normal generic nodes. Whenever a generic node is appended to a specialised one, we can walk the generic nodes specialising them using the parent as the factory. This is, to a degree, what happens inside each renderer anyway.
I think the language logic would just be that the LHS of an append becomes the context for the
RHS. That context pushes throughifexpressions andforloops. The generic node factory is used inside let bindings, function bodies, call arguments, other unary and binary expressions. The simplifier could subsequently specialise literal generic nodes if they end up on the RHS of an append with a specialised LHS – for instance, if a function body is inlined.It's going to be important that specialised nodes are cheaply-clonable, as literal nodes are always cloned into the render tree and this may happen multiple times over, for instance if we have a literal node inside a dynamic loop. It will also be important that setting further attributes on a specialised node does The Right Thing.