Just thinking out loud (let me know if you have any thoughts about anything):
The command queue could be on the UI thread. Some commands might be synchronous (f.e. handling of a DOM element) and some might be asynchronous (f.e. sending a request to a worker to calculate world transformation matrices or to a worker to calculate physics), and all can be queued using something like async.queue
.
Hmm, but if many of the functions are asynchronous, then they can run in parallel. Perhaps the fastest updating things will simply callback sooner, and those things render sooner (f.e. at 60fps while some other things are slower). There could be a result listener that simply applies whatever results were sent back within the frame. SO it's possible that some slow animations will be slower, and some will be faster depending on when results come back from workers. If such a design is done properly, rendering could always be at 60fps, and only drawing speed appear to be slow depending on the specific animation.
Can each node have it's own worker? Perhaps they're a bunch of workers all connect together in the same structure as the scene graph? Would that make sense for a component's onUpdate to run in it's Node's worker? If it needs state from another node for it's calculation (f.e. the size or rotation of another node) it can request it asynchronously and eventually get the result back in order to completely it's calculation. I'm imagining something this might work really nicely when MessageChannels aren't coupled to the UI-thread like they are now (as described in that Chromium bug you linked us to above @oldschooljarvis).
I think perhaps each Node worker in this design I'm imagining can obviously hold things like it's local state and world state (local transforms, local opacity, etc, and world/final transforms, final opacity, etc).
I'm imagining that in order to calculate the world transform of some Node in a tree of linked Workers, that the mechanism that will do these calculations (be it in some separate worker for this purpose, or perhaps in a sidekick worker of the Node who's world-transform we're calculating) will asynchronously query all Nodes in the path leading to the Node in question to get the local transforms of each node. The results might arrive in any order, but when they all finally arrive, the calculation can be completed, and the result world transform saved back onto the Node in question.
The WebGL renderer, f.e., would need all the world transforms of each Node in order to render them in the 3D world, so that would happen separately in a similar fashion: querying all nodes for world transforms, and applying only transforms that have changed. There'd be some way to query for transforms of things we expect have been modified.
Maybe a Node can be marked "dirty" on the UI side whenever a user has called a function to modify some of that Node's state, that way the renderer can query that worker once per frame until the Node says it's clean? I'm not entirely clear on this part yet, I have fog here.
But what I am starting to see clearly is that if we separate everything into workers in small pieces (f.e. one Worker per Node) then it seems clearer how things can perform well without blocking the UI thread.
At first I was imagining having a single "SceneWorker" that contains all of it's worker-side nodes in (worker-side because there is also some UI-side "Node" that the end user always has a reference to, and which is the user's interface to that "Node") and would run all updates in it's own self.
But now that I'm thinking about it, I'm liking the idea of splitting things into one worker per Node, which I think might actually make the design easier to implement because then the UI-side "Node" that a user interacts with can have methods that simply call methods on the worker-side Node in an RPC-like fashion.
And now, when I think about something like a "Transition" that performs calculations on any number of Nodes at once, I can imaging that the Transition thing can be it's own worker that queries Nodes for info needed then calculates results to give back to Nodes, which Nodes can then use to update their own local and world transforms.
So here's an example of possible UI-side end-user API:
let node = new Node // `node` is interface for the end user in the UI-thread, but creates a worker behind the scenes?
let transition = new Transition(0) // Transition might make a new worker behind the scenes.
function someAsyncFunction(callback) { // some async function who's result we'll need in a calculation.
// ...
callback(result)
}
transition.to(2*Math.PI, {
duration: 5000,
curve: Curve.expoInOut,
deps: {
otherPosX: otherNode.position, // transition sees this is a NodeComponent, so gets that component for the following calculator function.
someNumber: someAsyncFunction // transition automatically knows to call someAsyncFunction on the UI-side each tick?
}, // list dependencies needed by the Transform, so that it knows what to query on the worker side?
calculator: function(currentValue, deps) { // calculator runs on the worker-side. ?
return { // return a result object, the results of some (possibly intense) calculations.
rotation: deps.otherPosX * deps.someNumber + currentValue + Math.random()*Math.PI/8
}
},
applier: function(results) { // used to apply results. Receives the result object of the calculator on the UI-side.
node.rotation.y = results.rotation // this tells the UI to send the update to the node's worker after caching it on the UI-side for DOM rendering? We know that at each frame we need to apply cached values, whatever may be.
}
}).loop()
What I was thinking there is that there's some way to tell the API what it needs in order to perform some calculations worker-side (in this case, guided by a transforming number) and how to apply the result to other parts of the UI-side API.
But, maybe having that transition
in it's own worker is overkill? I initially thought that by making a new transform with new Transform(node)
that it might be a component of the Node that it received in it's constructor, and operate on that Node, but then that seems to make it complicated to do calculations involving state from multiple nodes, so I came up with that deps
idea just now. But the problem with the applier
function is that the result has come back to the UI-thread, and if it needs to be applied to another node, then the result is being sent back into a worker. It'd be nice for the result to go straight to where it needs to go without having to route through the UI (save for messaging limitations, but at least not via end-user API calls). Any ideas?
The overall main point that I'm seeing is that if everything is async on a worker basis, then nothing can block the UI except if the scene graph is just so big that it takes longer than a frame to apply results (i.e. pass transforms to renderers). But calculation wise, nothing will happen on the UI-side (if our API is used properly).
I'm imagining that subtrees in the scene graph could take longer than a frame to update if they involve large physics calculations for example, while other portions of the scene graph tree might update quickly, which could produce the effect of some things having sub-60fps lag, and other things moving smoothly. That'd be interesting to actually see. I also fear some behavior like that would be funky in, for example, a first person shooter case.
So, that's what I'm gonna do. I'm gonna try to make absolutely everything asynchronous, and the easy way to make mathematical things asynchronous is obviously doing them in workers. 