Let me start out by saying, if I could justify writing in assembly code, I would. I am no lover of node-based scripting systems, for the most part because I’m often doing extremely complex stuff that would never lend itself well to a node based system.
HOWEVER
There are some places that node based systems should be used a lot more then they currently are.
To provide some context, I’ve been working on this idea for the perfect VR game engine, and I keep getting stuck on the question of “what scripting language should I use” because there’s never one that fits everything. While the smart thing to do would have just have been to go with Luajit over a year ago, I did the very WireWhiz thing to do and made a series of toy languages trying to figure out if there was a way to get everything I wanted in one language, even if I needed to build itself. Through this process I learned a lot, about how these things are made. I got some very good results too, though nothing good enough, and nothing new enough to justify people learning a new language just for my engine/tech-stack.
That was… until today! Don’t bother reading the rest of the thread (unless you want too) because I’ll go over what I said there here in greater detail, but do take a moment to say hi to my “rubber ducky” friend I made while pacing in the garden. If you’re imagining me blabbering to a millipede while frantically pacing back and forth, you have the correct idea.
I just had what I can only describe as a vision. I knew that I wanted to create a new scripting system, and I knew what requirements I had, but I didn’t yet know the form those needed to take. But I just figured it out. pic.twitter.com/9exr6jo9dB
— WireWhiz (@TheWireWhiz) May 15, 2024
Basically, through coding async systems in over 6 languages/paradigms I’ve realized a key fact. Text kinda sucks the moment that you start dealing with two things happening at once. Mainly when you want two things to talk to each-other with time sensitive and order dependent operations. I’m not saying that there aren’t some extremely good abstractions and techniques that can make trying to make async and multithreaded systems easier to work with, I’m saying that no matter how well you understand it, it still sucks, you’re just better at using it so it affects you less.
Let me clarify: The abstractions and solutions are good, but only for a text based programming paradigm. When the only thing you have to connect two points in your code every time you want to set up a sync point is a name or a variable, that’s a lot of friction. And when you’re not actively managing concurrency, you’re just using a promises/futures system that’s pretending to be synchronous. They’re tacked onto a system that is really good at step-by-step, not step-and-step-and-step-andWe’reNotSureWhichOneExecutesAndWhen.
So, what’s the alternative? I personally really like text, but I see the appeal of a node based scripting system when instead of defining logic, it instead defines flow.
If you think about it, the word “programming” is not tied to text. The agenda for an event is also referred to as a program. At the end of the day a “program” is a series of events or actions, and when programming for a computer it’s our job to give it a series of actions to do, at least until computers start thinking on their own completely.
Therefore I propose we create a Programming paradigm where there are both text-based and node-based components of the same scripting system. While if architected correctly, someone could feasibly stick to only the GUI nodes system, or text behavior and config files, they would best be used side-by side. You use text to define independent processes, which are then strung together using a GUI to manage concurrency, sync points, atomic write points, and basically anything else that would be better to be done visually. The crazy thing is, this is almost already how I visually map out a program while creating it, as a series of separate independent processes with points that connect them to other processes, the only new thing here would be getting that out of our brains, and on to the screen.
I haven’t fully mapped it out, but the initial list of concepts I’m toying with here are:
- “Threads”
- (Blue transparent cylinders)
- These would be series of procedures that are always guarantied to happen in sequence
- Every time you need some new concurrency, you branch off into another “thread”
- These threads can merge back together causing a sync point, where execution of those procedures waits until both parent threads are complete
- Procedures (Might also be called operations, scripts, etc.)
- (Grey cylinders inside threads)
- Procedures are where you define the behavior that is to be executed in a thread
- They can recursively contain other procedures, possibly down to individual operations like addition or subtraction.
- They cannot store any state, they can only ingest parameters, and output results. They can interact with longer-lived data stores, though I haven’t decided if they get direct access or only through parameters/outputs.
- Data Stores / Events
- (Yellow cylinders)
- Data stores can store data throughout the lifetime of a program
- depending on the context of the program graph, access is automatically atomized when needed
- there could be smaller read-once versions of data stores like events, that store the equivalent of a struct, and when written to, kick off a series of other threads with that struct as the parameter. Basically a form of message passing.
- Data stores can map to Wasm multi-memory stores, allowing runtime loaded scripts to be more secure, while still being fast.
It’s still a little fuzzy, and it would definitely take a real use case to flesh out all the actual requirements, that’s why I’m going to get to cooking. I know what I need this to do for now to be able to support the game engine I’m building, so I have an initial set of requirements, next step is to get to some prototypes. GUI is definitely going to be 2D at first, but I can easily see going as far as a VR IDE where when you zoom in on certain portions enough a text editor opens up, allowing you to edit code in the context of where it sits in the entire system.
Anyway, let me know what you think! It’s definitely weird and out there, but with how much potential this potentially has, I’m going to at least try it out.
