Wow, I did not expect my latest video to do as well as it did, I’ll be honest I didn’t get anywhere close to the amount of work I wanted to do done last week because of how in-shock I was.

But I can stare at YouTube stats forever, what happens next?

First I want to make sure we’re all clear on where we are, because the video, which was intended as an introduction to the goals of the Brane Project, doesn’t really talk about where we are right now and only very briefly talks about my roadmap for the project over the next year.

The current state of Brane Engine

Brane engine has been my main side project for 3 years, and while I’ve had the same goals for it all that time, I’ve greatly progressed as a programmer and architect since first starting. A lot of the project is still pretty messy, and we’re going to need to refactor almost all of it down to the build system to bring it up to what I would consider acceptable for a widely known and used open source project.

Editor

Currently we have a Unity style GUI, built with ImGui, Vulkan, and a from-scratch ECS system that I wrote to work with BraneScript 1.0. With the current editor you can import (some) gltf files with varying levels of success, I wrote the gltf reader myself a couple years back so it has a lot of unhandled cases that cause crashes. Once it detects a gltf in the project files it converts it into an “assembly”, this is Brane Engine’s equivalent of Unity Prefabs. You can edit the components on each object, add lights, reparent stuff, etc.

The engine doesn’t have a default material yet, so you’ll have to manually create a vertex & fragment shader, create a material that uses those, and then assign that material in the properties of the assembly. I’m pretty proud of materials so far though, the editor auto detects uniforms and input variables and exposes them in the editor, so passing textures to shaders is very doable.

After you’ve done whatever simple edits are possible you can use the Sync Window to upload any updated assets in the project to an AssetServer in a custom binary format. (The asset server ip and port are set in the editor’s config.json, not in the UI for some reason…)

The asset server is the only server binary I’ve implemented so far, but it has two jobs right now. It has a local sqlite database it uses to store both what assets it’s serving, and user logins. So you do have to log in from the editor when uploading assets, (you might have to manually set up the sqlite schema for this, I can’t fully remember).

Then after you have an asset on the server, you can test streaming it to the test client. Booting up the test client logs into the server set in it’s config.json with the credentials provided also in config.json (very secure yes). After which it loads the “default chunk” of the asset server. If all works correctly, you’ll see the vertices of the meshes in the chunk stream in very sifi like. Buuuut that’s about all the client can do right now.

So in short, we have a decent foundation for an editor, a bare-bones asset server, and the idea of a client. Yes, this VR engine doesn’t even have VR yet. I made a lot of progress in the first year of the project, however when I stared working on the scripting system I spent a good bit of time with a couple prototypes and different approaches, got pretty far on my last attempt before figuring out my type system wasn’t great, and then half way through that refactor I took a management position at the last startup I was at and basically lost all time to work on this around a year and a half ago.

But very recently I decided this is what I really want to be doing and picked it back up. Started a new attempt a scripting language, this time with a much more solid idea of how to build it right, and made the “My plan to save VR” video. So you can understand when I was moderately freaked out by how well it was doing, I was hoping to get everything in a little better of a place before everyone was looking at my 2 year old code lol. But realistically all that means is now I still have to fix it, but with an audience.

Next Steps

Largest things that need to be done right now are:

• Audit the current state of the repos
  As I’ve been working on mainly BraneScript, up to this point I’ve lost track of the state of the BraneEngine repo, though with all the new people trying to build it I’ve pushed changes so that it at least builds now. But what I need to do is merge relevant changes from the scripting branch over to main, as I was working on that for a couple months by itself.
• Refactor the build system
  • Dependency/Package management
    Right now the old branches of the project use both git submodules and conan (maybe even some .lib files directly embedded in the repo 😬). Git submodules isn’t ideal because you have to manually figure out how to link the content of that repo to the larger project, and conan was good until they started transitioning to version 2.0 and not all the packages the project needs are on that branch… so for now my plan is to migrate everything to vcpkg. It’s a pain how long it takes to download and build everything, ESPECIALLY LLVM, however it’s widely known and commonly supported, which is what I prioritize now.
  • Build System
    My cmake skills have gotten a lot better since I started this project, but I need to sit down and fully flesh out a deployment pipeline. Right now it’s inconsistent for multi-target generators, and while I have learned some cheese methods for forcing cmake to put binaries where I want them, a better route is probably to implement cmake install as a better way of handling static files that need to be placed relative to the final executables. Additionally, doing that will allow me to implement cpack to auto-generate installers. Main generator I’m looking at for that rn is QtIFW, which would look a little like this but with Brane branding:

• Fix GitHub CI, and build artifacts for releases
  Building on the last point, after the build pipeline is fixed up we’d want GitHub automatically running that so that instead of people having to clone and build themselves (and deal with the MANY dependencies of the project) they can download an installer from the latest push or release and be good to go from there. Ideally somewhere in there I also figure out how to sign executables to appease windows defender.
• Double check formatting and docgen standards
  I’ve previously set up clang-format for most repos, as well as Doxygen for BraneScript, but ideally I put down standards for how myself and other contributors interact with these systems, and do a pass over the codebase to make it conform to those standards. Which will likely result in a LOT of documentation focused comment writing. Would also be good to set up some CI to push Docxygen artifacts to a github page.

Roadmap

The thing everyone cares about

* All of these dates are guesstimates and the roadmap is always in flux, so keep in mind the further the dates are the more inaccurate my guesses are going to be. Also my estimates are generally given with the idea that nothing else happens in those weeks, which now that I’m not living in a 1 roommate apartment in Idaho anymore, can’t always be depended on.

Scripting

Until we get scripting working it’s not worth moving forward on any other feature, since scripting touches almost every other feature in some way.

I originally had a crazy goal to get scripting fully working by MIT Reality Hack, however now I’ve downgraded that to a “nice if it happens”. My next two months are looking very busy. Between some odd jobs, some (very) cool secret projects I’m working on for some others, and CES, I’m not sure how much I’ll get done in that time period.

In February after all the events are over, I’ll start streaming on a schedule several days a week. Days to be determined, but I want to experiment with a format where for the first half of the stream we plan out some architecture, and then in the second half we work on implementing that. For the last couple years my streaming formula has been “turn on the camera, boot up the editor, and wing it” but I’d like to be a lot more deliberate in the future.

My goal is to have scripting, in the editor executable and usable by end of February.

Entity Overhaul

Right now the engine uses a custom ECS runtime that I built to support V1 of BraneScript, which was designed in a completely different way then BraneScript 2.0. After the time I’ve had to think on it, I don’t think ECS is the way to go for the engine runtime. It’s unintuitive in the context of UGC and the ways VR is commonly built. It works great in RTS situations, but in VR it’s much more common for everything in a scene to be it’s own completely unique thing, so a system that at it’s core is attempting to group duplicate things is going to hider more then help.

So, over the next couple weeks (December-January) while we’re rebuilding the scripting system we also need to decide how we want to represent and perform operations on Entities/Objects in a way that fits the needs of the engine, and there’s a pretty good chance we invent a new paradigm. It’s important that we decide on that while in the middle of the scripting update, since it’ll determine how we make the scripting system and scheduler interact with it’s host program.

Estimated: 1-2 weeks

Editor Plugins / Editor Audit & Refactor

After scripting is implemented in the engine, I want to refactor the UI in such a way that plugins can be developed for it using scripts, and additionally refactor the existing windows to use that new system instead wherever it makes sense. Want to change your local UI? Just rewrite the UI in the same language used for the rest of the engine.

I also want to go though all of the editor code and implement better error handling, because the current approach is “just don’t throw errors”. Rust has taught me much better since then, and we’re going to oxidize this a little bit.

Estimated time: 2-3 weeks

Eventually this will also allow us to replace ImGui if we want to, since UI will be defined with an api rather then a direct interface. I want to experiment with the in-game UI system also being used for the editor once in-game UI is a thing.

Asset Server Audit/Refactor

Refactor identity to be managed through a DID protocol, and general cleanup.

I also want to embed a development asset server in the editor to allow for local development without having to wrangle three executables/processes at once. This might just be a menu in the editor that secretly spins up a child process, or a deeper integration, but ease of use is the first goal.

I also want to take a brief look to see if asset compression/decompression can be done easily

Estimated time: 1-3 weeks.

Client UI & XR

Refactor the client to support the new scripting system, add temporary ImGui support (as a placeholder until we make a something more photogenic) so that we can use a connection dialog instead of hardcoding or config files for the asset server.

Adding OpenXR to the client is also long overdue. So we’ll create a very simple character controller that can look around, move on the x/z axis (no collision), and in the case of VR see your hands. I’m a big fan of systems that let you swap from VR to desktop on the fly, and I’ll build this similar to how I have for other projects to allow that.

Thing I’m honestly dreading most is getting UI to work in VR, and hoping that whatever I come up with doesn’t become a permanent temporary solution. (It’s not going to be crazy difficult tho, just render ImGui to a plane and figure out some raycasting)

Estimated time: 1-3 weeks

Implement MVP Runtime/Multiplayer Servers

I have some placeholder code for runtime servers but there’s nothing of substance there yet. For multiplayer to work we have to implement these. Maybe I should just rename these to “Multiplayer servers” since runtime is a bit confusing?

Initially we just need to sync player positions, and add in a simple voice chat system. (guess we also add super simple audio to the client here)

After we have something simple working for a manually configured or hard coded configuration, from that point forward community tests become possible, even though at this point it would mostly be cube headed headsets floating next to whatever static geometry we’ve brought in as a chunk.

From there we implement a very “simple” server-authoritative rollback system for syncing values in scripts. Easier said then done, this will be a fair amount of work, and some of the most important.

Estimated time: 3-6 weeks

Graphics Audit / Refactor / Skinned Meshes / Material API

To get avatars fully working, we need skinned meshes. So this is a good stage for us to check where we’re at with the graphics pipeline, see what needs to be changed and optimized.

Add skinned meshes (Eaaaazy)

While we’re working on graphics I also want to take the time to make an api for scripts to be able to change material/shader properties at runtime.

Estimated time: 2-4 weeks

Implement MVP Avatar System

This is when we figure out how avatars work, determining where the line between player rig and avatar is, creating an IK system, and adding a simple input/interaction system to allow avatars to interact with the chunk that they’re in.

Estimated time: 2-3 weeks for IK, 1-2 week for scripts.

Add Steam Audio

Add spatialized audio, this will probably have some overlap with physics colliders, but I want to put this in first since it’s a lot simpler then physics.

Estimated: 1-2 weeks

Add Jolt Physics

Add interfaces to make things work with physics, as well as network said physics interactions. This is going to be very fun, and an absolute nightmare.

Estimated: Yeah I have no clue, this takes as much time as I let it.

Conclusion

That bring us to about 6-9 (nice, that was not intentional) months from now, assuming everything goes very smoothly, which it wont lol. There will be side quests, weeks off for travel, collaboration where I help out other projects, etc. But this should give you a rough idea of what I’m thinking in terms of order of things to build how long I (very) optimistically think it will take to build them. Keep in mind this is semi-rushing so that we can get to an MVP state where we’re all hanging out in a lobby with the majority of core features working and it’s not toooo jank.

After we have that MVP up we’ll go back and add usability and polish along every step of that journey. Things like creating a language server for brane script that enables in-editor highlighting and error reporting, seeing if we can improve gdb/lldb debugging support for scripts (thinking about that gives me an incredibly cursed idea for something called VRDB), making a custom UI framework, refining chunk loading, starting work on that node scripting system, etc.

Hopefully I’m not grossly underestimating the time it will take to do all this, I’m made more confident by all the people who have reached out offering to help though.

I hope to see you all in The Brane soon! I can already see the tears of server during the first load test.