Hello and welcome to the new first tutorial in the OpenFall series, the reason that I am making a new one is that before I was not taking into account future networking needs and it turns out that you need to build it from the ground up for Netcode, I’m keeping the old ones around as I think that they are still a good resource. Now without any further dillydallying on to the code:
This scene is built on the default cube tutorial in the docs that you can check out here. I also have an in-depth exploration of the code in the example scene here.
Plugins
Here are all the plugins that you will need, since we are using DOTS and the new XR system there are quite a few:
For DOTS make sure you have the Entities, Hybrid Renderer Package, Unity Physics, And Netcode Packages installed through the Package Manager.
For XR install the XR Plugin Management Package through the package manager and then in the project settings under XR plug-in management check the boxes for the headsets that you want your game to support.
Setting Up The Scene
We are going to set this up the same way it’s set up in the getting started example. Add in a camera if you don’t already have one, then create an empty game object called SharedData and add a convert to client-server entity script, then set the conversion target to client and server. What this does is make it so that everything parented to it is spawned on both the client and server, this is where you are going to want to put all your static terrain and objects or they won’t show up.
Next, add another empty game object as a child of the SharedData Object and name it GhostCollection, then add a Ghost Collection Authoring Component Script. This is what keeps track of the objects (ghosts) that are synced over the network. Set the root path to something like Scripts/Generated/Collection and the name prefix to Players (important as this script generates code and this will be part of the name of a class we reference later so remember what you set this to). We will come back to this object later once we make our player prefab.
Now we create our player prefab, this must be a single game object with no children or the entire editor will crash when we try to spawn it. (I spent several days trying to figure that out so you’re welcome.) This means that we will need to use some workarounds to spawn in children for this object but we will handle this later. Create an empty game object and name it ether player or pilot (depending on if you are following the series). Then make it a prefab and delete it from the scene. Now open the prefab so that we can continue editing it. Once you have it open add a ghost authoring component. This allows our object to sync variables and components over the network along with allowing us to use the rollback system. Set the Instantiation type to Owner Predicted (since we want to eliminate lag for our player). Next, we need to set the network ID, this is done through a component variable so lets code that real fast:
using Unity.Entities;
using Unity.NetCode;
[GenerateAuthoringComponent]
public struct PilotData : IComponentData //you can rename this PlayerData if you wish
{
[GhostDefaultField]//this makes sure that the ID is synced over the network
public int PlayerId;
public bool clientHasAuthority;//This is an optional helper variable that I added, we do not want to sync this one.
}
Add this to your player/pilot prefab just as you would any other script. Now click the update component list button on the ghost authoring component. (you want to do this anytime you add or edit components to make sure they are detected.) After doing this the ghost authoring component should automatically detect the PlayerID variable and you can now set the predicting player network id to it. Set the root path to something like /Scripts/Generated/Player or, if you put the prefab in a folder, ../Scripts/Generated/Player. Finally set the name to Pilot or Player, again just like the ghost collection this will be part of the name of a generated class we will reference later so again pay attention to what you name it. One last step, click the update component list button just to be safe and then click the generate code button.
Now that we have our player prefab go back to the ghost collection object and click the update ghost list button. Our ghost prefab should then be automatically detected and appear in the list. Now click the Generate Collection Code Button. Most of our scene setup is now done.
The last thing we need to do before we are ready to code is to quickly create a camera rig prefab with three empty game objects as children named Head, Left hand, and Right Hand. Add some cubes scaled to about 0.1 as a child of each of these objects so that we can see them. We don’t do anything more with this for now.
Now to get to the code:
The Code:
To start off we need to write some boilerplate to connect to/setup the server:
First up we create a system that will connect us to the server and make the server start listening for connections:
using Unity.Entities;
using Unity.NetCode;
using Unity.Networking.Transport;
using UnityEngine;
[UpdateInWorld(UpdateInWorld.TargetWorld.Default)]
public class Game : SystemBase
{
// Singleton component to trigger connections once from a control system
struct InitGameComponent : IComponentData
{
}
protected override void OnCreate()
{
RequireSingletonForUpdate<InitGameComponent>();
// Create singleton, require singleton for update so system runs once
EntityManager.CreateEntity(typeof(InitGameComponent));
}
protected override void OnUpdate()
{
// Destroy singleton to prevent system from running again
EntityManager.DestroyEntity(GetSingletonEntity<InitGameComponent>());
foreach (var world in World.All)
{
var network = world.GetExistingSystem<NetworkStreamReceiveSystem>();
if (world.GetExistingSystem<ClientSimulationSystemGroup>() != null)
{
Debug.Log("Client trying to connect to: " + "Your IP here");
NetworkEndPoint.TryParse("Your IP here", 7979, out NetworkEndPoint ep);
network.Connect(ep);
}
else if (world.GetExistingSystem<ServerSimulationSystemGroup>() != null)
{
// Server world automatically listens for connections from any host
Debug.Log("Server listening for connections");
NetworkEndPoint ep = NetworkEndPoint.AnyIpv4;
ep.Port = 7979;
network.Listen(ep);
}
}
}
}
Then we need to ask the server to spawn in our player prefab:
// When client has a connection with network id, go in game and tell server to also go in game
[UpdateInGroup(typeof(ClientSimulationSystemGroup))]
public class GoInGameClientSystem : SystemBase
{
protected override void OnUpdate()
{
EntityManager entityManager = EntityManager;
Entities.WithStructuralChanges().WithNone<NetworkStreamInGame>().ForEach((Entity ent, ref NetworkIdComponent id) =>
{
entityManager.AddComponent<NetworkStreamInGame>(ent);
var req = entityManager.CreateEntity();
entityManager.AddComponent<GoInGameRequest>(req);
entityManager.AddComponent<SendRpcCommandRequestComponent>(req);
entityManager.SetComponentData(req, new SendRpcCommandRequestComponent { TargetConnection = ent });
}).Run();
}
}
//The boilerplate for the RPC
[BurstCompile]
public struct GoInGameRequest : IRpcCommand
{
public void Deserialize(ref DataStreamReader reader)
{
}
public void Serialize(ref DataStreamWriter writer)
{
}
[BurstCompile]
private static void InvokeExecute(ref RpcExecutor.Parameters parameters)
{
RpcExecutor.ExecuteCreateRequestComponent<GoInGameRequest>(ref parameters);
}
static PortableFunctionPointer<RpcExecutor.ExecuteDelegate> InvokeExecuteFunctionPointer = new PortableFunctionPointer<RpcExecutor.ExecuteDelegate>(InvokeExecute);
public PortableFunctionPointer<RpcExecutor.ExecuteDelegate> CompileExecute()
{
return InvokeExecuteFunctionPointer;
}
}
// The system that makes the RPC request component transfer, this just needs to exist or nothing will happen
public class GoInGameRequestSystem : RpcCommandRequestSystem<GoInGameRequest>
{
}
Then we need to create a system so the server can respond to the request:
// When server receives go in game request, go in game and delete request
[UpdateInGroup(typeof(ServerSimulationSystemGroup))]
public class GoInGameServerSystem : SystemBase
{
protected override void OnUpdate()
{
EntityManager entityManager = EntityManager;
Entities.WithStructuralChanges().WithNone<SendRpcCommandRequestComponent>().ForEach((Entity reqEnt, ref GoInGameRequest req, ref ReceiveRpcCommandRequestComponent reqSrc) =>
{
entityManager.AddComponent<NetworkStreamInGame>(reqSrc.SourceConnection);
UnityEngine.Debug.Log(System.String.Format("Server setting connection {0} to in game", EntityManager.GetComponentData<NetworkIdComponent>(reqSrc.SourceConnection).Value));
var ghostCollection = GetSingleton<GhostPrefabCollectionComponent>();
var ghostId = PlayersGhostSerializerCollection.FindGhostType<PilotSnapshotData>();//this is where the names we chose in the ghost collection and ghost authoring component scripts come into play
var prefab = EntityManager.GetBuffer<GhostPrefabBuffer>(ghostCollection.serverPrefabs)[ghostId].Value;
var player = entityManager.Instantiate( prefab);
entityManager.SetComponentData( player, new PilotData { PlayerId = EntityManager.GetComponentData<NetworkIdComponent>(reqSrc.SourceConnection).Value });
entityManager.AddBuffer<PilotInput>( player);//we will code this later in the tutorial so ether comment out or ignore for now
entityManager.SetComponentData(reqSrc.SourceConnection, new CommandTargetComponent { targetEntity = player });
entityManager.DestroyEntity( reqEnt);
Debug.Log("Spawend Player");
}).Run();
}
}
And with that when we start our game it should spawn in our player prefab and sync it over the network. If you want more detail on what is going on check out this article. But now that we have the boilerplate out of the way we can move on to the interesting bits.
First things first, we need to address the problem of spawning in our camera rig as a child of the player and syncing the positions of the hands and head. As I said earlier the entire game and editor will crash if we try to load in any ghost prefab with children so we need a workaround. The way I handle this is by adding an empty component that only holds a reference to the camera rig prefab to our player. It will be detected on the first frame that it exists and the setup system will be run, spawning in camera rig and parenting it, and then the component will be removed causing the system to only run once.
To start off lets create an empty component that we can use to signal our setup system:
[GenerateAuthoringComponent]
public struct PilotSetupRequired : IComponentData
{
public Entity prefab;
}
Add this to the player prefab object and set the prefab variable to the CameraRig prefab we made earlier. Next, we need to make a way to store a reference to our camera rig so that we can reference it later when setting the positions of the head and controllers:
using Unity.Entities;
using Unity.Mathematics;
using Unity.NetCode;
[GenerateAuthoringComponent]
public struct CameraRigChild : IComponentData
{
//a reference to the camera rig base
public Entity Value;
}
Add this to the player prefab gameobject.
Now we can create a system to parent this prefab to our player when they spawn in:
[UpdateInGroup(typeof(ClientAndServerSimulationSystemGroup))]
public class PilotSetup : SystemBase
{
protected override void OnUpdate()
{
var entityManger = EntityManager;
Entities.WithStructuralChanges().ForEach((Entity entity, in PilotSetupRequired setup) =>
{
//create the prefab
Entity cameraRig = entityManger.Instantiate(setup.prefab);
//parent it to the player
entityManger.AddComponent<Parent>(cameraRig);
entityManger.AddComponent<LocalToParent>(cameraRig);
entityManger.SetComponentData(cameraRig, new Parent { Value = entity});
//set a reference to the CameraRig so we can find it from systems referencing the player entity
entityManger.SetComponentData(entity, new CameraRigChild { Value = cameraRig });
UnityEngine.Debug.Log("set up pilot on world: " + entityManger.World.Name);
//remove the component so this system only runs once
entityManger.RemoveComponent<PilotSetupRequired>(entity);
}).Run();
}
}
Now that we have that set up we can move on to making the entities follow the player’s movement, but you may have realized a problem with all this: DOTS doesn’t have camera entities. This means that we need to use the ordinary camera script on a game object approach. But then how do we move it? Simple, we just create a singleton that stores a reference to the player entity, get the current head position, add that to the player position, and then we have our camera position. So here’s the code to do that:
using System.Collections.Generic;
using Unity.Entities;
using Unity.Mathematics;
using Unity.Transforms;
using UnityEngine;
using UnityEngine.XR;
using UnityEngine.XR.Management;
public class HeadCamera : MonoBehaviour
{
private Camera cam;
public void Start()
{
cam = GetComponent<Camera>();//store our camera
XRGeneralSettings.Instance.Manager.activeLoader.GetLoadedSubsystem<XRInputSubsystem>().TrySetTrackingOriginMode(TrackingOriginModeFlags.Floor); //set the tracking mode to avoid strangeness
//we always want the head to have the most up to date position possible so we run this right before rendering
Application.onBeforeRender += () => {
//find our head device
var heads = new List<InputDevice>();
InputDevices.GetDevicesAtXRNode(XRNode.Head, heads);
InputDevice head = new InputDevice();
if (heads.Count == 1)
{
head = heads[0];
}
//get it's position and rotation
head.TryGetFeatureValue(CommonUsages.devicePosition, out Vector3 headPosition);
head.TryGetFeatureValue(CommonUsages.deviceRotation, out Quaternion headRotation);
var ent = CameraRigEntity.Value;
if(ent != Entity.Null)//if we have the player position
{
var entMan = CameraRigEntity.World.EntityManager;
transform.position = entMan.GetComponentData<LocalToWorld>(ent).Position + (float3)headPosition;
transform.rotation = headRotation * entMan.GetComponentData<LocalToWorld>(ent).Rotation;
}
else//otherwise assume we are at (0,0,0)
{
transform.position = headPosition;
transform.rotation = headRotation;
}
};
}
}
//singleton references to our player
public static class CameraRigEntity
{
public static Entity Value;
public static World World;//store the world so we can get the right entity manager
}
//And finaly a simple system to set the singleton reference:
[UpdateInGroup(typeof(ClientSimulationSystemGroup))]
public class CameraRigLinker : SystemBase
{
protected override void OnUpdate()
{
var entityManger = EntityManager;
if (CameraRigEntity.Value == Entity.Null)
Entities.WithoutBurst().ForEach((Entity entity, in PilotData pilot, in CameraRigChild cameraRig) =>
{
if (pilot.clientHasAuthority)
{
CameraRigEntity.Value = entity;
CameraRigEntity.World = entityManger.World;
}
}).Run();
}
}
Add this to your main camera object or whatever camera you want to follow the position of the head.
Now that we can see we can move on to moving the hands. Now what we do next might seem a little counter-intuitive but trust me, it is the way to go. To get the networking rollback system to work correctly we need to store the positions of our controllers in an input buffer and then set our positions from the input buffer. The reason we can’t just always use the latest values is that the rollback system re-simulates the past to eliminate lag. (It’s a terrible explanation I know, just go with it) So first we need to create the input buffer itself, this buffer is also where we will put movement input in the future. The code also tells the buffer how to serialize and serialize itself so if you have any really weird de-syncing issues this is a good place to look.
public struct PilotInput : ICommandData<PilotInput>
{
public uint Tick => tick;
public uint tick;
public Transform head;
public Transform leftHand;
public Transform rightHand;
[System.Serializable]
public struct Transform
{
public float3 position;
public quaternion rotation;
//to cut down on coding put all this in a struct function
public void Deserialize(ref DataStreamReader reader, int quantizatinon)
{
position.x = reader.ReadInt() / (float)quantizatinon;
position.y = reader.ReadInt() / (float)quantizatinon;
position.z = reader.ReadInt() / (float)quantizatinon;
rotation.value.x = reader.ReadInt() / (float)quantizatinon;
rotation.value.y = reader.ReadInt() / (float)quantizatinon;
rotation.value.z = reader.ReadInt() / (float)quantizatinon;
rotation.value.w = reader.ReadInt() / (float)quantizatinon;
}
public void Serialize(ref DataStreamWriter writer, int quantizatinon)
{
writer.WriteInt((int)(position.x * quantizatinon));
writer.WriteInt((int)(position.y * quantizatinon));
writer.WriteInt((int)(position.z * quantizatinon));
writer.WriteInt((int)(rotation.value.x * quantizatinon));
writer.WriteInt((int)(rotation.value.y * quantizatinon));
writer.WriteInt((int)(rotation.value.z * quantizatinon));
writer.WriteInt((int)(rotation.value.w * quantizatinon));
}
}
public void Deserialize(uint tick, ref DataStreamReader reader)
{
//this is where we download data from the network
this.tick = tick;
//normaly we would need to send each individual float over the network but we made some functions to do this a lot easer.
head.Deserialize(ref reader, 10000);
leftHand.Deserialize(ref reader, 1000);
rightHand.Deserialize(ref reader, 1000);
}
public void Serialize(ref DataStreamWriter writer)
{
//this is where we upload our data to the network
writer.WriteInt((int)(movement.x * 100));
writer.WriteInt((int)(movement.y * 100));
writer.WriteInt(jumping ? 1 : 0);
head.Serialize(ref writer, 10000);
leftHand.Serialize(ref writer, 1000);
rightHand.Serialize(ref writer, 1000);
}
public void Deserialize(uint tick, ref DataStreamReader reader, PilotInput baseline, NetworkCompressionModel compressionModel)
{
Deserialize(tick, ref reader);
}
public void Serialize(ref DataStreamWriter writer, PilotInput baseline, NetworkCompressionModel compressionModel)
{
Serialize(ref writer);
}
}
//we need these two classes to exist for the buffer to be synced over the network
public class PilotSendCommandSystem : CommandSendSystem<PilotInput>
{
}
public class PilotReceiveCommandSystem : CommandReceiveSystem<PilotInput>
{
}
Now that we have our buffer we need to add it to our player and update it. On the server-side of the enter game code, we add it to the server-side player so all that’s left is to add it to our player, we can do this and update the code in the same code so let’s write this now:
[UpdateInGroup(typeof(ClientSimulationSystemGroup))]
public class PilotInputNetowrking : SystemBase
{
protected override void OnCreate()
{
//only update when we are connected
RequireSingletonForUpdate<NetworkIdComponent>();
RequireSingletonForUpdate<EnablePlayersGhostReceiveSystemComponent>();
}
protected override void OnUpdate()
{
//get our player
var localInput = GetSingleton<CommandTargetComponent>().targetEntity;
var entityManager = EntityManager;
//if we haven't set the singleton
if (localInput == Entity.Null)
{
//get our ID
var localPlayerId = GetSingleton<NetworkIdComponent>().Value;
Entities.WithStructuralChanges().WithNone<PilotInput>().ForEach((Entity ent, ref PilotData pilot ) =>
{
//find our player
if (pilot.PlayerId == localPlayerId)
{
UnityEngine.Debug.Log("Adding input buffer");
//set this variable to true so we don't need to always grab the ID later
pilot.clientHasAuthority = true;
//add the buffer
entityManager.AddBuffer<PilotInput>(ent);
//set the singleton so that this doesn't run again
entityManager.SetComponentData(GetSingletonEntity<CommandTargetComponent>(), new CommandTargetComponent { targetEntity = ent });
}
}).Run();
return;
}
//Debug.Log("updating input");
var input = default(PilotInput);
//get a blank struct
//set the tick for rollback
input.tick = World.GetExistingSystem<ClientSimulationSystemGroup>().ServerTick;
//behold the vast amout of boilerplate needed to find XR input devices
var heads = new List<InputDevice>();
InputDevices.GetDevicesAtXRNode(XRNode.Head, heads);
var leftHandDevices = new List<InputDevice>();
InputDevices.GetDevicesAtXRNode(XRNode.LeftHand, leftHandDevices);
var rightHandDevices = new List<InputDevice>();
InputDevices.GetDevicesAtXRNode(XRNode.RightHand, rightHandDevices);
InputDevice head = new InputDevice();
if (heads.Count == 1)
{
head = heads[0];
}
InputDevice leftHand = new InputDevice();
if (leftHandDevices.Count == 1)
{
leftHand = leftHandDevices[0];
}
InputDevice rightHand = new InputDevice();
if (rightHandDevices.Count == 1)
{
rightHand = rightHandDevices[0];
}
//get our values, I realize that I could directly input the struct but I didn't do it that way and I'm to lazy to change it yet
head.TryGetFeatureValue(CommonUsages.devicePosition, out Vector3 headPosition);
head.TryGetFeatureValue(CommonUsages.deviceRotation, out Quaternion headRotation);
leftHand.TryGetFeatureValue(CommonUsages.devicePosition, out Vector3 leftPosition);
leftHand.TryGetFeatureValue(CommonUsages.deviceRotation, out Quaternion leftRotation);
rightHand.TryGetFeatureValue(CommonUsages.devicePosition, out Vector3 rightPosition);
rightHand.TryGetFeatureValue(CommonUsages.deviceRotation, out Quaternion rightRotation);
//set struct values
input.head = new PilotInput.Transform
{
position = headPosition,
rotation = headRotation
};
input.leftHand = new PilotInput.Transform
{
position = leftPosition,
rotation = leftRotation
};
input.rightHand = new PilotInput.Transform
{
position = rightPosition,
rotation = rightRotation
};
//add our struct to our input buffer
var inputBuffer = EntityManager.GetBuffer<PilotInput>(localInput);
inputBuffer.AddCommandData(input);
}
}
And with that, all that’s left is to actually set the transforms of our head and hand objects. But before we do that we have one last component data struct to code:
[GenerateAuthoringComponent]
public struct CameraRigData : IComponentData
{
public Entity head;
public Entity leftHand;
public Entity rightHand;
}
Add this to the root of the camera rig prefab that we created about 1300 words ago then set the game object references according to name and we are ready to write our position update system:
[UpdateInGroup(typeof(GhostPredictionSystemGroup))]
public class CameraRigSystem : SystemBase
{
protected override void OnUpdate()
{
var group = World.GetExistingSystem<GhostPredictionSystemGroup>();
var tick = group.PredictingTick;
bool isFinal = group.IsFinalPredictionTick;
var deltaTime = Time.DeltaTime;
var entityManager = EntityManager;
Vector3 headPose = default;
Quaternion headRot = default;
Vector3 leftPose = default;
Quaternion leftRot = default;
Vector3 rightPose = default;
Quaternion rightRot = default;
if (isFinal && World.GetExistingSystem<ClientSimulationSystemGroup>() != null)//if this is the tick that we are showing to the player we're going to cheat and show them the most up to date values
{
var heads = new List<InputDevice>();
InputDevices.GetDevicesAtXRNode(XRNode.Head, heads);
var leftHandDevices = new List<InputDevice>();
InputDevices.GetDevicesAtXRNode(XRNode.LeftHand, leftHandDevices);
var rightHandDevices = new List<InputDevice>();
InputDevices.GetDevicesAtXRNode(XRNode.RightHand, rightHandDevices);
InputDevice head = new InputDevice();
if (heads.Count == 1)
{
head = heads[0];
}
Vector3 position;
Quaternion rotation;
head.TryGetFeatureValue(CommonUsages.devicePosition, out headPose);
head.TryGetFeatureValue(CommonUsages.deviceRotation, out headRot);
InputDevice leftHand = new InputDevice();
if (leftHandDevices.Count == 1)
{
leftHand = leftHandDevices[0];
}
leftHand.TryGetFeatureValue(CommonUsages.devicePosition, out leftPose);
leftHand.TryGetFeatureValue(CommonUsages.deviceRotation, out leftRot);
InputDevice rightHand = new InputDevice();
if (rightHandDevices.Count == 1)
{
rightHand = rightHandDevices[0];
}
rightHand.TryGetFeatureValue(CommonUsages.devicePosition, out rightPose);
rightHand.TryGetFeatureValue(CommonUsages.deviceRotation, out rightRot);
}
Entities.ForEach((Entity ent, DynamicBuffer<PilotInput> inputBuffer, ref PredictedGhostComponent prediction, in PilotData pilot, in CameraRigChild cameraRigChild) =>
{
if (cameraRigChild.Value == Entity.Null || !GhostPredictionSystemGroup.ShouldPredict(tick, prediction))
return;
inputBuffer.GetDataAtTick(tick, out PilotInput input);
if (pilot.clientHasAuthority && isFinal)//if this is the tick that we are showing to the player and this is the player then update to the latest values
{
input.head.position = headPose;
input.head.rotation = headRot;
input.leftHand.position = leftPose;
input.leftHand.rotation = leftRot;
input.rightHand.position = rightPose;
input.rightHand.rotation = rightRot;
}
var cameraRig = entityManager.GetComponentData<CameraRigData>(cameraRigChild.Value);
entityManager.SetComponentData(cameraRig.head, new Translation { Value = input.head.position });
entityManager.SetComponentData(cameraRig.head, new Rotation { Value = input.head.rotation });
entityManager.SetComponentData(cameraRig.leftHand, new Translation { Value = input.leftHand.position });
entityManager.SetComponentData(cameraRig.leftHand, new Rotation { Value = input.leftHand.rotation });
entityManager.SetComponentData(cameraRig.rightHand, new Translation { Value = input.rightHand.position });
entityManager.SetComponentData(cameraRig.rightHand, new Rotation { Value = input.rightHand.rotation });
}).Run();
}
}
And with that, we should have everything we need to get a “simple” networked scene working. With this, you should be able to run the project in the editor and a build on a separate computer and be able to see the headset/hands of the other player. I’m sorry if this tutorial was hard to follow, I know it was a lot, but I hope it can be a valuable resource for everyone. In the next one, we will add in movement so look forward to it, also I’ll be updating the project on this Github if you want to check out the latest version.
Thanks for sharing these tutorials. I started a similar ECS VR project and its amazing how even simple things can take hours to debug and track down because of lack of documentation. Without your previous tutorials I would still be stuck trying to figure out how to sync the XR rigs and unfortunatelly some of the things from those tutorials are already changed/broken with the new updates to the packages.