VR Body Tracking Part 2 | Arms.

This is part two of a three-part series so if you haven’t read the first one check it out here. Last time we got the head and torso working but the arms and legs were stationary leading to an eternal T-pose. While this might be useful for some situations it’s not ideal. In this tutorial, we are going to add control to our arms. Now when It comes to body tracking the arms are the most important thing to get right. Legs don’t matter as you aren’t actually walking, body and head don’t matter because you can’t really see them, but if your arm estimations are off you will know instantly and it will annoy you to no end. That’s why it’s important to get right.

How It Works

So to get started we need to get some IK for our arms. When I was programming this I tried every free IK asset on the store but I couldn’t find one that worked for the Robot Kyle asset and my specific needs, so I just programmed my own. (You can find a dedicated article for that one here.) But we are going to need to modify the code a little for it to work for this specific use case. In the code for It below, I added a public variable that it will update depending on whether or not the end bone can reach the target position or not, and the ability to rotate the end bone. It also adds a public function that adds the ability for other scripts to tell the IK to recalculate.

using UnityEngine;

[ExecuteInEditMode]
public class IK : MonoBehaviour
{
    public Transform Upper;
    public Transform Lower;
    public Transform End;
    public Transform Target;
    public Transform Pole;
    public float UpperElbowRotation;
    public float LowerElbowRotation;
    public bool SetRotation;
    public Vector3 OffsetEndRotation;
    // Start is called before the first frame update
    private float a;
    private float b;
    private float c;
    private Vector3 en;
    public bool CantReach;

    // Update is called once per frame
    void Update()
    {
        UpdateIK();
    }
    public void UpdateIK()
    {
        a = (Lower.position - Upper.position).magnitude;
        b = (End.position - Lower.position).magnitude;
        c = Vector3.Distance(Upper.position, Target.position);
        en = Vector3.Cross(Target.position - Upper.position, Pole.position - Upper.position);
        //Debug.Log("The angle is: " + CosAngle(a,b,c));
        Debug.DrawLine(Upper.position, Target.position);
        Debug.DrawLine((Upper.position + Target.position) / 2, Lower.position);
        //Debug.Log(en);
        Upper.rotation = Quaternion.LookRotation(Target.position - Upper.position, Quaternion.AngleAxis(UpperElbowRotation, Lower.position - Upper.position) * (en));
        Upper.rotation *= Quaternion.Inverse(Quaternion.FromToRotation(Vector3.forward, Lower.localPosition));
        Upper.rotation = Quaternion.AngleAxis(-CosAngle(a, c, b), -en) * Upper.rotation;

        Lower.rotation = Quaternion.LookRotation(Target.position - Lower.position, Quaternion.AngleAxis(LowerElbowRotation, End.position - Lower.position) * (en));
        Lower.rotation *= Quaternion.Inverse(Quaternion.FromToRotation(Vector3.forward, End.localPosition));
        if (SetRotation) End.rotation = Target.rotation * Quaternion.Euler(OffsetEndRotation.x, OffsetEndRotation.y, OffsetEndRotation.z);
        //Debug.Log("Distance is: "+(End.position - Target.position).magnitude );
        if ((End.position-Target.position).magnitude >.01)
        {
            CantReach = true;
        }
        else
        {
            CantReach = false;
        }
    }
    float CosAngle(float a, float b, float c) {
        if ( !float.IsNaN(Mathf.Acos((-(c * c) + (a * a) + (b * b)) / (-2 * a * b)) * Mathf.Rad2Deg))
        {
            return Mathf.Acos((-(c * c) + (a * a) + (b * b)) / (2 * a * b)) * Mathf.Rad2Deg;
        }
        else
        {
            return 1;
        }
    }
}

So now take this script and attach it to both Shoulder joints and fill in the options.

Mess with the rotations until it looks right, and for the pole create an empty GameObject as a child of both controllers (or a sphere like I did so you can see it.) and then move it directly back and down a little in the local y-direction.

Now at this point, you should be able to boot up the demo and it will work pretty well but you might notice as you look and move around sometimes your hands won’t be able to reach all the way to where your controllers are. Luckily we have a fix for this built into the IK asset. We need to make it so when the hand isn’t able to reach its target our body tracking script will rotate the body until it can. To do this we need to add some code to our body tracking script:

//add some new public varibles:
    public IK LeftArm;
    public IK RightArm;

//Then replace this in your current script
        if (Quaternion.Angle(Quaternion.Euler(0, BodyRoot.transform.rotation.eulerAngles.y, 0), Quaternion.Euler(0, Head.transform.rotation.eulerAngles.y, 0)) > 90)
            {
                BodyRoot.transform.rotation = Quaternion.RotateTowards(BodyRoot.transform.rotation, Quaternion.Euler(0, Head.transform.rotation.eulerAngles.y, 0) * TorsoRotation, 3);
            }

//With this:
        for (int i = 0; i < 5; i++)
        {
            if (Quaternion.Angle(Quaternion.Euler(0, BodyRoot.transform.rotation.eulerAngles.y, 0), Quaternion.Euler(0, Head.transform.rotation.eulerAngles.y, 0)) > 90)
            {
                BodyRoot.transform.rotation = Quaternion.RotateTowards(BodyRoot.transform.rotation, Quaternion.Euler(0, Head.transform.rotation.eulerAngles.y, 0) * TorsoRotation, 3);
            }
            else
            {
                if (RightArm.CantReach)
                {
                    BodyRoot.transform.rotation = Quaternion.RotateTowards(BodyRoot.transform.rotation, Quaternion.Euler(0, -90, 0) * Quaternion.Euler(0, Quaternion.FromToRotation(Vector3.forward, RightArm.Target.position - BodyRoot.transform.position).eulerAngles.y, 0) * TorsoRotation, 3);
                }
                if (LeftArm.CantReach)
                {
                    BodyRoot.transform.rotation = Quaternion.RotateTowards(BodyRoot.transform.rotation, Quaternion.Euler(0, 90, 0) * Quaternion.Euler(0, Quaternion.FromToRotation(Vector3.forward, LeftArm.Target.position - BodyRoot.transform.position).eulerAngles.y, 0) * TorsoRotation, 3);
                }
            }
            RightArm.UpdateIK();
            LeftArm.UpdateIK();
        }

After you do that and set the new variables it should be ready to go! If you have any questions feel free to comment or join the discord server (link in the sidebar)

And to know when the next one is coming out: