Meta researchers demonstrated that the Quest 2 can have full body tracking with no additional sensors, with the help of AI.
Meta researchers developed a system for full body tracking with Quest 2 that uses only the existing headset and controllers, and no additional sensors of any kind. The system is called QuestSim and uses AI that was trained on hours of motion captured subjects. After the AI system was trained, it could analyze the position of the headset and controllers, and factor in human physical limitations to make a reasonably accurate prediction of the body’s entire position:
The researchers tested QuestSim’s accuracy on two subjects of different heights, both of whom had not been analyzed previously. The result was that QuestSim was very accurate in conventional movements such as walking or running. It was less accurate in more unpredictable movements such as dancing, but still wasn’t wildly off in my opinion.
One issue is that the test subjects appeared to have average age and body compositions. It was not shown whether QuestSim could accurately predict the body movement when the subject was much heavier or lighter, or much older or younger. Nonetheless, the approach is promising and with more training data, QuestSim might become a practical method of full body tracking.