Easymocap to MetaHuman model using Rokoko Studio in Blender (Only free softwares)

Use the motion imported from Easymocap into MetaHuman Model

software used

- Blender (Free) https://www.blender.org/download/

- Python Miniconda(Free) https://docs.conda.io/en/latest/miniconda.html

- Easymocap(Free) https://github.com/zju3dv/EasyMocap

- Quixel Bridge (Free) https://quixel.com/bridge

- Unreal Engine (Free) https://www.unrealengine.com/en-US/

Addon for Blender used:

- Rokoko Studio (Free) https://github.com/Rokoko/rokoko-studio-live-blender

- Mocap Import (Free) https://gumroad.com/l/mocap_import

Instructions how to install Easymocap and use it

Playlist: https://youtube.com/playlist?list=PLYzax7LPWaADMqDjc9rnAOC3tsd35zZGL

Steps not in the video:

- get smplx model to use with mocap_import : https://www.youtube.com/watch?v=-vvSjAyiK_g

- instruction to change on easymocap to have finger movement: https://github.com/zju3dv/EasyMocap/issues/25#issuecomment-842266080

For patreons, links with files and better explanation: https://www.patreon.com/posts/51596911

- import using mocap_import: https://www.youtube.com/watch?v=tYkKdxJj0ds

- export the animation to BVH format

- Install Quixel Bridge

- Install Unreal Engine

- Download MetaHuman via Quixel Bridge

- Export MetaHuman from Quixel Bridge to Unreal Engine

Steps on the video:

In Unreal Engine

1- Open Metahuman blueprint inside Unreal Engine

———Repeat these steps for each body part or clothes, in the video were: Face, Torso, Legs and Feet)

2- On the model tree, select the parts of the body and click to navigate to that asset.

3- On UE main window right click on the Mesh object and choose Action > Export

4- choose a destination, and select only “Vertex Color”, “Map Skeletan Motion to Root”, leave the other options unchecked.

———-

In blender:

1- Import all the FBXs exported from Unreal Engine

After importing you will see that each fbx came parented to a null.

2- Unparent them from the null (Select the armature and on the 3d space screen, press “Alt P”) [repeat for all of them]

3- remove the null (we wont need them)

4- Unparent the mesh from the armatures (leave only the body armature and the head armature)

5- Delete the unparented armatures

6- Unparent the body mesh from its armature

7- Rename the Head and body armature (to make it easier to find them)

8- With the head armature selected, change to Edit Mode (to edit their bones)

9- find the Neck01 bone

10- Clear Parent “Alt P”

11- Erase all other bones that were below in Neck01 Hierarchy You will have only Neck01 and his descendents

12- go to Object mode

13- select the body armature (the yellow icon, because the green one is the bone) and the head armature

14- on 3d view, join them (CTRL J) (you will notice that the bones will be bellow the same armature, but the name of neck_01 changed to Neck_01.001 (because there is already a neck_01)

15- Expand Neck_01.001

16- Click on the new bone that appeared “FACIAL_C_Neck1Root”

17- On the bone menu, go to the parent option and change from “neck_01.001” to “neck_01” (just erase the “001”)

18- Expand neck02.001

19- Click on the new bone that appeared “FACIAL_C_Neck2Root”

20- On the parent option and change from “neck_02.001” to “neck_02” (again, just erase the “001”)

21- One last time expand “head.001”

22- Click on the new bone that appeared “FACIAL_C_FacialRoot”

23- On the parent option and change from “head.001” to “head” (ona last time, just erase the “001”)

24- Delete the remaining bones (the ones we were expanding) “neck_01.001”,neck_02.001” and “head.001”

—Repeat these steps for the remaining meshes that are not parented to the Armature

25- select an unparented mesh, and SHIFT select the armature

26- press “CTRL P” and select “Armature Deform - With Empty Groups”

—

27- change to pose mode, and test if the bones of the Legs, arms, neck, head are working properly. On the video you can see I had problem on the deformation of the head.

27.1 - If you have some deformation problem, like the bone is not working, got to object mode, select that mesh and go to the constraint menu on the left side (the wrench icon)

27.2 - there should have a already aconstraint, on the object option, select the armature we are working on the scene, and that part that was not deforming when moving the bone, should work now.

27.3- save the blend file if you are worried to loose all this work that was done :)

28- import BVH created after using mocap_import

If you have problem importing. To have the correct orientation, try changing the Transform Foward and Up. In the video, what worked was Foward = Y, Up = Z

29- Put Both armatures in edit mode.

30- Put the BVH skeleton roughly over the 3d model, dont have to be exact

31- Back in object mode, open Rokoko addon on the retargeting option and select source and target armatures

32-  Rebuild Bone list

33- create your mapping for the bones between bvh and MetaHuman bones, or download from our patreon 

34- Click use Pose: Rest

35- Click ReTarget animation.

36- Appreciate the result :)