cebstudios

RealTime MOCAP with webcam Mocap PE data import ALPHA 0.1 (MediaPipe)

Added 2021-05-01 21:41:25 +0000 UTC

This Alpha version is focused on the RealTime Development

This version 0.1 aims to bring the power of MediaPipe to use for realtime MOCAP with a simple webcam.

Instructions.

To use it you must install the addon on blender, and you will also need a python installation.

The part to use with python you need to extract the file "mediapipe_py_server.py" to a folder that you will have to run via python

We suggest the installation of Miniconda but you can use vanila Python

Having the python installed you will need to install the Mediapipe python package, to do that you can use the command:

pip install mediapipe

Using the addon:

First you click the prepare skeleton.

then you run the "mediapipe_py_server.py" and click on "Media Pipe RealTime" on the addon

The capture will start.

Then you can cancel it.

Now you can resize the bones to better fit the points that are the references captured from the webcam.

To resize the bones, the easier way is to use the "SK Multi" value and click "Reload MP Skeleton for RT". For me, values between 2.7 and 3 works better.

If you want to fit it even better, you can choose the Debug panel, that gives you the possibility to ajust each limb.

After fitting the bone to the point reference, you can start to use the realtime capture.

If you want to record it, you can enable the "Record Realtime" and click the "MediaPipe RealTIme" ands it will record the movements on the timeline.

You can choose were it will start recording on "Frame to start Recording" the default is 1

PS.: Always open the console window, because blender can be unresponsive. If blender becomes unresponsive, on the console window, click CTRL+C and it will stop what it is doing, and blender will become usable again

PS2.: if you have problems starting the realtime, change the values of Buffer and Refresh rate.

Testing in our machines the best buffer values were 512 and 1024

and the best refresh rate were 0.1 and 0.2.

Hope you enjoy, and if you can support us please be a patron