Capture Motion Capture (.bvh) from Videos with DeepMotion.
--
DeepMotion absolutely blew my mind this past weekend while I was experimenting with it. It uses AI to extract motion from a subject in a video or image and converts it to a motion capture file that you can use with properly rigged 3D models.
The first example is me uploading an image with 3D Pose, DeepMotion scanned the image and allowed me to pose my 3d model in the same pose as the image. The model started as an A-Pose model and once I uploaded the image, it was able to mimic the image in question.
The next feature was way more impressive. This time I uploaded a video, and selected 3D animation. DeepMotion was able to translate the motion of the subject in the entire video. What resulted was a perfect rendition of the video with my 3d model.
Here is a link to the video to see this thing in action.
DeepMotion made it SUPER simple to take an image OR video and apply that motion to a 3D model. I found that video quality doesn't really change much as well, ie 720p vs 1080p.
Motion capture is a technique used in the film and video game industries to record and digitize human movements. BVH (Biovision Hierarchy) is a file format commonly used to store motion capture data. It is a text-based format that describes the hierarchical structure of a skeletal rig, as well as the motion of each joint over time.
There are many websites and software tools available that can extract motion from a subject in a video and convert it to BVH files for use in motion capture 3D models. These tools often use computer vision algorithms to analyze the video and track the movement of specific points on the subject’s body. The resulting motion data can then be used to drive the movement of a 3D character or avatar in real time, or it can be edited and polished in a 3D animation software package.
DeepMotion is a company that provides AI-powered markerless motion capture and real-time 3D body tracking solutions for the animation industry. Their technology allows users to transform their animation pipeline and create more realistic digital characters. The team at DeepMotion is made up of experienced professionals from companies such as Blizzard, Pixar, Disney, ROBLOX, Microsoft, Crystal Dynamics, and Ubisoft. DeepMotion has been a leader in the field of Motion Intelligence since 2014 and is dedicated to advancing the capabilities of motion technology.
DeepMotion has also introduced a beta version of the Rotoscope Pose Editor, a tool that allows users to refine the AI motion capture results by tracing the motion of a character over input video footage. This can help to more accurately mimic the motion of the subject in the video. The company has also added a feature that allows users to adjust the iris tracking sensitivity for face tracking, and has updated the previous “Default Pose” feature to the “Fallback Pose” to provide a pose for characters when joints are not detected. The Fallback Pose can now be set to a sitting position, in addition to T, A, or I poses. Additionally, Studio and Enterprise plans now support 240 FPS video uploads, and the Foot Locking feature has been enabled for 3D Poses. Custom Enterprise Plans are also available for users with higher volume animation needs.
Thanks for reading! If you made it this far, hit that follow button!
Check me out on Twitter.
