Single camera motion capture

Kinetix AI allows for single camera motion capture. Our AI deep learning algorithms can extract any human motion from any single camera video.

Kinetix AI will analyze the video sent by the player, identify the human that's appearing, and read their movement, in 3D (recreating depth). Once the movement is fully analyzed, it will be recreated in 3D to generate an animation file. Then, the animation is retargeted on any custom avatar.

Full body movement captation

Kinetix tracks body movements at specific positions and then name them at each critical joints (e.g. feet, ankle, knee, shoulder, elbow). To get the best final output, Kinetix AI is making predictions accurate at the centimeter scale.

Hands movement captation

Hands movements are even more complex to detect but nothing’s impossible! At the moment, Kinetix AI is spotting if the fists are clenched, hands and fingers are opened or fingers are detached (celebrating victories ✌️for example). More complex hands movements are still hard to detect but Kinetix keeps improving its models at large scale.

Half-Body captation

Kinetix AI can also detect and isolate half body movements while keeping the other half static during the transposition process.

Which means you can create both lower body or upper body isolated animations while outputting a full body animation in the end.

Facial Expression Captation

Facial motion is currently not recognized by our algorithm.

Stay tuned! The feature is being developed and will arrive soon.

Welcome to the Kinetix documentation. You will find tutorials and technical information to help you navigate our technologies and integrate our User-Generated Emote feature in your game or app.

With the User-Generated Emote feature, players can create a custom emote from any video, directly in the game, and play it on their avatar. All it takes is a phone camera or any video for Kinetix's AI to create and play an animation in-game.

Quick start: Integrate Kinetix's User-Generated Emote & leverage the power of User-Generated Emotes in 2 steps only!

Integrate Kinetix into your game engine

Kinetix's User-Generated Emote feature is designed to be seamlessly integrated into any of the most popular game engine available on the market.

To integrate Kinetix's User-Generated Emote, you can choose from 2 options:

• An all-in-one, plug-and-play solution with the Kinetix SDK

• A more flexible solution with the Kinetix API

Choose the integration method that fits your needs best:

Quicklinks

Emote standards that maximize game compatibility

Kinetix has carefully benchmarked every emote available on the market to come up with these emote standards that match any avatar-based game constraints to ensure an optimal gamer experience (example: avoid collisions) and the full compatibility of any emotes distributed through our technologies.

Emotes metadatas for an optimal visibility

• Name: Every Emote has a unique name.

• Mature content: Y/N, useful metadata for moderation purposes

• Emote duration: Kinetix emotes have a duration in between 1 to 10 sec

• Emote Icon: All Kinetix emotes come with an associated icon for an optimal emote listing and distribution

• Emote Gif: All Classic Emotes come with an associated gif for an optimal apprehension of the Emote

• Emote ID: A unique ID composed of 36 characters to identify any Emote available.

Established with a passion for redefining the way players connect, express, and engage, Kinetix have embarked on a journey to enhance the gaming experience. We believe in the power of AI to serve in-game self-expression, customization, socialization and limitless creativity.

Our mission is to empower game developers to unlock the full potential of their game. We're here to provide them with the tools and resources they need to elevate player engagement, keep the games fresh and dynamic, and immerse their players in social, customizable, and expressive gaming world.

We believe that players too, can be creators, through UGC. There is no greater tool than generative AI to empower players with fantastic creation opportunities, without requiring any technical difficulties. By giving players the tools to craft their emotes, Kinetix fosters a vibrant and inclusive community where player expression is celebrated.

To achieve this vision, we believe that there is no better asset than Emotes!

About Kinetix's User-Generated Emotes

We developed our AI-powered User-Generated Emote technology to enable any game or virtual world to benefit from the potential of UGC. Empowering players with the possibility to craft emotes increases engagement, socialization & customization in-game and unlock new revenue streams.

The User-Generated Emote feature is essentially an emote creator to be embedded directly in-game, allowing players to craft custom emotes from any video.

Bring customization & self-expression using Generative AI

Let your players create & play custom emotes directly in your game, using our AI-powered User-Generated Emote feature. Integrate it with just a few lines of code, and boost your game's engagement as you empower your players with unlimited expression capabilities.

Kinetix Emotes work with any humanoid avatar

Kinetix technologies include a proprietary retargeting algorithm that automatically transfers the motion of the emote to any human avatar in your game.

Optimized to maximize in-game performance

Our cloud-based technology is designed to enable an unlimited amount of Emotes to be played simultaneously without altering the game performance.

Also, multiple players can create and/or play User-Generated Emotes in the same time, without damaging their in-game experience or your servers.

Our technologies support your growth and empower monetization

Unlocked monetization capacity

Once onboarded with our User-Generated Emote feature, you can monetize the created emotes the way you want, if you want to! You possess all rights to monetize every emotes distributed via our technologies to your users.

Endless monetization opportunities

Kinetix's User-Generated Emote feature adapts to your revenue model and unlock various monetization opportunities based on emotes.

• In-game stores: Integrate User-Generated Emote creation tokens in your in-game store. Monetize User-Generated Emotes and make them exclusive and rare items that only your top players can showcase.

• Loot boxes and crates: diversify the content of your loot boxes by offering User-Generated Emote creation tokens.

• Battle Pass: Fill your Battle Pass with valuable items including User-Generated Emote tokens to diversify the content of your Pass.

Why you should leverage Kinetix's User-Generated Emote

Leveraging Kinetix's User-Generated Emote offers a multitude of compelling reasons to enhance your gaming experience and player engagement. Here are the key benefits:

1. Enhanced User Engagement: Emotes create memorable and shareable moments for players. By empowering them to express themselves with custom moves through their avatars, you cultivate a sense of personalization and social interaction, leading to increased player engagement and retention.

2. Immersive Gameplay Experience: User-Generated Emotes add a new layer of immersion to your games. They allow players to express emotions and interact with the virtual world in a more meaningful way, enhancing their overall experience and emotional connection to the game.

3. Cutting-Edge AI Technology: Kinetix's User-Generated Emote technology is driven by state of the art AI, setting it apart from traditional emote systems. Embracing AI technologies early makes your game a precursor!

4. Future-Proof Your Game: As AI continues to advance, Kinetix AI Emote technologies position your game at the forefront of innovation. Embracing AI-driven technologies lays the foundation for integrating AI-generated assets, worlds, and experiences into your gameplay, keeping your game relevant and exciting for years to come.

5. Effortless Integration: The Kinetix SDK is designed with simplicity in mind. Integrating our SDK into your existing game engine is seamless, saving your development team valuable time and resources. It allows you to focus on creating compelling gameplay and content while we handle the complexities of emote management.

6. Monetization Opportunities: UGC can open up new revenue streams for your game. With our technologies, you can offer players the chance to customize their experience and invest in their in-game identities.

7. Continued Support and Updates: By leveraging Kinetix's User-Generated Emote, you gain access to ongoing support and regular updates. We are dedicated to refining our technology and adding new features, ensuring that your game remains at the forefront of the industry.

With the User-Generated Emote feature, players can create a custom emote from any video, directly in the game, and play it on their avatar. All it takes is a phone camera or any video for Kinetix's AI to create and play an animation in-game.

This documentation page will guide you through the exciting possibilities of user-generated emotes and how to integrate them into your gameplay.

What is the User-Generated Emote feature?

User-Generated Emote is a unique UGC feature that enables players to create a custom emote from a video, directly in the game.

The feature relies on state-of-the-art deep learning AI algorithms that turn videos into animations. Thanks to this pioneering technology, players can bring any movement from a video into their game.

Boost your game's engagement as you turn your players into creators and allow for more content generation, self-expression, and customization.

Features included for players

1. Upload any video

Players can upload videos from different sources:

• Upload from device: it lets players pick a video stored locally on their device.

• Upload from the Internet: it lets players upload videos by simply pasting an URL link, allowing players to create an emote from a TikTok or Instagram video for instance.

• Self-record: players can directly use their phone camera to film themselves as they perform the movement they would like to turn into an emote.

2. Video trimming & cropping

Players can easily select the exact moment containing the movement they would like to turn into an emote. Likewise, they can crop their video to remove elements that could mislead the AI.

Video cropping and trimming allow players to optimize their video without requiring any video editing skills or software.

How does the User-Generated Emote feature work?

Since its inception, Kinetix has always thrived to democratize access to 3D animation creation with AI. To do so, the team has designed state of the art machine learning models that are seamlessly integrated into game-oriented product bricks to make emote creation as easy as recording a video.

The overall User-Generated Emote process can be synthetized in 4 main steps:

1. Computer vision - Motion Detection

Kinetix AI will analyze the video sent by the player, identify the human that's appearing, and read their movement, in 3D (recreating depth).

Learn more about Kinetix's video-to-animation AI technology:

2. Video Moderation

Kinetix designed a Moderation layer to prevent players from adding unwanted content in the game. It consists in analyzing the video sent by the player, and removing any prohibited movement detected.

Let Kinetix take care of moderation

Kinetix has designed a robust and comprehensive moderation system to review and approve user-generated emotes while keeping you in the loop.

3. 3D animation file generation

Once the video has been approved by the moderation, the movement extraction begins. Our AI will create an animation file containing the exact same movement as the one detected in the video. The 3D animation file will be created on Kinetix's standard avatar.

4. Retargeting

Since players want to play their custom custom on their avatar, Kinetix AI needs to apply the generated animation to the player's avatar.

At Kinetix, we developed our own proprietary retargeting technology to guarantees full compatibility with any humanoid character.

Our automated retargeting not only adapts to the avatar's skeleton like any other traditional retargeting method, but also considers the character's mesh, which allows our AI to adapt the adjust the generated emote based on the avatar's specific body shape.

Learn more about Kinetix's retargeting technology:

Boost your game's engagement with the User-Generated Emote feature

Personalized Gaming Experience

• Bring Your World into the Game: Players can infuse their in-game experience with a personal touch by creating emotes from content they love and are passionate about. This direct control over in-game expressions allows players to integrate their personal interests, favorite moments, or trending content into their gaming world, making every emote a personalized statement.

• Creative Freedom: Offering players the ability to craft emotes from their chosen content empowers them with unparalleled creative freedom. This not only enhances their attachment to the game but also encourages a sense of ownership and pride in their in-game representation.

Enhanced Player Connection

• Foster Community Interaction: Custom emotes become a new medium for players to communicate, share, and celebrate with each other, strengthening the game's community and social bonds.

• Encourage Continuous Play: The novelty and enjoyment of creating and using personal emotes motivate players to return regularly, maintaining high levels of engagement and long-term interest in the game.

• Community Co-Creation: This feature fosters a collaborative environment where players contribute to the game’s evolving culture. Shared creativity leads to a dynamic and ever-changing community, bonded over shared interests and collective content creation.

Elevated Fun Factor

• Transformative Entertainment: The fun of translating real-life movements into game-specific emotes adds a layer of entertainment that’s both engaging and immersive, keeping players invested in the game.

• Viral Potential: Players are likely to share their innovative emotes on social platforms, drawing attention to the game and attracting new players who want to partake in the fun and creativity.

Embrace the full potential of User-Generated Content

Unlock Emote Creation Tools

You can now integrate as many User-Generated Emotes as you want without any downside on your games performances nor player immersion.

Emote Integration in Gameplay

Integrate User-Generated Emotes seamlessly into your gameplay experience, allowing gamers to create, own and use their custom emotes for any interactive or social activities.

Foster a vibrant community around user-generated emotes

Enrich your gameplay with User-Generated Emotes features and Implement features such as emote sharing, liking, and commenting to encourage social interaction and engagement.

What is rarer than a unique asset?

Make User-Generated Emotes an exclusive asset and monetize accordingly. We strongly recommend to consider UGE as the top tier in terms of Emote rarity.

User-Generated Emote pricing model

The pricing associated with Kinetix User-Generated Emote pipes is per API call. We charge with a fixed price every time our APIs are called, which corresponds to every emote generated by a player.

A standard pricing of €0.10 per emote generated by your players will be applied.

How to integrate the User-Generated Emote feature?

There are 2 ways to unlock User-Generated Emote features in your game:

• With Kinetix SDK: all-in-one solution that includes the access to the feature, in addition to multiple technologies to optimize the integration and run phase experiences. Kinetix SDK

• With the Emote Creation API endpoints: access the emote creation feature only. You have to create your own infrastructure around the feature to support it. Kinetix API

Conclusion

User-Generated Emotes unlock a world of creativity and self-expression for games and their players, elevating the gaming experience to new heights. By embracing user-generated content, you can create an inclusive and dynamic gaming community that thrives on player participation and engagement.

To discover how you can try the User-Generate Emote feature on your custom avatar, you can follow this video guide:

Learn how to try the User-Generated Emote feature

STEP 3: in the "Try UGC Emote" section of the Developer Portal, click on "create an animation", and then on "From a video".

Once your video has been sent, our AI will need a couple of minutes to process the User-Generated emote.

STEP 4: preview your user-generated emote on your custom avatar!

What is retargeting?

Emote retargeting is a powerful technique in the world of 3D animation. It allows you to transfer an animation from one avatar to another, regardless of their size, proportions, or skeletal structure. Instead of creating a unique animation for each character, emote retargeting streamlines the process by adapting a single emote to fit various avatars.

Kinetix's Proprietary Retargeting Algorithm

Each character in a 3D animation typically has a skeletal structure with bones and joints. All the Emotes from the Kinetix Emote Library and User-Generated Emotes are created for a specific and standard character with a standard skeleton.

Kinetix's proprietary algorithm is able to transfer the motion from Kinetix's standard avatar to the target's avatar, almost fully automatically. The algorithm will proactively identify the correspondence between the bones of the target skeleton and those of the source skeleton, and the user will then need to confirm and correct errors if necessary. Once the Bone Mapping is done, Kinetix's Retargeting Algorithm will adapt the motion to the target avatar's bones proportions to ensure that the movement is smooth and great.

Here is what the Kinetix Standard Skeleton looks like:

Mandatory bones:

Additional Bones

Hand bones

Bone Mapping

Bone Mapping is a crucial, semi-automated part of the retargeting process. It allows our AI to understand the target's skeletal structure and adapt the motion from the Kinetix Standard Avatar to the target avatar.

In a matter of seconds, our algorithm presents the results of this mapping, allowing the user to review and fine-tune the automatic bone mapping. This empowers users to correct any incorrectly assigned bones and map any unassigned bones, resulting in precise skeletal adaptation for animation, thereby saving time and ensuring animation accuracy.

Contact-Aware Retargeting

When target avatars feature intricate designs, such as unusual bone structures, very particular bone lengths, complex morphologies, or incorporate wearables, it can potentially lead to issues like inter-penetrations that diminish the overall output quality.

To address this concern, Kinetix has devised a unique retargeting method that factors in the character's mesh, proactively anticipating and mitigating contact errors. In this process, the algorithm strategically positions 45 key points among the character's vertices, effectively capturing the character's comprehensive morphology.

After correctly positioning the key points, our algorithm takes charge of the retargeting process. As usual, it seamlessly transfers the motion from the source's joints and bones to the target's skeletal structure, but this time, the motion is optimized while adhering to the constraints set by the key points.

Here is an output sample that highlights the value of the Contact-Aware retargeting:

Integrate Kinetix Unity SDK seamlessly!

For a quick integration, you can follow an accelerated flow:

1. Set-up Unity & Import the SDK

2. Quickstart - Unity SDK

User-Generated Emote sample scene - for tests only

Try the Kinetix emote creation experience

To let you create user-generated emotes in your game for testing purposes, Kinetix developed a sample scene that is an optional part of the SDK. It will guide you through the whole emote creation process so you can get started with the User-Generated Emote feature integration.

By clicking on the "Let's create an emote!" button, you will access an adapted version of the Progressive Web App (PWA) your players will use to create emotes. It the test version you have access to, you can only upload videos from your desktop (while players can also record themselves or paste a video link from the Internet).

Follow the flow, and upload a video to create an emote.

Summary

1. API & SDK: functionalities comparison

2. Additional work that remains once you've integrated the API or the SDK

3. Pre-requisites to integrate one or the other

4. Next steps to integrate

1. API VS SDK: functionalities comparison

With who Kinetix servers communicate:

With the SDK, the request (for exemple the video to process) will come directly from the end user.

For the API, Kinetix will receive these requests from your game server that will take them from the users.

Having the SDK directly communicating with players' clients offers strong benefits:

1. Reduced Latency: Direct communication between the Kinetix servers and the players' clients can significantly reduce latency. Without a game server acting as an intermediary, data travels a shorter path, resulting in faster response times and a smoother user experience, especially crucial for real-time interactions and animations.

2. Lower Server Load: By bypassing the game server, you reduce its workload. This can lead to lower operating costs, as the game server has fewer data processing tasks, potentially decreasing the need for extensive infrastructure and reducing the likelihood of bottlenecks during peak usage times.

3. Simplified Architecture: Direct SDK integration simplifies the overall system architecture. Fewer components in the communication chain mean there are fewer points of failure, which can enhance system reliability and ease the maintenance and troubleshooting processes.

The AI runs on Kinetix's servers

In both cases, the compute of the video is run on Kinetix's servers, never on your servers nor the end-user's computer.

Optimized Retargeting Technology

By using the SDK, you also have access to a real time retargeting that can be used to play an emote that is not known yet by our system.

Video Uploading Management

If you integrate the SDK, you will be able to access our pre-built mobile Progressive Web App (PWA). Players can access the PWA on their phone directly from the game by scanning a QR code. The PWA allows players to upload the videos they want to turn into an emote and send it to Kinetix. You can modify the UI/UX of the PWA as you want.

If you integrate our technology with the API REST method, you will have to build the PWA on your own.

Management of Emote Networking

Once the video has been transformed into an emote thanks to our technology, it is vital to send that emote in real time to every other users that will need to see it. This is taken care of in an optimized flow in our SDK, but that will need to be re implemented if you choose to integrate the API.

The networking system included with the SDK includes:

1. RAM optimizations:

   1. Cloud storage: Since players cannot store ALL the user-generated emotes played by all the players, emotes but be stored on a cloud server.

   2. Emote streaming: emotes are streamed into the game, not to overload players' RAM when a lot of user-generated emotes are played simultaneously. Emote streaming consists in dividing the emote files in small chunks, to allow players to download emotes chunk-by-chunk, instead of downloading multiple entire emote files. Downloading small chunks of multiple different emotes allow players to see multiple emotes simultaneously without any lag.

2. Server & player clients transfers optimizations:

   1. Send only the files players need: sending EVERY generated emote file to EVERY player would result in unreasonable amounts of data transferred. The SDK only sends the emote files when the players need it. For instance, when a player plays a user-generated emote, only him and the players nearby will stream-download the required emote file.

   2. Smart caching: our SDK includes optimizations to allow players to store on their RAM some emotes that are quite likely to be played again, in order to avoid multiple transfers of the same emote file in a short range of time.

3. In-game experience optimization: emote streaming and smart caching enable players to create, play, and see multiple user-generated emotes simultaneously. Without these two combined bricks, it's very unlikely that players would not experience lags and delays when a very high number of emotes are played in the same time in a local room (unless there's a way to pre-load emote files in your game).

If you choose to integrate the User-Generated Emote feature using the Kinetix API method, it is likely that you have to replicate some parts of the SDK's networking system, if not all of it.

Emote Moderation Management System

Kinetix helps you take care of the moderation of User-Generated Content added to your game.

Kinetix designed two moderation layers to ensure optimal safety for your game:

1. AI Moderation: the videos sent by the players are analyzed with AI. Any unwanted movement will be removed when the emote file is generated.

Integrating the User-Generated Emote feature with the SDK or the API will both let you access the UGC Management platform we have designed on the Developer Portal.

Only games that integrated the Kinetix SDK can access the AI Moderation for now.

2. Additional work that remains once you've integrated the API or the SDK

The SDK does not require any additional work from you for the User-Generated Emote to work in your game: the SDK is a complete infrastructure with prebuilt solutions. You may still want to customize the UI/UX, obviously.

After you've integrated the API, you still have to handle:

1. The system to allow the user to upload the video to be turned into an emote Video Uploading Management

2. The network system that will allow everyone in the room to see the emote in real time Management of Emote Networking

3. Pre-requisites to integrate Kinetix's User-Generated Emote Feature with the SDK or API

4. Next steps to integrate the SDK or the API

Follow the flow from these sections to integrate the SDK or API.

Unsure about the most relevant integration method? Let's chat!

1. Open your Unity project and open the Unity Package Manager Window on "Window > Package Manager".

1. In the Package Manager window click on the " + " icon in the top left corner and select Add Package From Git URL.

Paste in this URL and click "Add":

https://github.com/git-kinetix/kinetix-unity-sdk-packagemanager.git

1. In the Kinetix window, click on Package Manager.

1. Click on "Install Core Bundle"

The Kinetix SDK is a powerful cloud-based toolkit that lets you leverage our advanced User-Generated Emote technologies easily.

The SDK offers a complete infrastructure to integrate and manage the User-Generated Emote feature in your game or app

The User-Generated Emote feature

The Kinetix User-Generated Emote feature allows your players to create, share, and socialize with their own unique emotes whatever the game environment, adding a personalized touch to your gaming experience. Learn more about the User-Generated Emote feature.

The Kinetix SDK has been built around the User-Generated Emote feature, with simplicity in mind. The purpose of the SDK is to provide all the technologies, features, and scripts to integrate seamlessly the User-Generated Emote into any game.

The SDK's Core Package

The SDK's Core Package includes all the mandatory technologies and features a game needs to implement and leverage the User-Generated Emote feature at its fullest potential.

Flexible & easy integration

Kinetix has developed its SDK to be easily plugged with any avatar-based 3D project:

• No external server stream dependency. We share data packages your netcode can interact with. No need to be always connected with Kinetix's servers (learn more: Kinanim (Emote streaming)).

• It works with all animator systems. We generate animation at runtime for any avatars. You can play it by default as we override the animator’s system or plug it to your custom system (learn more in Animation System - Unity & Animation System - UE).

• The SDK matches with your design. Every modules' UI are provided as examples, you can customize them and create yours from scratch.

A nimble technical interface

Kinetix SDK is designed to seamlessly interface with any existing game infrastructure without interfering with Client or Game tech capabilities:

• Kinetix servers are handling most of the effort with content creation, storage, management, and optimization.

• Clients only carry the SDK - Core Package modules, essential to the SDK.

• Game server only handles the relay of serialized pose.

Ready to integrate?

Before you start

Dependencies

To import dependencies, access the Unity Package Manager (Window -> Package Manager), click the "+" button, and select Add Package from git URL.

Here are the dependencies to import (copy and past the links) :

• Newtonsoft (Compatible with any version) [com.unity.nuget.newtonsoft-json]

• Input System

  [com.unity.inputsystem]

While adding the Input System package, you may have a warning popup, select Yes.

As our UI uses new Input system, verify that your "Active Input Handling" is set to "Input System Package (New)" or "Both" in "Edit/Project Settings/Player/Other Settings/Configuration"

Account Implementation

Connecting your user

Each time an user logs in with their email or username in your game, call:

Disconnecting your user

And each time an user logs out, call:

Fetching your user's emotes

You can retrieve manually the user's emotes through the API of the core package via this method :

Watch this video tutorial to go through the first steps with Kinetix SDK

This video follows the flow of this section of the documentation. If you prefer to get started without watching the video, you can follow the exact flow of the documentation:

STEP 1:

STEP 2:

STEP 3:

STEP 4:

Kinanim is a groundbreaking file format developed by Kinetix, designed to revolutionize the way emotes are streamed in multiplayer games. It introduces a smart network-based caching mechanism that optimizes bandwidth usage, reduces server load, and enhances the overall gaming experience. With Kinanim, Kinetix can efficiently stream a large quantity of emotes, providing players with a seamless and responsive emote experience. Furthermore, Kinanim incorporates emote retargeting, making it a powerful all-in-one tool.

Key Features

• Network-Based Emote Streaming: Kinanim uses a network-based streaming approach to efficiently deliver emotes from Kinetix's servers to the SDK. It streams emotes in smaller, sequential portions, reducing the need for simultaneous data transmission of entire emotes.

• Optimized Bandwidth: The network-based approach significantly optimizes bandwidth usage, ensuring that emote streaming is efficient and does not consume excessive data. Also, Kinanim alleviates the server load by selectively sending emote segments as requested, resulting smooth Emote stream for multiple simultaneously played emotes.

• Great Responsiveness: Our system enables close to instantaneous responsiveness in emote streaming, allowing players to use emotes promptly without noticeable delays.

How Kinanim Works

Kinanim operates by streaming User-Generated Emotes from Kinetix's servers to the SDK, which will manage the attribution of the emote to the player's cache. When a player plays an user-generated emote, his client will not directly receive and read an animation file, but will rather receive portions of it through the SDK, including the specific character (avatar), that are transmitted via the network to their device in real time. Also, when a player plays an emote, the surrounding players will also download the played emote retargeted on the player's avatar so that everyone one will see the emote correctly.

In addition to this mechanism, the SDK includes a valuable feature that caches the most recent emotes played by the player. This feature from the SDK saves these emotes in the player's cache, reducing the need for repeated streaming of recently used emotes. By efficiently caching recent emotes, the SDK optimizes the player's experience and conserves both bandwidth and server resources.

This network-based streaming minimizes bandwidth usage and server load, ensuring that the emote is displayed promptly and without interruptions.

Kinanim Benefits

For Games & Digital Worlds:

• No need to hardcode thousands of emotes in the game, which would result in very heavy game files.

• All networking is handled by Kinetix (costs & know-how)

For Players:

• Kinanim ensures that emotes are displayed promptly and without interruptions, enabling players to interact with one another in real time.

• Players can own and play thousands of user-generated emotes with a simple push of a button. They do not necessarily need to select a few emotes to embed in an Emote Wheel as in Fortnite, for example.

Conclusion

With Kinanim, game developers can take their multiplayer gaming experience to the next level by providing efficient and responsive network-based emote streaming with emote retargeting, enabling players to customize their emote experience without affecting the game's performance.

You can initialize the SDK with parameters in the KinetixCoreConfiguration :

• GameAPIKey => Game API Key, obtainable via Dev portal

• PlayAutomaticallyAnimationOnAnimators => true if you want the SDK to handle the animator.

• EnableAnalytics => true if you want to help us and share general informations of how users use our SDK.

• ShowLogs => true if you want to log the behaviour of the SDK for debug purposes.

• EnableUGC => true if you want to enable UGC

• NetworkConfiguration => Customize network configuration

• CachedEmotesNb => Number of emotes cached in storage (0.5 MB / emote)

Our SDK offers functions in order to handle smoothly Unity's Animation system.

Local player:

To register/unregister your local player animator :

To stop animation on local player :

Remote Peers:

To register/unregister remote peer animators :

Using your custom uploaded avatar for Contact-Aware Retargeting

Then, when registering your local player, you can pass the UUID of one of your uploaded avatar, this will allow the SDK to automatically use your custom retargeted emotes.

NPC or other local avatars :

In addition to the local player or remote players, you can also register other avatars (to animate your NPCs or shop avatars for example)

The registering of the avatar returns a string representing the unique Id (UUID) to pass as parameter to the subsequent functions.

You then load the animations with the avatar UUID

And play the animation with

Loading the full emote and getting the duration :

To get the Duration of an emote, you must first get a KinetixClip object. You can easily get it via the code sample

Please note:

• You can have the duration in the OnSuccess callback

• The OnComplete callback ensures the full clip is available

Local Player

To register/unregister your local player avatars:

KinetixCore.Animation.RegisterLocalPlayerCustom(Avatar _Avatar, 
                                                Transform _RootTransform, 
                                                EExportType _ExportType)

KinetixCore.Animation.UnregisterLocalPlayer()

Get callbacks to get informed when an animation is played on local player:

event Action<AnimationIds> KinetixCore.Animation.OnPlayedAnimationLocalPlayer

To retrieve AnimationClip Legacy to and play it in your system for your local player:

KinetixCore.Animation.GetRetargetedAnimationClipLegacyForLocalPlayer(AnimationIds _AnimationIds, 
                                                                     Action<AnimationClip> _OnSuccess,
                                                                     Action _OnFailure = null)

Remote Peers

To register/unregister remote peer avatars:

KinetixCore.Network.RegisterRemotePeerCustom(string _RemotePeerUUID, 
                                             Transform _RootTransform, 
                                             EExportType _ExportType)

KinetixCore.Network.UnregisterRemotePeer(string _RemotePeerUUID)

KinetixCore.Network.UnregisterAllRemotePeers()

Get callbacks to get informed when an animation is played on remote peer:

event Action<string, AnimationIds> KinetixCore.Network.OnPlayedAnimationRemotePeer

To retrieve AnimationClip Legacy to and play it in your system for remote peer:

KinetixCore.Network.GetRetargetedAnimationClipLegacyForRemotePeer(string _RemotePeerUUID, 
                                                                  string _EmoteID,
                                                                  Action<AnimationClip> _OnSuccess, 
                                                                  Action _OnFailure = null)

NPC or other local avatars

In addition to the local player or remote players, you can also register other avatars (to animate your NPCs or shop avatars for example).

The registering of the avatar returns a string representing the unique Id (UUID) to pass as parameter to the subsequent functions.

string PlayerUUID = KinetixCore.Animation.RegisterLocalPlayerCustom(Avatar _Avatar, Transform _RootTransform, EExportType _ExportType)

You then load the animations for the avatar UUID:

KinetixCore.Animation.LoadAvatarAnimation(string _PlayerUUID, string _EmoteID, string _LockId, Action _OnSuccess = null)

And play the animation with:

KinetixCore.Animation.PlayAnimationOnAvatar(string _PlayerUUID, string _EmoteID)

When designing our SDK we focused on developing features that make you save time and allow you to easily integrate the User-Generated Emote feature within your game. Each of the Core Package modules are flexible so they match with any dev environment.

Learn more about the SDK's 3 Core Modules inAnimation system, #smart-networking & Account management.

Animation system

When loading a humanoid avatar, Kinetix's Core Package calls the associated user-generated emotes to handle the retargeting. The process consists of downloading the GLB file of the emotes and retargeting them with the registered avatar to generate Legacy AnimationClips that perfectly fit your avatar(s). The retargeting process is optimized to keep a constant frame rate of your application. Each time a new emote is loaded, it is added in the queue of the retargeting module and perform N actions per frame.

To avoid the conflict between generation of humanoid AnimationClip at runtime and the Animator, we developed a proprietary Animation System composed of "KinetixCharacterComponent", "ClipSampler" and "BlendAnimation" scripts. They are automatically added at the GameObject (Unity) / Actor (Unreal Engine) of your animator when you initialize your SDK.

That way, when a player triggers a user-generated emote in-game, our Core Package samples at runtime the pre-generated AnimationClip. It guarantees to always play high-quality emotes with constant level of performance. Once you have registered your animator, (Unity's animator, Unreal Engine's animator or custom animator) this process becomes seamless.

Emote networking & caching

Adding dynamically and playing multiple emotes simultaneously can be very tricky issue: in the vast majority of games, it's not possible to store thousands or millions of user-generated emotes in player's cache or to hardcode the emotes files, which leads to storing emotes on a cloud.

The concept is to cache the Peers ID by registering them via our SDK. A KinetixCharacterComponent is added to local and remote players. Using a simple implementation, you will be able to get the current pose of the local player and apply it to the remote player.

To ensure real-time communication between clients our solution compresses the volume of messages that are shared by only serializing poses from played emotes. This technique is +200 times more lightweight than sharing emotes files, and allows multiple players to play and see user-generated emotes simultaneously.

Account management

Managing and distributing the right content to the right gamer usually takes a lot of time and implies custom developments.

Kinetix's SDK include an Account management solution to fetch user-generated emotes and their data with players while guaranteeing their data privacy. It facilitates content distribution and monetization for game studios / developers while ensuring that players will easily retrieve their user-generated emote in their inventory in-game in a seamless way.

This Account management module also lets you track user-generated emotes usage metrics and optimize emotes distribution in your game accordingly (all metrics accessible from Developer Portal).

Before you start, check that you have successfully initialized the Core Package of the SDK:

How to link the SDK to my animation system?

How to setup the network and stream user-generated emotes:

How to link user-generated emotes and users in-game:

API Reference Core

How it works

When you play an animation, the event KinetixCore.Animation.OnAnimationStartOnLocalPlayerAnimator will be fired.

It allows you to use a callback to then pass the ID of the emote played to your network layer, and, using a RPC or a Sync Variable, to trigger the animation on the KinetixCharacterComponentRemote (see below).

For more information about the whole strategy, you can visit Kinanim (Emote streaming) and Emote networking & caching.

Network Implementation

On Remote Player Join

Once a remote peer connects in your same room or is already connected in the room, you can register them by giving their peer network IDs as argument, and their animator:

KinetixCore.Network.RegisterRemotePeerAnimator(string _RemotePeerUUID,
                                               Animator _Animator);

Then get and store the automatically added KinetixCharacterComponent

this.kcc = KinetixCore.Network.GetRemoteKCC(Id.ToString());

On Remote Player Leave

Once a remote peer disconnect in your same room, you can unregister them by giving in argument their peer network ID.

KinetixCore.Network.UnregisterRemotePeer(string _RemotePeerUUID);

On Local Player Leave

Once a the local peer disconnect of the room, you can unregister all remote peers.

KinetixCore.Network.UnregisterAllRemotePeers();

KCC Local vs Remote

We differenciate 2 types of KinetixCharacterComponent: Local and Remote. It's today mostly for semantics, as a Remote KCC will have the same capabilities as the Local one, but represents a Remote Peer.

Integration's exemples

Register and onboard your game

Authentication

How to get your Game API key?

There are 2 types of Keys:

• Limited - Free key used for tests and pre-production purposes. You won't be charged anything using that key in a pre-prod environment. Note that with a Limited key, the number of User-Generated Emote you can create is limited, to avoid issues.

1. To request an authentification Game API Key on the Developer Portal, you need to have a game space created. If you do not already have a game space created, head to your dashboard, and create your game space by clicking on "Create a game".

2. Then, click on the "Get API key" button, below "Activate your Game/App API Key.

1. Copy and save your Game API Key. You will have to use it soon!

Next, we'll finalize it to provide you with a complete template for integrating UGE into your game:

Let's review the different steps of the code together!

1. Reminder: Initialize the SDK and register a callback

Before Initializing the SDK (which is an asynchronous process), you can register a callback to call subsequent functions of the SDK:

Then, you can initialize the SDK with the configuration you want:

2. Register your local player

3. Link your game's account system with Kinetix SDK

We will focus on this part of the code:

We use the ConnectAccount method, establishing a link between the user account on your end and within Kinetix's system.

The userID you give to Kinetix must be reused each time you want to connect this specific user, and it must remain exclusive to them.

While it can be a hash or any string, we highly recommend anonymizing user data and refraining from incorporating any personal information.

4. Get ready to use User-Generated Emotes

Let's check the following code (inside the success callback of ConnectAccount)

As you can see, we provide a way to know when a link / token to the Web Application is invalid (either because of the security token expiration or after usage).

5. (After processing) Get the emote of your player

To get your player's emotes, you can call GetUserAnimationMetadatas, which will return an array of AnimationMetadata, containing all the info you will ever need (name, thumbnail, file links, etc...)

As you can see you can get the Id of an emote by accessing the Ids.UUID property of an AnimationMetadata

6. Play the emote

Once we have the id of the emote we want to play (and the local player has been registered), we can just call:

You've done it!

If you use Unity's Animator System :

If you use your own Animation System :

Our SDK allows comprehensive control over user-generated emotes with features such as trimming animations, pausing/resuming, setting elapsed time, modifying play rates, and looping animations. Note that these features are not automatically networked; you must send the relevant information over your network layer and call the appropriate methods using our "Remote implementation."

Trimming animation

Trimming animation allows you to choose a part of the emote to be played, instead of playing the whole one. Remove a part of the start of the animation, at the end of the animation or both simultaneously.

Resume / Pause

Resume and pause the playing of an emote, freezing the player avatar. The SDK will keep its state. (elapsed time, loop mode, play rate, animation timeline, etc...)

Go to Time (Set Elapsed time)

Enables you to go to a certain time of the sampler. The elapsed time is expressed in seconds.

Variable playrate / Play backward

Modify the speed at which frames are read by the SDK. A negative number will make the SDK play the emote backward.

Loop animation

Enable or disable loop mode for a player. When loop mode is enabled, the SDK will loop back to the start / end of the animation (depending on the playrate's sign)

How to let your players create a User-Generated Emote?

1. Using Kinetix Core, get the link to the Web Application that will let your players upload or record their video

1. Once in the web application, gamers can either record or upload a video to create their own Emote.

1. Once the video is uploaded, gamers can trim the portion of video they wish to extract to create their emote

1. After trimming the video, gamers can give an name to their emote, specify its suitability for a mature audience (avoiding any trolling, sexual or hateful behaviors), and consent to the Terms & Conditions.

1. Et voilà!

1. After successfully submitting the Emote, gamers can return to playing the game and wait for the end of the processing of the Emote (average wait time: 5 minutes).

2. The emote will be available automatically when fetching the user's emotes via

If you want to let your gamers create their own Emotes, you can either use the SDK exposed methods or use our Web API endpoints:

The goal here is to propose an implementation of the Root Motion system for controllers using a NavMeshAgent.

In this exemple the player controller is organized as follow:

• PlayerObject (contains the NavMeshAgent and the script VisualAvatar)

  • AvatarObject ("root" -> contains the Animator)

    • Mesh

    • Armature

      • Hips

When used, the root motion will transfer any movement of the Hips to the AvatarObject (the root).

The script VisualAvatar is responsible for updating the position of the PlayerObject from the position of the AvatarObject (see below)

[SerializeField] private Transform m_visual;
[SerializeField] private NavMeshAgent agent;
...

private void LateUpdate()
{
	if (!m_visual) return;

	if (photonView.AmOwner) {
		agent.nextPosition = transform.TransformPoint(m_visual.localPosition);
	}
	
	m_visual.localPosition = new Vector3(0, m_visual.localPosition.y, 0);
}

Please note 3 things here:

• m_visual is the transform of the AvatarObject

• The position update must be in LateUpdate, as the Root Motion system operates in the Update state of Unity

• In a networked environment, we want to set the position of the PlayerObject only if we are the owner of it

Root motion, if enabled, makes the parent object (the one with the Animator component) move instead of the hips of the armature. This allows collision detection in the implementation of the character controller.

Enabling the root motion

Root motion is enabled for a character via the following overloads when registering the players' animators:

And

Configuring the root motion

Example of configuration:

A few options are available :

• ApplyHipsYPos => Sets the root Y position to be the hips Y local position

• ApplyHipsXAndZPos => Sets the root X and Z positions to be the hips X and Z position

• BackToInitialPose => If set to true, the avatar will revert to its initial position when the emote has finished playing

• BakeIntoPoseXZ => If set to true, the emote will be played in place for the X and Z axis, meaning the avatar won't move horizontally during the animation

• BakeIntoPoseY => If set to true, the emote will be played in place for the Y axis, meaning the avatar won't move vertically during the animation

Setup an Avatar Mask

Create an Avatar mask.

Follow these steps in project view : Right Click> Create > Kinetix > AvatarMask

In the inspector, you can click on each individual node to enable (in green) or disable (in red) the bone. Hovering the node will show the name of the bone.

For the Mask to be applied on a character, you need to call the corrisponding SetMask method.

Runtime Avatar Mask Setup

The KinetixMask can be modified in runtime using the method SetEnabled

Example:

Runtime Create Avatar Mask

PhotonConnectManager is a global script attached one time to get information if the local player joined a room and then instantiate its prefab.

Player is a script attached to the player prefab and will register and unregister if its local player or a remote player.

Introduction

Our SDK allows inverse kinematic control when playing a kinetix animation. The IK has multiple features such as :

• Hint (or "Pole Vector"): These are positions like "left knee" or "right elbow" used by the IK to orient the elbow or knee in the correct direction.

• Target: These are, as their name suggest, the target position and rotation for a hand or foot.

• Auto-retargeted hand and foot rotation : Our IK ensure that if you have multiple rigs in your virtual world, the IK rotation will be the same for both characters.

• Adjust Hips : This feature let you lock hands or foots into place by moving the hips

The IK target rotation is described as follow : For the hands, the forward vector points toward the fingers (we average each finger to get a forward direction). The down vector points toward the palm of the hand.

For the feet, the forward vector points toward the toes, parallel to the ground. The down vector points toward the knee. (based on the T-Pose)

IK Event

Registering the OnBeforeIkEffect allows you bind to the Update of the Kinetix Animator before any IK is computed. This allows you to check for collisions or grounding and use IK methods.

Target Position

Hint Position

These are positions like "left knee" or "right elbow" used by the IK to orient the elbow or knee in the correct direction.

Global Target Rotation

Our IK ensure that if you have multiple rigs in your virtual world, the IK rotation will be the same for both characters. The global rotations are described as follow :

IK Weight

Adjust Hips

This feature let you lock hands or foots into place by moving the hips

Validation

Validating an emote will make it retrievable via the user's account (KinetixCore.Metadata.GetUserAnimationMetadatas)

private void OnProcessValidated(SdkApiProcess _Process)
    {
        KinetixCore.Process.ValidateEmoteProcess(_Process.Uuid.ToString(), (_Process) => {
            Debug.Log("Process validated emote will appear in the bag!");
        }, () => {
            Debug.LogError("Unable to validate the emote");
        });
    }

Previewing the emote

Before validating the emote, you may want to show it to the player:

public void PreviewProcessedEmote()
    {
         KinetixCore.Metadata.GetAnimationMetadataByAnimationIds(currentProcess.Emote.ToString(), (_Metadata) => {
            Debug.Log("Got animation metadata: " + _Metadata.Name);

            KinetixCore.Animation.PlayAnimationOnAvatar(previewAvatarID, _Metadata.Ids.UUID);
        }, () => {
            Debug.LogError("Unable to get animation metadata");
        });
    }

Retaking an emote

If a user is unsatisfied with their emote, you can allow them to replace it with a new emote, invalidating the previous process and restarting the flow from the begining with the old process as a parent.

private void OnProcessRejected(SdkApiProcess _Process)
    {
        KinetixCore.Process.RetakeEmoteProcess(_Process.Uuid.ToString(), (_RetakeUGCUrl) => {
            // Open the PWA with the retake Url
            Application.OpenURL(_RetakeUGCUrl);
        }, () => {
            Debug.LogError("Unable to retake the emote");
        });
    }

Opening the PWA

Once installed, you can follow these steps to integrate it and open the Kinetix PWA:

• Grab the prefab "CanvasWebViewPrefab" from the demo scenes of the plugin

• Use the following code to get the PWA url and open it

  Copy

   
      [SerializeField] private CanvasWebViewPrefab webView;
   
      
      public void BeginOpenPwa()
      {
          KinetixCore.UGC.ClearCachedUrl();
  
          // Example on how to get the link and pass it to the callback
          KinetixCore.UGC.GetUgcUrl(OpenPwa);
      }
  
      private void OpenPwa(string _Url)
      {
          StartCoroutine(OpenPwaCoroutine(_Url));
      }
  
      private IEnumerator OpenPwaCoroutine(string _Url)
      {
          webView.gameObject.SetActive(true);
          
          // Note that we await initialization
          yield return webView.WaitUntilInitialized();
  
          Debug.Log("Target url: " + _Url);
          webView.WebView.LoadUrl(_Url);
      }

Detecting end of the flow

Using the close ("X") buttons in the PWA

webView.WebView.CloseRequested += (_Sender, _EventArgs) => {
            Debug.Log("Received close request. Closing web view.");
            Debug.Log("We were on " + webView.WebView.Url);
            
            webView.gameObject.SetActive(false);
            
            // We can fetch the player processes manually if we were on the last url
            // Or just implement a polling (see below)
            if (webView.WebView.Url.Contains("/validation")
            {
                // Fetch processes
            }

};

Via polling

Detecting the end of the flow in the PWA can be done in a variety of ways, but the simplest is to poll the user's processes to detect if a new emote is being processed

    private IEnumerator FetchProcessesAtInterval()
    {
        GetPlayerProcesses();

        while (enabled)
        {
            yield return new WaitForSeconds(currentProcessFetchInterval);
            
            GetPlayerProcesses();
        }
    }

To detect a new valid Process, you can check the length of the processes after filtering them thanks to the property CanBeValidatedOrRejected

 
    private List<SdkApiProcess> GetFilteredProcesses(SdkApiProcess[] _Processes)
    {
        List<SdkApiProcess> filteredProcesses = new List<SdkApiProcess>();
        
        foreach (SdkApiProcess process in _Processes)
        {
            if (!process.CanBeValidatedOrRejected)
                continue;
                
            if (process.Step == "render_failed")
                continue;
            
            if (process.PostMLError == true)
                continue;
            
            filteredProcesses.Add(process);
        }

        return filteredProcesses;
    }

        
        

    private void RefreshProcessList(SdkApiProcess[] _Processes)
    {
        
        List<SdkApiProcess> filteredProcesses = GetFilteredProcesses(_Processes);

        int previousProcessCount = emoteProcessButtons.Count;
        
        if (filteredProcesses.Count > emoteProcessButtons.Count)
            OnNewProcessFound?.Invoke(filteredProcesses[0]); // The API result is ordered from most recent to oldest (DESC)
        
        // Close the PWA OnNewProcessFound
    }

Events

Public Methods