28 Nov

The interactive machine learning project that Gibson/Martelli have been working on with the Goldsmiths University project 4i, finally drops in the Epic marketplace. This open-source free plugin for Unreal Engine enables VR Devs and others to implement gesture interfaces in immersive experiences. We are using InteractML in our live-streaming mocap immersive experience DAZZLE. InteractML was funded by an Epic Megagrant and is entirely open source.
Get the tool here
See docs here
See it in action, with a video feat Sam Swain the tools programmer & Bruno Martelli here!

Bruno began using it to create a ‘semaphore recogniser’ to teach the user the semaphore flag code. This is a good use case and test for the IML system as the gestures needed to form each letter are specific and discrete. This semaphore tester has made it into the project as one of the demo files available here.

“Create machine learning systems using Blueprints. Choose from three machine learning algorithms; Classification, Regression, and Dynamic Timewarp. Build a training set by recording your input parameters, train the model with the accumulated examples, and then use the outputs of the running model to drive any in-engine systems or effects you like.

Teach the machine to recognize your movements and controls, and use it to drive your interactive experiences. 

Potential applications include:
Custom control schemes
Gesture recognition
Fuzzy control
Accessibility tools

Features:
Use machine learning techniques to drive your interactive experiences.
Choose from three algorithms: Classification, Regression, and Dynamic timewarp.
Build machine learning systems using Unreal Blueprints.
Use supervised learning to train the algorithms based on your chosen inputs.
Run the trained models to drive the visuals and systems in your world.
Manage model and training data with custom Asset types and UI.
Models can be trained and run asynchronously for the smooth operation of your application.
Model output can be a simple value or a set of configurable structured values.
Comprehensive demo project available showing off all algorithms and configurations.”

This project is a partnership between Goldsmiths, University of London, The University of Coventry, University of the Arts London, Gibson/Martelli and CodeLiberation, funded by the UKRI and an EPIC MEGAGRANT