Designed by Tera Nguyen

Hand-gesture interface and

physical therapy games

ROLE

Producer 

DURATION

16 weeks, 2018

TEAM

2 Engineers

3 Designers

CLIENT

CTRL-labs

series A, $67M 

PLATFORM

Electromyography wearable device,

Product ver.2017

TOOLS

Unity3D

Adobe Suite

G Suite

Trello

MY CONTRIBUTION
  • Developed competitive analysis to prioritize features that leverage the strengths of CTRL-labs's 2017 neural-signal device.

  • Created success metrics, organized user testing sessions and conducted 70+ user interviews

  • Built product backlog and optimized weekly feature discovery using Kanban method

  • Wrote documentation on design guidelines and reference artifacts for CTRL-labs's on-boarding developers

  • Collected developers feedback on client's hardware, SDK, and Intention Capture Engine

  • Handled client management and assisted in troubleshooting technical issues with the device

PROJECT SUMMARY

Three physical therapy games that showcase the strengths of client's Machine-Learning wearable device. 

Disclaimer: The device used in this project is an early prototype from 2017, which has no indication of CTRL-labs's current product in the market.

Snowboarding is an adventure game. Users define their own subtle hand or finger movements to steer the snowboard through the gates and avoid the Yetis. 

Cake Master is a cake-making game. Users use different forces to squeeze cream in 3 cake layers. 

Sound Mosaic is a synthesizer and visualizer experience, integrating the device with Kinect. Users make micro hand gestures to add sound filters to the background ambience.

THE CHALLENGES

Early prototype with technical issues

(e.g: high latency)

No documentation

My team of 6 only had 1 Machine-Learning focused programmer

MY PROCESS
[2] Product backlog and tasks prioritization

I created Product backlog (sample below) to drive the design directions for our prototypes.

After spending 1 month to research and building simple demos, we arrived at 3 use cases that showcase the device's strengths: continuous gesture recognition, force detection, and micro-gesture recognition. We spent each month designing an interactive game for each use case, following a 1-week Scrum cycle. The product roadmap was updated weekly after every build and user test. User stories were categorized into sections of priorities and placed into a vertical slice.

 

For this project, game mechanics were not as prioritized as design for usability and giving users a sense of control. 

[3] User interviews and testing

I organized weekly playtests with adult gamers, non-gamers and created success metrics for each. I documented all user feedback in Usability Testing Report 1  Usability Testing Report 2​  and many similar reports (images below). 

Recognizing individual fingers

Walking mechanism

Rotating gestures

Squeezing different forces

[4] Technical design document

Since our team was the first external developers for CTRL-labs and they did not have any documentation on their past experiments, I took it upon myself to write a Technical Design Document (images below). CTRL-labs later told me that they have used this document for their on-boarding engineers.

TAKEAWAYS
[1] What is a "done" task?

Delayed responses from the device made it challenging for me to understand whether my users' negative feedback was caused by our design or by the device's technical constraints. Our team struggled to define a task being "done." We learned to break down our goals into smaller milestones per week and record our progress to show to stakeholders. 

[2] Product management lessons
  • Have a clear product backlog and priorities so the team can focus on what's most important. Revisit it weekly.

  • Visually display thought process and prototyping goals in the project room to keep everyone's vision aligned.

  • Paper storyboarding does not always communicate user interactions well.

  • I learned how to communicate technical concepts, such as machine learning, in a non-technical way that artists, designers, and users would be able to understand.