Wearables provide a convenient way for receiving notifications when you cannot comfortably use a computer or mobile device. However, in some situations you might also need to input short messages without using voice commands. In this work, we explored some of the possibilities offered by the Thalmics’s Myo armband for menu selection and symbolic input. The Myo bracelet measures muscle activity through 8 electromyography sensors and arm spatial orientation by using a combination of three-axis gyroscope, magnetometer and accelerometer. The official API can discern 5 unique hand poses that are fairly invariant with respect to the arm position.
We have built our design around the two hand poses that could be joined continuously during selection: wave-in and wave-out. These two also happened to have less false positives during recognition, which allowed us to try more complex ideas. In the final design we combined orientation and pose information to expand the vocabulary of gestures that can be used for input in our design. The Myo rest position was used as a selection delimiter for the gestures (commit current selection).
The menu layout arranges items in 8 positions around a circle. Four of them can be directly selected by combination of wrist rotation plus wave pose (either one). The remaining 4 can be selected by just continuing the gesture after the initial selection. With the left or right menu items selected you can rotate clockwise or counter-clockwise to select an additional item before committing (coming to rest position).
We chose to assign 4 letters to each one of the 8 initial menu items. So the second level has 4 items with only a letter in each. Since erasing actions are less frequent than selecting letters, we were able to use the Fist pose to cancel input at the second level or erase a already entered letter. Using a circular layout for the menu allowed us to map the possible positions the hand can assume while rotating the arm with a wave-in or wave-out gesture. This mapping works by reinforcing the visual memory with the motor memory.
To test the design we have implemented a simple game where the user need correctly type words to win. One of the biggest challenges we observed was that Myo is still very sensitive to initial calibration. Obtaining a good calibration require some practice and a bad calibration affects the overall recognition of hand poses. The current firmware is also slow in detecting the rest position, which is sometimes a limiter for speed. Overall, within the aforementioned capabilities and limitations, we believe that design presented can be used as a basis for future techniques looking into hand/arm mobile input.