eBay announces HeadGaze, an open-source technology that uses head motion to navigate user interface on iPhone X.
The eBay supported reusable technology library that tracks head movement is available via open source on GitHub.com.
The first of its kind, this technology uses Apple ARKit and the iPhone X camera to track the head motion so that users can navigate hands-free on the iPhone X and subsequent models that which use iPhone X technology.
The eBay team responsible for this new technology built a model that creates a virtual stylus that follows the motion of the user’s head (up, down, side to side), taking the head’s 3D information from ARKit and applying 3D geometry mapping to get the location of the “cursor” on the screen.
In addition to that, the team designed and implemented new user interface widgets that sense and respond to the “cursor” interaction.
Similar to how a mouse navigates the cursor on a desktop, this design allows the user to point to any location on the screen with their head and activate designated “buttons”.
The eBay team further designed the new user interface components that register head gestures and navigate the technology accordingly.
For example, they sense the cursor movement and how long the cursor has been in one spot to trigger a clicking action.
Once activated, the buttons unlock different functions like scrolling the screen, moving between pages and selecting the product you wish to purchase, without touching the screen at all.
The fuse of the head-based control and the new UI widgets empowers the existing iOS app with hands-free interactions. The modular code design allows developers to easily integrate the features into their existing or future apps with a minimum code change.
eBay implemented the technology in the HeadSwipe app (also open-sourced via HeadGaze), as a test experience that allows users to swipe deals on eBay.
Specifically, this app empowers people to browse and buy their perfect items with simple head motions.
By pointing the head toward the up and down buttons, the app scrolls to different daily deals in different categories (e.g. a vertical swipe).
And by pointing the head toward the left and right buttons, the app swipes items one after another horizontally (e.g. a horizontal swipe) like a carousel.
AI for AT (Assistive Technology)
As part of the Partnership on AI, an association of businesses focused on establishing best practices for artificial intelligence systems and educating the public about AI, eBay is dedicated to investigating the profound impact that many expect AI will have on the world.
It is because of HeadGaze’s potential to make a tremendous impact on the lives of many people that eBay chose to offer this tool via open-source. The company wants to encourage developers to build more apps with Assistive Technology (AT) that don’t require screen touch.
While AT helps the disabled to perform some everyday tasks, there is no existing tool that considers their need when shopping online.
And with 39.5 million Americans currently considered physically disabled, eBay saw an opportunity to create a tool that would promote independence.
HeadGaze along with its example app HeadSwipe are now available on GitHub.com.
READ MORE: eBay Upgrades Their Image Search Tool
What do you think about eBay developing this technology? Please use the comments section below or head over to our Facebook Group.