Smartphones have become smarter. But the ways we interact with them, have not changed much.
Most of the time, we use our smartphones with our fingers. Other ways include gestures, stylus and voice. When we put that to the real world, things can be difficult if we don't have full use of our hands.
We can use our voice, but that is only limited to certain commands on certain apps.
While there are a lot of general-purpose tools for accessibility, like letting users control a cursor with their eyes or a joystick, nothing allows people to navigate sites like eBay or Amazon.
Here, eBay introduces another way for users to interact with their phones: using their head and gaze.
With a project called 'HeadGaze', eBay is showing that internet purchasing can be more accessible to just about anyone who can use a phone. So no hands, no problem.
The technology uses head tracking hardware and 3D sensors in the TrueDepth camera available on the iPhone X to enable hands-free shopping. HeadGaze can interpret different head movements, corresponding them to different actions.
It uses Apple's ARKit framework to create a "virtual stylus" that follows users' head movements. Combining that with an interface designed for shopping, users can just look at the right part of the screen to scroll, add items to their shopping cart and check out.
HeadGaze was created by Muratcan Çiçek, an eBay intern and PhD candidate at University of California, Santa Cruz. He relies on assistive technology every day to read, work and get around.
"As someone with extensive motor impairments, I don't have full control of my limbs," said Cicek in a blog post. "These limitations hinder my ability to perform everyday tasks, like going to the grocery store and shopping independently -- even though I have my own income."
Çiçek and his team wants HeadGaze to help people with motor impairments like him to shop online.
Çiçek shows that eBay users can navigate through categories and products by tilting their head all the way in various directions, or tilt partway down to buy, save or share.
While it's initial design is towards shopping, eBay is betting that developers can apply the technology to other apps and experiences.
Developers for example, can create apps where users can browse recipes without covering their phone with cooking ingredients. There can also be apps that can be interacted with, without touching or speaking. eBay is said to be working on the technology to create an eye-tracking interface that could lead to more complex hands-free gestures and might help people with disabilities to use mobile devices.
HeadGaze is a proof of concept from an eBay, showing that anyone should be able to navigate their devices with very little motor function, and nothing but head movements.
Apple’s front-facing hardware are originally meant for Face ID unlocking feature. But Çiçek prove there is much more than that. His creation isn't revolutionary, as there are apps and services that also allow this. But rarely that those apps are straightforward and extensible like HeadGaze.
With the app having a much broader applications as assistive technology for people with disabilities. and with the possibilities his creation can bring, Çiçek made the HeadGaze project available as an open source on GitHub. The codes are all in Swift and initially made for the iPhone X, but still, it’s a start.