Vysor Phone Control for Eye Gaze

As a 14 year old girl attending high school, it could be said that building and maintaining social relationships is one of the most important things in the world. Playing on the same apps, visiting the same websites and sending the same cat-filtered selfies as your friends is all part of the experience of growing up. So it came as no surprise when I was asked by a client if I could help her to get and use a mobile phone. 

This client is a very proficient user of EyeGaze. She uses a Gridpad with Grid 3 for her communication and a Surface Pro with the Tobii PC Eye Mini for her school work, browsing the internet and playing games. The initial conversation concerning her use of a mobile phone centred around using the ‘Grid Phone’ Gridset to allow her to make and receive calls and write text messages. This could also be extended to the use of WhatsApp via WhatsApp Desktop and the corresponding Gridset – whilst both of these solutions would serve a purpose, they didn’t feel advanced enough for her capabilities (and they were missing that cat filter…)

Offering total control over the phone using her most consistent access method felt like the best option, this is where Vysor came in.

Vysor is, according to their own tagline, ‘a window to your phone’. It is a piece of software that runs on any Windows or Mac device that allows you to mirror your phone screen to your computer and interact with it using your normal keyboard, mouse or alternative access method. In this case, we used a Surface Pro and Google Pixel 3. Using the free version of the software means that the phone must be plugged into the Surface in order to be used, given that both devices are mounted either on a wheelchair or a table mount; this didn’t feel too restrictive. If you were to try this with a different configuration that required a little more flexibility in positioning, the paid version is only $40 and allows wireless access. 

As far as the physical setup is concerned, it is almost plug and play once the software is installed on both the phone and the computer. The ‘almost’ element to it refers to the need to enable ‘USB debugging’ on the phone and install ‘ADB drivers’ on the computer. The latter is as straightforward as following the handy link from the Vysor software and doing what you’re told. The former, however, requires a little more searching and fiddling (or it did in my case, anyway!) It is worth mentioning at this point that the phone cable must be plugged directly into a USB port on your device, we found that the USB hub used on her Surface Pro was not able to passthrough sufficient power for both the phone and the EyeGaze camera, a powered USB hub would be needed for that. It worked perfectly plugged directly into the Gridpad.

USB debugging on your phone is the setting that allows your computer to take control of your phone – transferring files, sending and receiving commands etc. it is pretty safe if you are just using your own phone and your own computer but there are a few considerations to be made if you regularly plug your phone in all over the place with this setting enabled. Check out this article for more information if you are at all wary about this bit. 

To enable debugging, you need to find the ‘developer options’ on your Android phone, make yourself into a developer and flick the switch. I’m going to refer back to our friends at ‘How to Geek’ for the step-by-step instructions as they have summed it up quite nicely. Working in ‘developer mode’ has made little or no difference to the use of the phone other than allowing Vysor to take control, if you were to use this setup with someone with an inquisitive streak, it may be worth keeping an eye on as there are loads of things you can turn on and off within the develop settings that might be problematic. 

Once the settings are all in order and the phone is plugged into the device, interaction can begin. For a standard keyboard/mouse user it’s as simple as you’d hope; click, drag, type etc. For our EyeGaze user, an additional interface was required to enable her to access everything she wanted and, more importantly, to do so independently. 

As she is a Grid 3 user, this is the platform used to create the phone controls. Luckily, there was already a fairly substantial computer control Gridset in place that we have tailored over time to give her access to the tools and commands she uses regularly, this just needed some adaptations to work with the phone-specific commands.

The process of creating the commands was fairly simple as interaction with Vysor is possible via keyboard and mouse shortcuts. So, the ‘Home’ button is a single press of the mouse scroll wheel which I captured using the ‘advanced keyboard’ command on Grid 3. The ‘Unlock’ button is a combination of key presses ‘Space > wait 0.8 seconds > type PIN > Enter’ – this will only work if the phone is awake i.e. not a black screen, we had to alter the settings within Android to prevent it from going to sleep but given that it has to be plugged in to the Surface be used means that battery life isn’t a concern. The navigation of the home screens is achieved using the left and right cursor keys and our user’s ‘click’ is the Grid 3 ‘Zoom and Click’ for extra accuracy. 

Given that the Computer Control Gridset sits on top of all other windows when open, the keyboard we would normally use covered half of the phone screen, the solution for this was to split the keyboard into ‘left’ and ‘right’ as shown below – not my favourite fix but a working one for now! Any extended writing is generally done using her communication Gridset then copied across anyway so the slightly faffy half-keyboard is rarely used… 

The outcome of this is that the limits in the use of a phone for someone with such restricted physical access are almost entirely removed. We are yet to come up against something she would like to do with her phone that she is unable to; the simple addition of a mouse ‘drag’ command means Candy Crush is now a firm favourite, video calls with grandparents or friends from school are as commonplace as they should be for a 14 year old and the much-coveted cat filter can be applied to her heart’s content!

Please get in touch if you’d like a copy of the Gridset used here, it has not been uploaded yet as it is constantly being altered but we’re more than happy to share what we have so far if you want to give it a go yourself.