Advertisement
Features

Gaze: the driver-monitoring eye-tracking software which could be in your next car

We find out about latest eye-tracking tech that offers key journey info based on where you look as you drive

We all know that some cars can monitor the driver’s eyes and flash up a warning if they think they’re becoming drowsy or not paying attention. But what if the vehicle used the tech to get a better idea of what we’re looking at through the windscreen?

That’s the idea at the heart of the next generation of in-car systems from Nuance, a specialist in speech recognition whose tech is currently used by more than 60 firms. Called Gaze, the set-up has moved from static concept stage to real-world tests over the past six months, and Nuance’s automotive boss, Arnd Weil, says he’s in talks with car makers about bringing it to market, perhaps as soon as by 2020.

Advertisement - Article continues below

11 car safety systems to be mandatory by 2021

Our own experience of the set-up takes place on roads near Milan, Italy, where a rented Ford Galaxy has been modified to show off the concept. There’s a chunky tablet PC mounted on the dash and, more importantly, a computer gaming-spec eye tracking module taped to the top of the instrument panel.

Advertisement
Advertisement - Article continues below

As principal product manager Daniel Kindermann drives along the shore of Lake Maggiore, it’s clear the car is following his eyes, constantly monitoring where he’s looking. We pass a hotel and Kindermann asks: “What’s the rating on that business?” Within seconds, the system replies with not only the name, but also details on the style of hotel and its customer rating.

The same goes for restaurants; glancing sideways while driving along, Kindermann asks the system for details and gets ratings from review sites, plus a brief description. The next step, we’re told, would be the possibility to then call the business, for example, or to use an online service to reserve a room or table.

Advertisement - Article continues below

There are various systems at work here, of course. Our car has multiple microphones, allowing it to pick out individual occupants and work out who’s talking to it. Boxes of computer hardware fill the boot, much of it made by Nvidia, the graphics chip firm behind many infotainment and instrument displays.

Driverless cars: everything you need to know

Gaze mixes these systems with the eye tracking, uses the processing power to ‘back-track’ through time to the point where the user has glanced at the point of interest, then cross-references this with hi-res mapping and GPS data to work out what they were looking at when they asked the question.

Advertisement
Advertisement - Article continues below

The system will have the capacity to ‘learn’ what its users are viewing. At one point, Kindermann looks out towards a seafront location and asks the car to tell him about a restaurant, but it assumes he’s looking at an island in the distance. It doesn’t take long for him to correct the error, and a cloud server would flag up the area and help the system to second-guess the user more accurately in future.

Advertisement - Article continues below

There are other potential uses. Gaze could notice what you’re looking at inside the car. If a light flashes up on the dashboard, simply asking “Why has that light come on?” could prompt the system to give you safety info. It could also be used by insurance firms to monitor and grade drivers’ attentiveness, and increase or decrease their premiums as a result. 

Nuance is also working on gesture control, building on the tech found in BMW’s latest 7 and 5 Series models. Nils Lenke, Nuance’s director of corporate research, said: “We call it multimodal assistance. 

“You’re trying to accomplish one task but you could be using three ways of doing it: looking briefly at somewhere as you’re passing it, gesturing your hand towards it and at the same time, asking ‘Tell me more about this’. The system will combine these as inputs to give you a more accurate response.”

Weil admits that while the Gaze tech is approaching production standards, it may be a year or two before a brand can design the tracking sensors into a dash. 

In the meantime, Nuance’s R&D team is working on sentiment recognition. A car that not only knows what you’re looking at, but is capable of guessing why? Don’t bet against it. Not least because as Level 4 and 5 autonomy arrives in the next 10 to 15 years, we should have more time to look around us as we’re travelling. Helping us to make the most of it will be big business.

Enjoyed this article? Then why not read our previous feature on judging speed without a speedometer?

Advertisement
Advertisement

Most Popular

New Cupra el-Born electric hot hatch offers 310-mile range
Cupra el-Born

New Cupra el-Born electric hot hatch offers 310-mile range

The sporty Cupra el-Born is the Spanish brand’s first all-electric vehicle – and it has a maximum claimed range of 310 miles
8 Jul 2020
New Polestar 2 2020 review
Polestar 2 Hatchback

New Polestar 2 2020 review

We get behind the wheel of the new Polestar 2 to see if it can rival the Tesla Model 3
7 Jul 2020
New Kia e-Niro 2020 review
Kia e-Niro SUV

New Kia e-Niro 2020 review

All-electric Kia e-Niro mid-sized SUV 
benefits from same updates as hybrid Niro versions
9 Jul 2020