Ottawa company teaching autonomous vehicles to 'see' snow, drive in bad weather

Fahed Hassanat and his team at Ottawa-based Sensor Cortek get behind the wheel of cars covered from bumper to rooftop in sensors in order to solve one of Canada's biggest roadblocks to autonomous vehicle adoption: snow. Hassanat is shown in this undated handout. THE CANADIAN PRESS/HO-Sensor Cortek

When a major snowstorm hits Ottawa, most residents retreat indoors, griping about the weather and the heaping piles of snow they'll soon have to shovel.

But for Fahed Hassanat and his team at Sensor Cortek, a big dumping of snow is cause for excitement.

"The badder, the better," said the Ottawa-based software company's chief operating officer and head of engineering.

"Nowadays you don't get those heavy snowfalls as frequently as before, so whenever they happen, we just rush rush, rush."

Drivers for Sensor Cortek get behind the wheel of a car covered in sensors from bumper to rooftop and hit the road in order to solve one of Canada's biggest roadblocks to autonomous vehicle adoption: snow.

Snow can be hard to distinguish for sensors, which are often obscured and confused by bad weather, to detect, making it even more difficult to train self-driving vehicle software and algorithms.

And it's not just snow that poses a problem.

"You have the snow, you have the rain, you have the fog, you have the dust," said Hassanat.

To teach a car to cope with whatever Mother Nature throws its way, Sensor Cortek outfits vehicles with laser imaging, detection and ranging (Lidar) sensors, radar, cameras and advanced GPS systems.

Lidar sensors emit laser beams and capture the reflections of these beams from the environment to create clouds of points in 3D space that provide information about location of objects in that space. Radar relies on electromagnetic waves to capture information about surroundings.

"(Radar) is a very complex sensor ... but what's so good about the sensor is that it can be covered in mud," Hassanat said. "You can put it any weather condition and they will still operate."

Lidar and cameras are based on having line of sight, making them vulnerable to any obstruction between the sensor and the object it needs to detect, whereas radar doesn't need to "see" an object to detect it.

They generate about 10 gigabytes of data for every minute Sensor Cortek uses them, often at Area X.O, a private, 1,850-acre site in Ottawa with a 16-kilometre track, where equipment ranging from farming machinery and military tanks to emergency vehicles can be tested.

The work to get driverless cars ready to handle any weather is crucial because even the simplest car manoeuvers can be complicated by bad driving conditions and make for "terrible, dangerous scenarios," said William Melek, director of the University of Waterloo's RoboHub, a robotics and automation research hub.

"Think about autonomous cars trying to make right in a snowstorm in a busy intersection ... and you have cyclists, you can hardly see street signs or markings," he said.

"The information that is coming from your sensors or your data is completely unreliable, simply because the software is unable to process occluded images or cluttered image with a lot of noise."

For Sensor Cortek, no two drives are alike because its car encounters different numbers of pedestrians moving in varying ways, travels at changing speeds and finds itself in shifting weather conditions each time.

Sometimes Hassanat returns with a few inches of snow sitting on the sensors. Other times they're bare, but he's encountered winds or blowing snow and debris that affected sight lines.

After a drive, Sensor Cortek uses the captured data to train artificial neural network models, which later can be paired with the sensors to detect road users in all weather conditions and enable vehicles to make safer decisions.

The end goal for Sensor Cortek is creating a deep neural network and AI-based perception systems that can make autonomous vehicles -- and anything else needing sensors of the same nature -- "see" better in all visibility conditions and ultimately, operate safety.

Auto and tech companies developing autonomous cars have encountered difficulty in extreme weather conditions, Google owner Alphabet Inc. admitting its own driverless vehicle project had found snow a struggle as far back as 2015.

"It turns out in Mountain View it doesn't snow," the company's self-driving car project director Chris Urmson reportedly said of the car's California testing location at the annual Automotive News World Congress conference in Detroit that year.

Since then, there's been a steady flow of driverless car projects being abandoned for reasons ranging from cost benefit analyses to safety.

Automakers Ford Motor Co. and Volkswagen AG abandoned Argo AI, their autonomous vehicle company, last October, saying they don't see a path to profitability for the project.

Uber Technologies Inc. sold its self-driving car division, which had staff in Toronto, to autonomous vehicle startup Aurora in 2020, following an incident where one of its test vehicles struck and killed a pedestrian in Arizona.

Melek believes in the short-term the companies that keep at it will develop their cars to the point where some self-driving capabilities will be possible with human intervention, but he expects it to take many more years for a fully-autonomous vehicle to hit the road -- and Canadians to trust it.

"I wish I can have a more optimistic outlook, but I would say in my own personal assessment -- and I could be wrong -- we are maybe 15, 20 years out."

This report by The Canadian Press was first published March 10, 2023.