To Survive the Streets, Self-Driving Cars Must Learn to Think Like Humans

Next time you’re driving down the road or walking down the street, pause to consider how you read your surroundings. How you pay extra attention to the kid kicking a soccer ball around her front lawn and the slightly wobbly, nervous looking cyclist. How you deprioritize the woman striding toward the street, knowing she’s heading for the group of friends waving to her from the sidewalk.

You make these calls by drawing on a lifetime of social and cultural experience so ingrained you hardly need to think about it. But imagine you’re an autonomous car trying to do the same thing, without that accumulated knowledge or the shared humanity that lets you read others’ nuanced behavioral cues. Treating every pedestrian, cyclist, and vehicle as an obstacle to be avoided might keep you from hitting anything, but it could just as easily keep you from getting anywhere.

“We call it the freezing robot problem,” says Anca Dragan, who studies autonomy in UC Berkeley’s electric engineering and computer sciences department. “Anything the car could do is too risky, because there is some worst-case human action that would lead to a collision.”

Expect a thaw. Researchers like Dragan are tackling the challenges of interpreting—and predicting—human behavior to make self-driving cars safer and more efficient, but also more assertive. After all, if every machine screeches to a stop for every unpredictable human, we’ll have soon millions of terrified robots choking the streets.

To prevent the clog, those researchers are leaning on artificial intelligence and the ability to teach driving systems, through modeling and repetitive observation, what behaviors mean what, and how the system should react to them.

TU Delft

That begins with recognizing that people are not, in fact, obstacles. “Unlike, say, a tumbleweed moving along the street under the wind’s effect, people move because they make decisions,” Dragan says. “They want to do something, and they act to achieve it. We’re first looking into inferring what people want based on the actions they’ve been taking so far. So their actions are rational when seen from [that perspective], and would appear irrational when seen from the perspective of other intentions.”

Say a driver in the right lane of the freeway accelerates. The computer knows people should slow down as they approach exits, and can infer this person is likely to continue straight ahead instead of taking that upcoming off ramp. It’s a basic example that makes the point: Once computers can estimate what humans want and how they might achieve it, they can reasonably predict what they’ll do next, and react accordingly.

Machines en Scene

The key, even with machine learning, is to look beyond the individual elements of a scene. “It’s important to make strides there, but it’s only seeing part of what’s going on in a roadway setting,” says Melissa Cefkin, a design anthropologist at Nissan’s Silicon Valley R&D center. “We’re really good as human beings at recognizing certain kinds of behaviors that look one way to a machine, but in our social lens, it’s something else.”

Imagine you’re driving down a city block when you see a man walking toward the curb. The robot driver might calculate his speed and trajectory, determine he’s about to cross the street, and stop to avoid hitting him. But you see he’s holding car keys, and realize he’s stepping into the street to reach the driver’s side door of his parked car. You’ll slow down to be sure, but no need to stop traffic.

“The ways people move through the environment are already culturally and socially encoded,” Cefkin says. “It’s not always people-to-people interactions, but people interacting with things, too.”

Again, that’s a simple example. Cefkin points to what she calls the “multi-agent problem,” in which pedestrians and other drivers react to everyone around them. “If a pedestrian is going to cross in front of me, rather than looking at me they’re just as likely to look out into traffic for a gap,” Cefkin says. “So now I’m trying to figure out whether or not it’s safe to keep going based on what the rest of the traffic is going to do.”

Buying Time

If it seems the world is now headed for some sort of drivers-ed hellscape, don’t worry. Teaching AI-based autonomous systems to navigate the eternal weirdness of the human wilderness is tough, Cefkin says, but hardly impossible. In the Netherlands, where cities buzz with pedestrians and cyclists, researchers are doing the work. Dariu Gavrila, who researches intelligent vehicles at Delft University of Technology, training computers for the challenges like road debris, traffic police, and, things as unusual as someone pushing a cart down the middle of the street. The goal, he says, is to develop a more adaptive driving style for the machines—and thus enhancing social acceptance of the new hardware.

That work means factoring in the context around pedestrian traffic—proximity to curbs, the presence of driveways or public building entrances—and the norms of behavior in these environments. It extends all the way to individual movement, such as a person’s head looking one direction while their torso is pointing in another, and what that might mean. “Recognizing pedestrian intent can be a life saver,” Gavrila says. “We showed in real vehicle demonstration that an autonomous system can react up to one second faster than a human, without introducing false alarms.”

TU Delft

There are practical limits to what the computers can do, though. “This is no Minority Report,” Gavrila says—no one’s telling the future. “Uncertainty in future pedestrian or cyclist position rapidly increases with the prediction horizon, how many seconds in the future we’re trying to model. Basic behavior models already stop being useful after one second. More sophisticated behavior models might give us up to two seconds of predictability.”

Still, that second or two of warning might be all a computerized system needs, since it’s well within the scope of the human response times. But other autonomy experts think we might be setting our machines up to actually overthink every microsecond of driving.

“When you’re essentially trying to predict the future, that’s a massive computational task, and of course it just produces a probabilistic guess,” says Jack Weast, Intel’s chief systems architect for autonomous drive systems. “So rather than throw a supercomputer into every car, we just want to ensure that the car’s never going to hit any of those people anyway. It’s a much more economically scalable way of doing things.”

Getting Aggressive

There’s another wrinkle here. The ideal robocar won’t just comprehend its surroundings, it will understand how it itself changes the scene. Many robotic systems, Dragan says, come with a built-in flaw: Their makers assume the presence of an autonomous car won’t change how other actors move. “An autonomous car’s actions will influence human actions, whether we like it or not,” she says. “Cars need to start accounting for this influence.”

That’s why Dragan and her team have built a system that includes a model of human drivers’ responses to the car. “Our car is no longer ultra-defensive, because it knows it can trigger reactions from people, too,” she says. “Like other vehicles slowing down when our car merges in front of them. We’ve also looked at actively estimating human intentions, again by leveraging the autonomous car’s actions. In that case, our car might slow down gently to see if the person wants to be let in.”

That sort of assertiveness training will likely be key to traffic flow in the future. The key to a working robocar may be giving it not just human-like awareness, but a healthy dose of human-like entitlement.

Tech

Self-Driving Cars Can Soon Cruise This State’s Highways Without Anyone Inside

No human back-up driver would be required.

California regulators on Wednesday unveiled revised rules that would allow self-driving cars to travel the state’s highways without human drivers for the first time as early as next year, a move that won the support of automakers.

The new rules represent a compromise with automotive and technology companies, which had objected to many of the requirements previously proposed by the state.

The California rules could still conflict with proposed federal legislation that would largely bar states from regulating autonomous vehicles.

But they are a boost for automakers who want to be able to deploy vehicles without human controls in California.

More than 40 companies are testing self-driving vehicles in California with human controls, and most automakers have autonomous research centers in the state, which is the largest U.S. auto market.

The new rules are expected to take effect by June 2018, the state said.

Waymo, the self-driving car unit of Google parent company Alphabet, Ford Motor, Tesla, <a href=”http://fortune.com/2017/06/13/apple-tim-cook-car/”>Apple</a&gt;, and General Motors had sought changes in California.

Previous rules had demanded that firms submit safety assessment reports to state regulators and seek new approval for updated vehicles.

Existing rules also require a backup human driver to be in all driverless vehicles.

Wade Newton, a spokesman for trade group the Alliance of Automobile Manufacturers, said Wednesday it appeared that California had recognized that “certain onerous” requirements could delay deployment of self-driving technology.

“We appreciate the (state’s) attempts to streamline requirements consistent with the recently updated federal guidance,” Newton said.

But the Association of Global Automakers, a trade group representing mostly Asian and European automakers, said California did not go far enough.

“A special permit is still required to deploy, creating regulatory uncertainty and raising concerns about the ability of autonomous vehicles to cross state lines,” it said.

Companies would still need a California permit to test or deploy vehicles on state roads.

California would also require automakers and tech firms to record information about autonomous sensors in the 30 seconds before a collision. Vehicles must follow all state laws “except when necessary for the safety of the vehicle’s occupants” or other road users.

Consumer Watchdog criticized the revisions, saying California should stick to its earlier, stricter state requirements.

The group noted local communities could not block testing under the proposal.

Last week, a Senate panel approved a bill aimed at speeding the use of self-driving cars without human controls in the United States, a measure that also bars states from imposing regulatory road blocks.

Get Data Sheet, Fortune’s technology newsletter.

Automakers would be able to win exemptions from safety rules that require human controls if they met certain requirements. States could set rules on registration, licensing, liability, insurance and safety inspections, but not performance standards.

General Motors chief executive Mary Barra told Reuters on Tuesday that the federal legislation “allows us to get this technology on the road,” but declined to say when the automaker might seek approval for exemptions.

Tech

Uber vs Google: And now, the self-driving car war gets nasty

TwitterFacebook

Even the biggest, most powerful companies in the world are at the mercy of a grade-school concept: supply and demand.

That’s how a person like Anthony Levandowski can get paid $ 120 million by Google, leave to start his own company, watch that company get acquired by Uber just months later, and then find himself barred from working on self-driving car technology by a U.S. court.

That goes to show just how important Levandowski—and other engineers like him—truly are to the arms race around self-driving cars. 

“It’s not like [you] can hire any one of the thousands or tens of thousands of people who have a particular certification,” said Nidhi Kalra, who heads the RAND Corporation’s self-driving car policy work. “They’re looking for people who are innovators in the fields of machine learning, deep learning, LIDAR design—and there aren’t a lot of those people out there.” Read more…

More about Google, Uber, Autonomous Cars, Self Driving Cars, and Waymo


All articles