Toyota’s Vision of Autonomous Cars Is Not Exactly Driverless

(Bloomberg Businessweek) -- John Leonard strolls up to a drab one-story garage on the campus of MIT and unlocks the door. The building is remarkable only for the reflective windows, which make it impossible to peek inside. “If you were driving past in a taxi,” Leonard says, “would you think the future of Toyota is being designed here?”

Inside squats a silver Lexus LS 600hL sedan. It’s not just any Lexus—this one is vital to Toyota’s effort to develop driverless vehicles. Leonard, vice president for automated driving research at the almost three-year-old Toyota Research Institute, explains how the Lexus is jury-rigged with radars, video cameras, and lasers that can detect, identify, and react to objects up to 200 meters (656 feet) away—twice as far as a year ago. His charges steer the Toyota-built car around Cambridge, Mass., to capture data that can be used to create digital maps and try to extrapolate how a vehicle might behave without a human at the wheel. “It’s sort of like a science project,” Leonard says. “Stay on the road, don’t hit things, don’t get hit.”

If only it were that simple. Toyota Motor Corp., the world’s most valuable automaker, with a market capitalization of $200 billion, is behind in the race to create the vehicles of a maybe-not-so-distant future. Just four years ago, Akio Toyoda, the company’s president, was saying his company would pursue self-driving vehicles only after one beat a human driver—for instance, him—in a marathon road race. He’s not saying that anymore, because Toyota has too much to lose.

If the company fails to pick up the pace, Toyota could, in one version of the future, face the humiliation of becoming a mere steel-box supplier to upstarts such as Waymo and Baidu. Toyoda himself has singled out tech companies as “our new rivals, with speed many times greater than our own.” He added: “A life-or-death battle has begun in a world of unknowns.”

Alphabet Inc.’s Waymo plans to start a ride-hailing service with driverless taxis in Phoenix as early as this year. General Motors Co. recently drew a $2.25 billion investment from SoftBank Group Corp. for a robo-taxi rollout next year. Zoox Inc. says it will have self-driving cars ready for passengers as soon as 2020. The frenzy is reminiscent of the early 20th century, when hundreds of fledgling carmakers competed until almost all of them had failed or been consolidated.

As for Toyota, it invested almost $4 billion to start its research institute, headquartered in Los Altos, Calif., and established formal research links with MIT, the University of Michigan, and Stanford University. To advance its knowledge in artificial intelligence, robotics, and materials science, it has recruited self-driving stars such as Leonard, whose core expertise is in vehicles that steer themselves underwater, and Gill Pratt, who oversees research at the institute and around the world after years of running Darpa robot and autonomous vehicle challenges for the Department of Defense. In June, Toyota invested $1 billion in Grab Holdings Inc., the Southeast Asian ride-sharing giant; in August it agreed to invest $500 million in a deal with Uber Technologies Inc. to jointly create driverless vehicles. It’s even teaming up with nonautomotive partners to develop fleets of multipurpose driverless vehicles.

A paradox underlies these initiatives: Toyota doesn’t necessarily buy the hype about self-driving vehicles quickly taking control of roads in the U.S. and beyond. Leonard himself isn’t sold. “Taking me from Cambridge to Logan Airport with no driver in any Boston weather or traffic condition—that might not be in my lifetime,” he says. On its website, the research institute describes its goal as to “someday develop a vehicle that is incapable of causing a crash.” It doesn’t specify whether this uncrashable car would be driverless.

Toyota’s Vision of Autonomous Cars Is Not Exactly Driverless

Toyota’s self-driving vision isn’t really about getting rid of drivers. Rather, it’s about using autonomous and related technologies to make cars safer and more user-friendly, chockablock with features that help people stay productive while they remain, for the most part, behind the wheel. Toyoda, who competes regularly in road races and signs off on future vehicle designs only after personally testing them on a track, is betting that consumers’ love affair with the automobile is far from over.

The company certainly has the money to pursue whatever future it believes in—$50 billion in cash as of June 30, more than twice as much as GM. But autonomous driving technology is a distinctly American invention, and Toyota has long been known less as an innovator than as a superb manufacturer that figures out rivals’ inventions and does them better. That approach might not be tenable in the face of a cultural shift that Toyota executives say could be as dramatic as the one it faced in the 1930s, when it went from making weaving looms to building cars. “There’s a business need for us to become more like IT companies before the IT companies become more like us,” says Pratt. “The company is very strong now, so now is the time for us to make sure we are the ones who actually figure out all the cool stuff.”

Toyota’s Vision of Autonomous Cars Is Not Exactly Driverless

Strobe lights pulsed and music blared as a beaming Toyoda took the stage at CES in Las Vegas earlier this year. “It’s a great honor for a car guy like me to be here,” said Toyoda, whose grandfather led the company’s conversion to carmaker eight decades ago. Soon he was joined onstage by executives from, Uber, Pizza Hut, Mazda, and the Chinese ride-hailing company Didi Chuxing.

The six companies are unlikely partners in a product Toyoda said will be ready for the 2020 Tokyo Olympics and Paralympics: a bubble-shaped electric delivery van that, notwithstanding Toyota’s ambivalence about full autonomy, will be driverless. A fleet of “e-Palette business system” vans will be designed to shuttle people, deliver pizzas and packages, process credit cards, and convert on a dime into mobile shoe stores, hotel rooms, or emergency command centers. “Today, you have to travel to the store,” Toyoda said. “In the future, with e-Palette, the store will come to you.”

It was the splashiest public manifestation of a shift in thinking he’d had in 2014. By then, pre-Alphabet, pre-Waymo Google had hybrids fitted with self-driving gear zipping along California highways without anyone’s hands on the wheel—including some Prius and Lexus cars that Google had deployed without Toyota’s involvement.

Then Toyota joined early planning for the 2020 Olympics in Tokyo. Toyoda told Automotive News that he was surprised to learn that Paralympic athletes wanted to drive “cool cars, not just cars designed for handicapped people.” Autonomous and semiautonomous vehicles, he realized, could make getting around easier and more fun for the disabled, the elderly, and others for whom driving poses challenges.

He turned to Shigeki Tomoyama, a high-ranking company executive in Tokyo and longtime confidant whom Toyoda calls his “shogun.” Tomoyama is a devotee of kaizen, the core Toyota philosophy of making small, continual improvements to the manufacturing process in search of better quality and lower costs. He’s embraced tech enough that Jeff Liker, a University of Michigan professor and author of several books about Toyota, calls Tomoyama “Frankenstein”—half-Toyota, half-Silicon Valley. That probably overstates it. But Tomoyama set up Toyota’s first project for connecting cars to the internet almost two decades ago and says he’s flattered by the Frankenstein comparison because the monster was “an artificial construction but has a heart somewhere.” Virtually alone among Toyota executives, he posts almost daily on social media—pictures of meals he’s eaten, factories he’s visited, and fish he’s caught. His portfolio at the company ranges from big data to motor sports to developing a Lexus yacht.

Within months of being dispatched to ramp up Toyota’s driverless and driver-assistance endeavors, Tomoyama had lured Pratt from the U.S. Defense Department and launched the Toyota Research Institute. A year after founding TRI, he started Toyota Connected, a U.S. subsidiary that tries to turn the flood of data from Toyota’s web-connected cars into salable goods and services. Around the same time, Tomoyama started talking with Uber about the collaboration that became e-Palette.

The company hasn’t said where or when it will sell e-Palette vans beyond their Olympic debut. Perhaps that’s no surprise given Toyota’s ambivalence about the central technology. “Ninety percent of the consumers still do not expect autonomous driving,” says Masahiro Nakashima, chief operating officer at Toyota Connected. A Gartner Inc. study last year found that 55 percent of consumers surveyed wouldn’t ride in a fully autonomous car, but more than 70 percent would ride in one that was partially autonomous—some of the very customers Toyota is targeting.

Toyota’s Vision of Autonomous Cars Is Not Exactly Driverless

On a chilly spring morning, a four-wheeled vehicle the size of a small dog buzzes along a sidewalk on the MIT campus, dodging backpack-lugging students. Atop its flat chassis is a silver canister housing rotating lidar sensors that detect trees, utility poles, and students. A horizontal array of 3D stereo cameras sits where the little car’s windshield might be if it had one.

The TRI minicar steers itself while its gadgetry creates a high-resolution map of its surroundings. Stephen McGill, an institute scientist, trails behind with a joystick he uses in case the car threatens to veer into a pedestrian. Passing students barely notice, as if it’s routine to have miniature self-driving prototypes buzzing around campus.

The vehicle’s software is working to comprehend unpredictable human behavior based, for instance, on posture and where people may be looking. McGill will upload data about the car’s interactions with the environment, then fine-tune the software so it can propel vehicles more safely through pedestrians and across streets. In the future, this data will be used by bigger cars such as that experimental Lexus in the Cambridge garage.

Today, the minicar winds up in the classroom of McGill’s boss, Leonard, who’s teaching business management majors about electronics. A Mike Myers lookalike who spends 20 percent of his time working for MIT and the rest for the research institute, Leonard offers a short tutorial on how the small car works, then polls the students about the future of fully autonomous vehicles. Three-quarters expect to be using them regularly by 2030. Leonard thinks they might be disappointed.

It’s one thing to set driverless vehicles loose on closed-loop tracks, another to have them navigate cities, as Uber learned in Arizona in March, when one of its autonomous cars struck and killed a pedestrian. Busy metropolises present obstacles at once mundane and vexing. Clouds of tailpipe exhaust can appear as solid as an oncoming car during Boston winters. A FedEx van blocking half of a two-lane side street in Chicago could baffle a vehicle trained not to cross a center line. “This is really, really hard,” Leonard says.

Toyota is seeking a middle ground with a system it calls Guardian, which would harness the machine-intelligence and sensor capabilities that make full self-driving theoretically possible and bundle them in vehicles designed for human drivers. These cars and trucks would be able to see much farther ahead and behind, across multiple lanes of traffic, than any human would, and would be more adept at anticipating the behavior of other cars and pedestrians. Actual people would continue to steer and brake, but when Guardian detected potential danger, it would assume control and swerve, slow, stop, or otherwise act to avoid the problem.

A video produced by the institute shows Ryan Eustice, TRI’s senior vice president for automated driving, at the wheel of a Guardian car on a test track. Eustice lets the vehicle drift as he pretends to fall asleep. A dashboard camera spies his head drooping and eyes closing. In an instant, Guardian takes over and moves the car back into its lane. When Eustice perks up, he’s asked to tap the brake to resume driving.

This is what excites Leonard, not least because his teenage son recently learned to drive. “An order-of-magnitude reduction in fatalities should be technically possible,” he says. Instead of 40,000 traffic deaths a year, the U.S. could have 4,000. “Imagine if you had the most vigilant and capably trained driver in the world that could take over in a situation where a teenager took a curve too fast. Ten years from now, I dream Toyota gets letters from car owners saying, ‘My teenager was driving and did something stupid, and Guardian intervened. Thank you.’ ”

Toyota’s Vision of Autonomous Cars Is Not Exactly Driverless

Zack Hicks is one Toyota executive who says driverless technologies are certainly coming—but he adds they aren’t really the point. “It’s not the differentiator when everybody has it,” says Hicks, chief executive officer of Toyota Connected North America, based in a cavernous office in Plano, Texas. “The differentiator is the use of data, how you’re connecting the vehicle with commerce and other vehicles.”

Hicks and his young crew of programmers in the Connected skunkworks are attempting to create a supersmart web browser on wheels. They can upload data from 500 sensors on each of Toyota’s millions of internet-connected cars every 200 milliseconds—for a total of more than 7 million points of data per day from each vehicle. The sensors measure speed, location, brake pressure, steering angle, fuel levels, tire pressure, and hundreds of other metrics that indicate, for instance, whether the driver tends to brake harder than most.

Toyota’s Vision of Autonomous Cars Is Not Exactly Driverless

The information could help Toyota sell cars by making them safer and enhancing customer experiences. Your Camry notices you forgot your cellphone and asks if you want to retrieve it. You’re late for a meeting, and your RAV4 arranges for a Skype connection (after you park, of course). You get a cellphone alert that the left rear tire on your Corolla needs air, along with a map to the nearest service station. Toyota is betting that such features will keep consumers buying.

The company is seeking ways to turn data into cash. In March, Toyota sold 10,000 web-linked vehicles designed for Avis Budget Group Inc. Data on fuel levels and odometer readings save the car rental company money by expediting customer returns: Instead of having Avis employees walk parking lots manually scanning vehicles, Connected’s system can take inventory in an instant. Diagnostic codes regularly zapped to Avis could identify “a $4 repair that prohibits a $40 repair from being needed,” says Jeff Kaelin, Avis’s vice president for customer experience and product development. Toyota collects a monthly fee in addition to what it makes from selling Avis the cars.

In Honolulu, Connected has teamed with Servco Pacific Inc., the owner of Toyota, Lexus, and Subaru dealerships throughout the Hawaiian Islands, to test a car-sharing business called Hui (Hawaiian for “group”). Servco supplies vehicles, while Connected provides apps that let customers unlock cars with their mobile phones. User data, of course, gets sent to Connected. Starting at $9.95 an hour or $79.60 a day, customers can grab a car at one of 25 parking stations with a credit card and smartphone. Servco pays Toyota a monthly fee.

Toyota’s Vision of Autonomous Cars Is Not Exactly Driverless

These are small-bore businesses that will grow, or not, depending on how enthusiastically consumers gravitate from private car ownership. Every major automaker is working on similar technology, which makes up what the industry is calling mobility-as-a-service. A shift to the mobility business “could replace the industry’s traditional emphasis on ‘moving metal’ with new schemes to capture greater profits per mile or per trip,” McKinsey & Co. said in a report last October. McKinsey has predicted that the global mobility market could amount to as much as $750 billion by 2030.

Toyota could establish a lucrative place in this new world with Guardian technologies applied not just to human-driven vehicles but also to robo-taxi fleets operated by Toyota itself and partners such as Uber. With several years of driverless testing complete and a newly declared willingness to share technology, Uber could significantly boost Toyota’s efforts. Tomoyama goes so far as to boast that having Uber’s self-driving technology “will probably make for the world’s safest self-driving car.”

First, though, Toyota must convince not only itself but its traditional suppliers and dealers that simply moving metal will no longer suffice. “We aren’t giving up on our main business, but we can’t vaguely just continue to make cars,” Tomoyama says. “For everyone to share this sense of crisis is still a challenge.” —With Nao Sano, Kevin Buckland, Eric Newcomer, and Craig Trudell

To contact the editor responsible for this story: Daniel Ferrara at, Jeremy Keehn

©2018 Bloomberg L.P.