Baidu’s new AI simulator gives robotaxis a crash course in rare road hazards

For autonomous vehicles, the hardest lessons to learn are the ones that can test the skills of even the most experienced human drivers: an errant cyclist darting through a red light, or an animal bolting into oncoming traffic from behind a corner.

The Chinese tech giant Baidu has developed a new AI-driven simulator to prepare its Apollo robotaxis for just such rare but high-stakes road hazards, as the firm looks to gain an edge in the fast-growing self-driving taxi market.

The system, Apollo World, can build complete virtual journeys from scratch, allowing the company to prepare vehicles for scenarios that are difficult to capture safely in real-world testing, according to executives.

On Tuesday, Baidu’s Intelligent Driving Group showcased the new “world model”, highlighting how the simulator reacts in real time to the vehicle’s steering, braking and acceleration, generating synchronised camera images and lidar data.

“We hope our world model can evolve from simulating a single scenario to simulating the execution of an entire task,” said Yang Ze, a distinguished architect and head of world models at the group.

In technical demonstrations, Yang showed Apollo World generating high-risk hazards, including cyclists cutting across intersections and dogs running into a robotaxi’s path. Engineers could tweak the behaviours of virtual road users to stress-test the driving software’s responses across varied conditions.

The platform is key to Baidu’s push into cross-border road networks. During its demo, the company simulated virtual trips in Shenzhen and Hong Kong under changing traffic densities.

Adapting robotaxis to diverse geographies used to mean manually hardcoding local road rules – a cumbersome process that limited scalability, said Wang Liang, chief architect at Baidu’s Intelligent Driving Group and chairman of the Apollo technical committee.

Road conditions in different cities were often highly different, Wang said, pointing to Chongqing’s dense, multi-tiered road network, left-hand driving in London and Hong Kong, and Dubai motorists’ aggressive driving style.

Baidu’s next-generation architecture instead relies on end-to-end data learning, training models to translate sensory inputs into driving actions. Real-world data gathered by its robotaxi fleet is fed back into Apollo World to improve the system, according to Yang.

The Beijing-based company’s project reflects a broader shift across the global autonomous driving sector towards AI-generated training environments.

In the US, Alphabet-owned Waymo introduced a world model in February built on Google DeepMind’s Genie 3, generating lidar and camera streams of dangerous conditions such as snow on San Francisco’s Golden Gate Bridge and cars driving on the wrong side of the road.

In June, chipmaker Nvidia presented OmniDreams, a world model designed to evaluate autonomous driving software interactively in real time.

“The ultimate goal of autonomous driving is to take passengers safely from any point A to any point B,” Baidu’s Yang said, “rather than just handling an individual scenario, passing through an intersection or completing an overtaking manoeuvre.”