Self-improving AI the key step towards AGI but safety is key: Z.ai founder

Using artificial intelligence to build even smarter systems will make artificial general intelligence (AGI) more achievable, though it would need to be done safely, leading Chinese computer scientist Tang Jie said at an event in Shanghai on Saturday.

The remarks underscore how AI companies in the US and China are both chasing the holy grail of recursive self-improvement (RSI), where increasingly powerful AI systems also become better at improving themselves in an ever-accelerating “loop”.

“[Recursive self-improvement] is the main research problem for [achieving] AGI,” said Tang, who is a computer science professor at Tsinghua University and co-founder of Beijing-based artificial intelligence model company Z.ai, developer of the GLM family of models.

AGI refers to a hypothetical AI system that matches or surpasses human beings in all cognitive tasks.

In recent months, leading US AI start-ups OpenAI and Anthropic have both said that they are starting to see the early signs of RSI. In September, Claude developer Anthropic released a report showing that AI was leading 26 per cent of internal research and development on AI development in August, up from around 1 per cent in March.

Tang did not disclose how much of Z.ai’s R&D was now being done by AI. However, he said in his keynote speech at the World Laureates Forum that RSI represents a “new paradigm” of R&D that would accelerate AI development even more than it has been.

Earlier this year, Tang was one of the first Chinese AI experts and industry leaders to identify RSI as the next stage for AI development.

Last month, Z.ai said that its latest GLM-5.3 model helped optimise its own “inference” infrastructure – the software needed to serve a model quickly and efficiently – for Chinese chips, which had not previously been done before at the same scale.

In his speech on Saturday, Tang said that as frontier AI models shifted from being simple chatbots to AI agents tasked with complex scientific and research projects, it was logical to turn to AI systems themselves for help, given that human researchers were facing greater engineering difficulties as well as financial constraints.

RSI was a major theme of this year’s World Laureates Forum, an annual gathering of leading global scientists, with several Chinese industry and academic experts sharing presentations about how AI was accelerating the design and manufacturing of semiconductors, which are themselves used to power AI systems.

However, some experts have warned that RSI could lead to humans losing control over AI as the development process accelerates too quickly for humans to understand or intervene.

Acknowledging such concerns, Tang called for human researchers to be “humble” in the face of RSI, arguing that the responsibility for ensuring AI progresses safely should be “shared”.

“Probably … the world should organise something together so that we can guarantee the safety for not just one country but for all the world,” he said.

Tang spun Z.ai, also known as Zhipu AI, out of his Tsinghua University laboratory in 2019, with many of his former and current students going on to become core researchers at his company.

Several of his students have become key figures in China’s AI industry in their own right, including Moonshot AI CEO Yang Zhilin.