China’s leading artificial intelligence (AI) start-ups are looking to chase down new opportunities in their field, following the preview of an advanced new series of large language models (LLMs) – the technology underpinning generative AI services – by ChatGPT creator OpenAI, according to executives at Alibaba Cloud‘s Apsara Conference in Hangzhou.
OpenAI o1, previewed earlier this month by the Microsoft-backed firm, represents a so-called generative pre-trained transformer model, designed “to reason through complex tasks and solve harder problems than previous models in science, coding and maths”, according to the San Francisco-based company.
“[OpenAI o1] is indeed of great significance,” Yang Zhilin, founder of Moonshot AI, said on Thursday at the main forum of the three-day conference, which runs until Saturday. Alibaba Cloud is the digital technology backbone of Alibaba Group Holding, owner of the South China Morning Post.
Do you have questions about the biggest topics and trends from around the world? Get the answers with SCMP Knowledge, our new platform of curated content with explainers, FAQs, analyses and infographics brought to you by our award-winning team.
“The most important question here is whether you can do further scaling through reinforcement learning, which completely raises the upper limit of AI,” Yang said, referring to the so-called scaling law on how an LLM’s performance improves as its size and training data increases.
Moonshot AI founder and chief executive Yang Zhilin. Photo: Weibo alt=Moonshot AI founder and chief executive Yang Zhilin. Photo: Weibo>
The 31-year-old Moonshot AI founder expected OpenAI o1 to “cause some changes in the structure of many industries and create new opportunities for start-ups”.
According to an update by OpenAI this week, the new models were trained “to spend more time thinking through problems before they respond, much like a person would”.
Through training, it can “learn to refine their thinking process, try different strategies and recognise their mistakes”, the US firm said.
“If [a company] reaches a certain computing power threshold, it can make basic innovations in algorithms and even breakthroughs in foundational models,” Yang said.
This development, Yang said earlier this week, represents a paradigm shift because LLM developers, which have run out of organic data to train their models, are now turning to a technique known as reinforcement learning to recreate thought processes and in turn, generate more data.
Echoing that view, AI start-up StepFun‘s founder and chief executive, Jiang Daxin, said reinforcement learning “has been generalised to a higher stage” by OpenAI o1.
Jiang said OpenAI o1 is expected to boost opportunities for more innovation in the development of foundational models and AI applications.
An industry executive addresses the audience at the main forum of Alibaba Cloud’s three-day Apsara Conference in Hangzhou, capital of eastern Zhejiang province, on September 19, 2024. Photo: Xinhua alt=An industry executive addresses the audience at the main forum of Alibaba Cloud’s three-day Apsara Conference in Hangzhou, capital of eastern Zhejiang province, on September 19, 2024. Photo: Xinhua>
Still, he pointed out that computing power remains a problem for start-ups. US trade sanctions restrict the availability of advanced semiconductors, such as those from Nvidia, for Chinese firms’ AI development projects.
“The computing power required is still not small, especially when we pursue a generalised reasoning model,” Jiang said. “But if the goal we are pursuing is artificial general intelligence [AGI], we will stick to it no matter how much the cost is.”
While existing AI technologies all function within a set of predetermined parameters, AGI covers the development of systems with autonomous self-control, a degree of self-understanding and the ability to learn new skills.
No more than 10 AI start-ups – including the four so-called “AI tigers” – in China will be able to double down on investments in reinforcement learning, according to a Baichuan AI staff member at the conference who declined to be named.
These AI tigers – Moonshot AI, Baichuan AI, Zhipu AI and MiniMax – are all focused on LLM development.
This article originally appeared in the South China Morning Post (SCMP), the most authoritative voice reporting on China and Asia for more than a century. For more SCMP stories, please explore the SCMP app or visit the SCMP’s Facebook and Twitter pages. Copyright © 2024 South China Morning Post Publishers Ltd. All rights reserved.
Copyright (c) 2024. South China Morning Post Publishers Ltd. All rights reserved.
