Chinese AI can benefit from bigger models, more data, says start-up founder

The founder of Stepfun, which develops foundational models, has set its sights on improving multimodal models as China faces curtailed access to advanced chips

Reading Time:2 minutes

People walk by signage for the World Artificial Intelligence Conference in Shanghai on July 3, 2024. Photo: AP

Published: 7:00am, 8 Jul 2024

The founder of a Shanghai-based artificial intelligence (AI) start-up is putting his faith in the “scaling laws” of large language model (LLM) development, despite China’s disadvantages in investment and advanced chips.

Jiang Daxin, a Microsoft veteran who founded and now runs Stepfun, said at a side event for the World Artificial Intelligence Conference (WAIC) in Shanghai that LLMs will eventually reach hundreds of trillions of parameters.

Scaling laws seek to measure the relationship between AI model performance and their parameters. They generally show improved performance from larger models, more data, and greater computational resources, but with diminishing returns. Tech giants have been splurging on the most advanced tech – most notably Nvidia chips like the H100s – to eke out any possible edge in performance.

“The upgrades of OpenAI’s GPT series, which powers ChatGPT, along with the substantial investments in supercomputing centres by tech giants such as Amazon, Microsoft and Meta, all validate the effectiveness of scaling laws,” Jiang said in a talk on Saturday. “Going forward, issues like the availability of data, human resources, and concerns about return on investment may impact the pace of scaling laws.”

02:12

Inside a Chinese factory that makes humanoid robots with enhanced facial movements

China’s Big Tech firms and start-ups alike have rushed to launch their own LLMs since OpenAI launched ChatGPT in late 2022. There are more than 200 AI models in the country, including Tongyi Qianwen by Alibaba Group Holding and Ernie by internet search giant Baidu. Alibaba owns the South China Morning Post.