The big AI model of mobile phones is coming, and the storage will come to 16GB+512GB?

Infineon / Mitsubishi / Fuji / Semikron / Eupec / IXYS

The big AI model of mobile phones is coming, and the storage will come to 16GB+512GB?

Posted Date: 2024-01-21

Electronic Enthusiast Network reported (Text/Huang Jingjing) that mobile phone processor manufacturers were the first to lead the implementation of large AI models on mobile phones. Since the second half of last year, Qualcomm and MediaTek have successively released their latest processors, both of which have begun to support large AI models.

Snapdragon 8 Gen3 is Qualcomm’s first mobile platform designed with generative AI as its core. Snapdragon 8 Gen 3 is based on TSMC’s 4nm process and adopts an eight-core architecture design of one 3.3GHz Cortex-X4 + five 3.2GHz Cortex-A720 + two 2.3GHz Cortex-A530. With the support of powerful CPU, GPU and NPU, Snapdragon 8 Gen can run generative AI on the device side, which supports models with more than 10 billion parameters, such as Meta Llama2/Baichuan, and supports 20 tokens/second.

MediaTek Dimensity 9300 adopts a full large-core design. The CPU includes 4 Cortex-X4 large cores and 4 Cortex-A720 large cores with a main frequency of 2.0GHz. The APU 790 integrated in the Dimensity 9300 has a built-in generative AI engine, which is adapted to the Transformer model for operator acceleration, and can generate images within 1 second.

Combined with memory hardware compression technology NeuroPilot Compression to reduce the occupation of terminal memory by large AI models, Dimensity 9300 supports running large AI models with 1 billion, 7 billion, and 13 billion parameters on the terminal. And Dimensity 9300 successfully ran a large AI model with 33 billion parameters.

After the release of these processors, mobile phone manufacturers have successively launched flagship smartphones equipped with these latest processors, and of course, without exception, they have begun to be equipped with large AI models.

Mobile AIBig models emerge

According to statistics from Electronic Enthusiast Network, flagship smartphones released from the end of last year to January this year include Xiaomi 14 series, vivo X100 series, OPPO X7 series, Honor Magic 6 series, etc., as well as the Huami Mate60 series released earlier. They all already support the operation and application of large AI models. Mobile phone manufacturers have a very positive attitude towards embracing large AI models. On the basis that the main processor supports AI functions, they have launched self-developed large AI models.

For example, Xiaomi's MiLM large model supports 6.4 billion parameters, and there is also a device-side large model that supports 1.3 billion parameters.

Vivo released five self-developed "Blue Heart" AI large models, including 175 billion, 130 billion, 70 billion, 7 billion, and 1 billion with five different parameter sizes, and open sourced the 7 billion "Blue Heart" large model. The X100 series is equipped with a large model with 7 billion parameters.

With the support of AndesGPT large model with 7 billion parameters and large language, OPPO terminal has ultra-fast first word response speed. Users can experience more than 100 capabilities such as the new AIGC elimination function, the first AI large model voice summary, and text generation pictures.

Of course, we need to trace the progress of the large AI model on mobile phones. In fact, as early as August last year, Huawei Mate60 Pro was released and announced that it would be connected to the Pangu artificial intelligence large model to provide consumers with a smarter interactive experience.

With the support of AI large models, smartphones have become more intelligent, with more convenient and interesting applications. AI large models have become a major selling point of smartphones and will also be the trend of innovation and development of smartphones in the future.

How much storage capacity is required?

We have seen that these smartphones based on Qualcomm and MediaTek's flagship chips are more concentrated in supporting 7 billion AI large model parameters. So what is the storage capacity of these flagship models? According to statistics, the lowest is 8GB+256GB, and the highest is 16GB+1TB. The storage capacity of 16GB+512GB is the most concentrated.

Judging from the memory capacity of mobile phone AI large models, OPPO imaging product director Zhang Xuan previously gave data in an interview that the normal model size of a 7 billion large model is 28GB, which can be compressed to the minimum after compression and lightweighting. About 3.9GB.

Other mobile phone manufacturers have similar views on this data. Xie Weiqin, director of the Vivo AI Solution Center, previously revealed that running a 1 billion model requires at least 1G of memory, a 7 billion model requires 4GB of memory, and a 13 billion model currently requires more than 7GB of memory.

We cannot but mention lightweight and memory compression technology here. When MediaTek introduced the Dimensity 9300's support for generative AI, it specifically mentioned its unique memory hardware compression technology and NeuroPilot Compression mixed-precision INT4 quantization technology. Using INT4 quantization, a large model with 13 billion parameters can be compressed to 13GB, and then memory hardware compression technology can be used to further reduce storage capacity.

We calculate that a 7 billion large model requires 4GB after compression, plus 6GB App keep-alive and 4GB Android OS. In other words, a smartphone that supports a 7 billion parameter large model needs at least 14GB of memory to implement image generation functions, etc. .

If we want to support models with over 10 billion parameters in the future and implement functions such as digital AI assistants, smartphone memory will need at least 17GB under current conditions.

Currently, domestic smartphones are already equipped with up to 24GB of memory, such as OnePlus 12, OnePlus Ace 3, Realme GT5, etc., which already provide 24GB versions.

In terms of memory specifications, the memory of current flagship smartphones is mainly LPDDR5X. The vivo X100 series is the first to be equipped with Hynix LPDDR5T memory (to emphasize its ultra-high speed, SK Hynix added the word "Turbo Turbo" after the "LPDDR5" specification name). Compared with LPDDR5X memory, the reading speed is increased by 13%. Up to 9.6Gbps.

In terms of flash memory specifications, it is mainly UFS4.0. UFS is a full-duplex interface that allows devices to read and write at the same time. At the standard level, it can help devices achieve high-speed reading and writing with lower power consumption. The theoretical speed of UFS4.0 reaches 4,640MB/s. As mentioned before, the latest flagship smartphones are equipped with 512GB of flash memory capacity.

With the advancement of large AI models on mobile phones, the capacity and performance requirements of mobile phone processors, memory, and flash memory will be upgraded to a higher level. Increasing capacity is only one aspect. Whether it is storage chip manufacturers, mobile phone processors, or terminal manufacturers, more innovations are needed in storage technology, system optimization, etc.

Smartphones are picking up, AIMobile phones drive storage market growth

The smartphone market is finally experiencing growth after a long period of slump.

Counterpoint data shows that in October 2023, after global smartphone sales declined year-on-year for 27 consecutive months, first-time sales transaction volume (i.e. retail sales) increased by 5% year-on-year. In the fourth quarter of 2023, smartphone shipments grew by 3% year-on-year, reaching 312 million units. Counterpoint predicts that global smartphone shipments are expected to grow by 3% year-on-year in 2024.

IDC predicts that China's smartphone market shipments will reach 287 million units in 2024, a year-on-year increase of 3.6%, achieving year-on-year growth for the first time since 2021.

Just like the PC market has high hopes for AI PCs, AI phones will become a strong growth driver for the smartphone market. AI mobile phones will drive the penetration of LPDDR5 and UFS 4.0 and accelerate the recovery of the storage industry.

#big #model #mobile #phones #coming #storage #16GB512GB