Apple Ai
Yesterday Apple announced their new AI (Apple Intelligence) concept.
There are several aspects of their announcement that are interesting.
The two points that stand out to me are their partnership with OpenAI, and their new Private Cloud Compute infrastructure.
Elon Musk is apoplectic. He said that Apple devices will no longer be allowed on site at any of his businesses (Tesla, SpaceX and Twitter) and any visitors will be required to deposit their iPhones into a “Faraday cage” when visiting (basically a metal box that blocks access to the radio signals required to use cellular data and make phone calls).
There are lots of reasons for him to be mad. He is reported to have raised about $24 billion for his new AI venture, called xAI. He is so focused on this venture that he has taken the inventory of 12,000 NVidia H100 processors (priced at about $40k each) scheduled to be delivered to Tesla, and redirected them to his venture.
NVidia has a market cap of $3 Trillion today, but analysts have recently gotten a lot of press by saying it will more than triple in size by 2030 ”or sooner”.
The reason for this is that almost all AI startups (and established companies like Tesla) use NVidia processors. Google has its own Tensor Processing Units (TPUs) but they are not nearly as powerful as the NVidia’s H100 technology.
NVidia has been widely assumed to be the only game in town, but Apple’s new partnership with OpenAI could upend the assumptions around that, for several reasons.
Apple and NVidia are in the interesting position of being the two largest customers of TSMC (Taiwan Semiconductor Manufacturing Company). Unlike Intel, TSMC doesn’t design their own chips. They manufacture silicon chips at a massive scale for custom designs for other tech companies. Even Intel has announced that they will be using TSMC to make chips for them.
Apple uses TSMC chips to make their Apple Silicon that powers all their products: their phones, their laptops, their 3D VR glasses (remember those?) and now, the data centers that run their private compute cloud.
Power Efficiency — Unlike Intel, Apple designs their chips to minimize power. For the first 30 years of the computing industry, nobody cared about power. But in mobile devices, power is critical. Nobody wants a phone that will run out of power in a couple of hours. It turns out this is also critical for the data centers that run cloud computing services.
Security — Apple has worked to position themselves as providing products that protect your privacy. One of the ways they do this is by designing digital locks into their chips. Their face and fingerprint recognition technology use a special “secure enclave” to protect the keys that unlock the rest of the computer. They are using this feature in their private compute cloud to help offset fears people have about AI technology stealing personal information. NVidia and Google have not addressed this risk.
Apple claims that the private compute cloud is optional, and can run on the powerful chips inside their phones, but a lot of what people want to do requires a lot more processing power than you can carry in your pocket. Apple has been falling behind for several years because of their refusal to send their customers’ personal information into cloud servers, but now their technology is powerful enough for them to provide potentially useful cloud computing.
OpenAI now have the opportunity to eliminate their dependency on NVidia.
Some interesting data points
How to Build an AI Data Center
Data center trends
In the early 2000s, a single rack in a data center might use one kilowatt of power. Today, typical racks in an enterprise data center use 10 kilowatts or less, and in a hyperscaler data center, that might reach 20 kilowatts or more. Similarly, 10 years ago, nearly all data centers used fewer than 10 megawatts, but a large data center today will use 100 megawatts or more. And companies are building large campuses with multiple individual data centers, pushing total power demand into the gigawatt range. Amazon’s much-reported purchase of a nuclear-powered data center was one such campus; it included an existing 48 MW data center and enough room for expansion to reach 960 MW in total capacity.
...data centers are already some of the largest consumers of electricity in some markets. In Ireland, for example, data centers use almost 18% of electricity, which could increase to 30% by 2028. In Virginia, the largest market for data centers in the world, 24% of the power sold by Virginia Power goes to data centers
Influence of AI
While a rack in a typical data center will consume on the order of 5 to 10 kilowatts of power, a rack in an Nvidia superPOD data center containing 32 H100s (special graphics processing units, or GPUs, designed for AI workloads that Nvidia is selling by the millions) can consume more than 40 kilowatts. And while Nvidia’s new GB200 NVL72 can train and run AI models more efficiently, it consumes much more power in an absolute sense, using an astonishing 120 kilowatts per rack.
Not only is this amount of power far more than what most existing data centers were designed to deliver, but the amount of exhaust heat begins to bump against the boundaries of what traditional, air-based cooling systems can effectively remove.
Internet traffic took roughly 10 years to increase by a factor of 20, but cutting-edge AI models are getting four to seven times as computationally intensive every year. Data center projections by SemiAnalysis, which take into account factors such as current and projected AI chip orders, tech company capital expenditure plans, and existing data center power consumption and PUE, suggest that global data center power consumption will more than triple by 2030, reaching 4.5% of global electricity demand.