China’s AI Engineers Are Secretly Accessing Banned Nvidia Chips, WSJ, Aug. 26, 2024.
Brokers are making overseas computing power available and offering a high degree of anonymity
Chinese artificial-intelligence developers have found a way to use the most advanced American chips without bringing them to China.
They are working with brokers to access computing power overseas, sometimes masking their identity using techniques from the cryptocurrency world.
The tactic comes in response to U.S. export controls that have prevented Chinese companies from directly importing sought after AI chips developed by U.S.-based Nvidia NVDA -2.72%decrease; red down pointing triangle. While it is still possible for Chinese users to physically bring Nvidia’s chips to China by tapping a network of gray-market sellers, the process is cumbersome and can’t supply all the needs of big users.
One entrepreneur helping Chinese companies overcome the hurdles is Derek Aw, a former bitcoin miner. He persuaded investors in Dubai and the U.S. to fund the purchase of AI servers housing Nvidia’s powerful H100 chips.
In June, Aw’s company loaded more than 300 servers with the chips into a data center in Brisbane, Australia. Three weeks later, the servers began processing AI algorithms for a company in Beijing.
“There is demand. There is profit. Naturally someone will provide the supply,” Aw said.
Renting far away computing power is nothing new, and many global companies shuffle data around the world using U.S. companies’ services such as Google Cloud, Microsoft Azure and Amazon Web Services. However, those companies, like banks, have “Know Your Customer” policies that may make it difficult for some Chinese customers to obtain the most advanced computing power.
The buyers and sellers of computing power and the middlemen connecting them aren’t breaking any laws, lawyers familiar with U.S. sanctions say. Washington has targeted exports of advanced chips, equipment and technology, but cloud companies say the export rules don’t restrict Chinese companies or their foreign affiliates from accessing U.S. cloud services using Nvidia chips.
The Commerce Department in January proposed a rule that seeks to prevent malicious foreign entities from using U.S. cloud computing services for activities including training large AI models. U.S. cloud companies argue that the rule won’t prevent abuse and could instead undermine customer trust and weaken their competitiveness.
A smart contract
In platforms used by Aw and others, the billing and payment methods are designed to give the participants a high degree of anonymity.
Buyers and sellers of computing power use a “smart contract” in which the terms are set in a publicly accessible digital record book. The parties to the contract are identified only by a series of letters and numbers and the buyer pays with cryptocurrency.
The process extends the anonymity of cryptocurrency to the contract itself, with both using the digital record-keeping technology known as blockchain. Aw said even he might not know the real identity of the buyer. As a further mask, he and others said Chinese AI companies often make transactions through subsidiaries in Singapore or elsewhere.
“Since late last year, there has been a significant increase in the number of Chinese customers on our platform,” Aw said. “I often get asked if we have Nvidia’s chips.”
Nvidia declined to comment. The company has said it complies with U.S. export controls and expects its partners to do the same. It is scheduled to report its quarterly earnings on Wednesday.
Decentralized GPUs
Platforms such as Aw’s have emerged in the past two years to take advantage of slower activity in the cryptocurrency field, which has freed up some computers previously used to mine digital currencies such as bitcoin. These platforms try to gather up computing power scattered across the globe and rent it out to AI developers.
The service of selling scattered computing power is called a decentralized GPU model, referring to graphics-processing units. Nvidia’s GPUs are widely used in AI applications and coveted on these platforms. Since the U.S. restricted advanced chips from being sold to China in 2022, more Chinese customers have been going to decentralized platforms for computing power, operators say.
Joseph Tse, who until recently worked for a Shanghai-based AI startup, said his former employer turned to a decentralized GPU service after it found itself blocked from renting computing power from Amazon Web Services. The service arranged for more than 400 servers at a data center in California with Nvidia’s H100 chips to help the company Tse worked for to train its AI model, he said.
Tse said the service didn’t differ much from cloud computing offered by the likes of Amazon or Google, but he said the risk was higher because the blockchain system might have flaws allowing code and data to be stolen.
“Blockchain does protect the user’s privacy, but it’s therefore difficult to hold someone accountable if something goes wrong,” Tse said. “But we didn’t have a lot of options. We had to try everything to survive.”
One decentralized GPU provider with more than 40,000 chips in its network, io.net, advertises in its user guide that it doesn’t impose know-your-customer restrictions. This “allows users to access GPU supply and deploy clusters in less than 90 seconds,” it says.
Building bigger clusters
At an AI industry fair in Singapore in June, at least three decentralized GPU companies were touting platforms that provide untrammeled access to affordable Nvidia computing power around the world. All said they had customers from China.
Startups and individual developers use the decentralized platforms to build and operate small AI applications that don’t require heavy-duty computer power or split-second responses. However, decentralized networks typically can’t train large AI models such as the one that powers ChatGPT. Those require thousands of chips to transmit data among themselves quickly.
That issue is why operators such as Aw are building bigger clusters of processors in one place, often with the needs of specific customers in mind.
Edge Matrix Computing, founded in 2022, is one company looking to build bigger chip clusters. EMC has connected more than 3,000 GPUs in its decentralized network, including Nvidia chips used for AI training that are subject to U.S. export controls. Users purchase access to the chips with EMC’s tokens.
EMC said it was soliciting investors to buy Nvidia H100 chips—each of which can cost as much as a Cadillac sedan—and assemble them in a data center for more intensive AI training. Users buying computing power in bulk could pay less than $2 for each hour of H100 use, it said.
At a Senate committee hearing in July, Sen. John Kennedy (R., La.) argued that the Commerce Department has allowed Chinese users to exploit loopholes in chip-export restrictions.
“It appears that the steady flow of advanced microchips into China continues. That flow must stop,” Kennedy wrote in a letter to Commerce Secretary Gina Raimondo.
At the hearing, Thea Kendler, assistant secretary of Commerce for export administration, said her department was closely tracking illicit procurement networks and “it’s something we are cracking down on.”
Meanwhile, Aw is raising more money from a group of investors in Saudi Arabia and South Korea. They plan to build a cluster of Nvidia’s latest Blackwell chips for another Singapore company with a Chinese parent.
“No one is breaking the export controls,” Aw said. “Legally speaking, they are Singapore companies.”
https://www.axios.com/2023/08/23/why-nvidia-stock-price-up-chart-revenue-q2-2023