The Bible of AI

From a blue pixel to an AI server: where does gallium nitride save energy?

From a blue pixel to an AI server: where does gallium nitride save energy?

Where does gallium nitride reduce heat between a blue pixel and an AI server? In a display, the useful output is light. Researchers compared tiny blue emitters at the same optical output per unit area and measured how warm their surfaces became. In a server, the useful output of a power converter is electricity delivered to the next stage. Two matched converters show what happens when gallium nitride takes over the switching job from silicon. A separate test lets us calculate the watts still lost while delivering 480 watts to a load. Nitride transistors have also been joined into logic gates and a memory cell. Each experiment puts a different limit on what we can claim. Together they show where this semiconductor already earns its place, where silicon remains part of the device, and what to measure before promising lower energy per AI response.

From a blue pixel to an AI server: where does gallium nitride save energy? Leer más »

Why surround the channel? From FinFET fins to RibbonFET nanosheets

Why surround the channel? From FinFET fins to RibbonFET nanosheets

A transistor that is supposed to be off can still pass current when its channel gets very short. Why does wrapping the gate around thin sheets of silicon help, and when could an AI chip benefit? A cross-section compares the three controlled faces of a FinFET with the gate that surrounds each channel in a nanosheet transistor. Measurements on an early three-sheet device show what it means to control current with a small change in voltage. Intel’s later tests on working CPU cores reveal something a device measurement alone cannot tell us. The voltage that reaches a transistor depends on the network supplying it. A matrix engine might use that margin to calculate at lower voltage. A model waiting for weights from memory presents a different test. Which part of a response could actually get quicker, and how much energy would it take?

Why surround the channel? From FinFET fins to RibbonFET nanosheets Leer más »

What can stacked CFET transistors change for AI chips?

What can stacked CFET transistors change for AI chips?

What could stacked CFET transistors change for an AI chip? A logic gate normally places two complementary switches beside one another. Moving one above its partner can bring neighbouring gates closer and shorten the wires that carry intermediate results through a matrix engine. Intel and imec have tested stacked devices in silicon. TSMC has demonstrated a ring oscillator and a working memory cell. A separate study lets us ask what those results might mean for inference. Its researchers laid out the same arithmetic array with two transistor technologies and compared wire length, footprint and power at different operating points. A smaller array could run faster, yet draw more power when it did. The practical question is whether an accelerator can use that extra margin to deliver the same model output with less energy once memory traffic and cooling are counted.

What can stacked CFET transistors change for AI chips? Leer más »

Why does an AI accelerator wait for memory? Inside high-bandwidth memory (HBM)

Why does an AI accelerator wait for memory? Inside high-bandwidth memory (HBM)

High-bandwidth memory (HBM) sits beside an AI accelerator and feeds the units that do the arithmetic. Why can a processor still spend time waiting for data, even when the model fits in memory? Follow a token from the interconnect of Huawei’s Atlas 960E to the weights an expert needs to read. Then look at an experiment on an NVIDIA A100 in which the same attention calculation finished sooner despite doing more arithmetic. The difference lay in how often intermediate results crossed the boundary between the processor and its memory. Specifications from AMD and Google help distinguish how much data a chip can hold from how quickly it can read them. For someone using a language model, the final test is whether these changes shorten the wait between tokens or let more people receive timely answers.

Why does an AI accelerator wait for memory? Inside high-bandwidth memory (HBM) Leer más »

When does a faster link make AI faster? Huawei’s Ascend 960DT and Atlas 960E

When does a faster link make AI faster? Huawei’s Ascend 960DT and Atlas 960E

Huawei’s Ascend 960DT and Atlas 960E raise a practical question for AI. When parts of a language model run on different chips, how much of the wait for the next token comes from moving data between them? Follow one token as its numerical representation travels to a remote expert and the result comes back. Huawei’s design brings optics close to the electronics and gives the accelerators a shared way to address memory. An earlier system, CloudMatrix384, offers something the new design still needs: published measurements showing how the pace of an individual reply changes as more requests share the machine. We examine what those results establish, what Huawei has announced for the Atlas 960E, and how a test at fixed response time could show whether a faster link delivers more useful work for each joule.

When does a faster link make AI faster? Huawei’s Ascend 960DT and Atlas 960E Leer más »