Tech-update

Here is a synthesis of the key architectural strategies and the "trade-offs" identified for each vendor.

---

1. AMD: The "Open" Ecosystem Strategy

AMD's goal is to neutralize NVIDIA's software moat (CUDA) by moving the industry toward a cross-vendor "lingua franca."

• The Bet: Triton. By making Triton the default kernel path in PyTorch, AMD ensures that code written in Python DSLs works on both CUDA and ROCm, eliminating the need for manual, hand-tuned assembly for every new chip.
• The Edge: HBM Capacity. AMD has consistently led or matched NVIDIA in on-package memory (e.g., MI300X/355X), allowing them to fit larger models on fewer chips.
• The Weakness: The "Long Tail" of Research. While standard workloads (Llama inference) are parity-adjacent, novel research kernels (like FlashAttention-4) still require hand-written ports, leaving AMD trailing NVIDIA's frontier labs by months.

2. Cerebras: The "Wafer-Scale" Radical

Cerebras views the act of cutting a wafer into small dies as a "tax" that creates the memory wall.

• The Bet: SRAM-Only / No HBM. By building one giant chip (WSE-3), they place all memory (SRAM) one cycle away from the compute. They treat the wafer as a flat plane of 900,000 cores.
• The Edge: Batch-1 Latency. Because they don't have to move weights from HBM to a cache, they offer the fastest independently measured decode speeds in the industry.
• The Weakness: SRAM Scaling. SRAM density has stopped scaling relative to logic. To run a frontier-scale model, you need a fleet of wafers, making the cost-per-token prohibitively high compared to GPUs.

3. AWS Trainium: The "Vertical" Integrator

Amazon’s approach is that of a "fast-follower" focusing on the economics of the cloud rather than the prestige of the spec sheet.

• The Bet: Specialized Silicon for Collectives. Unlike GPUs, where communication (All-Reduce) competes with compute for SM cycles, Trainium uses dedicated CC-Cores to handle networking in parallel with math.
• The Edge: Price-Performance Integration. By owning the chip, the Nitro network, and the cloud API, AWS can undercut NVIDIA on price while providing a "turnkey" experience for anchor tenants like Anthropic.
• The Weakness: Compiler Fragility. Relying on a software-managed memory hierarchy (no caches) means that if the compiler makes a mistake in the schedule, there is no hardware fallback to hide the latency.

4. Groq: The "Deterministic" Specialist

Groq removes all uncertainty from the hardware, shifting the entire burden of scheduling to the compiler.

• The Bet: Absolute Determinism. No caches, no branch predictors, no arbiters. Every instruction and data movement is scheduled to a specific clock cycle.
• The Edge: The "Switchless" Cluster. Because the timing is deterministic, the network itself is scheduled. A cluster of Groq LPUs behaves like one giant, clock-exact processor.
• The Weakness: Capacity. Like Cerebras, Groq uses SRAM. Fitting a 70B model requires hundreds of chips, making it a "latency monster" but a "throughput/cost nightmare." (Note: The text notes NVIDIA eventually licensed this tech to use as a co-processor).

---

Summary Comparison Table: Architectural Philosophies

Final Insight: The Convergence

The analysis suggests a fascinating trend: The industry is splitting between "Throughput Machines" and "Latency Machines."

• NVIDIA, AMD, and AWS are building throughput machines (maximizing tokens per dollar via batching and HBM).
• Cerebras and Groq are building latency machines (maximizing tokens per second per user).

The "climax" of the text is the revelation that NVIDIA eventually incorporates the LPU (Groq) logic, suggesting that the ultimate AI system will be a hybrid: a large-memory GPU for attention/weights, paired with a deterministic SRAM co-processor for the feed-forward layers.