Loading…
Glossary term
The high-speed links that connect chips and servers so they can work together.
An interconnect is the high-speed link between chips, servers and racks. Training a large model means thousands of chips constantly exchanging results, so slow links leave expensive processors waiting.
Technologies include NVLink within a server and InfiniBand or fast Ethernet between servers.
At large scale, networking can be as important as the chips themselves.