TCP

A new engineering write-up proposes routing large language model traffic across multiple inference providers using congestion-control ideas borrowed from TCP. The approach dynamically shifts requests toward providers that respond faster or fail less, similar to how TCP manages network congestion. Readers on Hacker News are discussing the technical trade-offs of applying networking concepts to AI inference routing.