How on earth does OpenAI (or Anthropic) answer 800 million people at once
Datacenter GPUs, NVLink fabrics, liquid cooling and serving software. This is how an LLM answers you in 2 seconds while 3 million other people are using it.
anthropic
1
Article
