OpenAI scales Habitat storage for more than 1 billion ChatGPT users
OpenAI detailed how its Habitat storage platform scales Azure Cosmos DB for more than 1 billion users across 40 regions. Habitat stores over 500 petabytes and moved from a shared container to isolated containers before returning to a multi-tenant model.
OpenAI described how it is scaling Habitat, the online storage platform that provides fast access to data for its products for more than 1 billion users across 40 geographic regions.
Habitat handles more than 500 petabytes of data and is designed to respond in milliseconds. In the second part of a technical series, OpenAI detailed its work on Azure Cosmos DB, the storage component underneath the platform.
From shared storage to isolation
Habitat initially routed client workloads through one default Azure Cosmos DB container per account. Shared use led to collisions on common partition keys, including user_id and workspace_id, and allowed unusually heavy tenant workloads to affect unrelated use cases.
The practical limits were enforced by physical partitions, not only logical partition keys. As data volumes grew and physical partitions split, total container throughput stayed unchanged, reducing the effective capacity available per partition. Active workloads could therefore lose throughput capacity even when request rates remained stable.
OpenAI began assigning separate containers to critical new use cases. This reduced the blast radius of incidents, limiting failures to individual services or features rather than the wider platform.
Launches required advance capacity planning
Dedicated containers required teams to plan capacity separately. A workload could rise from 1,000 RU/s during testing to 1 million RU/s at launch, forcing teams to pre-scale resources. When they failed to do so, engineers had to increase capacity on launch day.
OpenAI said the former shared container absorbed sharp growth in an individual use case more easily because the change represented a smaller share of aggregate usage. Dedicated containers improved isolation but removed that workload-mixing benefit.
Habitat eventually returned to a multi-tenant container by default, relying where possible on Azure Cosmos DB’s built-in primitives. OpenAI described its multi-tenancy work as still being at an early stage while supporting a broad range of use cases.
What we know
- Habitat serves more than 1 billion users across 40 regions.
- The platform stores over 500 petabytes of data.
- Workloads could jump from 1,000 to 1 million RU/s at launch.
- OpenAI returned to a multi-tenant container by default.
Trust: Primary source
- The story cites a primary statement. It establishes what its author said, but does not guarantee every claim is true.
- Sources: OpenAI.
COMMUNITY
Discussion
Sign in to join the discussion.
Loading comments…