storage controller: use Utilization in scheduling decisions #7798
Labels
c/storage/controller
Component: Storage Controller
t/feature
Issue type: feature, for new features or requests
Currently, the scheduler orders nodes according to constraints, and then within nodes which have the same AffinityScore, we select the one with the fewest shards assigned to it.
That works pretty well at small scale and when we're mainly dealing with big tenants who have big shards such that disk space and shard count are strongly correlated.
In order to make the storage controller work well with large numbers of tiny tenants, we'll need to use the Utilization of a pageserver (already stored in Node courtesy of the heartbeater) in the scheduling.
This ticket isn't about making the scheduler perfect, just to make sure that we do something sensible if we have e.g. one node with 10 shards whose utilization is full, and another node with 1000 shards whose utilization is really low because all those 1000 shards are little 32mb ones.
The text was updated successfully, but these errors were encountered: