Thread-based Load Balancing¶
In Gurobi version 12, a new load balancing method was introduced for clusters with multiple nodes, providing finer control over job allocation.
Prior to version 12, the load balancing was done solely based on the maximum number of concurrent
jobs. The compute server would prefer running a job where the least number of jobs were already running,
and would queue jobs above the maximum number of concurrent jobs, the JOBLIMIT.
With this approach, a node could still become overloaded, if the sum of threads used by all jobs was above the
number of physical cores. This new load balancing approach looks at the requirements of each job at a
lower granularity using threads.
Administrators can now specify the thread limit for each node, which defines the maximum number
of threads that can be reserved by all concurrent running jobs using the NODE_THREADLIMIT
configuration parameter in grb_rs.cnf.
This ensures that the maximum number of threads reserved for concurrent jobs remains within
the physical capacity of each node.
It is also important to specify the node thread limit
in all nodes of the cluster to avoid inconsistency in the load balancing process. Finally, while we generally recommend
to use the same hardware for all nodes in the cluster to get consistent results, if the hardware is different,
the node thread limit can be set in each node as necessary.
Submitted jobs can then specify the maximum number of threads they are allowed to use, referred
to as thread reservation. The actual number of threads used by a job may vary depending on
the phase or algorithm selected to solve.
The thread reservation can be set using the ThreadLimit Gurobi
environment parameter or the `--thread-limit command-line flag when submitting a batch with
grbcluster. If the thread reservation is not specified, or if client uses a version prior to 12.0,
the thread reservation will default to the node’s thread limit divided by the number of allowed
concurrent jobs, with a minimum of 1 thread.
For example, if the selected configuration allows a maximum of 16 threads per node and the JOBLIMIT
is set as the default (2), then for every job that does not have a thread reservation, the default will be 8.
Once a job is submitted, the nodes will collaborate to find an appropriate placement where there is enough threads available. If several nodes are possible, the system will prefer the nodes with the most available threads, and the least number of running jobs (in this order). Note that if a job is submitted with a thread reservation exceeding the thread limit of all nodes, it will be rejected. Otherwise, the job will be queued until a node with enough available threads is found. The load-balancing algorithm is greedy and will select the first job it can run.