Distributed Optimization#
Distributed optimization in Gurobi enables the use of multiple computers connected via a network to solve optimization problems. This contrasts with concurrent optimization, which leverages multiple processor cores on a single computer. However, each distributed machine can also perform concurrent optimization internally.
To use distributed optimization, you must configure either Gurobi Remote Services or Gurobi Instant Cloud.
Gurobi offers the following distributed algorithms:
Distributed MIP - Divides the work of solving a single Mixed Integer Programming (MIP) model across multiple machines.
Distributed Concurrent - Uses multiple machines running different algorithmic strategies in parallel to solve LP or MIP models.
Distributed Tuning - Distributes the task of tuning model parameters across multiple machines.
Distributed MIP#
With Distributed MIP, several machines collaborate to solve a single MIP model. Gurobi begins with a “ramp-up” phase. In this phase, each distributed worker solves the problem concurrently. At a certain stage, Gurobi selects the most promising branch-and-bound tree created by these workers (also known as “racing ramp-up”). This search tree serves as the starting point for the distributed phase. From there, the different machines explore different parts of the tree and synchronize their progress.
This method is especially effective for models that generate large but shallow search trees, allowing efficient parallel exploration.
Configuration
Instant Cloud: Distributed optimization is automatically enabled when adding distributed workers to a pool (i.e., setting a value larger than 0 for distributed workers per server). No other parameter configuration is needed.
Remote Service: Set the parameter DistributedMIPJobs to enable this feature.
For logging behavior specific to Distributed MIP, refer to the Distributed MIP Logging section.
Distributed Concurrent#
Distributed Concurrent solves the same model on multiple machines using different algorithmic strategies. By default, Gurobi chooses the parameter settings used for each independent solve automatically. You can create concurrent environments to choose your own parameter settings (refer to the concurrent optimization section for details). The intent of concurrent MIP solving is to introduce additional diversity into the MIP search. By bringing the resources of multiple machines to bear on a single model, this approach can sometimes solve models much faster than a single machine.
Configuration
Instant Cloud: Configure the number of distributed workers per server and set the parameter ConcurrentJobs to enable distributed concurrent optimization.
Remote Service: Set the parameter ConcurrentJobs to enable distributed concurrent optimization.
The value for ConcurrentJobs is typically number of workers + 1 (including the compute server).
Distributed Tuning#
Gurobi’s parameter tuning tool automates the search for optimal parameter settings to improve model performance. Since tuning requires solving the model many times with varied settings, distributing this task across machines can significantly reduce tuning time.
Configuration
Determinism#
Determinism refers to the ability to achieve reproducible and consistent results across multiple runs of the same optimization problem.
For Distributed MIP, the algorithm maintains deterministic behavior when using identical input parameters, model data, and machine configurations and if no time dependent behavior is used (e.g., by using parameter TimeLimit).
However, Distributed Concurrent optimization inherits the non-deterministic characteristics of regular concurrent optimization, refer to the determinism section on concurrent optimization for more details.
Additional Remarks#
The technical details on how setting up distributed algorithms with the Gurobi Remote Service are discussed in this section of the Remote Services Guide. Please note that you need a license that has distributed enabled on the manager machine to use Distributed MIP or Distributed Concurrent. Distributed Tuning is available with any remote server license by default. On Gurobi Instant Cloud, distributed optimization can be enabled in the pool settings, any Instant Cloud license supports this.
Please note that one compute server node can only run a single distributed job simultaneously. Distributed jobs are never queued, so they may fail if there is no capacity at the time they are submitted.
The Distributed Algorithm Considerations section of the Remote Services Guide discusses further aspects to keep in mind:
For distributed algorithms, it is important that all worker machines operate at similar speeds. If there are speed differences, the results of distributed tuning may be unreliable. Additionally, in distributed MIP mode, faster machines may experience long idle times as they wait for slower ones to catch up.
The distributed MIP and distributed concurrent solvers are restricted in the available callback codes, see Distributed Algorithm Considerations for further details.