First Distributed Optimization in the Cloud#
The Gurobi Instant Cloud makes it easy to launch a cluster of machines for distributed optimization. This guide will walk you through the process of completing your first distributed solve in the Cloud.
1. Open the Instant Cloud Manager#
Go to cloud.gurobi.com. If you are not logged in, you will be prompted for your credentials. If you do not have an account, please register and contact Gurobi to request an evaluation license.
2. Create a pool with distributed workers#
In the Instant Cloud Manager, go to the ‘Pools’ section and click on the create pool button:
Then, select the ‘License’ tab and set the number of workers to 2.
Finally, create the new pool. Note that a default name is assigned for you such as pool1.
3. Download the pool license file#
The list of pools is displayed in the Instant Cloud Manager and your license file is ready to be downloaded with the following button.
The license file contains the default access ID and secret key for the selected pool. You just have to place this file in your home directory which takes precedence, or in one of the following shared locations:
C:\gurobi\ on Windows
/opt/gurobi/ on Linux
/Library/gurobi/ on Mac OS X
In case you previously had a license file installed, please make sure to
replace it, or set the environment variable GRB_LICENSE_FILE
to
point to the cloud license file, it will override the default locations.
4. Solve#
You can try to solve a MIP MPS file provided with the Gurobi distribution. Here is an example on Mac OS X:
$ gurobi_cl /Library/gurobi1001/macos_universal2/examples/data/misc07.mps
Waiting for cloud server to start..........
Capacity available on '999999-pool1' cloud pool - connecting...
Established HTTPS encrypted connection
Gurobi Optimizer version 10.0.1 build v10.0.1rc0 (mac64[x86])
Copyright (c) 2023, Gurobi Optimization, LLC
Read MPS format model from file /Library/gurobi1001/macos_universal2/examples/data/misc07.mps
Reading time = 0.47 seconds
MISC07: 212 rows, 260 columns, 8619 nonzeros
Optimize a model with 212 rows, 260 columns and 8619 nonzeros
Coefficient statistics:
Matrix range [1e+00, 7e+02]
Objective range [1e+00, 1e+00]
Bounds range [1e+00, 1e+00]
RHS range [1e+00, 3e+02]
Starting distributed worker jobs...
Using Compute Server as first worker - running now
Started distributed worker on ip-52-91-137-123
Started distributed worker on ip-54-159-77-110
Distributed MIP job count: 3
Nodes | Current Node | Objective Bounds | Work
Expl Unexpl | Obj Depth IntInf | Incumbent BestBd Gap | ParUtil Time
H 0 4155.0000000 - - 3s
H 0 3610.0000000 - - 3s
H 0 3500.0000000 1415.00000 59.6% 3s
H 0 2940.0000000 1415.00000 51.9% 3s
H 0 2810.0000000 1415.00000 49.6% 4s
24 22 2810.00000 1544.28571 45.0% 99% 4s
1114 475 2810.00000 1926.66667 31.4% 99% 5s
Ramp-up phase complete - continuing with instance 1 (best bd 2175)
7533 931 1492.85714 0 48 2810.00000 2175.00000 22.6% 99% 7s
15311 0 2785.00000 21 13 2810.00000 2810.00000 0.00% 93% 9s
Cutting planes:
Cover: 2
Clique: 4
MIR: 17
Zero half: 10
Runtime breakdown:
Active: 8.09s (88%)
Sync: 0.81s (9%)
Comm: 0.28s (3%)
Explored 15311 nodes (152346 simplex iterations) in 9.17 seconds
Distributed MIP job count: 3
Optimal solution found (tolerance 1.00e-04)
Best objective 2.810000000000e+03, best bound 2.810000000000e+03, gap 0.0%
Within this log, we have highlighted in bold some important steps. First, the client automatically connects to the Instant Cloud server and checks for the pool status. As the machines are not launched yet, Instant Cloud starts the machines and the client reports that it is waiting until capacity is available.
Then the Gurobi Optimizer detects that the pool is setup with 2 distributed workers. So it automatically starts the solve in distributed mode with 3 workers (the master compute server counts as one worker as well).
5. Terminate the pool (optional)#
The machine will auto-terminate once it stays idle for a duration limit called the idle shutdown. The default idle shutdown is 60 minutes, and it can be changed in the settings of the pools and your preferences. Otherwise, you can terminate the pool manually in the Instant Cloud Manager, by selecting the created pool and clicking on the terminate button.