Hide navigation sidebar
Hide table of contents sidebar
Back to top
Toggle table of contents sidebar

# Batch Optimization

Batch optimization is a feature available with the Gurobi Cluster Manager. It allows a client program to build an optimization model, submit it as a batch request to a Compute Server cluster (through a Cluster Manager), and later check on the status of the request and retrieve the solution. Once a batch is submitted to the Cluster Manager, it is identified through a unique BatchID. The client program (or any other program) can use this ID to query the BatchStatus of the batch (submitted, completed, etc.). Once the batch has completed and a solution is available, the client can retrieve that solution as a JSON string.

This section explains the steps required to perform the various tasks listed above. We’ll use the batchmode.py Python example, which is included with the distribution, to illustrate these steps. Complete examples for all supported languages can also be found in the Example Tour.

## Setting Up a Batch Environment

Recall that the first step in building an optimization model is to create a Gurobi environment. An environment provides a number of configuration options; among them is an option to indicate where the model should be solved. You can solve a model locally, on a Compute Server, or using a Gurobi Instant Cloud server. If you have a Cluster Manager installed, you also have the option of using batch optimization.

To use batch optimization, you should configure your environment as if you will be using a Compute Server through a Cluster Manager. You’ll need to set the CSManager parameter to point to your Cluster Manager, and provide a valid Username and ServerPassword. The difference is that you will also need to set the CSBatchMode parameter to 1. This will cause the client to build the model locally, and only submit it to the server once a call to the `optimizeBatch` method is made. This is in contrast to a standard Compute Server job, where the connection to the server is established immediately and the model is actually built on the server.

The following shows how you might set up your environment for batch optimization (in Python):

    env = gp.Env(empty=True)
    env.setParam("LogFile", "batchmode.log")
    env.setParam("CSManager", "http://localhost:61080")
    env.setParam("UserName", "gurobi")
    env.setParam("ServerPassword", "pass")
    env.setParam("CSBatchMode", 1)

Note that you can also use CSAPIAccessID and CSAPISecret (instead of Username and ServerPassword) to connect to a Cluster Manager.

## Tagging Variables or Constraints

Batch optimization separates the process of building a model from the process of retrieving and acting on its solution. For example, you can build your model on one machine, submit a batch request, and then use the resulting BatchID to retrieve the solution from a completely different machine.

Of course, disconnecting a model from its solution introduces a mapping problem: the process that retrieves the solution needs to know how to map the elements of the solution back to the corresponding elements of the model. This is done through tags. When a model is built, the user associates unique strings with the variables and constraints of interest in the model. Solution values are then associated with these strings. If the user doesn’t provide a tag for a model element, no solution information is stored or returned for that element.

You can tag variables (using the VTag attribute), linear constraints (using the CTag attribute), and quadratic constraints (using the QCTag attribute). We should point out that solutions to mixed-integer models don’t contain any constraint information, so constraint tags have no effect for such models.

For details on the information that is available in the solution in different situations, please refer to the JSON solution format section.

Here’s a simple example that tags the first 10 variables in a model:

    # Define tags for some variables in order to access their values later
    for count, v in enumerate(model.getVars()):
        v.VTag = f"Variable{count}"
        if count >= 10:
            break

## Submitting a Batch Optimization Request

Once you have built your model and tagged the elements of interest, you are ready to submit your batch request. This is done by invoking the `optimizeBatch` method (e.g., `optimizeBatch` in Python). This method returns a BatchID string which is used for later queries. Here’s a simple example:

    # Submit batch request
    batchID = model.optimizeBatch()

## Interacting with Batch Requests

You can use a BatchID string to ask the Cluster Manager for more information about the corresponding batch. Specifically, you can query the BatchStatus for that batch, and if the batch is complete you can retrieve the computed solution as a JSON string.

Your first step in using a BatchID to gather more information is to create a Gurobi environment that enables you to connect to your Cluster Manager. This is done in this line of our Python example:

    with setupbatchenv().start() as env, gp.Batch(batchID, env) as batch:

The `setupbatchenv` method creates an environment with the CSManager, Username, and ServerPassword parameters set to appropriate values.

With this environment and our BatchID, we can now create a `Batch object` (by calling the `Batch` constructor in the above code segment) that holds information about the batch.

That Batch object can be used to query the BatchStatus:

    # Setup and start environment, create local Batch handle object
    with setupbatchenv().start() as env, gp.Batch(batchID, env) as batch:
        starttime = time.time()
        while batch.BatchStatus == GRB.BATCH_SUBMITTED:
            # Abort this batch if it is taking too long
            curtime = time.time()
            if curtime - starttime > maxwaittime:
                batch.abort()
                break

            # Wait for two seconds
            time.sleep(2)

            # Update the resident attribute cache of the Batch object with the
            # latest values from the cluster manager.
            batch.update()

            # If the batch failed, we retry it
            if batch.BatchStatus == GRB.BATCH_FAILED:
                batch.retry()

It can also be used to perform various operations on the batch, including aborting or retrying the batch.

Once a batch has been completed, you can query the solution and all related attributes for tagged elements in the model by retrieving the associated JSON solution string (or by saving it into a file):

    print("JSON solution:")
    # Get JSON solution as string, create dict from it
    sol = json.loads(batch.getJSONSolution())

By default, the Cluster Manager will keep the solution for the model and other information for a while (the exact retention policy is set by the Cluster Manager). You can ask the Cluster Manager to discard information for a batch by explicitly calling the `discard` method:

    # Remove batch request from manager
    batch.discard()

No further queries on that batch are possible after this has been done.

## Interpreting the JSON Solution

Once you have retrieved a JSON solution string, you can use a JSON parser to retrieve solution information for individual variables and constraints. This parser isn’t included in the Gurobi library. Rather, programming languages have libraries for doing this. The appropriate package in Python is (not surprisingly) called `json`. The following provides a simple example of how this library can be used to parse a JSON solution string and extract a few pieces of solution information:

    # Get JSON solution as string, create dict from it
    sol = json.loads(batch.getJSONSolution())

    # Pretty printing the general solution information
    print(json.dumps(sol["SolutionInfo"], indent=4))

Note that you may have to set the parameter JSONSolDetail to 1 to see all relevant solution data, like pool solution values. Consult the JSON solution format description for details.

## Limitations

It is currently not possible to use multi-objective environments via the Batch Optimization feature. Concurrent Environments and the Parameter Tuning Tool are also not available via the Batch Optimization feature.

Help and Feedback

Press `/` or click to search this documentation | Gurobi website | Gurobi community forum
Go to other Gurobi documentation | Gurobi download | Contact support