A/B Testing¶
Problem Statement¶
Choose from a pool of A/B variants and allocate different user segments to them to minimize user disruption while upholding statistical significance.
Full example scenario
You are a backend engineer at a software company preparing to run A/B tests on a new feature. The goal is to determine the optimal selection of 3 out of 10 possible test variants (V1 to V10), and to allocate different user segments to those chosen variants, in order to minimize total user disruption.
Choose exactly 3 out of the 10 possible variants.
Each chosen variant must have at least 100 users from each user segment, for statistical significance.
All users from each segment must be allocated to the variants that end up being chosen.
Within each user segment, the difference in allocation between any two chosen variants cannot be more than 300.
The user segments and disruption costs can be found in
ab_testing.csv, with the columnsUser Segment,Total Users,Disruption in V1,Disruption in V2,...,Disruption in V10.
What an optimal solution looks like
Which users will test which variant.
New Users
1,000 users
Regular Users
1,500 users
Power Users
500 users
Variant #1
unused
Variant #3
1,133
533 New
300 Regular
300 Power
Variant #5
933
233 New
600 Regular
100 Power
Variant #8
934
234 New
600 Regular
100 Power
Variant #10
unused
Built with Modeler¶
The problem above is stated as a single shot: you hand over the whole problem and get a model back. Modeler takes the other route — you start with the business question you actually have, and it works with you until the problem is pinned down, the data is in, the code runs, and the answer holds up when the problem changes.
The tabs below walk the same A/B testing problem through that route, one Modeler capability at a time.
Instead of writing the whole prompt up front, you start with the business question you actually have. Modeler searches the knowledge base, asks one clarifying question at a time — always the most important one first — and tells you which parts of the model are still missing. The options it offers come from examples in the knowledge base and from how similar problems are usually modeled in your industry, so you can answer with a number instead of prose.
We’re a backend engineering team about to run A/B tests on a new feature. We have several candidate test variants and multiple user segments, and we need to decide which variants to actually test and how to split our user segments across them. We want to keep user disruption as low as possible while still running a valid experiment. Can you help us build a model for this?
This is a joint variant-selection and allocation problem, not a plain allocation problem — the number of variants to pick is itself a decision. Let me check the knowledge base for similar A/B testing setups before asking anything, so the first question can offer realistic, grounded options rather than a generic one.
Great, this is a well-suited problem for mathematical optimization. I found a closely related reference in the knowledge base that will help guide the model design. Let me start gathering the details to build a precise specification for your use case.
Question 1: How many variants do you have in your candidate pool, and how many do you want to actually select and run in the experiment?
For example, a common setup is selecting 3 variants out of a pool of 10. Do you have a fixed number to select, or is it a range (e.g., “at least 2, at most 5”)?
We select exactly 3 out of 10 candidate variants (V1-V10). We have 3 user segments: New Users (1000 total users), Regular Users (1500 total users), and Power Users (500 total users) — every user in a segment has to end up allocated to one of the 3 chosen variants, no one left out. Each variant we actually pick needs at least 100 users from each segment for the result to be statistically meaningful. One more thing: within a segment, we don’t want the allocation to be too lopsided across the chosen variants — the difference between the biggest and smallest allocation for a segment shouldn’t exceed 300 users. And I’d also like the output to show, per segment, exactly how many users go to each chosen variant, not just which variants were picked.
That answer settles the selection count, the three segments and their totals, the minimum-users floor, and — volunteered on top — a balance rule and an output requirement. Recording all of it. What is still open is the one thing that actually drives the objective: how “disruption” is measured. Without a cost figure per segment-variant pair there is nothing to minimize yet, so that has to be the next question.
Good, the spec is saved. Now the most important piece remaining is the objective.
You mentioned keeping “user disruption as low as possible” — how would you like to define disruption quantitatively? Here are the most common interpretations, pick the one that fits best or describe your own:
Minimize total users exposed to the test — keep the overall footprint of the experiment as small as possible.
Minimize allocation imbalance — minimize the total spread (max minus min allocation) across variants, summed over all segments.
Minimize the maximum single allocation — avoid any one variant receiving a disproportionately large share of users in any segment.
Minimize deviation from a target allocation — each variant should receive as close to an equal share of each segment as possible, and you minimize the total deviation from that ideal split.
Modeler updates the specification after every answer, even while it is still incomplete — so you can always see what it has understood and what it is still waiting on. The objective question above is answered with the data file in the Bringing in your data tab, since the numbers that define “disruption” and the file that carries them arrive together.
Modeler’s last open question was how to define disruption. Rather than typing
numbers into the chat, attach ab_testing.csv and answer both at once — the
file’s own columns settle the objective, since they are exactly the per-user
disruption costs the objective needs to minimize.
User Segment,Total Users,Disruption in V1,Disruption in V2,Disruption in V3,Disruption in V4,Disruption in V5,Disruption in V6,Disruption in V7,Disruption in V8,Disruption in V9,Disruption in V10
New Users,1000,3,4,2,5,3,4,2,3,5,4
Regular Users,1500,2,1,3,2,1,2,3,1,2,3
Power Users,500,1,2,1,3,2,1,1,2,3,2
ab_testing.csv attached
Here’s our data file. Each row is a user segment with its total user count, and the “Disruption in V1”…”Disruption in V10” columns give a per-user disruption cost for putting a user from that segment into that variant — these come from historical A/B test feedback scores, higher means more disruptive. So to answer your objective question: minimize the total weighted disruption, i.e. sum over all segment-variant assignments of (disruption cost) x (number of users allocated).
Reading the attachment. One row per segment, one column per variant, so this is a 3 x 10 cost matrix in wide format — matching the 3 segments already in the specification and the 10-variant candidate pool from Question 1. The stated objective is a direct weighted sum over this matrix, which resolves the open objective question exactly — nothing about it is ambiguous or needs a follow-up. Adding a “Disruption cost” entity to the Input section and recording the objective.
Got it — a 3 x 10 disruption-cost matrix, keyed by segment and variant. I have added a Disruption cost entity to the specification’s Input section and recorded the objective: minimize the total weighted disruption, summed over every segment-variant pair that ends up with a nonzero allocation.
That was the last open item. The specification is complete. Shall I proceed to Step 2: Model Implementation using Python with gurobipy?
Where your files live
Everything in the conversation — the file you attached and every artifact Modeler generates from it — lands in the conversation workspace as an attachment. Each one opens in a side panel next to the chat, so you can read the specification while you keep answering questions, or scroll the code while you ask for a change. From that panel you can copy the contents or download the file.
By the end of this conversation the workspace holds six items:
Attachment |
What it is |
|---|---|
|
The disruption-cost data you attached. |
|
The specification, rewritten after every answer. |
|
The |
|
The |
|
Output of the test runs, including the solver logs. |
|
The mathematical formulation, generated on request. |
Starting from files you already have
The same upload path runs in reverse. Attach the files of an existing
optimization project — a specification, a model script, a test file, or any
subset — and ask Modeler to check the project. It completes the
specification where it is thin (asking clarifying questions, as above), fills in
a missing implementation or test suite, reconciles code that has drifted from the
specification, and runs the tests. Point it at a model written for another solver
or another language and it recovers a specification from that file, asks you to
confirm it, then reimplements the model in gurobipy.
The specification is the artifact the questions are building toward, and the one everything else is generated from. Modeler rewrites it after every answer — even while it is still incomplete — so it doubles as the running record of what has been agreed and what is still open. It follows a fixed seven-section structure and is written entirely in business terms: no mathematics, so the person who owns the problem can read it and say whether it is right.
1. Overview
A backend engineering team needs to run an A/B test on a new feature. There are 10 candidate variants (V1–V10) and 3 user segments: New Users, Regular Users, and Power Users. The model selects exactly 3 variants to run and then allocates every user in each segment across those 3 chosen variants, such that the experiment is statistically valid and total user disruption is minimized.
2. Objective
Minimize the total weighted disruption across all segment-variant allocations, defined as the sum over all (segment, variant) pairs of the per-user disruption cost multiplied by the number of users allocated to that variant from that segment. Disruption costs are derived from historical A/B test feedback scores and vary by segment and variant; higher scores indicate greater user impact.
3. Constraints
C1 — Exactly 3 variants selected. Exactly 3 variants must be selected from the pool of 10 candidates. Running more dilutes the traffic each variant gets; running fewer than 3 was ruled out up front by the team.
C2 — Full allocation of each segment. Every user in each segment must be assigned to one of the 3 selected variants — no user may be left unallocated. The sum of allocations across all variants must equal the total user count for each segment.
C3 — No allocation to unselected variants. Users can only be allocated to variants that have been selected. If a variant is not selected, it receives zero users from every segment.
C4 — Minimum users per variant per segment. Each selected variant must receive at least 100 users from each segment, so that the result is statistically meaningful for every segment the experiment claims to cover.
C5 — Balance constraint. Within each segment, the difference between the largest and smallest allocation across the chosen variants must not exceed 300 users, so no single variant ends up dominating or starving a segment’s traffic.
4. Input
Entity |
Fields |
Values |
|---|---|---|
Variant |
variant id |
10 candidates, V1 through V10 |
User segment |
segment name, total users |
New Users: 1000, Regular Users: 1500, Power Users: 500 |
Disruption cost |
segment, variant id, per-user cost |
3 x 10 matrix, supplied in |
Total users across all three segments is 3,000, to be split across whichever 3 of the 10 variants the model selects.
5. Output
The 3 variants selected to run in the experiment.
The allocation: how many users from each segment go to each selected variant.
The total weighted disruption achieved.
6. Test scenarios
ID |
Scenario |
Acceptance criteria |
|---|---|---|
T1 |
Baseline — the data above. |
Selects V3, V5, V8. Total weighted disruption 5,267. |
T2 |
Disruption cost of V3, V5 and V8 raised to 10 across every segment. |
The recommendation changes: V1, V2, V7 are selected instead. Total weighted disruption 6,000. |
T3 |
Disruption cost of the 7 non-selected variants (V1, V2, V4, V6, V7, V9, V10) raised by 1. |
Selection and allocation are unchanged from T1 — V3, V5, V8 remain optimal at 5,267. |
T4 |
Balance spread tightened to 0 (variants must split each segment exactly evenly). |
Reported as infeasible — 1,000 and 500 users do not divide evenly by 3. |
T5 |
Compare against a naive heuristic: pick the 3 variants with the lowest average disruption cost, then split each segment evenly across them. |
Naive heuristic selects V1, V3, V5 at a cost of 6,333. The optimized plan (T1) beats it by 1,066 (16.8%). |
7. Out of scope
Variants are not evaluated for qualitative similarity or feature overlap.
No constraint on which specific variants must or must not be selected.
No multi-objective handling (e.g., balancing disruption against statistical power).
No time-based or sequential A/B test scheduling.
No fractional user allocations (all allocations are integer-valued).
With the specification settled, Modeler writes the gurobipy implementation.
This is the version that the Cloud solve tab shows running against the full
test suite — the same file, not a draft that preceded it.
"""
A/B Test Variant Selection and User Allocation Optimizer
=========================================================
Selects exactly K variants from a candidate pool and allocates users from
each segment across the chosen variants to minimize total weighted disruption,
subject to full-allocation, statistical-validity, and balance constraints.
Implements the specification in ab_testing_modeler_spec.rst.
"""
import gurobipy as gp
from gurobipy import GRB
import pandas as pd
class ABTestOptimizer:
"""
Optimizer for A/B test variant selection and user allocation.
Decision variables
------------------
y[j] : binary — 1 if variant j is selected, 0 otherwise
x[s, j] : integer — number of users from segment s allocated to variant j
Objective
---------
Minimize total weighted disruption:
sum_{s, j} disruption_cost[s, j] * x[s, j]
"""
def __init__(self, env):
"""Initialize optimizer with a Gurobi environment."""
self.env = env
self.model = None
# Data attributes
self.segments = []
self.variants = []
self.total_users = {} # segment -> int
self.disruption_costs = {} # (segment, variant) -> float
self.num_variants_to_select = 3
self.min_users_per_variant = 100
self.max_allocation_spread = 300
# Decision variable containers
self._y = None # variant selection binaries
self._x = None # user allocation integers
def __enter__(self):
"""Enter context manager."""
return self
def __exit__(self, exc_type, exc_val, exc_tb):
"""Dispose the Gurobi model on exit."""
if self.model is not None:
self.model.dispose()
return False
# ------------------------------------------------------------------
# Data loading
# ------------------------------------------------------------------
def set_data(
self,
segments: list,
total_users: dict,
variants: list,
disruption_costs: dict,
num_variants_to_select: int = 3,
min_users_per_variant: int = 100,
max_allocation_spread: int = 300,
):
"""
Load experiment data and parameters into the optimizer.
Parameters
----------
segments : list of segment names
total_users : dict mapping segment -> total user count
variants : list of variant identifiers
disruption_costs : dict mapping (segment, variant) -> per-user disruption cost
num_variants_to_select : number of variants to select (default 3)
min_users_per_variant : min users per selected variant per segment (default 100)
max_allocation_spread : max allowed difference between largest and smallest
allocation within a segment across chosen variants (default 300)
"""
self.segments = segments
self.total_users = total_users
self.variants = variants
self.disruption_costs = disruption_costs
self.num_variants_to_select = num_variants_to_select
self.min_users_per_variant = min_users_per_variant
self.max_allocation_spread = max_allocation_spread
# ------------------------------------------------------------------
# Model building
# ------------------------------------------------------------------
def build_model(self):
"""Construct the MIP model: variables, objective, and constraints."""
self.model = gp.Model("ABTestOptimization", env=self.env)
self._create_variables()
self._set_objective()
self._add_constraints()
def _create_variables(self):
"""Declare binary selection and integer allocation decision variables."""
# y[j] = 1 if variant j is selected for the experiment
self._y = self.model.addVars(
self.variants, vtype=GRB.BINARY, name="select"
)
# x[s, j] = number of users from segment s assigned to variant j
self._x = self.model.addVars(
self.segments, self.variants, vtype=GRB.INTEGER, lb=0, name="alloc"
)
def _set_objective(self):
"""Minimize total weighted disruption across all segment-variant pairs."""
self.model.setObjective(
gp.quicksum(
self.disruption_costs[s, j] * self._x[s, j]
for s in self.segments
for j in self.variants
),
GRB.MINIMIZE,
)
def _add_constraints(self):
"""Add all business constraints to the model."""
segments = self.segments
variants = self.variants
# Big-M: largest possible allocation in any single cell
M = max(self.total_users.values())
# C1: Exactly K variants must be selected
self.model.addConstr(
gp.quicksum(self._y[j] for j in variants) == self.num_variants_to_select,
name="c1_select_k_variants",
)
# C2: Every user in each segment must be allocated (full coverage)
for s in segments:
self.model.addConstr(
gp.quicksum(self._x[s, j] for j in variants) == self.total_users[s],
name=f"c2_full_alloc_{s}",
)
# C3: Allocations to unselected variants must be zero
for s in segments:
for j in variants:
self.model.addConstr(
self._x[s, j] <= M * self._y[j],
name=f"c3_no_alloc_unselected_{s}_{j}",
)
# C4: Each selected variant must receive at least min_users per segment
for s in segments:
for j in variants:
self.model.addConstr(
self._x[s, j] >= self.min_users_per_variant * self._y[j],
name=f"c4_min_users_{s}_{j}",
)
# C5: Balance — pairwise spread within each segment <= max_allocation_spread
# Linearized with big-M: active only when y[j]=1 and y[k]=1
for s in segments:
for j in variants:
for k in variants:
if j != k:
self.model.addConstr(
self._x[s, j] - self._x[s, k]
<= self.max_allocation_spread
+ M * (2 - self._y[j] - self._y[k]),
name=f"c5_balance_{s}_{j}_{k}",
)
# ------------------------------------------------------------------
# Solving
# ------------------------------------------------------------------
def solve(self):
"""Run the optimizer and return the Gurobi status code."""
self.model.optimize()
return self.model.Status
# ------------------------------------------------------------------
# Solution retrieval
# ------------------------------------------------------------------
def get_solution(self) -> dict:
"""
Extract and return the solution as a structured dictionary.
Returns
-------
dict with keys:
status : Gurobi status code
is_optimal : True if an optimal solution was found
objective_value : total weighted disruption (or None if infeasible)
selected_variants: list of selected variant ids
allocation : dict of segment -> variant -> users allocated
"""
status = self.model.Status
is_optimal = status == GRB.OPTIMAL
if not is_optimal:
return {
"status": status,
"is_optimal": False,
"objective_value": None,
"selected_variants": [],
"allocation": {},
}
selected_variants = [j for j in self.variants if self._y[j].X > 0.5]
allocation = {
s: {j: round(self._x[s, j].X) for j in selected_variants}
for s in self.segments
}
return {
"status": status,
"is_optimal": True,
"objective_value": self.model.ObjVal,
"selected_variants": selected_variants,
"allocation": allocation,
}
# ------------------------------------------------------------------
# Display
# ------------------------------------------------------------------
def display_results(self):
"""Print a formatted summary of the optimal solution."""
solution = self.get_solution()
if not solution["is_optimal"]:
print(f"No optimal solution found. Gurobi status: {solution['status']}")
return
sel = solution["selected_variants"]
print("\n" + "=" * 60)
print("A/B Test Optimization Results")
print("=" * 60)
print(f"Selected Variants : {', '.join(sel)}")
print(f"Total Weighted Disruption: {solution['objective_value']:.2f}")
print("\nUser Allocation per Segment:")
col_w = 12
header = f"{'Segment':<20}" + "".join(f"{j:>{col_w}}" for j in sel) + f"{'Total':>{col_w}}"
print(header)
print("-" * len(header))
for s in self.segments:
row = f"{s:<20}"
total = 0
for j in sel:
val = solution["allocation"][s].get(j, 0)
row += f"{val:>{col_w}}"
total += val
row += f"{total:>{col_w}}"
print(row)
print("=" * 60)
# ----------------------------------------------------------------------
# Data loading helper
# ----------------------------------------------------------------------
def load_csv_data(filepath: str):
"""Parse the A/B test CSV into the data structures expected by ABTestOptimizer."""
df = pd.read_csv(filepath)
segments = df["User Segment"].tolist()
total_users = dict(zip(df["User Segment"], df["Total Users"]))
variant_cols = [c for c in df.columns if c.startswith("Disruption in ")]
variants = [c.replace("Disruption in ", "") for c in variant_cols]
disruption_costs = {}
for _, row in df.iterrows():
s = row["User Segment"]
for col, v in zip(variant_cols, variants):
disruption_costs[s, v] = float(row[col])
return segments, total_users, variants, disruption_costs
# ----------------------------------------------------------------------
# Main — demonstrates the optimizer with the provided dataset
# ----------------------------------------------------------------------
def main():
segments, total_users, variants, disruption_costs = load_csv_data("ab_testing.csv")
with gp.Env(empty=True) as env:
env.setParam("OutputFlag", 1)
env.setParam("TimeLimit", 60)
env.start()
with ABTestOptimizer(env) as optimizer:
optimizer.set_data(
segments=segments,
total_users=total_users,
variants=variants,
disruption_costs=disruption_costs,
num_variants_to_select=3,
min_users_per_variant=100,
max_allocation_spread=300,
)
optimizer.build_model()
optimizer.solve()
optimizer.display_results()
if __name__ == "__main__":
main()
Alongside the implementation, Modeler writes a pytest suite with one test
class per scenario in the specification. Each class references its scenario
identifier, and uses the small hard-coded segment data and a short solver time
limit so the suite runs in well under a second.
"""
Unit tests for the A/B Test Variant Selection and User Allocation Optimizer.
Covers all scenarios defined in the specification (T1-T5).
"""
import time
import pytest
from ab_testing_modeler import ABTestOptimizer, load_csv_data
import gurobipy as gp
TOLERANCE = 1e-6
TIME_LIMIT = 10 # seconds per test (Gurobi)
# ---------------------------------------------------------------------------
# Module-level constants shared across test classes
# ---------------------------------------------------------------------------
_FULL_SEGMENTS = ["New Users", "Regular Users", "Power Users"]
_FULL_TOTAL_USERS = {"New Users": 1000, "Regular Users": 1500, "Power Users": 500}
_FULL_VARIANTS = [f"V{i}" for i in range(1, 11)]
# T2: raise V3/V5/V8 to 10, keep everything else at baseline
_T2_COSTS_OVERRIDE = {v: 10.0 for v in ["V3", "V5", "V8"]}
# T3: raise the 7 non-selected variants by 1, keep V3/V5/V8 at baseline
_T3_COSTS_BUMP = {v: 1 for v in ["V1", "V2", "V4", "V6", "V7", "V9", "V10"]}
# ---------------------------------------------------------------------------
# Shared helpers
# ---------------------------------------------------------------------------
def make_env():
"""Create a silent, time-limited Gurobi environment for testing."""
env = gp.Env(empty=True)
env.setParam("OutputFlag", 0)
env.setParam("TimeLimit", TIME_LIMIT)
env.start()
return env
def run_optimizer(segments, total_users, variants, disruption_costs,
num_variants_to_select=3, min_users_per_variant=100,
max_allocation_spread=300):
"""Convenience wrapper: build, solve, and return solution dict."""
with make_env() as env:
with ABTestOptimizer(env) as opt:
opt.set_data(
segments=segments,
total_users=total_users,
variants=variants,
disruption_costs=disruption_costs,
num_variants_to_select=num_variants_to_select,
min_users_per_variant=min_users_per_variant,
max_allocation_spread=max_allocation_spread,
)
opt.build_model()
opt.solve()
return opt.get_solution()
def build_costs_from_csv(overrides=None, bumps=None):
"""
Load baseline costs from CSV, apply per-variant scalar overrides and/or
additive bumps (dicts of variant -> value).
"""
_, _, _, dc = load_csv_data("ab_testing.csv")
dc = dict(dc)
if overrides:
for v, new_cost in overrides.items():
for s in _FULL_SEGMENTS:
dc[s, v] = float(new_cost)
if bumps:
for v, delta in bumps.items():
for s in _FULL_SEGMENTS:
dc[s, v] = dc[s, v] + delta
return dc
def naive_heuristic_objective(disruption_costs, segments, total_users, variants, k=3):
"""
Compute the objective value of the naive heuristic:
- Select the k variants with the lowest average disruption cost
(ties broken by alphabetical order of variant id).
- Allocate each segment evenly (floor division; remainder to first variant).
Returns (naive_obj, naive_variants, naive_allocation).
"""
avg_cost = {v: sum(disruption_costs[s, v] for s in segments) / len(segments)
for v in variants}
naive_variants = sorted(variants, key=lambda v: (avg_cost[v], v))[:k]
alloc = {}
for s in segments:
n = total_users[s]
base, rem = divmod(n, k)
alloc[s] = {v: base + (1 if i < rem else 0)
for i, v in enumerate(naive_variants)}
naive_obj = sum(
disruption_costs[s, v] * alloc[s][v]
for s in segments for v in naive_variants
)
return naive_obj, naive_variants, alloc
# ---------------------------------------------------------------------------
# T1: Baseline — standard optimization with provided data
# ---------------------------------------------------------------------------
class TestT1Baseline:
"""T1 — Full dataset, default parameters. Reference solution: V3, V5, V8 / 5267."""
EXPECTED_VARIANTS = {"V3", "V5", "V8"}
EXPECTED_OBJECTIVE = 5267.0
EXPECTED_ALLOCATION = {
"New Users": {"V3": 533, "V5": 233, "V8": 234},
"Regular Users": {"V3": 300, "V5": 600, "V8": 600},
"Power Users": {"V3": 300, "V5": 100, "V8": 100},
}
@pytest.fixture(scope="class")
def solution(self):
costs = build_costs_from_csv()
t0 = time.time()
sol = run_optimizer(_FULL_SEGMENTS, _FULL_TOTAL_USERS, _FULL_VARIANTS, costs)
assert time.time() - t0 < TIME_LIMIT, "T1 exceeded time limit"
return sol
def test_t1_is_optimal(self, solution):
"""T1: Model must find an optimal solution."""
assert solution["is_optimal"] is True
def test_t1_selected_variants(self, solution):
"""T1: Selected variants must be exactly {V3, V5, V8}."""
assert set(solution["selected_variants"]) == self.EXPECTED_VARIANTS
def test_t1_objective(self, solution):
"""T1: Total weighted disruption must equal 5267."""
assert abs(solution["objective_value"] - self.EXPECTED_OBJECTIVE) <= TOLERANCE
def test_t1_full_allocation(self, solution):
"""T1: Every user in each segment must be allocated (C2)."""
for s in _FULL_SEGMENTS:
allocated = sum(solution["allocation"][s].values())
assert abs(allocated - _FULL_TOTAL_USERS[s]) <= TOLERANCE
def test_t1_min_users_per_variant(self, solution):
"""T1: Each selected variant must receive >= 100 users per segment (C4)."""
for s in _FULL_SEGMENTS:
for j in solution["selected_variants"]:
assert solution["allocation"][s][j] >= 100 - TOLERANCE, (
f"Segment '{s}', variant '{j}': {solution['allocation'][s][j]} < 100"
)
def test_t1_balance_constraint(self, solution):
"""T1: Spread within each segment must not exceed 300 (C5)."""
for s in _FULL_SEGMENTS:
allocs = list(solution["allocation"][s].values())
assert max(allocs) - min(allocs) <= 300 + TOLERANCE
def test_t1_exact_allocation(self, solution):
"""T1: Cell-level allocation must match the verified reference solution."""
for s in _FULL_SEGMENTS:
for v in self.EXPECTED_VARIANTS:
assert solution["allocation"][s][v] == self.EXPECTED_ALLOCATION[s][v], (
f"Mismatch at ({s}, {v}): "
f"got {solution['allocation'][s][v]}, "
f"expected {self.EXPECTED_ALLOCATION[s][v]}"
)
# ---------------------------------------------------------------------------
# T2: Selection-changing perturbation
# ---------------------------------------------------------------------------
class TestT2SelectionChanging:
"""T2 — Raise V3/V5/V8 costs to 10; optimizer must switch to V1, V2, V7."""
EXPECTED_VARIANTS = {"V1", "V2", "V7"}
EXPECTED_OBJECTIVE = 6000.0
@pytest.fixture(scope="class")
def solution(self):
costs = build_costs_from_csv(overrides=_T2_COSTS_OVERRIDE)
t0 = time.time()
sol = run_optimizer(_FULL_SEGMENTS, _FULL_TOTAL_USERS, _FULL_VARIANTS, costs)
assert time.time() - t0 < TIME_LIMIT, "T2 exceeded time limit"
return sol
def test_t2_is_optimal(self, solution):
"""T2: Model must find an optimal solution after perturbation."""
assert solution["is_optimal"] is True
def test_t2_selected_variants_changed(self, solution):
"""T2: Baseline variants V3/V5/V8 must NOT be selected after cost raise."""
assert set(solution["selected_variants"]).isdisjoint({"V3", "V5", "V8"})
def test_t2_selected_variants_exact(self, solution):
"""T2: New selected variants must be exactly {V1, V2, V7}."""
assert set(solution["selected_variants"]) == self.EXPECTED_VARIANTS
def test_t2_objective(self, solution):
"""T2: Total weighted disruption must equal 6000."""
assert abs(solution["objective_value"] - self.EXPECTED_OBJECTIVE) <= TOLERANCE
def test_t2_full_allocation(self, solution):
"""T2: Every user in each segment must still be allocated (C2)."""
for s in _FULL_SEGMENTS:
allocated = sum(solution["allocation"][s].values())
assert abs(allocated - _FULL_TOTAL_USERS[s]) <= TOLERANCE
def test_t2_balance_constraint(self, solution):
"""T2: Spread within each segment must not exceed 300 (C5)."""
for s in _FULL_SEGMENTS:
allocs = list(solution["allocation"][s].values())
assert max(allocs) - min(allocs) <= 300 + TOLERANCE
# ---------------------------------------------------------------------------
# T3: Robustness perturbation (selection unchanged)
# ---------------------------------------------------------------------------
class TestT3Robustness:
"""T3 — Raise non-selected variant costs by 1; V3/V5/V8 must still be selected."""
EXPECTED_VARIANTS = {"V3", "V5", "V8"}
EXPECTED_OBJECTIVE = 5267.0
EXPECTED_ALLOCATION = {
"New Users": {"V3": 533, "V5": 233, "V8": 234},
"Regular Users": {"V3": 300, "V5": 600, "V8": 600},
"Power Users": {"V3": 300, "V5": 100, "V8": 100},
}
@pytest.fixture(scope="class")
def solution(self):
costs = build_costs_from_csv(bumps=_T3_COSTS_BUMP)
t0 = time.time()
sol = run_optimizer(_FULL_SEGMENTS, _FULL_TOTAL_USERS, _FULL_VARIANTS, costs)
assert time.time() - t0 < TIME_LIMIT, "T3 exceeded time limit"
return sol
def test_t3_is_optimal(self, solution):
"""T3: Model must find an optimal solution after robustness perturbation."""
assert solution["is_optimal"] is True
def test_t3_selected_variants_unchanged(self, solution):
"""T3: Selected variants must remain {V3, V5, V8} after non-selected cost bump."""
assert set(solution["selected_variants"]) == self.EXPECTED_VARIANTS
def test_t3_objective_unchanged(self, solution):
"""T3: Objective must remain 5267 (V3/V5/V8 costs unchanged)."""
assert abs(solution["objective_value"] - self.EXPECTED_OBJECTIVE) <= TOLERANCE
def test_t3_allocation_identical_to_baseline(self, solution):
"""T3: Allocation must be identical to the baseline solution."""
for s in _FULL_SEGMENTS:
for v in self.EXPECTED_VARIANTS:
assert solution["allocation"][s][v] == self.EXPECTED_ALLOCATION[s][v], (
f"Mismatch at ({s}, {v}): "
f"got {solution['allocation'][s][v]}, "
f"expected {self.EXPECTED_ALLOCATION[s][v]}"
)
def test_t3_full_allocation(self, solution):
"""T3: Every user in each segment must still be allocated (C2)."""
for s in _FULL_SEGMENTS:
allocated = sum(solution["allocation"][s].values())
assert abs(allocated - _FULL_TOTAL_USERS[s]) <= TOLERANCE
# ---------------------------------------------------------------------------
# T4: Infeasible — over-tight balance constraint (max_allocation_spread = 0)
# ---------------------------------------------------------------------------
class TestT4Infeasible:
"""T4 — max_allocation_spread=0 makes equal integer split impossible for
1000 and 500 users split across 3 variants."""
@pytest.fixture(scope="class")
def solution(self):
costs = build_costs_from_csv()
t0 = time.time()
sol = run_optimizer(
_FULL_SEGMENTS, _FULL_TOTAL_USERS, _FULL_VARIANTS, costs,
max_allocation_spread=0,
)
assert time.time() - t0 < TIME_LIMIT, "T4 exceeded time limit"
return sol
def test_t4_not_optimal(self, solution):
"""T4: Model must NOT find an optimal solution (infeasible)."""
assert solution["is_optimal"] is False
def test_t4_no_objective_value(self, solution):
"""T4: objective_value must be None when infeasible."""
assert solution["objective_value"] is None
def test_t4_no_selected_variants(self, solution):
"""T4: selected_variants must be empty when infeasible."""
assert solution["selected_variants"] == []
# ---------------------------------------------------------------------------
# T5: Comparison against naive heuristic baseline
# ---------------------------------------------------------------------------
class TestT5NaiveBaseline:
"""T5 — Verify the optimizer beats the naive lowest-average-cost heuristic.
Naive picks V1, V3, V5 (alphabetical tiebreak on avg cost 2.0) and splits
evenly -> objective 6333. Optimizer achieves 5267 -> 1066 improvement (16.8%).
"""
EXPECTED_NAIVE_VARIANTS = ["V1", "V3", "V5"]
EXPECTED_NAIVE_OBJECTIVE = 6333.0
EXPECTED_OPT_OBJECTIVE = 5267.0
EXPECTED_IMPROVEMENT = 1066.0
MIN_IMPROVEMENT_PCT = 15.0
@pytest.fixture(scope="class")
def costs(self):
return build_costs_from_csv()
@pytest.fixture(scope="class")
def naive_result(self, costs):
naive_obj, naive_variants, naive_alloc = naive_heuristic_objective(
costs, _FULL_SEGMENTS, _FULL_TOTAL_USERS, _FULL_VARIANTS, k=3
)
return naive_obj, naive_variants, naive_alloc
@pytest.fixture(scope="class")
def opt_solution(self, costs):
t0 = time.time()
sol = run_optimizer(_FULL_SEGMENTS, _FULL_TOTAL_USERS, _FULL_VARIANTS, costs)
assert time.time() - t0 < TIME_LIMIT, "T5 opt exceeded time limit"
return sol
def test_t5_naive_variants(self, naive_result):
"""T5: Naive heuristic must select V1, V3, V5 (alphabetical tiebreak)."""
_, naive_variants, _ = naive_result
assert naive_variants == self.EXPECTED_NAIVE_VARIANTS
def test_t5_naive_objective(self, naive_result):
"""T5: Naive heuristic objective must equal 6333."""
naive_obj, _, _ = naive_result
assert abs(naive_obj - self.EXPECTED_NAIVE_OBJECTIVE) <= TOLERANCE
def test_t5_naive_full_allocation(self, naive_result):
"""T5: Naive heuristic must allocate all users in every segment (C2)."""
_, _, naive_alloc = naive_result
for s in _FULL_SEGMENTS:
assert sum(naive_alloc[s].values()) == _FULL_TOTAL_USERS[s]
def test_t5_opt_is_optimal(self, opt_solution):
"""T5: Optimizer must find an optimal solution."""
assert opt_solution["is_optimal"] is True
def test_t5_opt_objective(self, opt_solution):
"""T5: Optimizer objective must equal 5267."""
assert abs(opt_solution["objective_value"] - self.EXPECTED_OPT_OBJECTIVE) <= TOLERANCE
def test_t5_improvement_absolute(self, naive_result, opt_solution):
"""T5: Absolute improvement over naive must equal 1066."""
naive_obj, _, _ = naive_result
improvement = naive_obj - opt_solution["objective_value"]
assert abs(improvement - self.EXPECTED_IMPROVEMENT) <= TOLERANCE
def test_t5_improvement_percentage(self, naive_result, opt_solution):
"""T5: Relative improvement over naive must be >= 15%."""
naive_obj, _, _ = naive_result
pct = (naive_obj - opt_solution["objective_value"]) / naive_obj * 100
assert pct >= self.MIN_IMPROVEMENT_PCT
Generating code and tests is the easy half. Modeler also runs the suite in an isolated environment created fresh for this conversation, solving against Gurobi in the cloud — you do not need Gurobi installed locally, and you do not need a license of your own to see the model solve. It reads the output and tells you honestly whether it passed, rather than a plausible-looking summary.
Yes, that looks right — please finalize the specification, implement the model in gurobipy, write the pytest test suite covering the scenarios, run it, and report the results. Also derive the mathematical formulation (LaTeX) for the model once it’s built.
The specification already settled every open question after the data
upload — there is no remaining implementation detail to check with the
user before writing code. Proceeding straight to implementation: the
ABTestOptimizer class, a pytest suite with one class per scenario
(T1-T5), and then the formulation once the tests pass.
Running the test suite.
The run
All four deliverables are complete. Here is a full summary of what was built and the results.
$ pytest test_ab_testing_modeler.py -q
............................
28 passed in 0.67s
Gurobi Optimizer version 12.x (linux64)
Optimize a model with 91 rows, 40 columns and 700 nonzeros
Variable types: 30 continuous, 10 integer (10 binary)
...
Explored 1 nodes in 0.01 seconds (0.00 work units)
Best objective 5.267000000000e+03, best bound 5.267000000000e+03, gap 0.0000%
28/28 tests passed on the first run — no fix was needed this time.
Optimal solution (T1, baseline)
Selected variants: V3, V5, V8 — total weighted disruption 5,267:
Segment |
V3 |
V5 |
V8 |
|---|---|---|---|
New Users |
533 |
233 |
234 |
Regular Users |
300 |
600 |
600 |
Power Users |
300 |
100 |
100 |
That beats the naive “lowest-average-cost” heuristic from scenario T5 — which would pick V1, V3, V5 and split each segment evenly — by 1,066 (16.8%).
Files created
File |
Description |
|---|---|
|
Complete problem specification |
|
Input disruption-cost dataset |
|
Gurobipy implementation ( |
|
Pytest suite covering scenarios T1-T5 |
Test results
Test |
Scenario |
Result |
|---|---|---|
T1 |
Baseline — selects V3, V5, V8 at 5,267 |
PASSED |
T2 |
Raising V3/V5/V8 cost to 10 flips the pick to V1, V2, V7 at 6,000 |
PASSED |
T3 |
Raising non-selected costs by 1 leaves V3, V5, V8 unchanged |
PASSED |
T4 |
Zero balance spread is infeasible |
PASSED |
T5 |
Optimized plan beats the naive heuristic by 1,066 (16.8%) |
PASSED |
Let me know if you would like any changes, such as adjusting the variant pool, adding a new constraint, or generating the mathematical formulation.
That log lands in the workspace as execution.log alongside the code, so the
solve is auditable rather than asserted.
The loop ends one of two ways. Either every test passes, as here — or Modeler concludes a failure reflects a genuine contradiction between the specification, the implementation and the tests. In that case it stops instead of guessing, tells you what it found, and asks which side is the one to correct. That second path is what the Changing the model tab demonstrates, once the compliance rule for V9 exposes an ambiguity of its own.
What the cloud environment gives you, and what it does not
The environment is built for validating a model, not for running production
workloads. Python 3 is available with gurobipy, gurobipy-pandas,
numpy, pandas, scipy and pytest pre-installed. Within that:
Solves use a Gurobi trial license, capped at 2,000 variables and 2,000 constraints. This model, at 40 columns and 91 rows, is nowhere near the cap — but a full-scale version with more segments or a larger variant pool could be.
Execution is limited to 60 seconds per command, with tight CPU and memory limits. This is why the generated tests use small hard-coded data and a short solver time limit.
Additional packages cannot be installed, there is no internet access, and Compute Server and Instant Cloud cannot be used.
So treat a passing suite as evidence that the model is right, not as a performance measurement. Download the implementation and run it against your own license and your own full data set for that. See Solving limits for the details.
Business problems do not hold still. The specification above puts constraints on which variants may be selected out of scope — until legal flags that one specific variant cannot be shown to one specific segment. Ask for the change in plain language and Modeler updates the specification first, then propagates it to the implementation and the tests, in that order.
One more requirement just came in from legal: variant V9 must never be shown to Power Users at all, for compliance reasons — though it’s still fine to test V9 with the other segments. Can you update the model for this?
This is a scope change, not a data correction: the specification currently lists “no constraint on which specific variants must or must not be selected” under Out of scope. The new rule is narrower than banning V9 outright — it forbids one (segment, variant) cell, not the variant itself, so V9 stays a selectable candidate. That interacts with C4, which currently requires every selected variant to receive at least 100 users from every segment, including Power Users. Enforcing both as written is a direct contradiction whenever V9 is selected, so C4 has to be updated too, not just have a new constraint bolted on. Updating the specification first, then the implementation and tests.
That moves this from out of scope into the model. I have updated the specification:
segment_variant_exclusions field on Experiment Parameters, defaulting to [("Power Users", "V9")].One thing to flag before I implement it: because the exclusion is per-segment rather than per-variant, V9 remains fully eligible for selection — it can still win New Users and Regular Users traffic. If what you actually meant was that V9 should never be tested at all, that is a different, stronger rule (removing V9 from the candidate pool entirely), and I’d implement that differently.
No, that’s right — V9 can still run, it just can’t touch Power Users. Go ahead.
Specification, then code, then tests
The specification changes first. Constraint C6 reads:
C6 — Compliance exclusions. Certain (segment, variant) pairs are explicitly
forbidden for legal or compliance reasons. The allocation of users from a
forbidden segment to the corresponding variant must be zero, regardless of
whether that variant is selected. Currently: variant V9 must receive zero
users from the Power Users segment.
The implementation follows the specification, not the other way around — three
surgical changes to _add_constraints, not a rewrite:
def _add_constraints(self):
"""Add all business constraints to the model."""
segments = self.segments
variants = self.variants
+ exclusions = set(self.segment_variant_exclusions)
# Big-M: largest possible allocation in any single cell
M = max(self.total_users.values())
# C1: Exactly K variants must be selected
...
- # C4: Each selected variant must receive at least min_users per segment
for s in segments:
for j in variants:
- self.model.addConstr(
- self._x[s, j] >= self.min_users_per_variant * self._y[j],
- name=f"c4_min_users_{s}_{j}",
- )
+ if (s, j) not in exclusions:
+ self.model.addConstr(
+ self._x[s, j] >= self.min_users_per_variant * self._y[j],
+ name=f"c4_min_users_{s}_{j}",
+ )
# C5: Balance — pairwise spread within each segment <= max_allocation_spread
for s in segments:
for j in variants:
for k in variants:
- if j != k:
+ if j != k and (s, j) not in exclusions and (s, k) not in exclusions:
self.model.addConstr(
self._x[s, j] - self._x[s, k]
<= self.max_allocation_spread
+ M * (2 - self._y[j] - self._y[k]),
name=f"c5_balance_{s}_{j}_{k}",
)
+ # C6: Compliance exclusions — forbidden (segment, variant) allocations
+ # are fixed to zero regardless of whether the variant is selected.
+ for s, j in exclusions:
+ if s in segments and j in variants:
+ self.model.addConstr(
+ self._x[s, j] == 0,
+ name=f"c6_compliance_{s}_{j}",
+ )
A new test class covers the C6 contract directly, forcing V9 to be selected so the exclusion actually gets exercised rather than trivially satisfied:
class TestC6ComplianceExclusion:
"""C6 contract — V9 has cost 0 (forces selection), yet Power Users must get 0."""
@pytest.fixture(scope="class")
def solution(self):
# V9 priced at 0, everything else at 999, to force V9 into the pick.
costs = {(s, v): (0.0 if v == "V9" else 999.0)
for s in _FULL_SEGMENTS for v in _FULL_VARIANTS}
return run_optimizer(
_FULL_SEGMENTS, _FULL_TOTAL_USERS, _FULL_VARIANTS, costs,
segment_variant_exclusions=[("Power Users", "V9")],
)
def test_c6_v9_is_selected(self, solution):
"""C6: V9 must be selected (zero cost makes it dominant)."""
assert "V9" in solution["selected_variants"]
def test_c6_power_users_v9_is_zero(self, solution):
"""C6: Power Users must receive zero allocation to V9."""
assert solution["allocation"]["Power Users"]["V9"] == 0
def test_c6_new_users_v9_nonzero(self, solution):
"""C6: New Users must be allocated to V9 (no exclusion applies)."""
assert solution["allocation"]["New Users"]["V9"] > 0
Modeler re-runs the whole suite, not just the new class — 34 tests pass, confirming the exclusion holds without breaking any of T1-T5.
Where it checks its own reasoning
Before accepting the change, it is worth pushing back on the choice Modeler made silently: it waived C4 for the excluded pair rather than barring V9 from selection outright. Asked to justify that instead of just implementing it, Modeler lays out both readings side by side:
Reading A (implemented) |
Reading B (alternative) |
|
|---|---|---|
V9 selectable? |
Yes |
No |
Matches “fine for other segments”? |
Yes |
No |
C4 waiver needed for the excluded pair? |
Yes |
Not applicable — V9 simply excluded |
Reading B would make V9 unselectable in every circumstance, which contradicts the legal team’s own wording — “it’s still fine to test V9 with the other segments” explicitly preserves V9’s eligibility. Structurally, C2 (full allocation) only requires every user in a segment to reach some selected variant, not every selected variant to serve every segment — so a selected V9 simply routes zero Power Users and picks up its share of New Users and Regular Users instead. Waiving C4 for the excluded pair is not a business judgment call, it is a necessity: requiring at least 100 Power Users for a cell that C6 fixes to zero would be a direct contradiction, not a competing valid interpretation. Would you like to proceed with the current implementation as-is, Orhan?
Yes, proceed with Reading A.
This is the payoff of writing a spec-first tool: rather than silently applying whichever reading it happened to pick, Modeler can be asked to show its reasoning, and does so by tracing the conflict back to the specific constraints (C2, C4, C6) that made one reading structurally necessary rather than a preference. On a problem where both readings were genuinely valid — say, if the compliance rule had been ambiguous about whether V9 could be tested at all — this is the same moment where Modeler would stop and ask before implementing either one, instead of picking silently.
The specification, the implementation and the tests never mention mathematics — that is deliberate, so the business owner can read them. But you may still need the formulation: to review the model, to document it, or to hand it to someone who thinks in constraints rather than in code. Ask at any point and Modeler derives one from the current specification and implementation, in Markdown with LaTeX, and saves it to the workspace with everything else.
Sets and indices
\(s \in S\): user segments, \(|S| = 3\).
\(j, k \in V\): candidate variants, \(|V| = 10\).
Parameters
\(N_s\): total number of users in segment \(s \in S\).
\(c_{s,j}\): per-user disruption cost for assigning a user from segment \(s\) to variant \(j\).
\(K\): number of variants to select (fixed at 3).
\(L\): minimum users a selected variant must receive from each segment (fixed at 100).
\(D\): maximum allowed allocation spread within a segment across selected variants (fixed at 300).
\(M\): big-M constant, \(M = \max_{s \in S} N_s\).
Decision variables
\(y_j \in \{0, 1\}\): 1 if variant \(j \in V\) is selected, 0 otherwise.
\(x_{s,j} \in \mathbb{Z}_{\geq 0}\): number of users from segment \(s \in S\) allocated to variant \(j \in V\).
Objective function
Minimize the total weighted disruption across all segment-variant allocations:
Constraints
C1 — Exactly K variants selected.
C2 — Full allocation of each segment.
C3 — No allocation to unselected variants.
C4 — Minimum users per selected variant per segment.
C5 — Balance constraint. For any two variants \(j\) and \(k\), the difference in their allocations within a segment must not exceed \(D\). Enforced pairwise with a big-M relaxation that deactivates the constraint unless both variants are selected:
Complete model
Additional notes
The model is a Mixed-Integer Linear Program: binary variables \(y_j\) select variants, general integer variables \(x_{s,j}\) allocate users, and both the objective and every constraint are linear.
With the provided data (\(|S|=3\), \(|V|=10\), \(M=1500\)), the model has \(10\) binary and \(30\) integer variables, and \(1 + 3 + 30 + 30 + 270 = 334\) constraints — C5’s pairwise linearization accounts for the vast majority of that count.
The big-M value \(M = \max_s N_s\) is tight: it is the largest value any \(x_{s,j}\) could ever take, bounded above by \(N_s\) through C2 and C3. A tighter per-segment big-M, \(M_s = N_s\), in C3 and C5 would strengthen the LP relaxation and can help solver performance on larger instances.
Note that this is the formulation of the original model, before the compliance-exclusion constraint (C6) added in the Changing the model tab, which fixes specific \((s, j)\) cells to zero and correspondingly narrows the index sets of C4 and C5.
### Sets and indices
- $s \in S$: user segments, $|S| = 3$.
- $j, k \in V$: candidate variants, $|V| = 10$.
### Parameters
- $N_s$: total number of users in segment $s \in S$.
- $c_{s,j}$: per-user disruption cost for assigning a user from segment $s$ to variant $j$.
- $K$: number of variants to select (fixed at 3).
- $L$: minimum users a selected variant must receive from each segment (fixed at 100).
- $D$: maximum allowed allocation spread within a segment across selected variants (fixed at 300).
- $M$: big-M constant, $M = \max_{s \in S} N_s$.
### Decision variables
- $y_j \in \{0, 1\}$: 1 if variant $j \in V$ is selected, 0 otherwise.
- $x_{s,j} \in \mathbb{Z}_{\geq 0}$: number of users from segment $s \in S$ allocated to variant $j \in V$.
### Objective function
Minimize the total weighted disruption across all segment-variant allocations:
$$
\min \; \sum_{s \in S} \sum_{j \in V} c_{s,j} \, x_{s,j}
$$
### Constraints
**C1 — Exactly K variants selected.**
$$
\sum_{j \in V} y_j = K
$$
**C2 — Full allocation of each segment.**
$$
\sum_{j \in V} x_{s,j} = N_s \qquad \forall s \in S
$$
**C3 — No allocation to unselected variants.**
$$
x_{s,j} \leq M \, y_j \qquad \forall s \in S, \, j \in V
$$
**C4 — Minimum users per selected variant per segment.**
$$
x_{s,j} \geq L \, y_j \qquad \forall s \in S, \, j \in V
$$
**C5 — Balance constraint.** For any two variants $j$ and $k$, the difference
in their allocations within a segment must not exceed $D$. Enforced pairwise
with a big-M relaxation that deactivates the constraint unless both variants
are selected:
$$
x_{s,j} - x_{s,k} \leq D + M \left(2 - y_j - y_k\right) \qquad \forall s \in S, \, j \in V, \, k \in V, \, j \neq k
$$
### Complete model
$$
\begin{aligned}
\min \quad & \sum_{s \in S} \sum_{j \in V} c_{s,j} \, x_{s,j} \\
\text{s.t.} \quad & \sum_{j \in V} y_j = K & \text{(C1)} \\
& \sum_{j \in V} x_{s,j} = N_s & \forall s \in S & \text{(C2)} \\
& x_{s,j} \leq M \, y_j & \forall s \in S, \, j \in V & \text{(C3)} \\
& x_{s,j} \geq L \, y_j & \forall s \in S, \, j \in V & \text{(C4)} \\
& x_{s,j} - x_{s,k} \leq D + M(2 - y_j - y_k) & \forall s \in S, \, j \neq k \in V & \text{(C5)} \\
& y_j \in \{0, 1\} & \forall j \in V \\
& x_{s,j} \in \mathbb{Z}_{\geq 0} & \forall s \in S, \, j \in V
\end{aligned}
$$
### Additional notes
- The model is a Mixed-Integer Linear Program: binary variables $y_j$ select
variants, general integer variables $x_{s,j}$ allocate users, and both the
objective and every constraint are linear.
- With the provided data ($|S|=3$, $|V|=10$, $M=1500$), the model has $10$
binary and $30$ integer variables, and $1 + 3 + 30 + 30 + 270 = 334$
constraints — C5's pairwise linearization accounts for the vast majority of
that count.
- The big-M value $M = \max_s N_s$ is tight: it is the largest value any
$x_{s,j}$ could ever take, bounded above by $N_s$ through C2 and C3. A
tighter per-segment big-M, $M_s = N_s$, in C3 and C5 would strengthen the LP
relaxation and can help solver performance on larger instances.
Note that this is the formulation of the original model, before the
compliance-exclusion constraint (C6) added in the **Changing the model** tab,
which fixes specific $(s, j)$ cells to zero and correspondingly narrows the
index sets of C4 and C5.