Counterbalancing & Assignment

Within-subjects structure — factors, crossing, sampling, shuffling — decides what one participant sees and in what order (see Experiment Design). Counterbalancing is the orthogonal question: across many participants, who gets which condition, kept balanced and reproducible. In PsyCloud it's declared, not left to chance, and it's deterministic — a participant's assignment can be reconstructed from the seed.

Two layers that work together

Counterbalancing in PsyCloud has two cooperating pieces:

1. A per-trial assignment step — assign_between. Inside a phase's pipeline, it filters the trial table down to the rows matching this participant's assigned cell. Methods:
   • hash — deterministic from the participant id; no server state required.
   • quota — balanced toward target counts (server-tracked).
   • random — seeded random pick.
2. A bundle-level allocation policy — cb_random / cb_rotation. This declares the factor's levels and the algorithm the server uses to allocate each new participant to a cell, keeping the cells balanced as people arrive.

The policy decides which cell a participant is in; assign_between applies that cell to the trial table at run time. Most studies use both (as below); a quick pilot can use assign_between(method = "hash") alone.

Declare it

# Per-trial: filter the practice/main rows to this participant's mapping cell.
exp |>
  phase("practice", label = "Practice") |>
  trials(both_mappings(practice_rows)) |>
  assign_between(factor = "mapping", method = "hash")
 
# ... later, at the bundle level: declare the balanced allocation policy.
exp |>
  bundle(
    with_auto_ids = TRUE,
    counterbalance = cb_random(mapping = c("A", "B"))
  )

Random vs. rotation

The policy's algorithm decides how participants are spread across cells:

• cb_random assigns each participant at random, weighted so the cells converge to the target proportions over many participants. Assignment is unpredictable; at small N a run can be a little uneven — it's balanced in expectation, not exactly.
• cb_rotation assigns by cycling through the cells (the order reshuffled each full cycle), so the cells are evened out after each complete cycle — the tightest balance, best at small/medium N. If the next scheduled cell is unavailable (e.g. a hard quota is full) it falls back to the balancing (minimization) algorithm.

Rule of thumb: reach for cb_rotation when you want maximally even cells throughout (small/medium N, or cells that must fill in lockstep), and cb_random when you want unpredictable assignment and N is large enough that balance-in-expectation is fine.

# Random: ~50/50 in expectation
bundle(counterbalance = cb_random(mapping = c("A", "B")))
 
# Rotation: as-even-as-possible at every point (tighter at small N)
bundle(counterbalance = cb_rotation(mapping = c("A", "B")))
 
# Multi-factor (2x2): the two axes are crossed into 4 cells
bundle(counterbalance = cb_rotation(
  mapping = c("A", "B"),
  framing = c("gain", "loss")
))

For weighted splits, quotas, balance targets, or streak-avoidance, drop to the lower-level cb_policy() builder. Cells are counted by assigned participants by default (accounting_unit can count started / completed / passed_qc instead).

The both_mappings(...) / practice_rows helpers above are illustrative — see the full runnable study in the repo's examples/high-level/task-switching-gate.

How balance actually happens (hosted vs. preview)

This distinction matters and is easy to get wrong:

• Hosted runs (real participants). At session start the server allocates the participant using the policy's algorithm — cb_rotation cycles through cells, cb_random samples toward the target proportions, and a minimization policy picks the least-filled cell — and records each decision in a ledger-backed allocation store so the running cell counts persist across sessions. The chosen cell is fixed for the session and passed to the runtime, where assign_between applies it. This is where real balance is enforced. (How abandoned sessions are handled depends on the path — see Known limits below.)
• Preview / simulate (no server). There's no ledger, so PsyCloud makes a stable, seeded per-participant pick across the declared levels — enough to exercise the design and reproduce a given participant, but not cross-participant balanced.
• Compiled bundles. A compiled bundle has no participant, so compilation pins one representative cell (the first level of each between-subject axis) consistently across all scopes. Inspect a compiled artifact and you'll see that single cell — not the live per-participant allocation.

Reproducibility

There are two different guarantees, depending on where assignment happens:

• Preview / simulate is seeded — assignment and trial order derive from a deterministic chain (study + participant + namespace), so they reproduce offline from the seed alone.
• Hosted assignment is ledger-backed — the server records each participant's allocation, so you reconstruct it from the stored assignment plus the policy, not the seed alone (balanced allocation depends on arrival order and counts, which a seed can't reproduce by itself).

The within-subject trial order for a given participant is seed-reproducible in both cases. Pair the stored assignment with a preregistered analysis script for fully reproducible results. Changing randomNamespace in the design's determinism settings deliberately invalidates all previously generated streams.

Order counterbalancing (Latin square)

For within-session order effects (e.g. block order), PsyCloud's allocators include Latin-square schemes — cyclic rotation and balanced Williams designs (which control first-order carryover). These are selected through the allocation policy rather than assign_between.

In Studio

When a phase contains an assign_between step, Studio surfaces a between-subjects factor banner in Trial Design with a per-factor method selector — Balanced quota (recommended), Deterministic hash, or Random. Advanced balancing (targets, Latin-square / balanced blocks, sequential effects) is configured through the counterbalance wizard reached from the assignment inspector.

Validation & checking allocation

• psycloud validate <bundle-or-study> and psycloud inspect <bundle-or-study> --section counterbalance check the policy structure (schema 0.1.1): factor ids, non-empty levels, proportions, quotas.
• On a live run, Monitor shows allocation status. Data exports carry an assignment_hash per session, and the participant's cell shows up as the filtered factor values in the trial rows (after assign_between) — there's no separate assigned-cell column, so confirm balance from those.

Known limits (alpha)

Next