# Synthetic discovery study Study artifact accompanying *When Game Discovery Has a Deadline*. It contains no Steam user records and makes no requests to Steam. ## Reproduce Use Python 3.12 and the NumPy version in `requirements.txt`. The captured run records exact versions in `results\manifest.json`. From this directory: ```powershell python -m unittest -v test_study python study.py --output results-reproduced python analyse.py --results results-reproduced ``` Use a new output directory. The simulation refuses to overwrite one that already exists. The full protocol has 512 synthetic worlds and 5,696 policy runs. Expect the full run to take time on a single CPU. Analysis regenerates both SVG figures inside the selected results directory. The initial ten-policy protocol was saved before the first experiment. Its locked SHA-256 was: ```text 68ef4d215474f6e7fa7e051a1212282e0d343377d691d2c337560488a6e682c0 ``` The strong random-then-greedy results exposed a missing simple baseline. Version 2 adds greedy-from-start after the initial run. Every policy and condition is rerun. Seeds and the original policy parameters are unchanged. `protocol.json` records this amendment; the final manifest records its hash. These are local protocol locks, not a public preregistration. The amended protocol SHA-256 is: ```text 736a8d5fb05f6d0c795a0b164f93183f0259deafe860d3c026bc4d39848d970f ``` ## Files | File | Purpose | | --- | --- | | `protocol.json` | Seeds, scenarios, distributions, policies and metric definitions | | `study.py` | Candidate generation, delayed observations and policy execution | | `analyse.py` | Integrity checks, paired bootstrap summaries and figures | | `test_study.py` | Focused tests of decision boundaries and accounting | | `results\raw.csv` | One row per complete policy run | | `results\daily.csv` | Cumulative checkpoints at day boundaries | | `results\summary.csv` | Means and marginal 95% intervals | | `results\paired-vs-thompson.csv` | Paired differences and their 95% intervals | | `results\paired-event-vs-end-tail.csv` | Matched continuation comparison: event-and-end tails minus the end-only tail | | `results\count-bounds.csv` | Arithmetic count-coverage bounds and uniform-sampling calculation | | `results\manifest.json` | Simulation version, protocol hash and raw-file hashes | | `results\analysis-manifest.json` | Analysis hash and summary-file hashes | | `sources.json` | Sources and the limited claims they support | The manuscript is in the parent directory. Results are synthetic evidence, not measurements of the Steam catalogue or its recommendation traffic. ## Decision model Each item has one latent favourable audience. Its response probability in that audience is drawn from `Beta(2, 2)`; other audience probabilities come from `Beta(1, 19)`. Items' latent audiences are uniformly distributed. Player opportunities follow the declared audience shares. An observed label identifies the latent audience with the scenario's stated accuracy. An incorrect label is selected uniformly from the other audiences. Every request has the same 64 candidate opportunities under all policies: 48 distinct items with its observed label and 16 distinct catalogue items, excluding the first 48. The latter 16 can also have the matching label. Candidate generation never reads latent response probabilities. Every audience-item pair has a separate Beta belief. The prior mean is 0.4 for a matching observed label and 0.1 otherwise, with concentration 2 by default. These are imperfect working beliefs, not a correctly specified posterior over the hidden audience-mixture generator. Warm items are selected uniformly, independently of quality. Each has 12 pre-event observations allocated using the scenario's audience shares. Those observations are outside the event budget and are available to every policy. Warm cases do not model fan traffic or popularity-driven selection. One day contains four batches. Outcomes arrive at the next boundary plus the stated extra delay. Consequently, zero extra delay still means batched feedback, not per-request instantaneous learning. For each request the environment draws one uniform random value. The selected item's response is one if that value is below its latent response probability. Sharing this exogenous draw across policies supplies common random numbers; policies cannot inspect it before acting. ## Policies All policies receive only observed labels, delivered outcomes and candidate sets. The evaluator alone uses latent probabilities. - Uniform chooses randomly within each candidate set. - Greedy from start uses the largest posterior mean from the first batch. - Random 1/2/3 days initially chooses randomly, then uses posterior means. Means continue updating after the switch. This is random-then-greedy, not a reproduction of a Steam implementation. - Thompson samples an independent Beta value for each candidate's audience-item pair on each request and chooses the largest. - Thompson + terminal tail uses means when new feedback could no longer inform a subsequent decision inside the objective horizon. - Quota 1/3/5 selects a matching-label pair with the lowest allocated count below the quota, breaking ties using Thompson draws. Otherwise it uses Thompson. Pending and same-batch reservations count when scheduling. - Quota 3 + terminal tail adds the same no-further-learning gate. - The continuation-only event-tail control stops sampling near the event boundary, resumes afterwards, and also uses a terminal tail at the actual end. It isolates treating the event boundary as an additional deadline. Quota scheduling counts delivered observations plus displays whose feedback is pending, including reservations within the current batch. The reported floor metric counts delivered observations, including pre-event observations. ## Metrics `event_utility` is the mean latent response probability of selected items over event opportunities. It reduces binary reward noise, but is accessible only in a synthetic evaluation. Actual sampled rewards are retained too. `cold_relevant_coverage` considers only items with no pre-event observations in any audience. Among those items, it is the fraction of audience-item pairs with latent probability at least 0.5 that receive an event exposure. Pairs on warm items are excluded even if the pair itself has no observations. This is exposure coverage, rather than user-level recall or proof of learning. `target_floor_coverage` is the fraction of items whose observed-label pair has at least three delivered observations by event close. Three is an arbitrary count target, not a statistical confidence standard. `top_one_percent_share` is the share of event displays allocated to the most-displayed ceiling of 1% of items. `late_feedback_fraction` counts event responses not yet delivered at close. Responses delivered at the closing boundary are counted as delivered, although they cannot affect another event decision. The candidate oracle knows all latent probabilities, but can choose only among each request's 64 candidates. It is an unattainable reference for this model, not an implementable baseline. The continuation conditions retain the same items and response probabilities for seven additional days at the same attention rate. Event, post-event and combined utility are reported separately. To isolate the extra event-tail window, compare the event-and-end policy with the end-only-tail policy. Comparing it with ordinary Thompson also changes behaviour near the end of the continuation. ## Uncertainty and limits Intervals are percentile bootstrap intervals for means across the 32 world seeds, using 10,000 resamples. Differences preserve seed pairing. They describe Monte Carlo uncertainty conditional on the model, not uncertainty about Steam or the adequacy of the model. Comparisons are exploratory, with no multiple-comparison significance claims. The matched continuation comparison was added during manuscript QA. It uses the existing paired runs; no simulation policies or world seeds were changed. Both continuation conditions have lower post-event response under event-and-end tails than under the end-only tail, with a higher combined response over the equally weighted event and continuation. The model has no repeated-user fatigue, shared learning across items, varying demo lengths, external publicity or social effects. Its fixed candidate generator does not adapt to policy outcomes. It cannot measure production throughput, actual recommendation quality or developer revenue. ## Publication and licensing Published on 22 October 2026: [When Game Discovery Has a Deadline](https://jasondoyle.ie/whitepapers/when-game-discovery-has-a-deadline/). Python software is MIT-licensed. The report and original study data are CC BY 4.0. File-level terms and notices are in the parent directory's `LICENSES.txt` and `THIRD-PARTY-NOTICES.txt`. The experiment does not redistribute third-party catalogue records.