BanditRLlib
Lean gate passed before this site build; local proof declarations are shown as compiled.

Teaching chapter · canonical scope Planned

10. Automation, resources, and open routes

The proof harness, task vocabulary, resource stopping leaves, literature registry, a partial source-frozen delayed-feedback audit, and planned BwK, preference, robust, federated, neural-bandit, and sharp KL-asymptotic work.

Orientation

Who should read this. Read this chapter to contribute a new route or understand what is deliberately not claimed.

Learning goals

  • Use the same status vocabulary for source declarations, theorem cards, tasks, and blockers.
  • Distinguish a stopping-time foundation from a completed resource-constrained algorithm theorem.
  • Move a literature target through task packets, proof obligations, compilation, and synchronized documentation.

Textbook crosswalk

Read the mathematics before the Lean interface

The Book Map is a curated formalization curriculum anchored in Bandit Algorithms, not a chapter-for-chapter reproduction of one book. Page numbers below use its free online edition; companion papers cover algorithm-specific results.

Primary spine · free online edition

Bandit Algorithms

Tor Lattimore and Csaba Szepesvári

Location
Parts VII–VIII as a background index
Pages
online pp. 358–538
Open the source
contribution flow

How an open theorem becomes a contribution

  1. Fix the mathematical target

    Record the exact source, assumptions, constants, and intended Lean signature.

  2. Retrieve before proving

    Search BanditRLlib, Mathlib, LML theorem cards, and paper/textbook evidence without upgrading cards to proofs.

  3. Split a proof leaf

    Choose one narrow obligation with a clear compile gate and explicit dependencies.

  4. Compile and review

    Only a passing local Lean gate plus review can promote the leaf to compiled status.

  5. Publish the boundary

    Synchronize the declaration index, teaching note, dependencies, and remaining gaps.

No single source theorem. This chapter is a cross-source automation and research frontier, not a completed chapter from one textbook. Compiled infrastructure, blocked algorithm routes, theorem cards, and planned work remain visibly separate.

Natural-language and Lean side by side

The mathematical readings are explanatory summaries. The exact generated Lean statement and its source link remain authoritative for hypotheses, types, constants, and indexing.

Lean declaration

BanditRLProof.Budget.isStoppingTime_budgetExhaustionTime_of_adapted

Compiled

Plain-English statement. The first time an adapted cumulative spending process reaches a fixed budget is a stopping time.

Mathematical reading. The first time an adapted cumulative spending process reaches a fixed budget is a stopping time.
Intuition
Whether the budget has been exhausted by time n can be decided from information available by time n.
Why it is needed
Any rigorous bandits-with-knapsacks route must stop at a random, history-observable exhaustion time.
Place in the proof
This is a compiled foundation leaf, not a BwK model or regret theorem.
Proof idea
Express exhaustion as Mathlib's hittingAfter construction and use adaptedness to prove measurability of each level event.
Lean reading notes
The stopping time takes values in WithTop Nat, allowing the budget never to be reached.
Teaching dependencies
No direct teaching dependency recorded.
Exact Lean statement
theorem isStoppingTime_budgetExhaustionTime_of_adapted {Omega : Type u} [mOmega : MeasurableSpace Omega] {F : Filtration Nat mOmega} {spent : Nat -> Omega -> Nat} (budget : Nat) (hspent : Adapted F spent) : IsStoppingTime F (budgetExhaustionTime spent budget)
Lean declaration

BanditRLProof.HarnessTask

Compiled

Plain-English statement. A harness task records the target, task kind, status, source and scenario cards, profile, required artifacts, and acceptance gates used by the automation system.

Mathematical reading. A harness task records the target, task kind, status, source and scenario cards, profile, required artifacts, and acceptance gates used by the automation system.
Intuition
Proof automation is safer when completion criteria are data that can be inspected, rather than an informal instruction that can drift across runs.
Why it is needed
The website uses the same status discipline: a theorem card, a planned target, and a compiled Lean theorem are visibly different objects.
Place in the proof
This structure belongs to the automation layer rather than the mathematical bandit theory.
Proof idea
It is a structure definition. The default Lean gate later instantiates the acceptance condition with the repository's build command.
Lean reading notes
Inductive TaskStatus and TaskKind values make invalid status strings unrepresentable inside Lean-side harness data.
Teaching dependencies
No direct teaching dependency recorded.
Exact Lean statement
structure HarnessTask where

Chapter implementation status

MilestoneStatusLean declarationRemaining gap
Direct LeanMachineLearning toolchain identityBlockedNo local declaration yetReconcile ABRL's Lean 4.29.1 and Mathlib v4.29.1 environment with the recorded LML seed's newer Lean/Mathlib toolchain in an isolated migration build.
Add a pinned LML dependency and compile the real LeanMachineLearning.Online.Bandit.Algorithms.ETC and UCB imports.
Consume the actual upstream symbols in ABRL wrapper theorems without copied or shadow declarations.
Pass the complete Lean, test, license, notice, attribution, and website gates on the unified toolchain.
Budget-exhaustion stopping timeCompiledBanditRLProof.Budget.isStoppingTime_budgetExhaustionTime_of_adapted
Bandits-with-knapsacks regret theoremBlockedBanditRLProof.Budget.isStoppingTime_budgetExhaustionTime_of_adaptedResource-consumption and feasibility model.
Primal-dual comparison.
Final resource-constrained regret assembly.
Source-faithful delayed-feedback accountingCompiledBanditRLProof.DelayedFeedback.observedBefore
BanditRLProof.DelayedFeedback.outstandingAt
BanditRLProof.DelayedFeedback.observedBefore_disjoint_outstandingAt
BanditRLProof.DelayedFeedback.observedBefore_union_outstandingAt
BanditRLProof.DelayedFeedback.card_observedBefore_add_card_outstandingAt
BanditRLProof.DelayedFeedback.outstandingCount
BanditRLProof.DelayedFeedback.maxOutstandingBeforeThrough
BanditRLProof.DelayedFeedback.outstandingCount_le_round
BanditRLProof.DelayedFeedback.outstandingCount_le_maxOutstandingBeforeThrough
BanditRLProof.DelayedFeedback.oneBasedDelayShift
BanditRLProof.DelayedFeedback.paperMissingAtEnd
BanditRLProof.DelayedFeedback.paperMissingAtEnd_eq_outstandingAt_oneBasedDelayShift
BanditRLProof.DelayedFeedback.paperMissingCount
BanditRLProof.DelayedFeedback.paperMissingCount_eq_outstandingCount_oneBasedDelayShift
BanditRLProof.DelayedFeedback.paperMissingCount_le_round
BanditRLProof.DelayedFeedback.paperSigmaMaxThrough
BanditRLProof.DelayedFeedback.paperMissingCount_le_paperSigmaMaxThrough
Causal action-time view and new-feedback processingCompiledBanditRLProof.DelayedFeedback.ActionTimeView
BanditRLProof.DelayedFeedback.ActionTimeView.ext
BanditRLProof.DelayedFeedback.actionTimeViewAt
BanditRLProof.DelayedFeedback.CausalDecisionRule
BanditRLProof.DelayedFeedback.actionTimeViewAt_pastAction_of_lt
BanditRLProof.DelayedFeedback.actionTimeViewAt_pastAction_of_not_lt
BanditRLProof.DelayedFeedback.actionTimeViewAt_observedLoss_of_mem
BanditRLProof.DelayedFeedback.actionTimeViewAt_observedLoss_of_not_mem
BanditRLProof.DelayedFeedback.actionTimeViewAt_outstanding_loss_hidden
BanditRLProof.DelayedFeedback.actionTimeViewAt_eq_of_observation_equivalent
BanditRLProof.DelayedFeedback.causalDecision_eq_of_observation_equivalent
BanditRLProof.DelayedFeedback.newlyObservedBefore
BanditRLProof.DelayedFeedback.observedBefore_mono
BanditRLProof.DelayedFeedback.processed_disjoint_newlyObservedBefore
BanditRLProof.DelayedFeedback.processed_union_newlyObservedBefore
BanditRLProof.DelayedFeedback.processAllNew
BanditRLProof.DelayedFeedback.processAllNew_eq_observedBefore
BanditRLProof.DelayedFeedback.previousObservedBefore_subset_current
BanditRLProof.DelayedFeedback.processAllNew_from_previous_eq_current
BanditRLProof.DelayedFeedback.outstandingAt_disjoint_newlyObservedBefore
A measurable stochastic policy kernel and recursively generated action law depending only on this view.
The source's simultaneous-arrival order and BSC/EAP state-transition invariants.
Delayed SAPO active-arm allocation leafCompiledBanditRLProof.DelayedFeedback.inactiveArms
BanditRLProof.DelayedFeedback.activeEqualShare
BanditRLProof.DelayedFeedback.delayedSAPOProbability
BanditRLProof.DelayedFeedback.delayedSAPOProbability_of_active
BanditRLProof.DelayedFeedback.delayedSAPOProbability_of_inactive
BanditRLProof.DelayedFeedback.activeEqualShare_nonneg
BanditRLProof.DelayedFeedback.delayedSAPOProbability_nonneg
BanditRLProof.DelayedFeedback.sum_delayedSAPOProbability_eq_one
A source-faithful EAP state and proof that every update preserves nonnegative inactive mass at most one.
A measurable sampling kernel using this vector on the recursively generated delayed-feedback history; the one-round probability measure is compiled downstream.
Optimal-arm survival and causal one-round action lawCompiledBanditRLProof.DelayedFeedback.DelayedSAPOEliminationSnapshot
BanditRLProof.DelayedFeedback.DelayedSAPOEliminationSnapshot.eliminated
BanditRLProof.DelayedFeedback.DelayedSAPOEliminationSnapshot.remainingActive
BanditRLProof.DelayedFeedback.DelayedSAPOEliminationSnapshot.mem_eliminated_iff
BanditRLProof.DelayedFeedback.DelayedSAPOEliminationSnapshot.mem_remainingActive_iff
BanditRLProof.DelayedFeedback.DelayedSAPOEliminationSnapshot.OptimalArmSurvivalCertificate
BanditRLProof.DelayedFeedback.DelayedSAPOEliminationSnapshot.optimal_mem_remainingActive_of_certificate
BanditRLProof.DelayedFeedback.DelayedSAPOEliminationSnapshot.remainingActive_nonempty_of_certificate
BanditRLProof.DelayedFeedback.DelayedSAPOEliminationSnapshot.sum_delayedSAPOProbability_after_elimination_eq_one
BanditRLProof.DelayedFeedback.DelayedSAPOAllocation
BanditRLProof.DelayedFeedback.DelayedSAPOAllocation.probability
BanditRLProof.DelayedFeedback.DelayedSAPOAllocation.probability_nonnegative
BanditRLProof.DelayedFeedback.DelayedSAPOAllocation.sum_probability_eq_one
BanditRLProof.DelayedFeedback.DelayedSAPOAllocation.finiteActionDistribution
BanditRLProof.DelayedFeedback.DelayedSAPOAllocation.actionMeasure
BanditRLProof.DelayedFeedback.DelayedSAPOAllocation.actionMeasure_isProbabilityMeasure
BanditRLProof.DelayedFeedback.causalDelayedSAPOActionMeasureRule
BanditRLProof.DelayedFeedback.causalDelayedSAPOActionMeasureRule_isProbabilityMeasure
BanditRLProof.DelayedFeedback.causalDelayedSAPOActionMeasureRule_eq_of_observation_equivalent
The Definition-D.1 count, phase, error, and delay clauses, their probability bound, and full recursive source Lemma D.9.
EAP preservation of its inactive-probability premises.
Coordinate measurability, a Markov kernel over generated histories, and recursive delayed trajectory generation.
Source-shaped good-event projection for optimal-arm survivalCompiledBanditRLProof.DelayedFeedback.DelayedSAPOSourceConfidenceSnapshot
BanditRLProof.DelayedFeedback.DelayedSAPOSourceConfidenceSnapshot.sourceUcbStar
BanditRLProof.DelayedFeedback.DelayedSAPOSourceConfidenceSnapshot.EliminationGoodEvent
BanditRLProof.DelayedFeedback.DelayedSAPOSourceConfidenceSnapshot.optimalMean_le_ucbStar_of_eliminationGoodEvent
BanditRLProof.DelayedFeedback.DelayedSAPOSourceConfidenceSnapshot.optimalArmSurvivalCertificate_of_eliminationGoodEvent
BanditRLProof.DelayedFeedback.DelayedSAPOSourceConfidenceSnapshot.optimal_mem_remainingActive_of_eliminationGoodEvent
BanditRLProof.DelayedFeedback.DelayedSAPOSourceConfidenceSnapshot.eliminationGoodEventSet
BanditRLProof.DelayedFeedback.DelayedSAPOSourceConfidenceSnapshot.optimalSurvivalEventSet
BanditRLProof.DelayedFeedback.DelayedSAPOSourceConfidenceSnapshot.eliminationGoodEventSet_subset_optimalSurvivalEventSet
BanditRLProof.DelayedFeedback.DelayedSAPOSourceConfidenceSnapshot.measure_optimalSurvivalEventSet_compl_le
BanditRLProof.DelayedFeedback.DelayedSAPOSourceConfidenceSnapshot.measure_optimalSurvivalEventSet_compl_le_of_goodEvent
The full Definition D.1 count, phase, error, and delay clauses and their measurable simultaneous event.
The D.2--D.7 concentration/counting lemmas that produce the six component probability bounds.
Persistence across the recursive Delayed SAPO state machine and the stochastic/adversarial regret endpoints.
Corollary-D.8 union assembly to D.9 survivalCompiledBanditRLProof.DelayedFeedback.DelayedSAPOGoodEventComponent
BanditRLProof.DelayedFeedback.DelayedSAPOGoodEventFailureFamily
BanditRLProof.DelayedFeedback.DelayedSAPOGoodEventFailureFamily.componentFailure
BanditRLProof.DelayedFeedback.DelayedSAPOGoodEventFailureFamily.failureSet
BanditRLProof.DelayedFeedback.DelayedSAPOGoodEventFailureFamily.sourceGoodEventSet
BanditRLProof.DelayedFeedback.DelayedSAPOGoodEventFailureFamily.sourceGoodEventSet_compl
BanditRLProof.DelayedFeedback.DelayedSAPOGoodEventFailureFamily.measure_sourceGoodEventSet_compl_le_sum
BanditRLProof.DelayedFeedback.DelayedSAPOGoodEventFailureFamily.quadraticFailureBudget
BanditRLProof.DelayedFeedback.DelayedSAPOGoodEventFailureFamily.linearFailureBudget
BanditRLProof.DelayedFeedback.DelayedSAPOGoodEventFailureFamily.quadraticFailureBudget_le_linearFailureBudget
BanditRLProof.DelayedFeedback.DelayedSAPOGoodEventFailureFamily.measure_sourceGoodEventSet_compl_le_nine_div
BanditRLProof.DelayedFeedback.DelayedSAPOGoodEventFailureFamily.measure_eliminationGoodEventSet_compl_le_nine_div
BanditRLProof.DelayedFeedback.DelayedSAPOGoodEventFailureFamily.measure_optimalSurvivalEventSet_compl_le_nine_div
Source-faithful random variables, events, and proofs of Lemmas D.2--D.7 on one generated Delayed SAPO law.
A proved projection from the complete Definition-D.1 event to the compiled elimination slice.
Recursive optimal-arm persistence and either paper-level regret endpoint.
Lemma-D.10/D.12 width-direction diagnostic and conditional same-snapshot skeletonPartialBanditRLProof.DelayedFeedback.sourceEmpiricalWidthScale
BanditRLProof.DelayedFeedback.sourceEmpiricalWidthScale_antitone
BanditRLProof.DelayedFeedback.one_le_ten_mul_sourceEmpiricalWidthScale_of_count_le_96_mul_scale
BanditRLProof.DelayedFeedback.one_le_ten_mul_sourceEmpiricalWidthScale_two_log_of_small_count
BanditRLProof.DelayedFeedback.sourceEmpiricalWidthScale_one_one
BanditRLProof.DelayedFeedback.sourceEmpiricalWidthScale_one_four
BanditRLProof.DelayedFeedback.not_sourceEmpiricalWidthScale_one_le_four
BanditRLProof.DelayedFeedback.not_sourceEmpiricalWidthScale_horizon_four_one_le_four
BanditRLProof.DelayedFeedback.eight_mul_empiricalWidth_lt_gap_of_mem_eliminated
BanditRLProof.DelayedFeedback.gap_le_sixteen_mul_empiricalWidth_of_mem_remainingActive
BanditRLProof.DelayedFeedback.gap_le_sixteen_mul_empiricalWidth_of_mem_remainingActive_of_large_or_small_count
BanditRLProof.DelayedFeedback.gap_le_twenty_mul_gap_at_earlier_elimination_snapshot
BanditRLProof.DelayedFeedback.gap_le_twenty_mul_gap_at_earlier_elimination_snapshot_of_large_or_small_count
BanditRLProof.DelayedFeedback.DelayedSAPOD10D12GapOrderingContract
BanditRLProof.DelayedFeedback.DelayedSAPOD10D12GapOrderingContract.surrogateGap
BanditRLProof.DelayedFeedback.DelayedSAPOD10D12GapOrderingContract.surrogateGap_le_gap
BanditRLProof.DelayedFeedback.DelayedSAPOD10D12GapOrderingContract.gap_le_two_mul_surrogateGap
BanditRLProof.DelayedFeedback.DelayedSAPOD10D12GapOrderingContract.d12_gap_ordering_chain
BanditRLProof.DelayedFeedback.DelayedSAPOD10D12GapOrderingContract.gap_le_twenty_mul_gap_of_eliminationPrefixIndex_le
A recursive Delayed SAPO processed trace that produces the actual large/small-count branch certificate, the source-width/count equality in the small branch, the current-UCB and optimal-to-later factor-three edges in the large branch, and the later-to-earlier factor-ten comparison used by the conditional same-snapshot skeleton.
A source amendment or author clarification for the intended printed D.10 prefix-to-elimination width step; the compiled conditional skeleton bypasses rather than validates that step.
Unconditional source Lemmas D.10/D.12, main-text Lemma 4.2, Theorem 4.1, and either regret endpoint.
Source-frozen delayed best-of-both-worlds endpoint auditPartialBanditRLProof.DelayedFeedback.SameAlgorithmMultiRegimeContract
BanditRLProof.DelayedFeedback.SameAlgorithmMultiRegimeContract.stochasticClaim
BanditRLProof.DelayedFeedback.SameAlgorithmMultiRegimeContract.adversarialClaim
BanditRLProof.DelayedFeedback.SameAlgorithmMultiRegimeContract.stochasticClaim_iff_shared_fields
BanditRLProof.DelayedFeedback.SameAlgorithmMultiRegimeContract.adversarialClaim_iff_shared_fields
The complete Definition-D.1 event, its D.2--D.7 component probability producers, Delayed SAPO BSC/EAP state machines, switching rule, and ordered update semantics.
A measurable causal randomized sampling kernel and recursively generated delayed-feedback trajectory law; only the one-round measure-valued rule now compiles.
Recursive branch/count/width producers for the large/small-count conditional D.12 skeleton, the same-prefix factor-ten comparison, a clarification or amendment of the printed D.10 transport, and an unconditional D.12 / main-text Lemma 4.2 bridge.
The stochastic-instance and oblivious-adversarial regret endpoints for the same algorithm identity.
Balanced target-drift controlled evaluationPlannedNo local declaration yetFreeze the provider, immutable model version, replicate semantics, token/tool/build/time/cost budgets, retry and missing-run policies.
Publish and freeze the final production checker image from a reviewed candidate, freeze the provider image and commands, and pass all seven bound checker probes: network denial, host-sentinel protection, operator-ground-truth absence, controller/worker output protection, patched-source/controller-input read-only enforcement, mounted-input/cidfile protection, and background-process reaping under the exact image.
Pass the preregistered real-provider/real-sandbox one-case-by-three-condition smoke, excluded only from the primary analysis.
Complete the frozen-model source-absent wording control and independent blind wording review; freeze hash-verified source paths, grader identities, the sealed pack digest, and blind-packet procedure.
Execute and neutrally check all 450 matched runs, complete independent grading, and analyze the frozen target-level endpoints.

Open boundaries

  • The conservative generated KL-UCB finite-time route has named compiled declarations. The source-frozen delayed-bandit audit now compiles accounting, causal-view, new-arrival processing, probability allocation, the deterministic optimal-arm-survival core, and a causal one-round action measure. EAP/BSC state preservation, a measurable recursive trajectory, and stochastic/adversarial regret endpoints remain blocked. Full BwK/primal-dual regret, dueling, robust, federated, and neural-bandit routes remain planned or partial.
  • Direct LeanMachineLearning declaration identity remains blocked on a deliberate cross-toolchain migration and real upstream-symbol import; local theorem-card-shaped ETC/UCB proofs do not satisfy that gate.
  • The harness records completion gates but does not replace Lean elaboration or mathematical review.

All Lean modules in this chapter

Open the complete module list (14 modules)