Restarting a convex solver halves the radius and preserves a finite budget
AutoSamplingTheory.TechnicalLemmas.Analysis.RestartReduction.radius_accuracy_and_cost · theorem · Teaching coverage
Statement
E is a complete real inner-product space and f:E→R is C1 and alpha-strongly convex on E, with the alpha/2 chord convention. A global minimizer z is supplied. Alpha,beta,R,epsilon are strictly positive real numbers. The initial point satisfies ||x0-z||<=R. R is a supplied positive upper radius, not an exact unknown distance. A(y,r,delta) returns a point and a natural certified cost. For every y and positive r,delta with ||y-z||<=r, its output gap is <=delta and its cost is <=phi(beta*r²/delta). Phi:R→N is arbitrary; monotonicity is not assumed. N is a natural number, including zero, and alpha*(R/2^N)²<=epsilon. Set r_k=R/2^k and define the actual state/cost recursion and final call below. The recursion has the stated radius and cumulative cost, and its final output has gap at most epsilon.
All objects and hypotheses
- E is a complete real inner-product space and f:E→R is C1 and alpha-strongly convex on E, with the alpha/2 chord convention. A global minimizer z is supplied.
- Alpha,beta,R,epsilon are strictly positive real numbers. The initial point satisfies ||x0-z||<=R. R is a supplied positive upper radius, not an exact unknown distance.
- A(y,r,delta) returns a point and a natural certified cost. For every y and positive r,delta with ||y-z||<=r, its output gap is <=delta and its cost is <=phi(beta*r²/delta). Phi:R→N is arbitrary; monotonicity is not assumed.
- N is a natural number, including zero, and alpha*(R/2^N)²<=epsilon. Set r_k=R/2^k and define the actual state/cost recursion and final call below.
Mathematical proof
1. Derive the sharp quadratic growth bound
Fermat gives the genuine gradient at z equal to zero. Apply the existing strong-convexity first-order lower bound at z. The resulting alpha/2 factor is essential: the older midpoint-only alpha/4 bound would not give the prescribed halving constant.
Corresponding Lean step
StrongConvexFirstOrder.firstOrder_lower_bound_of_strongConvexOn; IsLocalMin.fderiv_eq_zero and actual HasGradientAt from C1.
2. Induct on the actual restarted calls
Assume the actual current point has distance at most r_k. Positivity of R and alpha makes r_k and delta_k=alpha*r_k²/8 positive, so the base solver contract applies. Its objective guarantee combined with quadratic growth yields squared distance at most r_k²/4. Nonnegative norms and r_k>0 then give radius r_k/2=r_(k+1). The initial radius is the supplied hypothesis.
Corresponding Lean step
Nat.rec and natural induction; solver contract, positive-radius cancellation, pow_succ and div_mul_eq_div_div.
3. Accumulate the certified stage costs
At every stage the prescribed radius appears in both the tolerance and the solver budget. Their ratio is exactly 8beta/alpha, so no monotonicity of phi is needed. The recursion adds the returned cost to the accumulated natural count. Induction gives c_N<=N phi(8beta/alpha).
Corresponding Lean step
field_simp uses r_k nonzero; Nat.add_le_add and Nat.add_mul propagate the actual accumulated count.
4. Make the final polishing call on a justified radius
The chosen horizon gives r_N<=sqrt(epsilon/alpha). Pass that positive upper radius, together with tolerance epsilon, to the same solver at the actual state x_N. Its output has objective gap at most epsilon. The final budget argument simplifies exactly to beta/alpha. Add this final returned cost, including when N=0.
Corresponding Lean step
Real.le_sqrt_of_sq_le, Real.sqrt_pos, Real.sq_sqrt; final hA call and Nat.add_le_add.
Lean statement · radius_accuracy_and_cost
Actual scheduled state/cost recursion and final solver call.
Braces mark parameters Lean can infer; square brackets request structures such as a measurable space or probability measure. Named hypotheses are mathematical premises, not facts established by this declaration. Section parameters are described in the mathematical hypotheses above; the module link retains their exact source context.
theorem radius_accuracy_and_cost {f : E → ℝ} {α β R ε : ℝ}
(hf : ContDiff ℝ 1 f) (hsc : StrongConvexOn univ α f)
(hα : 0 < α) (hβ : 0 < β) (hR : 0 < R) (hε : 0 < ε)
{z x₀ : E} (hmin : IsMinOn f univ z) (hx₀ : ‖x₀-z‖ ≤ R)
(A : E → ℝ → ℝ → E × ℕ) (φ : ℝ → ℕ)
(hA : ∀ y r δ, 0 < r → 0 < δ → ‖y-z‖ ≤ r →
f (A y r δ).1 - f z ≤ δ ∧ (A y r δ).2 ≤ φ (β*r^2/δ))
(N : ℕ) (hN : α * (R / (2:ℝ)^N)^2 ≤ ε) :
let r := fun k : ℕ => R / (2:ℝ)^k
let run : ℕ → E × ℕ := Nat.rec (x₀, 0) (fun k p =>
let w := A p.1 (r k) (α*(r k)^2/8)
(w.1, p.2 + w.2))
let last := A (run N).1 (Real.sqrt (ε/α)) ε
‖(run N).1-z‖ ≤ r N ∧ (run N).2 ≤ N * φ (8*(β/α)) ∧
f last.1 - f z ≤ ε ∧ (run N).2 + last.2 ≤ N * φ (8*(β/α)) + φ (β/α)Lean proof · radius_accuracy_and_cost
Fermat gives the genuine gradient at z equal to zero. Apply the existing strong-convexity first-order lower bound at z. The resulting alpha/2 factor is essential: the older midpoint-only alpha/4 bound would not give the prescribed halving constant. Assume the actual current point has distance at most r_k. Positivity of R and alpha makes r_k and delta_k=alpha*r_k²/8 positive, so the base solver contract applies. Its objective guarantee combined with quadratic growth yields squared distance at most r_k²/4. Nonnegative norms and r_k>0 then give radius r_k/2=r_(k+1). The initial radius is the supplied hypothesis. At every stage the prescribed radius appears in both the tolerance and the solver budget. Their ratio is exactly 8beta/alpha, so no monotonicity of phi is needed. The recursion adds the returned cost to the accumulated natural count. Induction gives c_N<=N phi(8beta/alpha). The chosen horizon gives r_N<=sqrt(epsilon/alpha). Pass that positive upper radius, together with tolerance epsilon, to the same solver at the actual state x_N. Its output has objective gap at most epsilon. The final budget argument simplifies exactly to beta/alpha. Add this final returned cost, including when N=0.
Braces mark parameters Lean can infer; square brackets request structures such as a measurable space or probability measure. Named hypotheses are mathematical premises, not facts established by this declaration. Section parameters are described in the mathematical hypotheses above; the module link retains their exact source context.
theorem radius_accuracy_and_cost {f : E → ℝ} {α β R ε : ℝ}
(hf : ContDiff ℝ 1 f) (hsc : StrongConvexOn univ α f)
(hα : 0 < α) (hβ : 0 < β) (hR : 0 < R) (hε : 0 < ε)
{z x₀ : E} (hmin : IsMinOn f univ z) (hx₀ : ‖x₀-z‖ ≤ R)
(A : E → ℝ → ℝ → E × ℕ) (φ : ℝ → ℕ)
(hA : ∀ y r δ, 0 < r → 0 < δ → ‖y-z‖ ≤ r →
f (A y r δ).1 - f z ≤ δ ∧ (A y r δ).2 ≤ φ (β*r^2/δ))
(N : ℕ) (hN : α * (R / (2:ℝ)^N)^2 ≤ ε) :
let r := fun k : ℕ => R / (2:ℝ)^k
let run : ℕ → E × ℕ := Nat.rec (x₀, 0) (fun k p =>
let w := A p.1 (r k) (α*(r k)^2/8)
(w.1, p.2 + w.2))
let last := A (run N).1 (Real.sqrt (ε/α)) ε
‖(run N).1-z‖ ≤ r N ∧ (run N).2 ≤ N * φ (8*(β/α)) ∧
f last.1 - f z ≤ ε ∧ (run N).2 + last.2 ≤ N * φ (8*(β/α)) + φ (β/α) := by
let r := fun k : ℕ => R / (2:ℝ)^k
let run : ℕ → E × ℕ := Nat.rec (x₀, 0) (fun k p =>
let w := A p.1 (r k) (α*(r k)^2/8)
(w.1, p.2 + w.2))
change ‖(run N).1-z‖ ≤ r N ∧ (run N).2 ≤ N * φ (8*(β/α)) ∧ _
have hr (k : ℕ) : 0 < r k := div_pos hR (pow_pos (by norm_num) _)
have hgrad : gradient f z = 0 := by
simp [gradient, (hmin.isLocalMin Filter.univ_mem).fderiv_eq_zero]
have hqg (y : E) : α/2 * ‖y-z‖^2 ≤ f y - f z := by
have h := StrongConvexFirstOrder.firstOrder_lower_bound_of_strongConvexOn hsc
(fun x _ => (hf.differentiable_one x).hasGradientAt) (mem_univ z) (mem_univ y)
rw [hgrad, inner_zero_left] at h
linarith
have hind (k : ℕ) : ‖(run k).1-z‖ ≤ r k ∧ (run k).2 ≤ k * φ (8*(β/α)) := by
induction k with
| zero => simpa [run, r] using hx₀
| succ k ih =>
have hδ : 0 < α*(r k)^2/8 := by positivity
have hw := hA (run k).1 (r k) (α*(r k)^2/8) (hr k) hδ ih.1
have hid : β*(r k)^2 / (α*(r k)^2/8) = 8*(β/α) := by
field_simp [ne_of_gt (hr k)]
rw [hid] at hw
have hs : r (k+1) = r k / 2 := by dsimp [r]; rw [pow_succ, div_mul_eq_div_div]
constructor
· change ‖(A (run k).1 (r k) (α*(r k)^2/8)).1-z‖ ≤ r (k+1)
rw [hs]
have hg := (hqg (A (run k).1 (r k) (α*(r k)^2/8)).1).trans hw.1
have hsquare : ‖(A (run k).1 (r k) (α*(r k)^2/8)).1-z‖^2 ≤ (r k/2)^2 := by
nlinarith
nlinarith [norm_nonneg ((A (run k).1 (r k) (α*(r k)^2/8)).1-z), hr k]
· change (run k).2 + (A (run k).1 (r k) (α*(r k)^2/8)).2 ≤ (k+1) * φ (8*(β/α))
simpa only [Nat.add_mul, Nat.one_mul] using Nat.add_le_add ih.2 hw.2
have hs : r N ≤ Real.sqrt (ε/α) := by
apply Real.le_sqrt_of_sq_le
exact (le_div_iff₀ hα).mpr (by nlinarith [hN])
have hlast := hA (run N).1 (Real.sqrt (ε/α)) ε
(Real.sqrt_pos.mpr (div_pos hε hα)) hε ((hind N).1.trans hs)
have hid : β * (Real.sqrt (ε/α))^2 / ε = β/α := by
rw [Real.sq_sqrt (div_nonneg hε.le hα.le)]
field_simp
rw [hid] at hlast
exact ⟨(hind N).1, (hind N).2, hlast.1, Nat.add_le_add (hind N).2 hlast.2⟩
end AutoSamplingTheory.TechnicalLemmas.Analysis.RestartReductionScope and omitted-condition boundaries
- This is the finite restart construction and certified cost composition in the proof of Lemma4.1. It is not a new proof of correctness of every possible base solver, a formal first-order-oracle machine, or the full class-level asymptotic reduction. The solver contract is exactly the assumed ingredient being transformed.
- The formal reduction works for C1 strongly convex Hilbert objectives admitting the stated base solver contract. Source beta-smoothness motivates that contract; it is not independently inferred from beta or from an arbitrary returned pair. No supplied quadratic-growth or stage-radius invariant replaces the proof.
- R>0 avoids zero-tolerance calls. An exactly minimizing initial point is allowed by any positive upper-radius certificate, but the R=0 stopping branch is outside this declaration. N=0 still makes the distinct final polishing call; the radius condition alone is not used as an objective-accuracy certificate.
- The radius supplied to a base call is a known upper bound, consistent with the opening definition in Section4.1. The final call uses sqrt(epsilon/alpha) rather than the unknown exact distance; its cost is exactly bounded by phi(kappa) without monotonicity.
- No logarithmic round-count normalization or absorption of phi(kappa) into phi(8kappa) is claimed. A global bound must account for the last call when the logarithm is small or nonpositive; an asymptotic small-tolerance convention and any phi comparison must be stated when formalized.
- The concrete test uses actual gradient descent on x²/2 at step1/2. Each base call executes ceil(r²/delta)+1 gradient updates and reports that count; the existing convex value theorem proves its uniform contract. All restart counts, including zero, exercise the real recursion and final call. No hidden access to the supplied minimizer is passed to A.
- No minimizer-existence, reverse regularization reduction, oracle lower bound, Riemannian/stochastic method or companion-paper completion is asserted.
Source and reuse
ASTIS parents called
Mathlib API called (external library)
- IsLocalMin.fderiv_eq_zero
- Nat.rec
- pow_succ
- Real.le_sqrt_of_sq_le
- Real.sqrt_pos
- Real.sq_sqrt
- Nat.add_le_add
Mathematical sources
- Chewi Lemma4.1 and preceding solver convention — Finite restart radius schedule, per-call budget and final accuracy; full logarithmic reduction remains separate.
- Actual gradient-descent base solver test — A nonstationary quadratic optimizer validates the base contract using the existing convex GD rate, then consumes this restart theorem.
ASTIS prose is not a quotation or a source-equivalence certificate. Definitions and aliases are explained as constructions, not counted as new mathematical proofs.