Samplinglib
Lean gate not recorded for this source state main · 0e31a3cda412

RestartReduction: mathematical reading route

Read the statements and derivations in source order. Every result has its own optional Lean statement and proof. Source assumptions, library reuse and unproved boundaries are kept explicit.

  1. Restarting a convex solver halves the radius and preserves a finite budget
ASTIS mathematical exposition

Restarting a convex solver halves the radius and preserves a finite budget

AutoSamplingTheory.TechnicalLemmas.Analysis.RestartReduction.radius_accuracy_and_cost · theorem · Teaching coverage

Statement

E is a complete real inner-product space and f:E→R is C1 and alpha-strongly convex on E, with the alpha/2 chord convention. A global minimizer z is supplied. Alpha,beta,R,epsilon are strictly positive real numbers. The initial point satisfies ||x0-z||<=R. R is a supplied positive upper radius, not an exact unknown distance. A(y,r,delta) returns a point and a natural certified cost. For every y and positive r,delta with ||y-z||<=r, its output gap is <=delta and its cost is <=phi(beta*r²/delta). Phi:R→N is arbitrary; monotonicity is not assumed. N is a natural number, including zero, and alpha*(R/2^N)²<=epsilon. Set r_k=R/2^k and define the actual state/cost recursion and final call below. The recursion has the stated radius and cumulative cost, and its final output has gap at most epsilon.

\[\begin{gathered}r_k=R/2^k,\quad (x_0,c_0)=(x_0,0),\\(w_k,q_k)=A(x_k,r_k,\alpha r_k^2/8),\quad(x_{k+1},c_{k+1})=(w_k,c_k+q_k),\\(x_{\rm out},q_{\rm out})=A(x_N,\sqrt{\varepsilon/\alpha},\varepsilon),\\\|x_N-z\|\le r_N,\quad c_N\le N\phi(8\beta/\alpha),\\ f(x_{\rm out})-f(z)\le\varepsilon,\quad c_N+q_{\rm out}\le N\phi(8\beta/\alpha)+\phi(\beta/\alpha).\end{gathered}\]

All objects and hypotheses

  • E is a complete real inner-product space and f:E→R is C1 and alpha-strongly convex on E, with the alpha/2 chord convention. A global minimizer z is supplied.
  • Alpha,beta,R,epsilon are strictly positive real numbers. The initial point satisfies ||x0-z||<=R. R is a supplied positive upper radius, not an exact unknown distance.
  • A(y,r,delta) returns a point and a natural certified cost. For every y and positive r,delta with ||y-z||<=r, its output gap is <=delta and its cost is <=phi(beta*r²/delta). Phi:R→N is arbitrary; monotonicity is not assumed.
  • N is a natural number, including zero, and alpha*(R/2^N)²<=epsilon. Set r_k=R/2^k and define the actual state/cost recursion and final call below.

Mathematical proof

1. Derive the sharp quadratic growth bound

Fermat gives the genuine gradient at z equal to zero. Apply the existing strong-convexity first-order lower bound at z. The resulting alpha/2 factor is essential: the older midpoint-only alpha/4 bound would not give the prescribed halving constant.

\[\nabla f(z)=0,\qquad \frac{\alpha}{2}\|y-z\|^2\le f(y)-f(z).\]
Corresponding Lean step

StrongConvexFirstOrder.firstOrder_lower_bound_of_strongConvexOn; IsLocalMin.fderiv_eq_zero and actual HasGradientAt from C1.

2. Induct on the actual restarted calls

Assume the actual current point has distance at most r_k. Positivity of R and alpha makes r_k and delta_k=alpha*r_k²/8 positive, so the base solver contract applies. Its objective guarantee combined with quadratic growth yields squared distance at most r_k²/4. Nonnegative norms and r_k>0 then give radius r_k/2=r_(k+1). The initial radius is the supplied hypothesis.

\[\frac\alpha2\|x_{k+1}-z\|^2\le\frac{\alpha r_k^2}{8}\quad\Longrightarrow\quad\|x_{k+1}-z\|\le\frac{r_k}{2}=r_{k+1}.\]
Corresponding Lean step

Nat.rec and natural induction; solver contract, positive-radius cancellation, pow_succ and div_mul_eq_div_div.

3. Accumulate the certified stage costs

At every stage the prescribed radius appears in both the tolerance and the solver budget. Their ratio is exactly 8beta/alpha, so no monotonicity of phi is needed. The recursion adds the returned cost to the accumulated natural count. Induction gives c_N<=N phi(8beta/alpha).

\[\frac{\beta r_k^2}{\alpha r_k^2/8}=8\frac\beta\alpha,\qquad c_{k+1}=c_k+q_k,\qquad c_N\le N\phi(8\beta/\alpha).\]
Corresponding Lean step

field_simp uses r_k nonzero; Nat.add_le_add and Nat.add_mul propagate the actual accumulated count.

4. Make the final polishing call on a justified radius

The chosen horizon gives r_N<=sqrt(epsilon/alpha). Pass that positive upper radius, together with tolerance epsilon, to the same solver at the actual state x_N. Its output has objective gap at most epsilon. The final budget argument simplifies exactly to beta/alpha. Add this final returned cost, including when N=0.

\[\rho=\sqrt{\varepsilon/\alpha},\qquad\frac{\beta\rho^2}{\varepsilon}=\frac\beta\alpha,\qquad f(x_{\rm out})-f(z)\le\varepsilon,\qquad c_N+q_{\rm out}\le N\phi(8\beta/\alpha)+\phi(\beta/\alpha).\]
Corresponding Lean step

Real.le_sqrt_of_sq_le, Real.sqrt_pos, Real.sq_sqrt; final hA call and Nat.add_le_add.

Lean statement · radius_accuracy_and_cost

Actual scheduled state/cost recursion and final solver call.

Braces mark parameters Lean can infer; square brackets request structures such as a measurable space or probability measure. Named hypotheses are mathematical premises, not facts established by this declaration. Section parameters are described in the mathematical hypotheses above; the module link retains their exact source context.

theorem radius_accuracy_and_cost {f : E → ℝ} {α β R ε : ℝ}
    (hf : ContDiff ℝ 1 f) (hsc : StrongConvexOn univ α f)
    (hα : 0 < α) (hβ : 0 < β) (hR : 0 < R) (hε : 0 < ε)
    {z x₀ : E} (hmin : IsMinOn f univ z) (hx₀ : ‖x₀-z‖ ≤ R)
    (A : E → ℝ → ℝ → E × ℕ) (φ : ℝ → ℕ)
    (hA : ∀ y r δ, 0 < r → 0 < δ → ‖y-z‖ ≤ r →
      f (A y r δ).1 - f z ≤ δ ∧ (A y r δ).2 ≤ φ (β*r^2/δ))
    (N : ℕ) (hN : α * (R / (2:ℝ)^N)^2 ≤ ε) :
    let r := fun k : ℕ => R / (2:ℝ)^k
    let run : ℕ → E × ℕ := Nat.rec (x₀, 0) (fun k p =>
      let w := A p.1 (r k) (α*(r k)^2/8)
      (w.1, p.2 + w.2))
    let last := A (run N).1 (Real.sqrt (ε/α)) ε
    ‖(run N).1-z‖ ≤ r N ∧ (run N).2 ≤ N * φ (8*(β/α)) ∧
      f last.1 - f z ≤ ε ∧ (run N).2 + last.2 ≤ N * φ (8*(β/α)) + φ (β/α)

Exact module and namespace context

Lean proof · radius_accuracy_and_cost

Fermat gives the genuine gradient at z equal to zero. Apply the existing strong-convexity first-order lower bound at z. The resulting alpha/2 factor is essential: the older midpoint-only alpha/4 bound would not give the prescribed halving constant. Assume the actual current point has distance at most r_k. Positivity of R and alpha makes r_k and delta_k=alpha*r_k²/8 positive, so the base solver contract applies. Its objective guarantee combined with quadratic growth yields squared distance at most r_k²/4. Nonnegative norms and r_k>0 then give radius r_k/2=r_(k+1). The initial radius is the supplied hypothesis. At every stage the prescribed radius appears in both the tolerance and the solver budget. Their ratio is exactly 8beta/alpha, so no monotonicity of phi is needed. The recursion adds the returned cost to the accumulated natural count. Induction gives c_N<=N phi(8beta/alpha). The chosen horizon gives r_N<=sqrt(epsilon/alpha). Pass that positive upper radius, together with tolerance epsilon, to the same solver at the actual state x_N. Its output has objective gap at most epsilon. The final budget argument simplifies exactly to beta/alpha. Add this final returned cost, including when N=0.

Braces mark parameters Lean can infer; square brackets request structures such as a measurable space or probability measure. Named hypotheses are mathematical premises, not facts established by this declaration. Section parameters are described in the mathematical hypotheses above; the module link retains their exact source context.

theorem radius_accuracy_and_cost {f : E → ℝ} {α β R ε : ℝ}
    (hf : ContDiff ℝ 1 f) (hsc : StrongConvexOn univ α f)
    (hα : 0 < α) (hβ : 0 < β) (hR : 0 < R) (hε : 0 < ε)
    {z x₀ : E} (hmin : IsMinOn f univ z) (hx₀ : ‖x₀-z‖ ≤ R)
    (A : E → ℝ → ℝ → E × ℕ) (φ : ℝ → ℕ)
    (hA : ∀ y r δ, 0 < r → 0 < δ → ‖y-z‖ ≤ r →
      f (A y r δ).1 - f z ≤ δ ∧ (A y r δ).2 ≤ φ (β*r^2/δ))
    (N : ℕ) (hN : α * (R / (2:ℝ)^N)^2 ≤ ε) :
    let r := fun k : ℕ => R / (2:ℝ)^k
    let run : ℕ → E × ℕ := Nat.rec (x₀, 0) (fun k p =>
      let w := A p.1 (r k) (α*(r k)^2/8)
      (w.1, p.2 + w.2))
    let last := A (run N).1 (Real.sqrt (ε/α)) ε
    ‖(run N).1-z‖ ≤ r N ∧ (run N).2 ≤ N * φ (8*(β/α)) ∧
      f last.1 - f z ≤ ε ∧ (run N).2 + last.2 ≤ N * φ (8*(β/α)) + φ (β/α) := by
  let r := fun k : ℕ => R / (2:ℝ)^k
  let run : ℕ → E × ℕ := Nat.rec (x₀, 0) (fun k p =>
    let w := A p.1 (r k) (α*(r k)^2/8)
    (w.1, p.2 + w.2))
  change ‖(run N).1-z‖ ≤ r N ∧ (run N).2 ≤ N * φ (8*(β/α)) ∧ _
  have hr (k : ℕ) : 0 < r k := div_pos hR (pow_pos (by norm_num) _)
  have hgrad : gradient f z = 0 := by
    simp [gradient, (hmin.isLocalMin Filter.univ_mem).fderiv_eq_zero]
  have hqg (y : E) : α/2 * ‖y-z‖^2 ≤ f y - f z := by
    have h := StrongConvexFirstOrder.firstOrder_lower_bound_of_strongConvexOn hsc
      (fun x _ => (hf.differentiable_one x).hasGradientAt) (mem_univ z) (mem_univ y)
    rw [hgrad, inner_zero_left] at h
    linarith
  have hind (k : ℕ) : ‖(run k).1-z‖ ≤ r k ∧ (run k).2 ≤ k * φ (8*(β/α)) := by
    induction k with
    | zero => simpa [run, r] using hx₀
    | succ k ih =>
      have hδ : 0 < α*(r k)^2/8 := by positivity
      have hw := hA (run k).1 (r k) (α*(r k)^2/8) (hr k) hδ ih.1
      have hid : β*(r k)^2 / (α*(r k)^2/8) = 8*(β/α) := by
        field_simp [ne_of_gt (hr k)]
      rw [hid] at hw
      have hs : r (k+1) = r k / 2 := by dsimp [r]; rw [pow_succ, div_mul_eq_div_div]
      constructor
      · change ‖(A (run k).1 (r k) (α*(r k)^2/8)).1-z‖ ≤ r (k+1)
        rw [hs]
        have hg := (hqg (A (run k).1 (r k) (α*(r k)^2/8)).1).trans hw.1
        have hsquare : ‖(A (run k).1 (r k) (α*(r k)^2/8)).1-z‖^2 ≤ (r k/2)^2 := by
          nlinarith
        nlinarith [norm_nonneg ((A (run k).1 (r k) (α*(r k)^2/8)).1-z), hr k]
      · change (run k).2 + (A (run k).1 (r k) (α*(r k)^2/8)).2 ≤ (k+1) * φ (8*(β/α))
        simpa only [Nat.add_mul, Nat.one_mul] using Nat.add_le_add ih.2 hw.2
  have hs : r N ≤ Real.sqrt (ε/α) := by
    apply Real.le_sqrt_of_sq_le
    exact (le_div_iff₀ hα).mpr (by nlinarith [hN])
  have hlast := hA (run N).1 (Real.sqrt (ε/α)) ε
    (Real.sqrt_pos.mpr (div_pos hε hα)) hε ((hind N).1.trans hs)
  have hid : β * (Real.sqrt (ε/α))^2 / ε = β/α := by
    rw [Real.sq_sqrt (div_nonneg hε.le hα.le)]
    field_simp
  rw [hid] at hlast
  exact ⟨(hind N).1, (hind N).2, hlast.1, Nat.add_le_add (hind N).2 hlast.2⟩
end AutoSamplingTheory.TechnicalLemmas.Analysis.RestartReduction

Exact module and namespace context

Scope and omitted-condition boundaries

  • This is the finite restart construction and certified cost composition in the proof of Lemma4.1. It is not a new proof of correctness of every possible base solver, a formal first-order-oracle machine, or the full class-level asymptotic reduction. The solver contract is exactly the assumed ingredient being transformed.
  • The formal reduction works for C1 strongly convex Hilbert objectives admitting the stated base solver contract. Source beta-smoothness motivates that contract; it is not independently inferred from beta or from an arbitrary returned pair. No supplied quadratic-growth or stage-radius invariant replaces the proof.
  • R>0 avoids zero-tolerance calls. An exactly minimizing initial point is allowed by any positive upper-radius certificate, but the R=0 stopping branch is outside this declaration. N=0 still makes the distinct final polishing call; the radius condition alone is not used as an objective-accuracy certificate.
  • The radius supplied to a base call is a known upper bound, consistent with the opening definition in Section4.1. The final call uses sqrt(epsilon/alpha) rather than the unknown exact distance; its cost is exactly bounded by phi(kappa) without monotonicity.
  • No logarithmic round-count normalization or absorption of phi(kappa) into phi(8kappa) is claimed. A global bound must account for the last call when the logarithm is small or nonpositive; an asymptotic small-tolerance convention and any phi comparison must be stated when formalized.
  • The concrete test uses actual gradient descent on x²/2 at step1/2. Each base call executes ceil(r²/delta)+1 gradient updates and reports that count; the existing convex value theorem proves its uniform contract. All restart counts, including zero, exercise the real recursion and final call. No hidden access to the supplied minimizer is passed to A.
  • No minimizer-existence, reverse regularization reduction, oracle lower bound, Riemannian/stochastic method or companion-paper completion is asserted.

Source and reuse

ASTIS parents called

Mathlib API called (external library)

  • IsLocalMin.fderiv_eq_zero
  • Nat.rec
  • pow_succ
  • Real.le_sqrt_of_sq_le
  • Real.sqrt_pos
  • Real.sq_sqrt
  • Nat.add_le_add

Mathematical sources

ASTIS prose is not a quotation or a source-equivalence certificate. Definitions and aliases are explained as constructions, not counted as new mathematical proofs.