Samplinglib
Lean gate not recorded for this source state main · 0e31a3cda412

RestartLogComplexity: mathematical reading route

Read the statements and derivations in source order. Every result has its own optional Lean statement and proof. Source assumptions, library reuse and unproved boundaries are kept explicit.

  1. An explicit logarithmic horizon for restarted optimization
ASTIS mathematical exposition

An explicit logarithmic horizon for restarted optimization

AutoSamplingTheory.TechnicalLemmas.Analysis.RestartLogComplexity.logarithmic_accuracy_and_cost · theorem · Teaching coverage

Statement

E is a complete real inner-product space. The fixed function f:E→R is C1 and alpha-strongly convex on E with the alpha/2 convention. A global minimizer z is supplied. Alpha,beta,R,epsilon are positive; ||x0-z||<=R. R is a supplied upper radius. For every point y and positive radius r and tolerance delta with ||y-z||<=r, A(y,r,delta) returns a point whose objective gap is <=delta and a natural cost <=phi(beta*r²/delta). Phi:R→N is arbitrary. Beta is the positive budget parameter of this contract; beta-smoothness and an oracle implementation are not independently inferred. Let q=alpha R²/epsilon, L=log(q)/log4 and N=ceil(L) in the natural numbers, so a nonpositive L yields N=0. Define the actual restart states and final call below; C counts all returned stage costs plus the final cost. The stated radius, accuracy and cost bounds hold. Budget absorption and the small-error bound are conditional as detailed below.

\[\begin{gathered}q=\alpha R^2/\varepsilon,\quad\kappa=\beta/\alpha,\quad L=\log q/\log4,\quad N=\lceil L\rceil_{\mathbb N},\\r_k=R/2^k,\quad (x_0,c_0)=(x_0,0),\\(w_k,a_k)=A(x_k,r_k,\alpha r_k^2/8),\\(x_{k+1},c_{k+1})=(w_k,c_k+a_k),\\(x_{\rm out},a_{\rm out})=A(x_N,\sqrt{\varepsilon/\alpha},\varepsilon),\quad C=c_N+a_{\rm out},\\N\le\max\{0,L\}+1,\quad(N=0\Leftrightarrow\alpha R^2\le\varepsilon),\\\|x_N-z\|\le r_N,\quad f(x_{\rm out})-f(z)\le\varepsilon,\\C\le N\phi(8\kappa)+\phi(\kappa)\le(\max\{0,L\}+1)\phi(8\kappa)+\phi(\kappa).\end{gathered}\]

All objects and hypotheses

  • E is a complete real inner-product space. The fixed function f:E→R is C1 and alpha-strongly convex on E with the alpha/2 convention. A global minimizer z is supplied.
  • Alpha,beta,R,epsilon are positive; ||x0-z||<=R. R is a supplied upper radius. For every point y and positive radius r and tolerance delta with ||y-z||<=r, A(y,r,delta) returns a point whose objective gap is <=delta and a natural cost <=phi(beta*r²/delta).
  • Phi:R→N is arbitrary. Beta is the positive budget parameter of this contract; beta-smoothness and an oracle implementation are not independently inferred.
  • Let q=alpha R²/epsilon, L=log(q)/log4 and N=ceil(L) in the natural numbers, so a nonpositive L yields N=0. Define the actual restart states and final call below; C counts all returned stage costs plus the final cost.

Mathematical proof

1. Choose an adequate integer horizon

All logarithm arguments are positive. Natural ceiling gives L<=N. Multiply by the positive log4 and use monotonicity of log together with log(4^N)=N log4 to obtain q<=4^N. Since (2^N)²=4^N, this is exactly the horizon condition required by the finite restart theorem.

\[L\le N\ \Longrightarrow\ \log q\le N\log4=\log(4^N)\ \Longrightarrow\ \alpha(R/2^N)^2\le\varepsilon.\]
Corresponding Lean step

Nat.le_ceil; Real.log_pow and Real.log_le_log_iff on positive arguments; positive-denominator algebra.

2. Control rounding and the zero-round case

For L>=0, the natural ceiling is less than L+1. For L<0, it equals zero. Thus N<=max(0,L)+1 globally. More precisely, N=0 iff L<=0 iff q<=1 iff alpha R²<=epsilon. This certifies only that no halving stage is needed; the final solver call remains.

\[N\le\max\{0,L\}+1,\qquad N=0\ \Longleftrightarrow\ \alpha R^2\le\varepsilon.\]
Corresponding Lean step

Nat.ceil_lt_add_one requires nonnegative L; Nat.ceil_eq_zero handles the other branch. Real.log_le_log_iff relates q and1.

3. Consume the actual finite restart theorem

Apply the existing radius_accuracy_and_cost theorem at this N using the horizon just proved. Its Nat.rec is exactly the recursion below. It supplies the actual state radius, final output error, and the natural sum of all certified call costs. The new theorem does not assume a radius invariant or an adequate iteration count.

\[\|x_N-z\|\le R/2^N,\qquad f(x_{\rm out})-f(z)\le\varepsilon,\qquad C\le N\phi(8\kappa)+\phi(\kappa).\]
Corresponding Lean step

RestartReduction.radius_accuracy_and_cost; alpha/2 quadratic growth and the genuine solver recursion are reused, not reproved.

4. Keep the final cost while normalizing the stage count

Cast the natural cost inequality to real numbers. Both phi values are nonnegative, so replacing N by its upper bound preserves the inequality. This global logarithmic bound uses no comparison between the two phi values.

\[C\le(\max\{0,L\}+1)\phi(8\kappa)+\phi(\kappa).\]
Corresponding Lean step

Nat.cast_le, Nat.cast_add, Nat.cast_mul and multiplication by a nonnegative natural-valued budget.

5. State exactly when the source-shaped bound follows

Under the additional condition phi(kappa)<=phi(8kappa), the final cost can be absorbed with the displayed universal factor. The result is (max(0,L)+2)phi(8kappa). If additionally q>=4, then L>=1, so L+2<=3L. This gives the explicit constant3/log4 in front of phi(8kappa)logq. These are conditional consequences; no monotonicity of phi or all-accuracy logarithmic bound is silently asserted.

\[\begin{gathered}\phi(\kappa)\le\phi(8\kappa)\ \Longrightarrow\ C\le(\max\{0,L\}+2)\phi(8\kappa),\\\bigl(\phi(\kappa)\le\phi(8\kappa)\ \land\ q\ge4\bigr)\ \Longrightarrow\\C\le3L\phi(8\kappa)=\frac3{\log4}\phi(8\kappa)\log q.\end{gathered}\]
Corresponding Lean step

Order of natural casts, Real.log_le_log, division by positive log4, and nonnegative multiplication. The last implication is inside the phi-comparison implication.

Lean statement · logarithmic_accuracy_and_cost

The exact logarithmic count, recursive calls and conditional cost conclusions.

Braces mark parameters Lean can infer; square brackets request structures such as a measurable space or probability measure. Named hypotheses are mathematical premises, not facts established by this declaration. Section parameters are described in the mathematical hypotheses above; the module link retains their exact source context.

theorem logarithmic_accuracy_and_cost {f : E → ℝ} {α β R ε : ℝ}
    (hf : ContDiff ℝ 1 f) (hsc : StrongConvexOn univ α f)
    (hα : 0 < α) (hβ : 0 < β) (hR : 0 < R) (hε : 0 < ε)
    {z x₀ : E} (hmin : IsMinOn f univ z) (hx₀ : ‖x₀-z‖ ≤ R)
    (A : E → ℝ → ℝ → E × ℕ) (φ : ℝ → ℕ)
    (hA : ∀ y r δ, 0 < r → 0 < δ → ‖y-z‖ ≤ r →
      f (A y r δ).1 - f z ≤ δ ∧ (A y r δ).2 ≤ φ (β*r^2/δ)) :
    let L := Real.log (α*R^2/ε) / Real.log 4
    let N := ⌈L⌉₊
    let r := fun k : ℕ => R / (2:ℝ)^k
    let run : ℕ → E × ℕ := Nat.rec (x₀, 0) (fun k p =>
      let w := A p.1 (r k) (α*(r k)^2/8)
      (w.1, p.2 + w.2))
    let last := A (run N).1 (Real.sqrt (ε/α)) ε
    let C := (run N).2 + last.2
    (N : ℝ) ≤ max 0 L + 1 ∧ (N = 0 ↔ α*R^2 ≤ ε) ∧
      ‖(run N).1-z‖ ≤ r N ∧ f last.1 - f z ≤ ε ∧
      C ≤ N * φ (8*(β/α)) + φ (β/α) ∧
      (C : ℝ) ≤ (max 0 L + 1) * φ (8*(β/α)) + φ (β/α) ∧
      (φ (β/α) ≤ φ (8*(β/α)) →
        (C : ℝ) ≤ (max 0 L + 2) * φ (8*(β/α)) ∧
        (4 ≤ α*R^2/ε → (C : ℝ) ≤ 3*L*φ (8*(β/α))))

Exact module and namespace context

Lean proof · logarithmic_accuracy_and_cost

All logarithm arguments are positive. Natural ceiling gives L<=N. Multiply by the positive log4 and use monotonicity of log together with log(4^N)=N log4 to obtain q<=4^N. Since (2^N)²=4^N, this is exactly the horizon condition required by the finite restart theorem. For L>=0, the natural ceiling is less than L+1. For L<0, it equals zero. Thus N<=max(0,L)+1 globally. More precisely, N=0 iff L<=0 iff q<=1 iff alpha R²<=epsilon. This certifies only that no halving stage is needed; the final solver call remains. Apply the existing radius_accuracy_and_cost theorem at this N using the horizon just proved. Its Nat.rec is exactly the recursion below. It supplies the actual state radius, final output error, and the natural sum of all certified call costs. The new theorem does not assume a radius invariant or an adequate iteration count. Cast the natural cost inequality to real numbers. Both phi values are nonnegative, so replacing N by its upper bound preserves the inequality. This global logarithmic bound uses no comparison between the two phi values. Under the additional condition phi(kappa)<=phi(8kappa), the final cost can be absorbed with the displayed universal factor. The result is (max(0,L)+2)phi(8kappa). If additionally q>=4, then L>=1, so L+2<=3L. This gives the explicit constant3/log4 in front of phi(8kappa)logq. These are conditional consequences; no monotonicity of phi or all-accuracy logarithmic bound is silently asserted.

Braces mark parameters Lean can infer; square brackets request structures such as a measurable space or probability measure. Named hypotheses are mathematical premises, not facts established by this declaration. Section parameters are described in the mathematical hypotheses above; the module link retains their exact source context.

theorem logarithmic_accuracy_and_cost {f : E → ℝ} {α β R ε : ℝ}
    (hf : ContDiff ℝ 1 f) (hsc : StrongConvexOn univ α f)
    (hα : 0 < α) (hβ : 0 < β) (hR : 0 < R) (hε : 0 < ε)
    {z x₀ : E} (hmin : IsMinOn f univ z) (hx₀ : ‖x₀-z‖ ≤ R)
    (A : E → ℝ → ℝ → E × ℕ) (φ : ℝ → ℕ)
    (hA : ∀ y r δ, 0 < r → 0 < δ → ‖y-z‖ ≤ r →
      f (A y r δ).1 - f z ≤ δ ∧ (A y r δ).2 ≤ φ (β*r^2/δ)) :
    let L := Real.log (α*R^2/ε) / Real.log 4
    let N := ⌈L⌉₊
    let r := fun k : ℕ => R / (2:ℝ)^k
    let run : ℕ → E × ℕ := Nat.rec (x₀, 0) (fun k p =>
      let w := A p.1 (r k) (α*(r k)^2/8)
      (w.1, p.2 + w.2))
    let last := A (run N).1 (Real.sqrt (ε/α)) ε
    let C := (run N).2 + last.2
    (N : ℝ) ≤ max 0 L + 1 ∧ (N = 0 ↔ α*R^2 ≤ ε) ∧
      ‖(run N).1-z‖ ≤ r N ∧ f last.1 - f z ≤ ε ∧
      C ≤ N * φ (8*(β/α)) + φ (β/α) ∧
      (C : ℝ) ≤ (max 0 L + 1) * φ (8*(β/α)) + φ (β/α) ∧
      (φ (β/α) ≤ φ (8*(β/α)) →
        (C : ℝ) ≤ (max 0 L + 2) * φ (8*(β/α)) ∧
        (4 ≤ α*R^2/ε → (C : ℝ) ≤ 3*L*φ (8*(β/α)))) := by
  let L := Real.log (α*R^2/ε) / Real.log 4
  let N := ⌈L⌉₊
  have hlog : 0 < Real.log 4 := Real.log_pos (by norm_num)
  have hq : 0 < α*R^2/ε := by positivity
  have hround : (N : ℝ) ≤ max 0 L + 1 := by
    by_cases hL : 0 ≤ L
    · exact (Nat.ceil_lt_add_one hL).le.trans (by rw [max_eq_right hL])
    · have hz : N = 0 := Nat.ceil_eq_zero.mpr (le_of_not_ge hL)
      rw [hz, Nat.cast_zero, max_eq_left (le_of_not_ge hL)]
      norm_num
  have hzero : N = 0 ↔ α*R^2 ≤ ε := by
    change ⌈Real.log (α*R^2/ε) / Real.log 4⌉₊ = 0 ↔ _
    rw [Nat.ceil_eq_zero, div_le_iff₀ hlog, zero_mul]
    rw [← Real.log_one, Real.log_le_log_iff hq (by norm_num)]
    exact (div_le_iff₀ hε).trans (by simp)
  have hpow : α*R^2/ε ≤ (4:ℝ)^N := by
    apply (Real.log_le_log_iff hq (by positivity)).mp
    rw [Real.log_pow]
    exact (div_le_iff₀ hlog).mp (Nat.le_ceil L)
  have hid : ((2:ℝ)^N)^2 = (4:ℝ)^N := by
    rw [← pow_mul, Nat.mul_comm N 2, pow_mul]
    norm_num
  have hN : α*(R/(2:ℝ)^N)^2 ≤ ε := by
    rw [div_pow, ← mul_div_assoc, hid]
    apply (div_le_iff₀ (by positivity : 0 < (4:ℝ)^N)).mpr
    have h := (div_le_iff₀ hε).mp hpow
    nlinarith
  have hb := RestartReduction.radius_accuracy_and_cost hf hsc hα hβ hR hε
    hmin hx₀ A φ hA N hN
  let r := fun k : ℕ => R / (2:ℝ)^k
  let run : ℕ → E × ℕ := Nat.rec (x₀, 0) (fun k p =>
    let w := A p.1 (r k) (α*(r k)^2/8)
    (w.1, p.2 + w.2))
  let last := A (run N).1 (Real.sqrt (ε/α)) ε
  let C := (run N).2 + last.2
  have hcast : (C : ℝ) ≤ (N : ℝ) * φ (8*(β/α)) + φ (β/α) := by
    have hc : C ≤ N * φ (8*(β/α)) + φ (β/α) := hb.2.2.2
    simpa only [Nat.cast_add, Nat.cast_mul] using ((Nat.cast_le (α := ℝ)).mpr hc)
  have hm : (N : ℝ) * φ (8*(β/α)) ≤
      (max 0 L + 1) * φ (8*(β/α)) :=
    mul_le_mul_of_nonneg_right hround (Nat.cast_nonneg _)
  have hreal : (C : ℝ) ≤ (max 0 L + 1) * φ (8*(β/α)) + φ (β/α) := by
    linarith
  refine ⟨hround, hzero, hb.1, hb.2.2.1, hb.2.2.2, hreal, ?_⟩
  intro hφ
  have hp : (φ (β/α) : ℝ) ≤ φ (8*(β/α)) := by exact_mod_cast hφ
  have ha : (C : ℝ) ≤ (max 0 L + 2) * φ (8*(β/α)) := by nlinarith
  refine ⟨ha, ?_⟩
  intro hsmall
  have hL : 1 ≤ L := by
    apply (le_div_iff₀ hlog).mpr
    simpa using (Real.log_le_log (by norm_num : (0:ℝ)<4) hsmall)
  rw [max_eq_right (by linarith : 0 ≤ L)] at ha
  have hfactor : L+2 ≤ 3*L := by linarith
  exact ha.trans (mul_le_mul_of_nonneg_right hfactor (Nat.cast_nonneg _))

end AutoSamplingTheory.TechnicalLemmas.Analysis.RestartLogComplexity

Exact module and namespace context

Scope and omitted-condition boundaries

  • This closes logarithmic horizon selection and certified cost normalization for the actual fixed-objective restart construction. It does not formalize oracle execution, uniform algorithm existence over the source function class, or the entire Lemma4.1.
  • The positive radius and supplied minimizer/base-solver contract are inherited. Literal R=0 stopping, minimizer existence and reverse regularization remain separate. C1 Hilbert geometry generalizes the smooth Euclidean source specialization.
  • The unconditional bound retains both phi values and the final call for N=0. The displayed absorbed bound is an implication with phi(kappa)<=phi(8kappa); the pure logarithmic constant3/log4 also requires q>=4. These are sufficient conditions for these consequences, not an accepted repair of the original source.
  • No whole-lemma source equivalence or asymptotic first-order class theorem is claimed. An actual oracle interpretation needs an external implementation of the supplied A contract.
  • Tests instantiate a real step1/2 gradient-descent solver on x²/2, with each call executing and counting ceil(r²/delta)+1 updates. For every positive epsilon the theorem gives C<=9N+2, a global real bound9(max(0,L)+1)+2 and, for epsilon<=1/4, C<=27L. Separate ceiling checks cover q=4³,q=1/4 and q=1.

Source and reuse

ASTIS parents called

Mathlib API called (external library)

  • Nat.le_ceil
  • Nat.ceil_eq_zero
  • Nat.ceil_lt_add_one
  • Real.log_pos
  • Real.log_pow
  • Real.log_le_log_iff
  • Real.log_le_log
  • Nat.cast_le

Mathematical sources

ASTIS prose is not a quotation or a source-equivalence certificate. Definitions and aliases are explained as constructions, not counted as new mathematical proofs.