Samplinglib
Lean gate not recorded for this source state main · 0e31a3cda412
ASTIS mathematical exposition

Quadratics witness the convex function-value convergence order

AutoSamplingTheory.TechnicalLemmas.Analysis.ConvexGradientGapSharpness.quadratic_gap_lower_bound · theorem · Teaching coverage

Statement

For every real beta>0 and natural horizon N, the explicit positive scalar quadratic below is C2, mu-strongly convex, admits the global beta quadratic upper model and is minimized at0. Its actual gradient-descent iterate from1 at step1/beta has the following exact gap and lower bound.

\[\begin{gathered}\mu=\frac\beta{2(N+1)},\quad f(x)=\frac\mu2x^2,\quad T(x)=x-\beta^{-1}\nabla f(x),\quad x_N=T^N(1),\\ f(x_N)-f(0)=\frac\beta{4(N+1)}\left(1-\frac1{2(N+1)}\right)^{2N}\ge\frac\beta{16(N+1)}.\end{gathered}\]

All objects and hypotheses

  • Beta is real and strictly positive; N is natural, including0. The function and positive curvature are chosen from this N; each horizon may use a different function.
  • The ambient space is R. The conclusion certifies ContDiff R2 f, StrongConvexOn univ mu f, the global beta upper model f(y)<=f(x)+<gradient f(x),y−x>+beta||y−x||²/2 and IsMinOn f univ0.
  • T uses the true gradient, with constant positive step1/beta. Iteration starts at1, at distance1 from the certified minimizer0. T^0 is the identity.

Mathematical proof

1. Choose curvature after the horizon

For beta>0 and N natural set D=N+1 and mu=beta/(2D). Then D>=1 and0<mu<=beta. Reuse the compiled scalar-quadratic witness with both curvature endpoints equal to mu. Equality of the endpoints forces that witness to be exactly f(x)=mu x²/2. It supplies C2 regularity, mu strong convexity, minimizer0 and the genuine gradient trajectory.

\[D=N+1,\qquad \mu=\frac\beta{2D}>0,\qquad f(x)=\frac\mu2x^2,\qquad x_0=1.\]
Corresponding Lean step

GradientDescentSharpness.exists_quadratic_worst_case at equal endpoints; order and positive division.

2. Obtain the exact value along actual gradient iterates

At the prescribed step1/beta the scalar contraction is q=1−1/(2D), which is nonnegative. The existing norm identity gives norm(x_N)=q^N. Since the objective is mu times the squared norm divided by2 and f(0)=0, this is also an exact objective-gap formula. Enlarging the quadratic upper model from mu to beta verifies the advertised smoothness class.

\[q=1-\frac1{2D}\ge0,\quad x_N=T^N(1),\quad T(x)=x-\beta^{-1}\nabla f(x),\quad f(x_N)-f(0)=\frac\beta{4D}q^{2N}.\]
Corresponding Lean step

Existing actual norm identity; abs_mul_abs_self and scalar norm; nonnegative quadratic remainder when mu<=beta.

3. Keep a fixed fraction of the initial distance

Apply Bernoulli to q and exponent N. The resulting affine lower bound is (D+1)/(2D), at least1/2. Thus even after N steps the selected small curvature leaves at least half of the initial distance.

\[q^N\ge1+N(q-1)=\frac{D+1}{2D}\ge\frac12.\]
Corresponding Lean step

Mathlib one_add_mul_sub_le_pow; positive denominator arithmetic.

4. Convert distance retention into a gap lower bound

Squaring the nonnegative half-distance bound gives q^(2N)>=1/4. Multiply by the positive beta/(4D) in the exact objective identity. With the source upper rate for N>0, the resulting family obstructs any uniformly faster order at this prescribed step.

\[f(x_N)-f(0)=\frac\beta{4D}(q^N)^2\ge\frac\beta{16D}=\frac\beta{16(N+1)}.\]
Corresponding Lean step

pow_mul; square inequality and mul_le_mul_of_nonneg_left. The general upper bound is composed in the test, not assumed in this proof.

Lean statement · quadratic_gap_lower_bound

Positive beta and natural horizon determine one explicit scalar quadratic, all admissibility certificates, exact actual-iterate objective gap and beta/(16(N+1)) lower bound.

Braces mark parameters Lean can infer; square brackets request structures such as a measurable space or probability measure. Named hypotheses are mathematical premises, not facts established by this declaration. Section parameters are described in the mathematical hypotheses above; the module link retains their exact source context.

theorem quadratic_gap_lower_bound {β : ℝ} (hβ : 0 < β) (N : ℕ) :
    let μ := β / (2 * ((N : ℝ) + 1))
    let f : ℝ → ℝ := fun x => μ * x ^ 2 / 2
    0 < μ ∧ ContDiff ℝ 2 f ∧ StrongConvexOn univ μ f ∧
      (∀ x y, f y ≤ f x + inner ℝ (gradient f x) (y - x) + β / 2 * ‖y - x‖ ^ 2) ∧
      IsMinOn f univ 0 ∧
      let z := (fun x => x - (1 / β) * gradient f x)^[N] 1
      f z - f 0 = β / (4 * ((N : ℝ) + 1)) *
        (1 - 1 / (2 * ((N : ℝ) + 1))) ^ (2 * N) ∧
      β / (16 * ((N : ℝ) + 1)) ≤ f z - f 0

Exact module and namespace context

Lean proof · quadratic_gap_lower_bound

For beta>0 and N natural set D=N+1 and mu=beta/(2D). Then D>=1 and0<mu<=beta. Reuse the compiled scalar-quadratic witness with both curvature endpoints equal to mu. Equality of the endpoints forces that witness to be exactly f(x)=mu x²/2. It supplies C2 regularity, mu strong convexity, minimizer0 and the genuine gradient trajectory. At the prescribed step1/beta the scalar contraction is q=1−1/(2D), which is nonnegative. The existing norm identity gives norm(x_N)=q^N. Since the objective is mu times the squared norm divided by2 and f(0)=0, this is also an exact objective-gap formula. Enlarging the quadratic upper model from mu to beta verifies the advertised smoothness class. Apply Bernoulli to q and exponent N. The resulting affine lower bound is (D+1)/(2D), at least1/2. Thus even after N steps the selected small curvature leaves at least half of the initial distance. Squaring the nonnegative half-distance bound gives q^(2N)>=1/4. Multiply by the positive beta/(4D) in the exact objective identity. With the source upper rate for N>0, the resulting family obstructs any uniformly faster order at this prescribed step.

Braces mark parameters Lean can infer; square brackets request structures such as a measurable space or probability measure. Named hypotheses are mathematical premises, not facts established by this declaration. Section parameters are described in the mathematical hypotheses above; the module link retains their exact source context.

theorem quadratic_gap_lower_bound {β : ℝ} (hβ : 0 < β) (N : ℕ) :
    let μ := β / (2 * ((N : ℝ) + 1))
    let f : ℝ → ℝ := fun x => μ * x ^ 2 / 2
    0 < μ ∧ ContDiff ℝ 2 f ∧ StrongConvexOn univ μ f ∧
      (∀ x y, f y ≤ f x + inner ℝ (gradient f x) (y - x) + β / 2 * ‖y - x‖ ^ 2) ∧
      IsMinOn f univ 0 ∧
      let z := (fun x => x - (1 / β) * gradient f x)^[N] 1
      f z - f 0 = β / (4 * ((N : ℝ) + 1)) *
        (1 - 1 / (2 * ((N : ℝ) + 1))) ^ (2 * N) ∧
      β / (16 * ((N : ℝ) + 1)) ≤ f z - f 0 := by
  let D : ℝ := (N : ℝ) + 1
  have hD : 0 < D := by dsimp [D]; positivity
  have hD1 : 1 ≤ D := by dsimp [D]; exact le_add_of_nonneg_left (Nat.cast_nonneg N)
  let μ := β / (2 * D)
  have hμ : 0 < μ := div_pos hβ (by positivity)
  have hμβ : μ ≤ β := by
    apply (div_le_iff₀ (by positivity : 0 < 2 * D)).mpr
    nlinarith
  obtain ⟨ν, hν, _, hf, hc, hu, hm, hn⟩ :=
    exists_quadratic_worst_case hμ (le_refl μ) (1 / β)
  have hv : ν = μ := hν.elim id id
  subst ν
  let f : ℝ → ℝ := fun x => μ * x ^ 2 / 2
  let q : ℝ := 1 - 1 / (2 * D)
  have hq : 0 ≤ q := by
    have : 1 / (2 * D) ≤ 1 := (div_le_one (by positivity)).mpr (by linarith)
    dsimp [q]; linarith
  have he : 1 - 1 / β * μ = q := by
    dsimp [μ, q]; field_simp
  have hn' : ‖(fun x => x - (1 / β) * gradient f x)^[N] 1‖ = q ^ N := by
    simpa only [max_self, he, abs_of_nonneg hq] using (hn N).1
  have hpow : 1 / 2 ≤ q ^ N := by
    have hb := one_add_mul_sub_le_pow (by linarith : -1 ≤ q) N
    have heq : 1 + (N : ℝ) * (q - 1) = (D + 1) / (2 * D) := by
      dsimp [q, D]; field_simp; ring
    rw [heq] at hb
    have : (1 : ℝ) / 2 ≤ (D + 1) / (2 * D) := by
      apply (le_div_iff₀ (by positivity)).mpr
      linarith
    exact this.trans hb
  have huβ : ∀ x y, f y ≤ f x + inner ℝ (gradient f x) (y - x) + β / 2 * ‖y - x‖ ^ 2 := by
    intro x y
    have hh : f y ≤ f x + inner ℝ (gradient f x) (y - x) + μ / 2 * ‖y - x‖ ^ 2 := hu x y
    nlinarith [mul_nonneg (sub_nonneg.mpr hμβ) (sq_nonneg ‖y - x‖)]
  change 0 < μ ∧ _ ∧ _ ∧ _ ∧ _ ∧ _
  refine ⟨hμ, hf, hc, huβ, hm, ?_⟩
  dsimp only
  have hvalue : f ((fun x => x - (1 / β) * gradient f x)^[N] 1) - f 0 =
      β / (4 * D) * (q ^ N) ^ 2 := by
    have hs : ((fun x => x - (1 / β) * gradient f x)^[N] 1) ^ 2 = (q ^ N) ^ 2 := by
      simpa only [Real.norm_eq_abs, abs_mul_abs_self, sq] using congrArg (fun r : ℝ => r * r) hn'
    change μ * _ ^ 2 / 2 - μ * 0 ^ 2 / 2 = _
    rw [hs]
    dsimp [μ]; field_simp; ring
  rw [hvalue]
  change β / (4 * D) * (q ^ N) ^ 2 = β / (4 * D) * q ^ (2 * N) ∧
    β / (16 * D) ≤ β / (4 * D) * (q ^ N) ^ 2
  constructor
  · rw [Nat.mul_comm 2 N, pow_mul]
  · have hs : (1 : ℝ) / 4 ≤ (q ^ N) ^ 2 := by nlinarith
    have := mul_le_mul_of_nonneg_left hs (by positivity : 0 ≤ β / (4 * D))
    convert this using 1 <;> first | rfl | (field_simp; ring)

end AutoSamplingTheory.TechnicalLemmas.Analysis.ConvexGradientGapSharpness

Exact module and namespace context

Scope and omitted-condition boundaries

  • The horizon N is fixed before choosing the objective f_N. Its positive curvature mu_N shrinks with N. This is a family witnessing a worst-case bound over the convex beta-smooth class; it is not a single fixed strongly convex objective with an asymptotic reciprocal tail.
  • Beta is a valid upper curvature bound, not the tight smoothness constant of this scalar witness. The step is prescribed as1/beta, not tuned to its intrinsic curvature. Strong convexity with mu_N>0 implies ordinary convexity.
  • The constant1/16 is an ASTIS-derived convenient lower constant, not an optimal constant or a number printed in the exercise. For N>=1, beta/[16(N+1)]>=beta/(32N); comparison with the source upper bound beta/(2N) shows order sharpness at initial distance1. The theorem states the finite N+1 bound.
  • N=0 is included only as a nonsingular lower-bound identity: the actual initial gap is beta/4. It does not interpret the singular source1/N upper bound at zero.
  • No variable/adaptive-step result, general first-order oracle lower bound, full Exercise3.3 comparison list, acceleration claim or companion-paper completion follows. The function, actual gradients, smoothness model and minimizer are certified, not supplied as hypotheses.

Source and reuse

ASTIS parents called

Mathlib API called (external library)

  • one_add_mul_sub_le_pow
  • abs_mul_abs_self
  • pow_mul
  • mul_le_mul_of_nonneg_left

Mathematical sources

ASTIS prose is not a quotation or a source-equivalence certificate. Definitions and aliases are explained as constructions, not counted as new mathematical proofs.