Samplinglib
Lean gate not recorded for this source state main · 0e31a3cda412

ConvexGradientGapSharpness: mathematical reading route

Read the statements and derivations in source order. Every result has its own optional Lean statement and proof. Source assumptions, library reuse and unproved boundaries are kept explicit.

  1. Quadratics witness the convex function-value convergence order
ASTIS mathematical exposition

Quadratics witness the convex function-value convergence order

AutoSamplingTheory.TechnicalLemmas.Analysis.ConvexGradientGapSharpness.quadratic_gap_lower_bound · theorem · Teaching coverage

Statement

For every real beta>0 and natural horizon N, the explicit positive scalar quadratic below is C2, mu-strongly convex, admits the global beta quadratic upper model and is minimized at0. Its actual gradient-descent iterate from1 at step1/beta has the following exact gap and lower bound.

\[\begin{gathered}\mu=\frac\beta{2(N+1)},\quad f(x)=\frac\mu2x^2,\quad T(x)=x-\beta^{-1}\nabla f(x),\quad x_N=T^N(1),\\ f(x_N)-f(0)=\frac\beta{4(N+1)}\left(1-\frac1{2(N+1)}\right)^{2N}\ge\frac\beta{16(N+1)}.\end{gathered}\]

All objects and hypotheses

  • Beta is real and strictly positive; N is natural, including0. The function and positive curvature are chosen from this N; each horizon may use a different function.
  • The ambient space is R. The conclusion certifies ContDiff R2 f, StrongConvexOn univ mu f, the global beta upper model f(y)<=f(x)+<gradient f(x),y−x>+beta||y−x||²/2 and IsMinOn f univ0.
  • T uses the true gradient, with constant positive step1/beta. Iteration starts at1, at distance1 from the certified minimizer0. T^0 is the identity.

Mathematical proof

1. Choose curvature after the horizon

For beta>0 and N natural set D=N+1 and mu=beta/(2D). Then D>=1 and0<mu<=beta. Reuse the compiled scalar-quadratic witness with both curvature endpoints equal to mu. Equality of the endpoints forces that witness to be exactly f(x)=mu x²/2. It supplies C2 regularity, mu strong convexity, minimizer0 and the genuine gradient trajectory.

\[D=N+1,\qquad \mu=\frac\beta{2D}>0,\qquad f(x)=\frac\mu2x^2,\qquad x_0=1.\]
Corresponding Lean step

GradientDescentSharpness.exists_quadratic_worst_case at equal endpoints; order and positive division.

2. Obtain the exact value along actual gradient iterates

At the prescribed step1/beta the scalar contraction is q=1−1/(2D), which is nonnegative. The existing norm identity gives norm(x_N)=q^N. Since the objective is mu times the squared norm divided by2 and f(0)=0, this is also an exact objective-gap formula. Enlarging the quadratic upper model from mu to beta verifies the advertised smoothness class.

\[q=1-\frac1{2D}\ge0,\quad x_N=T^N(1),\quad T(x)=x-\beta^{-1}\nabla f(x),\quad f(x_N)-f(0)=\frac\beta{4D}q^{2N}.\]
Corresponding Lean step

Existing actual norm identity; abs_mul_abs_self and scalar norm; nonnegative quadratic remainder when mu<=beta.

3. Keep a fixed fraction of the initial distance

Apply Bernoulli to q and exponent N. The resulting affine lower bound is (D+1)/(2D), at least1/2. Thus even after N steps the selected small curvature leaves at least half of the initial distance.

\[q^N\ge1+N(q-1)=\frac{D+1}{2D}\ge\frac12.\]
Corresponding Lean step

Mathlib one_add_mul_sub_le_pow; positive denominator arithmetic.

4. Convert distance retention into a gap lower bound

Squaring the nonnegative half-distance bound gives q^(2N)>=1/4. Multiply by the positive beta/(4D) in the exact objective identity. With the source upper rate for N>0, the resulting family obstructs any uniformly faster order at this prescribed step.

\[f(x_N)-f(0)=\frac\beta{4D}(q^N)^2\ge\frac\beta{16D}=\frac\beta{16(N+1)}.\]
Corresponding Lean step

pow_mul; square inequality and mul_le_mul_of_nonneg_left. The general upper bound is composed in the test, not assumed in this proof.

Lean statement · quadratic_gap_lower_bound

Positive beta and natural horizon determine one explicit scalar quadratic, all admissibility certificates, exact actual-iterate objective gap and beta/(16(N+1)) lower bound.

Braces mark parameters Lean can infer; square brackets request structures such as a measurable space or probability measure. Named hypotheses are mathematical premises, not facts established by this declaration. Section parameters are described in the mathematical hypotheses above; the module link retains their exact source context.

theorem quadratic_gap_lower_bound {β : ℝ} (hβ : 0 < β) (N : ℕ) :
    let μ := β / (2 * ((N : ℝ) + 1))
    let f : ℝ → ℝ := fun x => μ * x ^ 2 / 2
    0 < μ ∧ ContDiff ℝ 2 f ∧ StrongConvexOn univ μ f ∧
      (∀ x y, f y ≤ f x + inner ℝ (gradient f x) (y - x) + β / 2 * ‖y - x‖ ^ 2) ∧
      IsMinOn f univ 0 ∧
      let z := (fun x => x - (1 / β) * gradient f x)^[N] 1
      f z - f 0 = β / (4 * ((N : ℝ) + 1)) *
        (1 - 1 / (2 * ((N : ℝ) + 1))) ^ (2 * N) ∧
      β / (16 * ((N : ℝ) + 1)) ≤ f z - f 0

Exact module and namespace context

Lean proof · quadratic_gap_lower_bound

For beta>0 and N natural set D=N+1 and mu=beta/(2D). Then D>=1 and0<mu<=beta. Reuse the compiled scalar-quadratic witness with both curvature endpoints equal to mu. Equality of the endpoints forces that witness to be exactly f(x)=mu x²/2. It supplies C2 regularity, mu strong convexity, minimizer0 and the genuine gradient trajectory. At the prescribed step1/beta the scalar contraction is q=1−1/(2D), which is nonnegative. The existing norm identity gives norm(x_N)=q^N. Since the objective is mu times the squared norm divided by2 and f(0)=0, this is also an exact objective-gap formula. Enlarging the quadratic upper model from mu to beta verifies the advertised smoothness class. Apply Bernoulli to q and exponent N. The resulting affine lower bound is (D+1)/(2D), at least1/2. Thus even after N steps the selected small curvature leaves at least half of the initial distance. Squaring the nonnegative half-distance bound gives q^(2N)>=1/4. Multiply by the positive beta/(4D) in the exact objective identity. With the source upper rate for N>0, the resulting family obstructs any uniformly faster order at this prescribed step.

Braces mark parameters Lean can infer; square brackets request structures such as a measurable space or probability measure. Named hypotheses are mathematical premises, not facts established by this declaration. Section parameters are described in the mathematical hypotheses above; the module link retains their exact source context.

theorem quadratic_gap_lower_bound {β : ℝ} (hβ : 0 < β) (N : ℕ) :
    let μ := β / (2 * ((N : ℝ) + 1))
    let f : ℝ → ℝ := fun x => μ * x ^ 2 / 2
    0 < μ ∧ ContDiff ℝ 2 f ∧ StrongConvexOn univ μ f ∧
      (∀ x y, f y ≤ f x + inner ℝ (gradient f x) (y - x) + β / 2 * ‖y - x‖ ^ 2) ∧
      IsMinOn f univ 0 ∧
      let z := (fun x => x - (1 / β) * gradient f x)^[N] 1
      f z - f 0 = β / (4 * ((N : ℝ) + 1)) *
        (1 - 1 / (2 * ((N : ℝ) + 1))) ^ (2 * N) ∧
      β / (16 * ((N : ℝ) + 1)) ≤ f z - f 0 := by
  let D : ℝ := (N : ℝ) + 1
  have hD : 0 < D := by dsimp [D]; positivity
  have hD1 : 1 ≤ D := by dsimp [D]; exact le_add_of_nonneg_left (Nat.cast_nonneg N)
  let μ := β / (2 * D)
  have hμ : 0 < μ := div_pos hβ (by positivity)
  have hμβ : μ ≤ β := by
    apply (div_le_iff₀ (by positivity : 0 < 2 * D)).mpr
    nlinarith
  obtain ⟨ν, hν, _, hf, hc, hu, hm, hn⟩ :=
    exists_quadratic_worst_case hμ (le_refl μ) (1 / β)
  have hv : ν = μ := hν.elim id id
  subst ν
  let f : ℝ → ℝ := fun x => μ * x ^ 2 / 2
  let q : ℝ := 1 - 1 / (2 * D)
  have hq : 0 ≤ q := by
    have : 1 / (2 * D) ≤ 1 := (div_le_one (by positivity)).mpr (by linarith)
    dsimp [q]; linarith
  have he : 1 - 1 / β * μ = q := by
    dsimp [μ, q]; field_simp
  have hn' : ‖(fun x => x - (1 / β) * gradient f x)^[N] 1‖ = q ^ N := by
    simpa only [max_self, he, abs_of_nonneg hq] using (hn N).1
  have hpow : 1 / 2 ≤ q ^ N := by
    have hb := one_add_mul_sub_le_pow (by linarith : -1 ≤ q) N
    have heq : 1 + (N : ℝ) * (q - 1) = (D + 1) / (2 * D) := by
      dsimp [q, D]; field_simp; ring
    rw [heq] at hb
    have : (1 : ℝ) / 2 ≤ (D + 1) / (2 * D) := by
      apply (le_div_iff₀ (by positivity)).mpr
      linarith
    exact this.trans hb
  have huβ : ∀ x y, f y ≤ f x + inner ℝ (gradient f x) (y - x) + β / 2 * ‖y - x‖ ^ 2 := by
    intro x y
    have hh : f y ≤ f x + inner ℝ (gradient f x) (y - x) + μ / 2 * ‖y - x‖ ^ 2 := hu x y
    nlinarith [mul_nonneg (sub_nonneg.mpr hμβ) (sq_nonneg ‖y - x‖)]
  change 0 < μ ∧ _ ∧ _ ∧ _ ∧ _ ∧ _
  refine ⟨hμ, hf, hc, huβ, hm, ?_⟩
  dsimp only
  have hvalue : f ((fun x => x - (1 / β) * gradient f x)^[N] 1) - f 0 =
      β / (4 * D) * (q ^ N) ^ 2 := by
    have hs : ((fun x => x - (1 / β) * gradient f x)^[N] 1) ^ 2 = (q ^ N) ^ 2 := by
      simpa only [Real.norm_eq_abs, abs_mul_abs_self, sq] using congrArg (fun r : ℝ => r * r) hn'
    change μ * _ ^ 2 / 2 - μ * 0 ^ 2 / 2 = _
    rw [hs]
    dsimp [μ]; field_simp; ring
  rw [hvalue]
  change β / (4 * D) * (q ^ N) ^ 2 = β / (4 * D) * q ^ (2 * N) ∧
    β / (16 * D) ≤ β / (4 * D) * (q ^ N) ^ 2
  constructor
  · rw [Nat.mul_comm 2 N, pow_mul]
  · have hs : (1 : ℝ) / 4 ≤ (q ^ N) ^ 2 := by nlinarith
    have := mul_le_mul_of_nonneg_left hs (by positivity : 0 ≤ β / (4 * D))
    convert this using 1 <;> first | rfl | (field_simp; ring)

end AutoSamplingTheory.TechnicalLemmas.Analysis.ConvexGradientGapSharpness

Exact module and namespace context

Scope and omitted-condition boundaries

  • The horizon N is fixed before choosing the objective f_N. Its positive curvature mu_N shrinks with N. This is a family witnessing a worst-case bound over the convex beta-smooth class; it is not a single fixed strongly convex objective with an asymptotic reciprocal tail.
  • Beta is a valid upper curvature bound, not the tight smoothness constant of this scalar witness. The step is prescribed as1/beta, not tuned to its intrinsic curvature. Strong convexity with mu_N>0 implies ordinary convexity.
  • The constant1/16 is an ASTIS-derived convenient lower constant, not an optimal constant or a number printed in the exercise. For N>=1, beta/[16(N+1)]>=beta/(32N); comparison with the source upper bound beta/(2N) shows order sharpness at initial distance1. The theorem states the finite N+1 bound.
  • N=0 is included only as a nonsingular lower-bound identity: the actual initial gap is beta/4. It does not interpret the singular source1/N upper bound at zero.
  • No variable/adaptive-step result, general first-order oracle lower bound, full Exercise3.3 comparison list, acceleration claim or companion-paper completion follows. The function, actual gradients, smoothness model and minimizer are certified, not supplied as hypotheses.

Source and reuse

ASTIS parents called

Mathlib API called (external library)

  • one_add_mul_sub_le_pow
  • abs_mul_abs_self
  • pow_mul
  • mul_le_mul_of_nonneg_left

Mathematical sources

ASTIS prose is not a quotation or a source-equivalence certificate. Definitions and aliases are explained as constructions, not counted as new mathematical proofs.