Quadratics witness the convex function-value convergence order
AutoSamplingTheory.TechnicalLemmas.Analysis.ConvexGradientGapSharpness.quadratic_gap_lower_bound · theorem · Teaching coverage
Mathematical statement
For every real beta>0 and natural horizon N, the explicit positive scalar quadratic below is C2, mu-strongly convex, admits the global beta quadratic upper model and is minimized at0. Its actual gradient-descent iterate from1 at step1/beta has the following exact gap and lower bound.
Lean statement · quadratic_gap_lower_bound
Positive beta and natural horizon determine one explicit scalar quadratic, all admissibility certificates, exact actual-iterate objective gap and beta/(16(N+1)) lower bound.
Braces mark parameters Lean can infer; square brackets request structures such as a measurable space or probability measure. Named hypotheses are mathematical premises, not facts established by this declaration. Section parameters are described in the mathematical hypotheses above; the module link retains their exact source context.
theorem quadratic_gap_lower_bound {β : ℝ} (hβ : 0 < β) (N : ℕ) :
let μ := β / (2 * ((N : ℝ) + 1))
let f : ℝ → ℝ := fun x => μ * x ^ 2 / 2
0 < μ ∧ ContDiff ℝ 2 f ∧ StrongConvexOn univ μ f ∧
(∀ x y, f y ≤ f x + inner ℝ (gradient f x) (y - x) + β / 2 * ‖y - x‖ ^ 2) ∧
IsMinOn f univ 0 ∧
let z := (fun x => x - (1 / β) * gradient f x)^[N] 1
f z - f 0 = β / (4 * ((N : ℝ) + 1)) *
(1 - 1 / (2 * ((N : ℝ) + 1))) ^ (2 * N) ∧
β / (16 * ((N : ℝ) + 1)) ≤ f z - f 0Proof roadmap
- 1Choose curvature after the horizon
For beta>0 and N natural set D=N+1 and mu=beta/(2D). Then D≥1 and0<mu≤beta. Reuse the compiled scalar-quadratic witness with both curvature endpoints equal to mu. Equality of the endpoints forces that witness to be exactly f(x)=mu x²/2. It supplies C2 regularity, mu strong convexity, minimizer0 and the genuine gradient trajectory.
- 2Obtain the exact value along actual gradient iterates
At the prescribed step1/beta the scalar contraction is q=1−1/(2D), which is nonnegative. The existing norm identity gives norm(x_N)=q^N. Since the objective is mu times the squared norm divided by2 and f(0)=0, this is also an exact objective-gap formula. Enlarging the quadratic upper model from mu to beta verifies the advertised smoothness class.
- 3Keep a fixed fraction of the initial distance
Apply Bernoulli to q and exponent N. The resulting affine lower bound is (D+1)/(2D), at least1/2. Thus even after N steps the selected small curvature leaves at least half of the initial distance.
- 4Convert distance retention into a gap lower bound
Squaring the nonnegative half-distance bound gives q^(2N)≥1/4. Multiply by the positive beta/(4D) in the exact objective identity. With the source upper rate for N>0, the resulting family obstructs any uniformly faster order at this prescribed step.
Full assumptions and notation
All objects and hypotheses
- Beta is real and strictly positive; N is natural, including0. The function and positive curvature are chosen from this N; each horizon may use a different function.
- The ambient space is R. The conclusion certifies ContDiff R2 f, StrongConvexOn univ mu f, the global beta upper model f(y)≤f(x)+<gradient f(x),y−x>+beta||y−x||²/2 and IsMinOn f univ0.
- T uses the true gradient, with constant positive step1/beta. Iteration starts at1, at distance1 from the certified minimizer0. T^0 is the identity.
Mathematical proof
1. Choose curvature after the horizon
For beta>0 and N natural set D=N+1 and mu=beta/(2D). Then D≥1 and0<mu≤beta. Reuse the compiled scalar-quadratic witness with both curvature endpoints equal to mu. Equality of the endpoints forces that witness to be exactly f(x)=mu x²/2. It supplies C2 regularity, mu strong convexity, minimizer0 and the genuine gradient trajectory.
Corresponding Lean step
GradientDescentSharpness.exists_quadratic_worst_case at equal endpoints; order and positive division.
2. Obtain the exact value along actual gradient iterates
At the prescribed step1/beta the scalar contraction is q=1−1/(2D), which is nonnegative. The existing norm identity gives norm(x_N)=q^N. Since the objective is mu times the squared norm divided by2 and f(0)=0, this is also an exact objective-gap formula. Enlarging the quadratic upper model from mu to beta verifies the advertised smoothness class.
Corresponding Lean step
Existing actual norm identity; abs_mul_abs_self and scalar norm; nonnegative quadratic remainder when mu≤beta.
3. Keep a fixed fraction of the initial distance
Apply Bernoulli to q and exponent N. The resulting affine lower bound is (D+1)/(2D), at least1/2. Thus even after N steps the selected small curvature leaves at least half of the initial distance.
Corresponding Lean step
Mathlib one_add_mul_sub_le_pow; positive denominator arithmetic.
4. Convert distance retention into a gap lower bound
Squaring the nonnegative half-distance bound gives q^(2N)≥1/4. Multiply by the positive beta/(4D) in the exact objective identity. With the source upper rate for N>0, the resulting family obstructs any uniformly faster order at this prescribed step.
Corresponding Lean step
pow_mul; square inequality and mul_le_mul_of_nonneg_left. The general upper bound is composed in the test, not assumed in this proof.
Lean verification
The exact proof begins folded immediately after the mathematical argument; the statement is folded beside the claim above.
Lean proof · quadratic_gap_lower_bound
For beta>0 and N natural set D=N+1 and mu=beta/(2D). Then D>=1 and0<mu<=beta. Reuse the compiled scalar-quadratic witness with both curvature endpoints equal to mu. Equality of the endpoints forces that witness to be exactly f(x)=mu x²/2. It supplies C2 regularity, mu strong convexity, minimizer0 and the genuine gradient trajectory. At the prescribed step1/beta the scalar contraction is q=1−1/(2D), which is nonnegative. The existing norm identity gives norm(x_N)=q^N. Since the objective is mu times the squared norm divided by2 and f(0)=0, this is also an exact objective-gap formula. Enlarging the quadratic upper model from mu to beta verifies the advertised smoothness class. Apply Bernoulli to q and exponent N. The resulting affine lower bound is (D+1)/(2D), at least1/2. Thus even after N steps the selected small curvature leaves at least half of the initial distance. Squaring the nonnegative half-distance bound gives q^(2N)>=1/4. Multiply by the positive beta/(4D) in the exact objective identity. With the source upper rate for N>0, the resulting family obstructs any uniformly faster order at this prescribed step.
Braces mark parameters Lean can infer; square brackets request structures such as a measurable space or probability measure. Named hypotheses are mathematical premises, not facts established by this declaration. Section parameters are described in the mathematical hypotheses above; the module link retains their exact source context.
theorem quadratic_gap_lower_bound {β : ℝ} (hβ : 0 < β) (N : ℕ) :
let μ := β / (2 * ((N : ℝ) + 1))
let f : ℝ → ℝ := fun x => μ * x ^ 2 / 2
0 < μ ∧ ContDiff ℝ 2 f ∧ StrongConvexOn univ μ f ∧
(∀ x y, f y ≤ f x + inner ℝ (gradient f x) (y - x) + β / 2 * ‖y - x‖ ^ 2) ∧
IsMinOn f univ 0 ∧
let z := (fun x => x - (1 / β) * gradient f x)^[N] 1
f z - f 0 = β / (4 * ((N : ℝ) + 1)) *
(1 - 1 / (2 * ((N : ℝ) + 1))) ^ (2 * N) ∧
β / (16 * ((N : ℝ) + 1)) ≤ f z - f 0 := by
let D : ℝ := (N : ℝ) + 1
have hD : 0 < D := by dsimp [D]; positivity
have hD1 : 1 ≤ D := by dsimp [D]; exact le_add_of_nonneg_left (Nat.cast_nonneg N)
let μ := β / (2 * D)
have hμ : 0 < μ := div_pos hβ (by positivity)
have hμβ : μ ≤ β := by
apply (div_le_iff₀ (by positivity : 0 < 2 * D)).mpr
nlinarith
obtain ⟨ν, hν, _, hf, hc, hu, hm, hn⟩ :=
exists_quadratic_worst_case hμ (le_refl μ) (1 / β)
have hv : ν = μ := hν.elim id id
subst ν
let f : ℝ → ℝ := fun x => μ * x ^ 2 / 2
let q : ℝ := 1 - 1 / (2 * D)
have hq : 0 ≤ q := by
have : 1 / (2 * D) ≤ 1 := (div_le_one (by positivity)).mpr (by linarith)
dsimp [q]; linarith
have he : 1 - 1 / β * μ = q := by
dsimp [μ, q]; field_simp
have hn' : ‖(fun x => x - (1 / β) * gradient f x)^[N] 1‖ = q ^ N := by
simpa only [max_self, he, abs_of_nonneg hq] using (hn N).1
have hpow : 1 / 2 ≤ q ^ N := by
have hb := one_add_mul_sub_le_pow (by linarith : -1 ≤ q) N
have heq : 1 + (N : ℝ) * (q - 1) = (D + 1) / (2 * D) := by
dsimp [q, D]; field_simp; ring
rw [heq] at hb
have : (1 : ℝ) / 2 ≤ (D + 1) / (2 * D) := by
apply (le_div_iff₀ (by positivity)).mpr
linarith
exact this.trans hb
have huβ : ∀ x y, f y ≤ f x + inner ℝ (gradient f x) (y - x) + β / 2 * ‖y - x‖ ^ 2 := by
intro x y
have hh : f y ≤ f x + inner ℝ (gradient f x) (y - x) + μ / 2 * ‖y - x‖ ^ 2 := hu x y
nlinarith [mul_nonneg (sub_nonneg.mpr hμβ) (sq_nonneg ‖y - x‖)]
change 0 < μ ∧ _ ∧ _ ∧ _ ∧ _ ∧ _
refine ⟨hμ, hf, hc, huβ, hm, ?_⟩
dsimp only
have hvalue : f ((fun x => x - (1 / β) * gradient f x)^[N] 1) - f 0 =
β / (4 * D) * (q ^ N) ^ 2 := by
have hs : ((fun x => x - (1 / β) * gradient f x)^[N] 1) ^ 2 = (q ^ N) ^ 2 := by
simpa only [Real.norm_eq_abs, abs_mul_abs_self, sq] using congrArg (fun r : ℝ => r * r) hn'
change μ * _ ^ 2 / 2 - μ * 0 ^ 2 / 2 = _
rw [hs]
dsimp [μ]; field_simp; ring
rw [hvalue]
change β / (4 * D) * (q ^ N) ^ 2 = β / (4 * D) * q ^ (2 * N) ∧
β / (16 * D) ≤ β / (4 * D) * (q ^ N) ^ 2
constructor
· rw [Nat.mul_comm 2 N, pow_mul]
· have hs : (1 : ℝ) / 4 ≤ (q ^ N) ^ 2 := by nlinarith
have := mul_le_mul_of_nonneg_left hs (by positivity : 0 ≤ β / (4 * D))
convert this using 1 <;> first | rfl | (field_simp; ring)
end AutoSamplingTheory.TechnicalLemmas.Analysis.ConvexGradientGapSharpnessScope and omitted-condition boundaries
- The horizon N is fixed before choosing the objective f_N. Its positive curvature mu_N shrinks with N. This is a family witnessing a worst-case bound over the convex beta-smooth class; it is not a single fixed strongly convex objective with an asymptotic reciprocal tail.
- Beta is a valid upper curvature bound, not the tight smoothness constant of this scalar witness. The step is prescribed as1/beta, not tuned to its intrinsic curvature. Strong convexity with mu_N>0 implies ordinary convexity.
- The constant1/16 is an ASTIS-derived convenient lower constant, not an optimal constant or a number printed in the exercise. For N≥1, beta/[16(N+1)]≥beta/(32N); comparison with the source upper bound beta/(2N) shows order sharpness at initial distance1. The theorem states the finite N+1 bound.
- N=0 is included only as a nonsingular lower-bound identity: the actual initial gap is beta/4. It does not interpret the singular source1/N upper bound at zero.
- No variable/adaptive-step result, general first-order oracle lower bound, full Exercise3.3 comparison list, acceleration claim or companion-paper completion follows. The function, actual gradients, smoothness model and minimizer are certified, not supplied as hypotheses.
Sources, dependencies, and reuse
Source and reuse
Samplinglib parents called
Mathlib API called (external library)
- one_add_mul_sub_le_pow
- abs_mul_abs_self
- pow_mul
- mul_le_mul_of_nonneg_left
Mathematical sources
- Chewi Exercise3.3 and Theorem3.4 — Selected convex objective-gap order comparison; explicit witness and constant are ASTIS derivations.
- Same-witness upper/lower bounds and boundary tests — Composes the new witness with the existing weighted upper-rate theorem; zero and positive horizon cases.
Samplinglib prose is not a quotation or a source-equivalence certificate. Definitions and aliases are explained as constructions, not counted as new mathematical proofs.