Quadratics witness the convex function-value convergence order
AutoSamplingTheory.TechnicalLemmas.Analysis.ConvexGradientGapSharpness.quadratic_gap_lower_bound · theorem · Teaching coverage
Statement
For every real beta>0 and natural horizon N, the explicit positive scalar quadratic below is C2, mu-strongly convex, admits the global beta quadratic upper model and is minimized at0. Its actual gradient-descent iterate from1 at step1/beta has the following exact gap and lower bound.
All objects and hypotheses
- Beta is real and strictly positive; N is natural, including0. The function and positive curvature are chosen from this N; each horizon may use a different function.
- The ambient space is R. The conclusion certifies ContDiff R2 f, StrongConvexOn univ mu f, the global beta upper model f(y)<=f(x)+<gradient f(x),y−x>+beta||y−x||²/2 and IsMinOn f univ0.
- T uses the true gradient, with constant positive step1/beta. Iteration starts at1, at distance1 from the certified minimizer0. T^0 is the identity.
Mathematical proof
1. Choose curvature after the horizon
For beta>0 and N natural set D=N+1 and mu=beta/(2D). Then D>=1 and0<mu<=beta. Reuse the compiled scalar-quadratic witness with both curvature endpoints equal to mu. Equality of the endpoints forces that witness to be exactly f(x)=mu x²/2. It supplies C2 regularity, mu strong convexity, minimizer0 and the genuine gradient trajectory.
Corresponding Lean step
GradientDescentSharpness.exists_quadratic_worst_case at equal endpoints; order and positive division.
2. Obtain the exact value along actual gradient iterates
At the prescribed step1/beta the scalar contraction is q=1−1/(2D), which is nonnegative. The existing norm identity gives norm(x_N)=q^N. Since the objective is mu times the squared norm divided by2 and f(0)=0, this is also an exact objective-gap formula. Enlarging the quadratic upper model from mu to beta verifies the advertised smoothness class.
Corresponding Lean step
Existing actual norm identity; abs_mul_abs_self and scalar norm; nonnegative quadratic remainder when mu<=beta.
3. Keep a fixed fraction of the initial distance
Apply Bernoulli to q and exponent N. The resulting affine lower bound is (D+1)/(2D), at least1/2. Thus even after N steps the selected small curvature leaves at least half of the initial distance.
Corresponding Lean step
Mathlib one_add_mul_sub_le_pow; positive denominator arithmetic.
4. Convert distance retention into a gap lower bound
Squaring the nonnegative half-distance bound gives q^(2N)>=1/4. Multiply by the positive beta/(4D) in the exact objective identity. With the source upper rate for N>0, the resulting family obstructs any uniformly faster order at this prescribed step.
Corresponding Lean step
pow_mul; square inequality and mul_le_mul_of_nonneg_left. The general upper bound is composed in the test, not assumed in this proof.
Lean statement · quadratic_gap_lower_bound
Positive beta and natural horizon determine one explicit scalar quadratic, all admissibility certificates, exact actual-iterate objective gap and beta/(16(N+1)) lower bound.
Braces mark parameters Lean can infer; square brackets request structures such as a measurable space or probability measure. Named hypotheses are mathematical premises, not facts established by this declaration. Section parameters are described in the mathematical hypotheses above; the module link retains their exact source context.
theorem quadratic_gap_lower_bound {β : ℝ} (hβ : 0 < β) (N : ℕ) :
let μ := β / (2 * ((N : ℝ) + 1))
let f : ℝ → ℝ := fun x => μ * x ^ 2 / 2
0 < μ ∧ ContDiff ℝ 2 f ∧ StrongConvexOn univ μ f ∧
(∀ x y, f y ≤ f x + inner ℝ (gradient f x) (y - x) + β / 2 * ‖y - x‖ ^ 2) ∧
IsMinOn f univ 0 ∧
let z := (fun x => x - (1 / β) * gradient f x)^[N] 1
f z - f 0 = β / (4 * ((N : ℝ) + 1)) *
(1 - 1 / (2 * ((N : ℝ) + 1))) ^ (2 * N) ∧
β / (16 * ((N : ℝ) + 1)) ≤ f z - f 0Lean proof · quadratic_gap_lower_bound
For beta>0 and N natural set D=N+1 and mu=beta/(2D). Then D>=1 and0<mu<=beta. Reuse the compiled scalar-quadratic witness with both curvature endpoints equal to mu. Equality of the endpoints forces that witness to be exactly f(x)=mu x²/2. It supplies C2 regularity, mu strong convexity, minimizer0 and the genuine gradient trajectory. At the prescribed step1/beta the scalar contraction is q=1−1/(2D), which is nonnegative. The existing norm identity gives norm(x_N)=q^N. Since the objective is mu times the squared norm divided by2 and f(0)=0, this is also an exact objective-gap formula. Enlarging the quadratic upper model from mu to beta verifies the advertised smoothness class. Apply Bernoulli to q and exponent N. The resulting affine lower bound is (D+1)/(2D), at least1/2. Thus even after N steps the selected small curvature leaves at least half of the initial distance. Squaring the nonnegative half-distance bound gives q^(2N)>=1/4. Multiply by the positive beta/(4D) in the exact objective identity. With the source upper rate for N>0, the resulting family obstructs any uniformly faster order at this prescribed step.
Braces mark parameters Lean can infer; square brackets request structures such as a measurable space or probability measure. Named hypotheses are mathematical premises, not facts established by this declaration. Section parameters are described in the mathematical hypotheses above; the module link retains their exact source context.
theorem quadratic_gap_lower_bound {β : ℝ} (hβ : 0 < β) (N : ℕ) :
let μ := β / (2 * ((N : ℝ) + 1))
let f : ℝ → ℝ := fun x => μ * x ^ 2 / 2
0 < μ ∧ ContDiff ℝ 2 f ∧ StrongConvexOn univ μ f ∧
(∀ x y, f y ≤ f x + inner ℝ (gradient f x) (y - x) + β / 2 * ‖y - x‖ ^ 2) ∧
IsMinOn f univ 0 ∧
let z := (fun x => x - (1 / β) * gradient f x)^[N] 1
f z - f 0 = β / (4 * ((N : ℝ) + 1)) *
(1 - 1 / (2 * ((N : ℝ) + 1))) ^ (2 * N) ∧
β / (16 * ((N : ℝ) + 1)) ≤ f z - f 0 := by
let D : ℝ := (N : ℝ) + 1
have hD : 0 < D := by dsimp [D]; positivity
have hD1 : 1 ≤ D := by dsimp [D]; exact le_add_of_nonneg_left (Nat.cast_nonneg N)
let μ := β / (2 * D)
have hμ : 0 < μ := div_pos hβ (by positivity)
have hμβ : μ ≤ β := by
apply (div_le_iff₀ (by positivity : 0 < 2 * D)).mpr
nlinarith
obtain ⟨ν, hν, _, hf, hc, hu, hm, hn⟩ :=
exists_quadratic_worst_case hμ (le_refl μ) (1 / β)
have hv : ν = μ := hν.elim id id
subst ν
let f : ℝ → ℝ := fun x => μ * x ^ 2 / 2
let q : ℝ := 1 - 1 / (2 * D)
have hq : 0 ≤ q := by
have : 1 / (2 * D) ≤ 1 := (div_le_one (by positivity)).mpr (by linarith)
dsimp [q]; linarith
have he : 1 - 1 / β * μ = q := by
dsimp [μ, q]; field_simp
have hn' : ‖(fun x => x - (1 / β) * gradient f x)^[N] 1‖ = q ^ N := by
simpa only [max_self, he, abs_of_nonneg hq] using (hn N).1
have hpow : 1 / 2 ≤ q ^ N := by
have hb := one_add_mul_sub_le_pow (by linarith : -1 ≤ q) N
have heq : 1 + (N : ℝ) * (q - 1) = (D + 1) / (2 * D) := by
dsimp [q, D]; field_simp; ring
rw [heq] at hb
have : (1 : ℝ) / 2 ≤ (D + 1) / (2 * D) := by
apply (le_div_iff₀ (by positivity)).mpr
linarith
exact this.trans hb
have huβ : ∀ x y, f y ≤ f x + inner ℝ (gradient f x) (y - x) + β / 2 * ‖y - x‖ ^ 2 := by
intro x y
have hh : f y ≤ f x + inner ℝ (gradient f x) (y - x) + μ / 2 * ‖y - x‖ ^ 2 := hu x y
nlinarith [mul_nonneg (sub_nonneg.mpr hμβ) (sq_nonneg ‖y - x‖)]
change 0 < μ ∧ _ ∧ _ ∧ _ ∧ _ ∧ _
refine ⟨hμ, hf, hc, huβ, hm, ?_⟩
dsimp only
have hvalue : f ((fun x => x - (1 / β) * gradient f x)^[N] 1) - f 0 =
β / (4 * D) * (q ^ N) ^ 2 := by
have hs : ((fun x => x - (1 / β) * gradient f x)^[N] 1) ^ 2 = (q ^ N) ^ 2 := by
simpa only [Real.norm_eq_abs, abs_mul_abs_self, sq] using congrArg (fun r : ℝ => r * r) hn'
change μ * _ ^ 2 / 2 - μ * 0 ^ 2 / 2 = _
rw [hs]
dsimp [μ]; field_simp; ring
rw [hvalue]
change β / (4 * D) * (q ^ N) ^ 2 = β / (4 * D) * q ^ (2 * N) ∧
β / (16 * D) ≤ β / (4 * D) * (q ^ N) ^ 2
constructor
· rw [Nat.mul_comm 2 N, pow_mul]
· have hs : (1 : ℝ) / 4 ≤ (q ^ N) ^ 2 := by nlinarith
have := mul_le_mul_of_nonneg_left hs (by positivity : 0 ≤ β / (4 * D))
convert this using 1 <;> first | rfl | (field_simp; ring)
end AutoSamplingTheory.TechnicalLemmas.Analysis.ConvexGradientGapSharpnessScope and omitted-condition boundaries
- The horizon N is fixed before choosing the objective f_N. Its positive curvature mu_N shrinks with N. This is a family witnessing a worst-case bound over the convex beta-smooth class; it is not a single fixed strongly convex objective with an asymptotic reciprocal tail.
- Beta is a valid upper curvature bound, not the tight smoothness constant of this scalar witness. The step is prescribed as1/beta, not tuned to its intrinsic curvature. Strong convexity with mu_N>0 implies ordinary convexity.
- The constant1/16 is an ASTIS-derived convenient lower constant, not an optimal constant or a number printed in the exercise. For N>=1, beta/[16(N+1)]>=beta/(32N); comparison with the source upper bound beta/(2N) shows order sharpness at initial distance1. The theorem states the finite N+1 bound.
- N=0 is included only as a nonsingular lower-bound identity: the actual initial gap is beta/4. It does not interpret the singular source1/N upper bound at zero.
- No variable/adaptive-step result, general first-order oracle lower bound, full Exercise3.3 comparison list, acceleration claim or companion-paper completion follows. The function, actual gradients, smoothness model and minimizer are certified, not supplied as hypotheses.
Source and reuse
ASTIS parents called
Mathlib API called (external library)
- one_add_mul_sub_le_pow
- abs_mul_abs_self
- pow_mul
- mul_le_mul_of_nonneg_left
Mathematical sources
- Chewi Exercise3.3 and Theorem3.4 — Selected convex objective-gap order comparison; explicit witness and constant are ASTIS derivations.
- Same-witness upper/lower bounds and boundary tests — Composes the new witness with the existing weighted upper-rate theorem; zero and positive horizon cases.
ASTIS prose is not a quotation or a source-equivalence certificate. Definitions and aliases are explained as constructions, not counted as new mathematical proofs.