Convex gradient flows have an explicit objective rate
AutoSamplingTheory.TechnicalLemmas.Analysis.GradientFlowValue.value_le · theorem · Teaching coverage
Statement
Let E be a complete real inner-product space, f:E→R be Frechet differentiable everywhere and globally alpha-convex, with alpha>=0 in the alpha/2 chord convention. Let z be a global minimizer and T>=0. Supply X:R→E continuous on [0,T], satisfying the actual right gradient ODE at every s in [0,T). Then at each strictly positive observation time t<=T, its objective gap has the following bound, with a separate zero-curvature case.
All objects and hypotheses
- Complete real inner-product space E; f is Frechet differentiable everywhere.
- Alpha>=0 and StrongConvexOn univ alpha f: f((1-lambda)x+lambda y)<=(1-lambda)f(x)+lambda f(y)-(alpha/2)lambda(1-lambda)norm(y-x) squared for all x,y and lambda in [0,1].
- A supplied z satisfies f(z)<=f(x) for every x:E; T>=0.
- X is continuous on [0,T]; for each s in [0,T), HasDerivWithinAt X (-gradient f(X(s))) (Ici s) s. The observation satisfies 0<t<=T.
Mathematical proof
1. Derive objective descent from the actual ODE
Represent the derivative of f by its gradient, compose with the actual right derivative of X, and obtain minus the squared gradient norm. On each subinterval [a,b], scalar Gronwall with zero coefficient and zero forcing proves f(X(b))<=f(X(a)). This uses continuity at both endpoints and only right derivatives before b.
Corresponding Lean step
DifferentiableAt.hasGradientAt, HasFDerivAt.comp_hasDerivWithinAt and le_gronwallBound_of_liminf_deriv_right_le with K=epsilon=0.
2. Obtain distance dissipation with the value gap
Fix an observation time t and put r(s)=norm(X(s)-z) squared, e(s)=f(X(s))-f(z). Differentiate r using the actual ODE. The existing first-order strong-convexity inequality at X(s) with comparator z gives r prime<=-alpha r-2e(s). Descent implies e(s)>=e(t) for s<=t, so the forcing can be bounded by the fixed constant -2e(t).
Corresponding Lean step
HasDerivWithinAt.norm_sq, StrongConvexFirstOrder.firstOrder_lower_bound_of_strongConvexOn, and the derived AntitoneOn objective.
3. Compare with a constant-forcing scalar equation
Apply existing scalar interval Gronwall to r on [0,t] with coefficient -alpha and forcing -2e(t). The scalar version permits negative forcing. Nonnegativity of r(t) gives the left side of each displayed bound. Alpha=0 is evaluated directly; no zero denominator is substituted into the other branch.
Corresponding Lean step
le_gronwallBound_of_liminf_deriv_right_le, gronwallBound_K0 and gronwallBound_of_K_ne_0.
4. Divide only by positive quantities
For t>0, the zero branch divides by 2t. In the positive branch multiply by alpha exp(alpha t), use exp(-alpha t)exp(alpha t)=1, then divide by 2(exp(alpha t)-1)>0. The supplied global minimum gives a nonnegative objective gap.
Corresponding Lean step
Real.exp_add, Real.one_lt_exp_iff, le_div_iff₀ and IsMinOn.
Lean statement · value_le
Positive-time objective gap bound with direct alpha=0 branch.
Braces mark parameters Lean can infer; square brackets request structures such as a measurable space or probability measure. Named hypotheses are mathematical premises, not facts established by this declaration. Section parameters are described in the mathematical hypotheses above; the module link retains their exact source context.
theorem value_le {f : E → ℝ} {X : ℝ → E} {z : E} {α T : ℝ}
(hα : 0 ≤ α) (_hT : 0 ≤ T) (hf : Differentiable ℝ f)
(hsc : StrongConvexOn univ α f) (hz : IsMinOn f univ z)
(hX : ContinuousOn X (Icc 0 T))
(hflow : ∀ t ∈ Ico 0 T, HasDerivWithinAt X (-gradient f (X t)) (Ici t) t) :
∀ t ∈ Ioc 0 T, 0 ≤ f (X t) - f z ∧
f (X t) - f z ≤ if α = 0 then ‖X 0 - z‖ ^ 2 / (2 * t)
else α / (2 * (Real.exp (α * t) - 1)) * ‖X 0 - z‖ ^ 2Lean proof · value_le
Represent the derivative of f by its gradient, compose with the actual right derivative of X, and obtain minus the squared gradient norm. On each subinterval [a,b], scalar Gronwall with zero coefficient and zero forcing proves f(X(b))<=f(X(a)). This uses continuity at both endpoints and only right derivatives before b. Fix an observation time t and put r(s)=norm(X(s)-z) squared, e(s)=f(X(s))-f(z). Differentiate r using the actual ODE. The existing first-order strong-convexity inequality at X(s) with comparator z gives r prime<=-alpha r-2e(s). Descent implies e(s)>=e(t) for s<=t, so the forcing can be bounded by the fixed constant -2e(t). Apply existing scalar interval Gronwall to r on [0,t] with coefficient -alpha and forcing -2e(t). The scalar version permits negative forcing. Nonnegativity of r(t) gives the left side of each displayed bound. Alpha=0 is evaluated directly; no zero denominator is substituted into the other branch. For t>0, the zero branch divides by 2t. In the positive branch multiply by alpha exp(alpha t), use exp(-alpha t)exp(alpha t)=1, then divide by 2(exp(alpha t)-1)>0. The supplied global minimum gives a nonnegative objective gap.
Braces mark parameters Lean can infer; square brackets request structures such as a measurable space or probability measure. Named hypotheses are mathematical premises, not facts established by this declaration. Section parameters are described in the mathematical hypotheses above; the module link retains their exact source context.
theorem value_le {f : E → ℝ} {X : ℝ → E} {z : E} {α T : ℝ}
(hα : 0 ≤ α) (_hT : 0 ≤ T) (hf : Differentiable ℝ f)
(hsc : StrongConvexOn univ α f) (hz : IsMinOn f univ z)
(hX : ContinuousOn X (Icc 0 T))
(hflow : ∀ t ∈ Ico 0 T, HasDerivWithinAt X (-gradient f (X t)) (Ici t) t) :
∀ t ∈ Ioc 0 T, 0 ≤ f (X t) - f z ∧
f (X t) - f z ≤ if α = 0 then ‖X 0 - z‖ ^ 2 / (2 * t)
else α / (2 * (Real.exp (α * t) - 1)) * ‖X 0 - z‖ ^ 2 := by
have hd (u : ℝ) (hu : u ∈ Ico 0 T) :
HasDerivWithinAt (fun s => f (X s)) (-‖gradient f (X u)‖ ^ 2) (Ici u) u := by
have hgrad : HasFDerivAt f (InnerProductSpace.toDual ℝ E (gradient f (X u))) (X u) :=
(hf (X u)).hasGradientAt
simpa only [Function.comp_def, InnerProductSpace.toDual_apply_apply,
inner_neg_right, real_inner_self_eq_norm_sq] using
hgrad.comp_hasDerivWithinAt u (hflow u hu)
have hmono : AntitoneOn (fun s => f (X s)) (Icc 0 T) := by
intro a ha b hb hab
have hc := (hf.continuous.comp_continuousOn hX).mono
(show Icc a b ⊆ Icc 0 T from fun u hu => ⟨ha.1.trans hu.1, hu.2.trans hb.2⟩)
have hg := le_gronwallBound_of_liminf_deriv_right_le
(f' := fun u => -‖gradient f (X u)‖ ^ 2) (δ := f (X a)) (K := 0) (ε := 0) hc
(fun u hu r hr => by
simpa [slope] using (hd u ⟨ha.1.trans hu.1, hu.2.trans_le hb.2⟩).liminf_right_slope_le hr)
le_rfl (fun u _ => by nlinarith [sq_nonneg ‖gradient f (X u)‖]) b ⟨hab, le_rfl⟩
simpa [gronwallBound_K0] using hg
have hdist (u : ℝ) (hu : u ∈ Ico 0 T) :
HasDerivWithinAt (fun s => ‖X s - z‖ ^ 2)
(-2 * inner ℝ (gradient f (X u)) (X u - z)) (Ici u) u := by
have h := ((hflow u hu).sub_const z).norm_sq
simp only [inner_neg_right, real_inner_comm] at h
convert h using 1; ring
intro t ht
refine ⟨sub_nonneg.mpr (hz (mem_univ _)), ?_⟩
have hbound := le_gronwallBound_of_liminf_deriv_right_le
(f := fun s => ‖X s - z‖ ^ 2)
(f' := fun u => -2 * inner ℝ (gradient f (X u)) (X u - z))
(δ := ‖X 0 - z‖ ^ 2) (K := -α) (ε := -2 * (f (X t) - f z))
(((hX.sub continuousOn_const).norm.pow 2).mono
(show Icc 0 t ⊆ Icc 0 T from fun u hu => ⟨hu.1, hu.2.trans ht.2⟩))
(fun u hu r hr => by
simpa [slope] using (hdist u ⟨hu.1, hu.2.trans_le ht.2⟩).liminf_right_slope_le hr)
le_rfl (fun u hu => by
have hs := StrongConvexFirstOrder.firstOrder_lower_bound_of_strongConvexOn
hsc (fun x _ => (hf x).hasGradientAt) (mem_univ (X u)) (mem_univ z)
rw [show z - X u = -(X u - z) by abel, inner_neg_right, norm_neg] at hs
have hm := hmono ⟨hu.1, hu.2.le.trans ht.2⟩ ⟨ht.1.le, ht.2⟩ hu.2.le
nlinarith) t ⟨ht.1.le, le_rfl⟩
have hn := (sq_nonneg ‖X t - z‖).trans hbound
by_cases ha : α = 0
· simp only [ha, neg_zero, gronwallBound_K0, sub_zero] at hn ⊢
apply (le_div_iff₀ (mul_pos (by norm_num) ht.1)).mpr
nlinarith
· rw [if_neg ha]
have hap : 0 < α := lt_of_le_of_ne hα (Ne.symm ha)
rw [gronwallBound_of_K_ne_0 (neg_ne_zero.mpr ha), sub_zero] at hn
have hmul := mul_nonneg hn hap.le
have heq : (‖X 0 - z‖ ^ 2 * Real.exp (-α * t) +
(-2 * (f (X t) - f z)) / -α * (Real.exp (-α * t) - 1)) * α =
α * ‖X 0 - z‖ ^ 2 * Real.exp (-α * t) +
2 * (f (X t) - f z) * (Real.exp (-α * t) - 1) := by
field_simp
rw [heq] at hmul
have he : Real.exp (-α * t) * Real.exp (α * t) = 1 := by
rw [← Real.exp_add, show -α * t + α * t = 0 by ring, Real.exp_zero]
have hp := mul_nonneg hmul (Real.exp_pos (α * t)).le
have hcancel : (α * ‖X 0 - z‖ ^ 2 * Real.exp (-α * t) +
2 * (f (X t) - f z) * (Real.exp (-α * t) - 1)) * Real.exp (α * t) =
α * ‖X 0 - z‖ ^ 2 - 2 * (f (X t) - f z) * (Real.exp (α * t) - 1) := by
calc
_ = α * ‖X 0 - z‖ ^ 2 * (Real.exp (-α * t) * Real.exp (α * t)) +
2 * (f (X t) - f z) * (Real.exp (-α * t) * Real.exp (α * t)) -
2 * (f (X t) - f z) * Real.exp (α * t) := by ring
_ = _ := by rw [he]; ring
rw [hcancel] at hp
rw [div_mul_eq_mul_div]
apply (le_div_iff₀ (mul_pos (by norm_num) (sub_pos.mpr (Real.one_lt_exp_iff.mpr
(mul_pos hap ht.1))))).mpr
nlinarith
end AutoSamplingTheory.TechnicalLemmas.Analysis.GradientFlowValueScope and omitted-condition boundaries
- The finite real-valued quotient is asserted only for 0<t<=T. The printed source says t>=0, although its denominator vanishes at t=0. The original source statement is retained and a separately reviewed positive-time clarification is exposed. Lean total division at zero is not the source limiting or infinite convention.
- Alpha>=0 and the alpha/2 chord normalization are preserved. Alpha=0 is proved directly with coefficient 1/(2t); alpha>0 uses alpha/(2(exp(alpha*t)-1)). No analytic alpha-to-zero limit theorem or sharpness claim is made.
- The source C2 Euclidean objective is generalized to an everywhere Frechet differentiable objective on a complete real Hilbert space. A global minimizer z and an actual continuous trajectory on [0,T] are supplied. Right gradient dynamics are assumed only on [0,T); no terminal or negative-time derivative, ODE existence, minimum existence, discrete method, stochastic flow or companion-paper result is asserted.
Source and reuse
ASTIS parents called
Mathlib API called (external library)
- HasFDerivAt.comp_hasDerivWithinAt
- HasDerivWithinAt.norm_sq
- HasDerivWithinAt.liminf_right_slope_le
- le_gronwallBound_of_liminf_deriv_right_le
- gronwallBound_K0
- gronwallBound_of_K_ne_0
Mathematical sources
- Chewi Theorem2.4 — Exact rate for positive observation times; printed t0 wording is singular and retained as a source qualification.
- Actual quadratic trajectory and endpoint witness — Nonstationary exp(-t) trajectory tests every curvature in [0,1], including both rate branches. A t0 test refutes interpreting the printed quotient through Lean total division.
ASTIS prose is not a quotation or a source-equivalence certificate. Definitions and aliases are explained as constructions, not counted as new mathematical proofs.