Documentation

SHSLib.Foundations.ClopperPearsonCoverage

Coverage of the Clopper–Pearson interval #

For n Bernoulli trials with unknown success probability p, the exact confidence interval built from the observed count c is

[p⁻(c), p⁺(c)], p⁻(c) = Q_{α/2}(c, n−c+1), p⁺(c) = Q_{1−α/2}(c+1, n−c),

with the conventions p⁻(0) = 0 and p⁺(n) = 1. This module defines those total endpoints (cpLo, cpHi) and proves the coverage guarantee

P(p⁻(C) ≤ p ≤ p⁺(C)) ≥ 1 − α, C ∼ Bin(n, p),

for every p ∈ [0,1] and every level 0 ≤ α ≤ 1.

The argument is the classical one. The counts at which the lower endpoint overshoots the truth form a set A; each c ∈ A satisfies P(Bin(n,p) ≥ c) < α/2, and A is contained in the upper tail above its own smallest element, whose probability is exactly one of those small numbers. The counts at which the upper endpoint undershoots are handled symmetrically with the largest element and a lower tail. No monotonicity of the endpoints is needed: containment in the tail above the minimum is automatic.

The mirror of the quantile/CDF adjunction #

theorem SHSLib.Trigamma.le_betaQuantile_iff {a b p x : ℝ} (ha : 0 < a) (hb : 0 < b) (hp0 : 0 ≤ p) (hp1 : p ≤ 1) (hx0 : 0 ≤ x) (hx1 : x ≤ 1) :
x ≤ betaQuantile a b p ↔ regBetaFn a b x ≤ p

The companion of betaQuantile_le_iff: for p, x ∈ [0,1], x ≤ Q(p) ↔ I_x(a,b) ≤ p.

The endpoints for an arbitrary observed count #

noncomputable def SHSLib.Trigamma.cpLo (n c : ℕ) (alpha : ℝ) :

The Clopper–Pearson lower endpoint for c successes in n trials, with the convention p⁻(0) = 0.

Equations
Instances For
    noncomputable def SHSLib.Trigamma.cpHi (n c : ℕ) (alpha : ℝ) :

    The Clopper–Pearson upper endpoint for c successes in n trials, with the convention p⁺(n) = 1.

    Equations
    Instances For
      @[simp]
      theorem SHSLib.Trigamma.cpLo_zero (n : ℕ) (alpha : ℝ) :
      cpLo n 0 alpha = 0
      @[simp]
      theorem SHSLib.Trigamma.cpHi_self (n : ℕ) (alpha : ℝ) :
      cpHi n n alpha = 1
      theorem SHSLib.Trigamma.cpLo_eq_clopperPearsonLower (j m : ℕ) (alpha : ℝ) :
      cpLo (j + m + 2) (j + 1) alpha = clopperPearsonLower j m alpha

      The lower endpoint agrees with clopperPearsonLower on the interior counts: c = j+1 successes in n = j+m+2 trials.

      theorem SHSLib.Trigamma.cpHi_eq_clopperPearsonUpper (j m : ℕ) (alpha : ℝ) :
      cpHi (j + m + 2) (j + 1) alpha = clopperPearsonUpper j m alpha

      The upper endpoint agrees with clopperPearsonUpper on the interior counts.

      theorem SHSLib.Trigamma.cpLo_le_iff {p alpha : ℝ} (hp0 : 0 ≤ p) (hp1 : p ≤ 1) (ha0 : 0 ≤ alpha) (ha1 : alpha ≤ 1) {n c : ℕ} (hc1 : 1 ≤ c) (hcn : c ≤ n) :
      cpLo n c alpha ≤ p ↔ alpha / 2 ≤ regBetaFn (↑c) (↑n - ↑c + 1) p

      Characterisation of the lower endpoint: for a positive count, p⁻(c) ≤ p exactly when the binomial upper tail at p is at least α/2.

      theorem SHSLib.Trigamma.le_cpHi_iff {p alpha : ℝ} (hp0 : 0 ≤ p) (hp1 : p ≤ 1) (ha0 : 0 ≤ alpha) (ha1 : alpha ≤ 1) {n c : ℕ} (hcn : c < n) :
      p ≤ cpHi n c alpha ↔ regBetaFn (↑c + 1) (↑n - ↑c) p ≤ 1 - alpha / 2

      Characterisation of the upper endpoint: for a count below n, p ≤ p⁺(c) exactly when the binomial upper tail above c at p is at most 1 − α/2.

      theorem SHSLib.Trigamma.binomial_measure_upperTail {p : ℝ} (hp0 : 0 ≤ p) (hp1 : p ≤ 1) {n c : ℕ} (hc1 : 1 ≤ c) (hcn : c ≤ n) :
      (PMF.binomial p.toNNReal ⋯ n).toMeasure (binomUpperTail (c - 1) n) = ENNReal.ofReal (regBetaFn (↑c) (↑n - ↑c + 1) p)

      The binomial mass of the upper tail {i ≥ c}, in closed form.

      theorem SHSLib.Trigamma.binomial_measure_lowerTail {p : ℝ} (hp0 : 0 ≤ p) (hp1 : p ≤ 1) {n c : ℕ} (hcn : c < n) :
      (PMF.binomial p.toNNReal ⋯ n).toMeasure (binomLowerTail c n) = 1 - ENNReal.ofReal (regBetaFn (↑c + 1) (↑n - ↑c) p)

      The binomial mass of the lower tail {i ≤ c}, in closed form.

      The coverage bound #

      theorem SHSLib.Trigamma.measure_lowerEndpoint_bad {p alpha : ℝ} (hp0 : 0 ≤ p) (hp1 : p ≤ 1) (ha0 : 0 ≤ alpha) (ha1 : alpha ≤ 1) (n : ℕ) :
      (PMF.binomial p.toNNReal ⋯ n).toMeasure {c : Fin (n + 1) | ¬cpLo n (↑c) alpha ≤ p} ≤ ENNReal.ofReal (alpha / 2)

      The counts at which the lower endpoint overshoots the true parameter carry probability at most α/2.

      theorem SHSLib.Trigamma.measure_upperEndpoint_bad {p alpha : ℝ} (hp0 : 0 ≤ p) (hp1 : p ≤ 1) (ha0 : 0 ≤ alpha) (ha1 : alpha ≤ 1) (n : ℕ) :
      (PMF.binomial p.toNNReal ⋯ n).toMeasure {c : Fin (n + 1) | ¬p ≤ cpHi n (↑c) alpha} ≤ ENNReal.ofReal (alpha / 2)

      The counts at which the upper endpoint undershoots the true parameter carry probability at most α/2.

      theorem SHSLib.Trigamma.clopperPearson_coverage {p alpha : ℝ} (hp0 : 0 ≤ p) (hp1 : p ≤ 1) (ha0 : 0 ≤ alpha) (ha1 : alpha ≤ 1) (n : ℕ) :
      ENNReal.ofReal (1 - alpha) ≤ (PMF.binomial p.toNNReal ⋯ n).toMeasure {c : Fin (n + 1) | cpLo n (↑c) alpha ≤ p ∧ p ≤ cpHi n (↑c) alpha}

      Coverage of the Clopper–Pearson interval. If the count C is binomial with n trials and success probability p, then the exact interval built from C contains p with probability at least 1 − α.