TWiki> Main Web>TWikiUsers>KenjiHamano>KHStatTools (2024-01-11, KenjiHamano) EditAttachPDF

Basic ideas

Statistics Forum

Documentation

Statistical Methods

StatAnalysis release

Statiatical Tools Git Repo

Stat tools

Exotics workshop

Weighted events

Significance

Zero events 

For the case n=0, for any background estimate (but no error on it), the 1-alpha CL upper limit on the number of signal events s using the CLs statistic is 
s(upper) = -ln(alpha) = -ln(1-CL).
So, 
CL = 0.95 --> s(upper 95% CL) = -ln(0.05) = 2.9957 ~ 3  (this is the 3 people talk about).
CL = 0.6827 ("1 sigma") --> s(upper 68.27% CL) = -ln(0.3173) = 1.14791

If you only consider a Poisson distribution for n and seek the frequentist confidence interval (no discussion on signal or background), the result is given in the PDG article
https://pdg.lbl.gov/2020/reviews/rpp2020-rev-statistics.pdf
equations 40.76a and 40.76b. 
There are two possible definitions of intervals here: one-sided and two-sided (assumed symmetric in coverage).

For the case n=0, one could argue that one needs a one-sided upper limit (since n cannot be lower than 0).
In this case, on obtains the same result as above:
95%CL:
Set 1-alpha_up = 0.95 -->
root [0] std::cout << 0.5*TMath::ChisquareQuantile(0.95,2) << std::endl;
2.99573
68.27%CL
Set 1-alpha_up = 0.6827 -->
root [1] std::cout << 0.5*TMath::ChisquareQuantile(0.6827,2) << std::endl;
1.14791

The case of a two-sided interval is what I believe you use for plotting purposes already.
For 68.27%CL one then sets
alpha_lo = alpha_up = (1-0.6827)/2 = 0.15865
And, for n = 0 (which is not really a two-sided problem...):
root [5] std::cout << "mu_lo = " << 0.5*TMath::ChisquareQuantile(0.15865,0) << std::endl;
mu_lo = 0
root [6] std::cout << "mu_hi = " << 0.5*TMath::ChisquareQuantile(1-0.15865,2) << std::endl;
mu_hi = 1.84105

I note that CMS quotes 0 -0 + ~1.84, see fig 3 of
https://cds.cern.ch/record/2718976/files/HIG-19-007-pas.pdf

I am not sure why they show 1.84, and not 1.14791.

For the case n=1, one MUST use the two-sided interval, and one gets
root [7] std::cout << "mu_lo = " << 0.5*TMath::ChisquareQuantile(0.15865,2) << std::endl;
mu_lo = 0.172748
root [8] std::cout << "mu_hi = " << 0.5*TMath::ChisquareQuantile(1-0.15865,4) << std::endl;
mu_hi = 3.29957

which is indeed what CMS uses on the same plot -- and what I believe you are using.

Naively, I would use

n=0  --> 0 (-0, +1.14791)   (one-sided, same results as CLs method for 0 background, that gives 3 for 95%CL)

n=1 --> 1 (-0.827252, +2.29957)  (double-sided)

etc.

Yes, the confusion is in the definition of what a 1sigma error is for a one-sided case (n=0).
I believe one sigma, by definition, must include 68.27% of cases.

For n=1,
1 (-0.827252, +2.29957) 
the interval includes 68.27% of cases.

For n=0,
0 (-0, +1.14791)  
the interval includes 68.27% of cases
and
0 (-0, +1.84105)
the interval includes 84.135%  of cases...

Global significance (Look elsewhere)

Errors on Errors

Bootstrap Method

Pruning & Smoothing

Limit contour

Functional Decomposition

Gaussian Process

Statistics Forum meetings

RooStats

Functional Form / Bump Hunting

SWift

PMGSystematicsTool

pyhf

TRooFit (Will Buttinger)

HistFitter

ResonanceFinder

TRExFitter

FrequentistFramework

HZZWorkspace

WSMaker

-- KenjiHamano - 2018-02-14

Edit | Attach | Watch | Print version | History: r59 < r58 < r57 < r56 < r55 | Backlinks | Raw View | WYSIWYG | More topic actions
Topic revision: r59 - 2024-01-11 - KenjiHamano
 
    • Cern Search Icon Cern Search
    • TWiki Search Icon TWiki Search
    • Google Search Icon Google Search

    Main All webs login

This site is powered by the TWiki collaboration platform Powered by PerlCopyright &© 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
or Ideas, requests, problems regarding TWiki? use Discourse or Send feedback