Here is my suggestion for how to iterate on these comments: ARC reviewer comments/questions are in regular text. Responses are in bold followed by initials. I have moved all style/grammar comments to the end.

Comments from Nick:

* Discussion title: Review of TOP-11-028

Dear TOP-11-028 Authors and Analyzers,

Here are my comments on the TOP-11-028 version of the PAS.

I'll put the main comments first then some suggested changes to the text.

0) Nice Analysis. This certainly should proceed to a PAS and then a paper. I see no substantial flaws in what has been done.

1) Abstract. the limit is given as 0.39% here, but 0.4% in the Conclusions. I would probably choose to use 0.39% in both places.

I agree (SW)

2) Table 1: I think we quote far too many significant figures in many numbers here. For example, "161226.12 +/-1269.73" I would change to "161666+/-1270" similarly "425.66 +/- 20.63" change to "425+/-21" and many others.

We see this comment repeated by all of the reviewers (SW)

3) Table 2: I would put the percent sign in the table itself. Also in "0.375+/-0.13" the number of significant figures doesn't match. Change to "0.37+/-0.13%" Same for other numbers under "Tight selection".

4) general comment: somewhere (first time) when saying xxx+/-yyy+/-zzz we should say which is statistical and which systematic. Is it CMS style to say this everywhere?

5) Table 3: There are many of these, but this one jumps out. "0.65+/-0.14+/-0.97" This would seem to imply that negative numbers of events are OK. We should decide how best to deal with these cases and use that style everywhere.

6) Table 3 Also, is 0 events background in the tight selection really allowed at one standard deviation? This would seem to be in conflict with the quoted WW background of "0.38 +/- 0.10 +/- 0.06". I don't think adding the uncertainties in quadrature is appropriate here.

7) p.2 Somewhere we should say that all three leptons are required to come from the same vertex explicitly.

8) p.3 Section 4. In the second sentence we say "We also expect BR(t->Zq) to be small..." I would move this to Section 1 Introduction to right before the sentence "We present a search for t-> Zq...." This explains why we chose to search in this channel.

9) p.5 Section 7 - Results We should say explicitly how the limit is calculated for the loose selection, particularly since this is the result we end up quoting as our final result.

Luca

Comments on TOP-11-028, PAS v3

line 165: as commented by Nick, how do you deal with negative background events? It's important in the background determination: do you use a log-normal for the corresponding nuisance parameter in the CLs determination? In this case, you could quote an asymmetric uncertainty (if the uncertainty on the logarithm of the background yield is normal -> the yield has asymmetric uncertainties)

line 165: does the estimate of 0.001 background events has an uncertainty? Please, quite it. Or just say it is negligible.

line 167, 168: please quote uncertainties on background estimates

line 169: "data driven estimation" -> "estimate from data"

line 172: "backgrounds" -> "background"; "error" -> "uncertainty"

line 190: "Cls method" -> "modified frequentist approach (CLs method)"

line 193,194: "This fraction is ...": not clear, pleaser rephrase and define more clearly what you mean.

line 195: "The calculated limit ..." -> "The 95\% CL upper limits are shown..."

line 203: "flavor changing" -> "flavor-changing"

line 206: "of the existence" -> "of the existence of the decay"

line 208: "The branching fraction of ..." -> "A branching fraction B(t->Zq) larger than..."

Fig 1: add in the caption a description of the different contributions (data are points with the error bars, the solid white histogram is the signal assuming B(t->Zq)=1\%, etc.)

Tab 1: please, reduce the number of significant digits

Tab 1: specify what are the four different blocks with appropriate labels

Tab 1: can you explain the difference between 126 \pm 11 and 194 (6 sigma) in data, and consequently 68 \pm 8 and 85 (3.4 sigma)? Is it a hint of new physics smile ? Is this reflected somewhere in Fig. 2? It's not clear to me which block corresponds to which selection...

Tab 4: maybe the b-tagging uncertainty could be presented as "8 (0)", and then the total uncertainty as "23 (21) for tight (loose)".

Francisco

In the introduction, it looks a bit odd you mention BR of O(10^-14) and then search in the O(10^-3). Are there models that make predictions in this range or close?

The reason for doing the search to this sensitivity is because this is all the sensitivity we have at the moment with the data that's on tape. However, there are a few models which make predictions of order 10^-5 or so. (SW)

lines 67-66, is the 4th lepton veto done with loose or tight lepton ID?

Tight, but I agree this should be made explicit (SW)

line 72-77, do you really need this cut? Is there any cosmic left after all other cuts, including vertex? Does it kill any signal?

The opening angle cut to reject cosmic ray muons is quite standard, and has essentially no effect on the signal. (SW)

lines 99-100, it's not clear to me what you intend to say here

Table 1, I agree that there are too many significant digits. Then the agreement is not always that good. Since you say that you use data-driven methods to obtain background, can this table be improved? BTW , second line of mumue, "total" column, the error seems to be wrong +-123, looks rather +-12.3

Figure 1, related to above, the agreement is not perfect. a) shows some excess in the high njets (where only signal is expected) (c) shows some excess in the Mw region. Again, if you use data-driven methods, can this be improved?

Table 2, is it really simulation only? doesn't it include the usual scale factors for trigger, lepton ID, btagging, etc?

For me, it includes all relevant scale factors--trigger efficiency, lepton id efficiency, b tagging efficiency

Figure 2, not obvious what you mean by "basic selection"

Section 5, are the WZ and ZZ backgrounds taken purely from data? Can't they be estimated or at least cross-checked with data? you mention DY is data driven, but don't explain at all how. Please explain or refer to other analyses. The last sentence on b-jets is rather cryptic...

Systematics, I think that more detail (or a reference) is needed of each of the systematic errors

Line 193-194, again you mention "on simulated" events, while I think is more than purely simulation (or should be)

Line 201, I think this kind of statement has to come from the beginning. It is included in the harmonization I asked. From the intro say that you propose two methods and you take one that gives slightly better performance. Then I would only quote limits for the chosen one.

Browsing the references, I see that most refer to PAS of year 2010, aren't there newer ones?

Paolo

1) Line 47: One number is given for each of the ee and mumu double lepton trigger efficiencies (97% and 90%). However, for the selections where the 3rd lepton is of the same type of the first two, the trigger efficiency should be higher, shouldn't it?

Yes, and in my AN where the selection efficiency is broken out more completely, this is explicit. The effect is fairly small, of order 3% more efficiency (SW)

2) lines 57-58 and lines 60-61: the description of the requirement of muons and electrons to be associated with the primary vertex could be made once for both. Actually, this is already done later on lines 73-75, so the first 2 sentences should simply be removed.

3) Was any optimization of the final event selections performed? The cuts on MET, HTs and the windows around the top mass value would be good candidates to be optimized to see if the potential of the 2 selections could improve.

4) line 134: Naively I would not have expected that the correct p_z solution could be found 100% of the times by choosing the less energetic of the 2 solutions (or by any other method). Could you please point me to the study or plots that show this?

The statement of "nearly" 100% reflects the fact that this number is statistics-limited. I did a study on the generated signal MC, and checked the reconstructed solutions against the MC truth, the lower energy solution was the correct one for 100% of ~3k events. I can convert that into a statistical error on the 100%, but I can't think of a way to really make a meaningful plot to show this. (SW)

5) Also, in the AN-11-358 dated 2012/01/20, lines 252-253 state something else: "the two solutions for p_z are examined to determine which one (in combination with the lepton) gives the closest value to the W mass". Don't the 2 solutions give the exact W mass by definition? What am I missing?

Yes, this is an error that will be corrected in the next version of the AN in a few days. (SW)

6) Sections 4.1 and 4.2: I would suggest to swap the order of presenting the 2 sections. The description of the loose selection should go first and the tight after. Various things would become clearer and easier to describe I believe, or at least clearer to me. For example, since you introduce the request on one b-jet in Section 4.1, it was not clear to me (without saying it explicitely in Sec. 4.2) that this is not asked in the loose selection. I would swap the tight/loose order everywhere else too, Tables, figures etc.

7) Section 5: The Z+jets bkg, where a jet could fake a lepton (an electron more than a muon), is mentioned in the AN lines 221-222, but not in the PAS. Could a sentence be added?

8) Section 5: The multijet background is presumably very small with 3 leptons, however even the background level is very low in this analysis. Was the multijet bkg checked in any way?

9) Line 182: Table 3 is pointed to for the systematic on the bkg, but only stat. errors are reported on the table.

10) Lines 196 and 200: what are the 1-sigma boundaries of the expected limits?

Minor or style comments:

---

STYLE/EDITORIAL Nick:

Editorial Suggestions. 1) p. 1 "properties within the standard model (SM) and as a probe of new physics." -> "properties both as a test of the standard model (SM) and as a probe of new physics."

2) p.1 "large enhancement, hence a model-independent..." -> "large enhancement, and provides a model-indepedent..."

3) p.1 "for a cleaner measurement at the expense..." -> "for a measurement with less background at the expense..."

4) p.2 "which associates with the most transverse activities." -> "associated with the most transverse energy."

5) p.3 "the Particle Flow Technique" -> "a Particle Flow Technique" "The anti-k_T clustering..." -> An anti-k_T clustering..."

6) matter of taste but I would spell out Drell-Yan everywhere and not use "DY".

Luca: line 6: why not simply: "where q is a u or c quark"

line 14: "This allow us" -> "This allows us"

line 15: "the boson" -> "the Z boson"

Section 2: for a PAS the detector section can be omitted, leaving just "the CMS detector is described elsewhere [5]"

line 41: "Drell-Yan (DY)" -> "Drell--Yan (DY) events", with double-dash

line 42: "single top events" -> "single-top events"

lines 52-66: do we need such level of details for a PAS? Those paragraphs could be probably shortened, leaving references to the PASes.

line 76: "cosmic ray background" -> "cosmic-ray background"

line 86: "interaction" -> "interactions"; add comma before "after" and after "interactions"

line 89: please, use a different notation for MET: either the default E^{\mathrm{miss}}_{\mathrm{T}} , or:

\newcommand{\met}{\ensuremath{{E\!\!\!/}_{\mathrm{T}}}\xspace}

line 103: "the dibosons includes" -> "Dibosons include"

line 104: "the single top is dominated by" -> "Single-top production is dominated by"

line 105: "data and the simulation" -> "data and simulation"

line 107: "the simulated events" -> "simulated events"

line 107: "Figure 1 shows the comparison in data and simulation of the distributions of ...."

line 110: "but it also includes..." -> "where the requirement of two or more jets is also applied"

line 123: "heavy flavor quark" -> "heavy-flavor quark". Please anticipate here the b-tagging requirement and reference from line 139

line 124: "heavy quark not checked": jargon. Please use something like: "no requirement on the identification as a b-jet..."

line 124: "require a minimum value on the scalar sum of..." -> "require a minimum value of HTs"

line 126 Add "The" at the beginning of the sentence; "backgrounds" -> "background"

line 129+3: "calculated from" -> "calculated as"

line 131: "under..." -> "imposing the constraint that the invariant mass of the lepton and the neutrino is equal to the W mass"

line 139: The b-jet requirement and reference should be anticipated to line 123, where it's used for the first time.

line 150: remove "the" before M_Wb

line 151: remove "value". Remove "The" before "M_Zj"

line 154: "Figure 2 shows the comparison of the distributions of M_Zj and M_Wb in data and simulation"

line 156: "shown" -> "are shown"

line 160: "unlikely" -> "is unlikely"

line 165: for 0.23 \pm 0.54 \pm 0.04 specify stat. and syst. contributions

Paolo: 11) line 37-38: the sentence "These events are generated .... event generator" could be removed and the citations of the generator programs simply moved to the next sentences.

12) line 42: "They are then simulated ..." --> "The CMS detector response is simulated using ..."

13) line 54 and line 60: for the muon case "strip and pixel detector" is used while for the electron case "silicon tracker" is used. The same wording could be used for both.

14) line 65: "to maintaining" -> "to maintain"

15) line 81-2: "articles" -> "particles"

16) line 86: "after correcting for multiple proton-proton interaction" could be expanded a bit, e.g. "after correcting for additional underlying activity due to multiple proton-proton interactions" or similar (and placed between commas).