Measurement of double parton scattering in proton-proton collision events with four jets with low transverse momentum at 13 TeV

Type B:

1. I have a question about phase space region I. It is explained in the
text why this region has better sensitivity to DPS, and why it is not
used for sigma_eff extraction because of the trigger thresholds. I do
not see explained, however, how the effect of the trigger is taken into
account for all of the analyses of region I distributions. Isn't there a
concern that the trigger thresholds above the kinematic cuts can affect
the comparison of data and predictions?

I do believe that we address this in section 5. There we mention the general approach for region I and region II. We use different single jet triggers in succesion and have determined the efficiency curves of these triggers, which we use to alter the event weight if the trigger is not fully efficient. The text reads:

The online selection of events was based on four single-jet triggers each requiring the presence of at least one jet with a p T above 30, 50, 80, or 100 GeV, and within | η | < 4.7. As the triggers have been prescaled, they are used in disjoint p T ranges. Offline requirements are imposed to ensure that the triggers are fully efficient, except for the trigger with the lowest p T threshold. In this last case, a correction as a function of the jet p T is applied to the selected event, effectively altering its weight. The trigger efficiency has been determined by comparing the performance of the jet trigger with a minimum bias trigger serving as an unbiased reference.


2. In the discussion of sigma_eff results, the comparison with the CMS 7
TeV result is interesting, any mabye more information could be provided.
For example, the difference between the 7 TeV and the 14 TeV 2->2,3,4
results is compable to the difference between the 14 TeV 2->2 and
2->2,3,4 results. Which processes were considered in extracting
sigma_eff at 7 TeV?

We refer to the paper at hand throughout our text. The sigma_eff value at 7 TeV was obtained by inclusively fitting Pythia 8 + Tune 4C to DPS sensitive data, where only a 2->2 ME has been used. The differences in the values of sigma_eff can also lie in the used tunes which we do refer to in the text (L607-608), the text reads:

the CMS measurement at a center-of-mass energy of 7 TeV [49], which is more in line with the results obtained with some of the models based on older UE tunes.


3. On line 613, "it is noteworthy that the result from the same-sign WW
analysis shows agreement with the results of all samples, while the J/y
meson pair production/y measurement only shows agreement with the NLO
2->2 models." The second part of the statement is valid, but I don't
think we can conclude much from the WW result since the uncertainties
are large enough to cover all of the 4-jet results.

We have modified the text to:
It has been shown that \sigmaeff is expected to be process independent for inclusive final states \cite{Seymour:2013sya}, therefore, it is noteworthy that the result the \cPJgy meson pair production measurement shows only agreement with the NLO $2\to 2$ models. The extracted \sigmaeff of the same-sign WW measurement shows agreement with all models due to the size of the errors on the measurement, indicating that further measurements in this channel are desireable before any further conclusions can be made.

Type A:

Line 11: I think you could omit "and their kinematic correlations". This
can be thought of as part of "final states with multiple jets" and the
sentence reads much better without this.

Line 33: "extent in which" -> "extent two which"

Line 299: the absolute value signs can be replaced with parenthesis
(equivalent, and more conventional)

Table 4 captions says "in the phase space region region I and region
II", while Table 3 says "in the phase space region I and region II". I
think it is better without the extra "region". Or you could say "in
region I and region II of phase space."

Line 617: "A measurement of the inclusive production of four-jet
events..." This should probably be "A study of the inclusive production
of four-jet events..." or "Measurements of kinematic distributions in
inclusive four-jet events..."

All changed accordingly.

COMMENTS TYPE B

figure 2-5: to be honest i am a bit confused about these figures. they seem to show 1) comparison of various distributions between data and some MC models in the top panel + 2) a ratio of MC/data in two lower panels - so some models are present in the top panel and some not, but no explanation is given (in the text) why this is done and why these particular models were chosen to be shown in the top panel and others only in the ratio. i could only assume that this is because showing all MCs in the top panel would make it too crowded and you do not want to introduce additional plots to show all MCs compared to data (for example have one plots for P8 and a separate one for H++ and H7). however - why these particular MCs made it to the top panel? is it random? this all should be somehow explain. i think line 398 would be a good place to say something like "Figures 2-5 show a comparison of the data to various MCs as a function of pt, eta and the DPS-sensitive observables. Some models are shown in the direct comparisons of these variable (and here explain why these one if there is an explanation) and all of them are represented in ration plots MC/data" --> or something like this.

We have opted to show the models that either use the most recent and up to date tune, use a dedicated DPS tune or one of the KaTie models for the on- and off-shell production.
We added the lines:

... Figures 2–5 show a comparison of the data to various MC models as a function of p T , η, and the DPS-sensitive observables.Three Pythia8 models and one HERWIG 7 model are shown in direct comparison with the data. These models employ the most recent CP5 and CH3 tunes, the dedicated DPS tune or combine PYTHIA 8 with a dipole-antenna shower provided by V INCIA , while all of the models are represented in the ratio plots.

the same comment is valid for figures 6-9.

Same as above, we have altered the text to:

Figures 6–9 show a comparison of the data to various MC models as a function of p T , η, and the DPS-sensitive observables. Four models are shown in direct comparison with the data. These models include one of the two on- and off-shell K A T IE models, the M AD G RAPH 5 a MC @ NLO LO sample interfaced with the CP5 tune and the POWHEG NLO 2 → 2 sample, while all of the models are represented in the ratio plots.

figures 2-5: please make the legend larger/bold (also the pt cuts); please place just "data" (not CMS data) first in the legend (on top); please make the x-axis label larger; please make the axis ticks values larger; i don't like the dashed lines - would it work with the solid lines as well?

All suggested changes have been adapted in the layout of the figures.

figure 2: on the most top left panel the y-axis digit "10^2" is partially cut off.

The axis range has been extended.

lines 399-400: i do not understand the first statement in this sentence. figure 2 does show that for low pt there is a large discrepancy of the data and MC cross sections (and it's much better for larger pt) but i don't understand how the figure explains that fact - and you write that the figure shows that the large cross sections are due to the abundance of lot-pt jets. to me the figure does not show any explanation why there are large discrepancies for low pt and not high pt and these lines seems to suggest that. or maybe i am confused and misunderstood the sentence - please clarify.

We do not try to offer any explenation at this point to why the low-pT region is less well described compared to the high-pT region. We simply try to state that these models fail in the low-pT region, while they succesfully describe the data at high-pT. What we try to state is that the discrepancy between the integrated cross sections is due to this abundance in low pT jets.
In the text we have now clarified that we are talking about the integrated cross section and not the differential one, the text now reads:

The pT spectra in Fig. 2, obtained for region I, show that the much larger integrated cross section of the MC models compared to the data is due to an abundance of low-pT jets, while for pT> 100 GeV the models show agreement within 50% of the data and even within the total uncertainty in the case of HERWIG7.

figure 6-9: please make the legend larger/bold (also the pt cuts); please place just "data" (not CMS data) first in the legend (on top); please make the axis label larger; please make the axis ticks values larger; i don't like the dashed lines - would it work with the solid lines as well?

All suggested changed have been adapted in the layout of the figures, except for the removal of the dashed lines. If we were to remove the dashed lines, the plots and ratio plots would get too crowded and less clear due to the overlap of the models in certain bins. We would opt to keeo pusing the dashed lines for clarity's sake.

figure 6: on the most top left panel the y-axis digit "10^2" is partially cut off.

The axis range has been extended.

figures 10-12: please make the legend larger/bold; please place just "data" (not CMS data) first in the legend (on top); please make the x-axis label larger; please make the axis ticks values larger.

All changes to the layout have been adapted.

figure 15: please make the legend for the predictions larger and bold.

Changed accordingly.

Type A:

line 23: The "m" is usually associated with a mass. I suggest you select a different name for the factor, maybe "p".

We follow the litterature and other papers with which we would like to stay conform.

lines 58,59: Change "whereafter it is fitted along with" to "and then fitted to".

If we change it like that, the scentence implies that the DPS template is fitted to the SPS template. We altered the scentence to:
... a pure DPS signal sample is reconstructed from data by randomly mixing two inclusive single-jet events into one inclusive DPS four-jet event, and is then fitted together with several SPS-only background MC models to the distributions obtained from inclusive four-jet data.

Title Change "Study" to "Measurement". This is more specific.

Abstract

line 1: Change "study" to "measurement".

line 6: Delete "It is found that".

Text

line 2: Change "the strong interaction" to "strong interactions".

line 4: Change "in" to "into".

line 5: Delete "on the one hand".

line 7: Delete "on the other" and change "latter" to "PDFs".

line 10: Delete "good".

line 12: Change "further studies" to "additional theoretical treatments".

line 16: Change "As such" to "Thus".

line 20: Add a comma after "(DPS)" and add "they" before "represent".

line 26: Change "while" to "whereas".

line 28: Change "while" to "whereas".

line 31: Change the colon to a semicolon.

line 32: Change "protons" to "proton".

line 33: Change "in" to "to".

line 36: Change "Several" to "Various".

line 38: Add "Refs." before "[5-9]".

line 41: Break into two sentences; end with "[20,21]." and start with "The ATLAS and CMS . . ".

Figure 1 caption line 2: Change "while" to "whereas".

Figure 1 caption line 3: Change "As" to "Since".

Figure 1 caption line 5: Change "to" to "with" after "compare".

lines 43,44: Move "at TeV [26]" after "performed".

line 47: Delete the comma after "2016".

line 48: Add a hyphen to "data-taking".

line 49: Change "(henceforward referred to a "pileup")" to "(pileup)". This is simply a definition of 'pileup".

lines 49-51: Write "This mitigates the confusion caused by pileup, but also enables us to include jets with low pt that would otherwise be assigned as pileup. As a result of the low pt jets, a custom calibration of the jet energy scale is required.". I think this is what you are saying.

lines 53-55: " . . .are studied by comparing the distributions of DPS-sensitive observables predicted by various Monte Carlo (MC) event generators with the distributions measured in data.".

line 55: Delete "the" before "differences".

line 63: Change "to" to "with" before "data".

line 64: Change "while" to "and".

line 66: Delete "and their size".

line 75: Delete "as compared to SPS".

line 79: Move "therefore" to the beginning of the sentence.

line 84: Change "As" to "Since" and change "prone" to "likely".

line 90: Add "Delta S" before "distribution".

lines 92,93: Write "Thus we anticipate that DPS events tend toward lower values of Delta S.".

line 125: Change "aims to reconstruct and identify" to "reconstructs and identifies".

lines 129-131: Write "particles in the jet is typically within 5 to 10% of the true momentum over the whole pt spectrum and detector acceptance, based on simulation.".

line 137: Move "typically" before "amounts".

line 145: Change "while" to "whereas".

lines 163,164: Change "activated. This option" to "activated, which".

line 178: Change "elements, allowing for the production of "elements to produce".

lines 184,185" Delete "from now on".

line 189: Change "while" to "whereas".

line 204: Change "has been" to "was".

line 206: Change "while" to "and".

line 209: Delete the hyphen from "nonzero" and add a comma after "kT".

line 210: Change "different" to "various".

line 211: change "to" to "with" after "compared".

line 212: Change "requirement" to "requirements" and "on" to "for".

line 213: Change "while" to "and".

line 214: Change "As" to "Since".

line 215: Change "on" to "for".

line 226: Change "allow the generation of" to "produce".

line 229: Change "as" to "because".

line 243: Change "as" to "Since".

line 253: Change "As" to "Because".

line 256: Change :has been" to "was".

line 259: Delete "that".

line 267: Change "As" to "Since".

line 268: Delete "in order".

line 273: Delete "considered".

lie 291+2: Change the colon to "because".

line 302: Change "has been" to "was" and delete "equal to".

line 303: Delete "found to be".

lines 305,306: Change "has been found eta be" to "is".

line 309: Delete "in order".

line 312: Add a comma after "analysis".

line 334: Delete "found to be".

line 336: Delete "in order".

line 334" Delete "in order".

line 348: Change "to" to "with" after "compared".

line 351: Delete "in order".

lines 368,369: Delete "food to be".

line 372: Change "As" to "Because".

line 373: Change "taken into account" to "included".

line 277: Change "reweighing" to "reweighting" and delete "in order".

line 379: Change "while" to "whereas".

Table 2 caption line 4: Add a colon after "error)" and delete "while".

Table 2 caption line 5: Change "on" to "in" after "uncertainty" and change "due to" to "because of".

Table 2 caption line 6: Add "events," after "detector-level (note the comma after "events").

line 386: Change "compare" to "compares" and change "to" to "with" after "data".

line 388+1: Change "have been" to "are".

line 388+2: Delete "found to be".

line 388+3 (Eq. 3): Add a comma after the equation.

line 388+4 (Eq. 4): Add a period after the equation.

line 394: Change "to" to "with" after "compared".

line 400: Change "is due to" to "comes from" and change "while" to "whereas".

lines 401,402: Add a colon after "data" and write " . . . data; for HERWIG 7 it is even within the total uncertainty.".

lines 402,403: Write "Figure 3, the pseudorapidity spectra, shows that a large part of the excess . . . ".

lines 405: Delete "in order".

line 408: Add a comma after "distribution".

line 410: Change "aims to reduce" to "reduces".

line 412: Change "single last bin, as" to "the highest bin, since".

Figure 2 caption line 4: Change "while" to "and".

Figure 3 caption line 4: Change "while" to "and".

Figure 4 caption line 4: Change "while" to "and".

Figure 4 caption line 5: Change "on" to "in" after "uncertainty".

Figure 5 caption line 4: Change "while" to "and".

line 415: Change "have been obtained" to "are observed".

line 416: Delete "the" after "predictions of" and change "observable from" to "for".

line 416-418: Write "Deviations of 10-20% occur between a model . . . . and the data.".

line 419,420: Write " . . . models differs significantly from data, and the MC-to-data ratio . . . ".

line 421: Change "due to" to "consistent with".

line 424: Change "while" to "although".

line 425: Add a hyphen to "angular-ordered".

line 426: Change "while" to "but" and change "for" to "by".

line 427: Change "lies at the base of" to "algorithm is responsible for".

line 429: Change "disentangle" to "untangle".

line 431: Add a comma after "data".

line 433: Delete "in order".

line 436: Change "to" to "with" after "compared" and change "samples, obtained" to "samples that are obtained".

line 444: Add "predicted" before "cross".

line 445: Change "drastically drop" to "sharply decrease".

line 450: Write "Figure 6 compares the pt spectra of the various models with the data.".

lines 452,453: Write "This may by explained because most jets originate from the 2 -> 4 matrix element . . . ".

line 458: Delete the comma after "PYTHIA88+VINCIA".

line 459: Change "lowering of the" to "lower" and change "similar description as" to "description similar to

lines 460,461: Write " . . . sample; the higher-order matrix element, including virtual corrections, contributes to a lower cross section.".

lines 461-464: Write "A comparison of the multi jet data samples . . . . previous section demonstrates that NLO . . . ".

line 465: Change "well" to "consistently".

line 466: Add a colon after models" and delete "i.e.".

line 467: Change "however" to "although".

line 473: Delete "From" and change "one may conclude" to "demonstrate".

line 477: Delete "indeed".

line 483: Add a hyphen to "angular-ordered".

Figure 6 caption line 4: Change "while" to "and".

Figure 7 caption line 4: Change "while" to "and".

Figure 8 caption line 4: Change "while" to "and".

Figure 9 caption line 4: Change "while" to "and".

line 490: Change "to" to "with" after "compared".

line 494: Change "to" to "with" after "compared".

line 495: Delete "Note that".

line 497: Change "as" to "because" and change "have been" to "are" and change "account for" to 'include".

line 499: Change "Comparing" to "A comparison of".

line 500: Change "one observes an" to "shows the".

line 504: Change "In comparison to" to "A comparison with" and change "one can conlude" to "shows".

line 505: Move "mainly" after "contribute".

line 508: Change "of at most" to "typically".

line 512: Change "as" to "since".

line 518: Add a hyphen to "SPS-only".

Figure 10 caption line 4: Change "while" to "and".

Figure 11 caption line 4: Change "while" to "and".

line 519: Delete "in order".

line 523: Delete "as".

line 524: Move "a" after "too large" and change "while" to "whereas".

line 529: Move "Therefore" to the beginning of the sentence.

line 533: Change "compared to" to "than" and change "while" to "whereas".

line 534: Delete "as".

line 547: Change "due to" to "because of".

line 557: Change "as" to "since".

line 558: Change "can be found" to "is presented" and add a comma after "14".

line 564: Change "was" to "is" and change "of" to "for".

Figure 12 caption line 4: Change "while" to "and".

Figure 13 caption line 3: Change "while" to "and".

line 565: Change "as" to "because" and delete hyphen in "nonphysical".

line 566: Change "while" to "and".

line 567: Change "The distinction" to "A distinction".

line 572: Change "while" to "whereas".

line 578: Change "as" to "since".

line 580: Delete "It is observed that" and change "are" too "is".

line 583: Change "while" to "whereas".

line 584: Change "is found" to "exists".

Figure 14 caption line 4: Change "As" to "Since".

Figure 14 caption line 5: Change "of" to "in" after "uncertainty".

line 589: Delete "very".

line 591: Change "to" to "with" after "compared".

line 593: Change "allowed" to "included".

line 601: Delete "and".

line 606: Change "found" to "reported" and change "while" to "whereas".

line 614: Move "only" after "shows".

line 620: Change "have been" to "are".

line 623: Change "can be" to "is".

line 627: Change "is found to have" to "has".

line 640: Delete "found to be".

line 649: Delete "in order".

Ref. 1: Delete "CMS-SMP-13-002, CERN-PH-EP-2014-068".

Ref. 14: Add "Submitted to JHEP".

Ref. 17: Change "tevatron" to "Tevatron".

Ref. 23: Add year = "2013".

Ref. 57: The reference is "EPJC 81(2021)312".

Ref. 76: Add "Accepted by EPJC".

Ref. 77: Add "reportNumber = "MAN/HEP/2013/20". I assume this is a Manchester preprint.

All changed accordingly.


Type B

lines 227,228: The size of DPS relative to SPS cross section should be a ratio, not a number with units (mb). Please clarify what 21.3 mb really represents.

We added some more details, the text now reads:
In \KATIE, \sigmaeff is a parameter that directly determines the size of the DPS contribution relative to the SPS cross section.
From eq.~\eqref{eq:DPSpocketformula} it is observed that \sigmaeff correlates the independent Processes A and B: the probability of the occurence of Process B whenever Process A is observed will decrease (the DPS cross section becomes smaller) whenever \sigmaeff increases, where the cross sections $\sigma_\mathrm{A}$ and $\sigma_\mathrm{B}$ of the independent Processes A and B are fixed values.

lines 233-247: I find the description of these samples is confusing.

For lines 238-240 we altered the text to:
An on-shell \KATIE sample is generated with an explicit DPS contribution that is obtained by overlaying two $2\to 2$ matrix elements, with the exact same generation parameters as the on-shell \KATIE LO sample from the multijet models.

lines 234: What DPS contribution is added?

We tried to clarify that the pure DPS sample that is mentioned in line 226-227 is added here. The text now reads:
A \PYTHIA{}8.240 sample is generated with the {CP5} tune and the {NNPDF2.3\_NNLO} PDFs. It is the aforementioned \PYTHIA~8 sample to which a pure DPS sample obtained by overlaying two $2\to 2$ matrix elements, is added.

line 236: Where is it mentioned?

We added a reference to section 4.1 where Pythia8 + CDPSTP8S1 -4j is discussed, the text now reads:
The second \PYTHIA{}8 sample that includes an explicit DPS contibution, is the one already mentioned in Sec.~\ref{subsec:P8H}, as the {CDPSTP8S1-4j} tune has been fitted to DPS-sensitive observables.

lines 404,405: The figures should indicate which which region is presented.

The figures do not mention the regions explicitly but rather show the pT cuts that have been used, therefore, they inderictly indicate which region is used. For region I the figures show pT1 ,pT2 ,pT3 ,pT4 ≥ 35,30,25,20 GeV, while for region II de figures show pT1 > 50 GeV, pT2,3,4 > 30 GeV.

line 424: What does "more correlated" mean? I think you are concluding they are less correlated, but you write what is observed. Then add a sentence what that implies.

We added an extra scentence as requested:
The models implementing a \PT-ordered parton shower describe the distribution in \Dphimin well, although yielding a distribution in \phiij that is more decorrelated than observed in data. The slope of the \Dphimin distribution obtained from the models with a \PT-ordered shower algorithm going to zero, overshoots the slope of the distribution obtained from data.

line 425: Same point. How can we tell the"correlation is too strong"? What do we see.

We added an additional scentence, the text now reads:
For models that use an angular-ordered or dipole-antenna parton shower, the \Dphimin correlation is too strong. The slope of the distributions \Dphimin distributions obtained from the models with an angular-ordered shower overshoot the distribution obtained from data when going to $\pi$, whereas the shape of the data is described more accurately by \phiij.

Type B comments (clarity, contents)

General:
It is strange to put the detector description between the observables and the MC for them.

To us this seemed as the most natural order. If another order it to be preferred please let us know.

"0.042 pb-1 " => "42 nb-1" ? Heavy-ionists do use nb-1, it is also ridiculous to use pb in the
luminosity although the measured cross sections are in μb (Eqs. 6-9) and mb (L562).

We have adapted 42 nb-1 now.

Main text
L3: "of jets" => "of hadron jets"

Changed accordingly.

L6: "probability to find a given parton" => "probability to find a parton with given properties" (it is
100% to find a quark or gluon there)

Changed accordingly.

Fig. 1: What is the meaning of the three different circle sizes? It would be nicer to have them all
equal.

The figure has been updated, all cirkels have the same size now.

L74: "The two softest jets are more likely to be in a back-to-back configuration" Why should the
soft gluons radiated by two scattered quarks be back-to-back? This has to be explained as it one of
the basic assumptions, turning up in all observables.

They are only more likely to be in a back to back configuration if they are produced in an independent scattering from the two hardest partons (DPS event) due to the conservation of momentum.
We have changed the text to:

The two softest jets are more likely to be in a back-to-back configuration when produced by DPS as compared to SPS since there is an increased probability that the two softest jets are produced in an independent scattering fron the two hardest jets and since the momentum should be conserved in the transverse plane. The increased probability leads to an enhanced DPS contribution around $\DphiS = \pi$.

L125: "reconstruction" => "reconstruction ({\em particle flow})"

Changed accordingly.

L154: In principle the sentence "These generated events..." should be in L150 as it should be related
to all simulations, not only to Pythia8 with CUETP8M1 and HERWIG ++ with CUETHS1. What
use is a simulated event sample with no detector effects? Of course, if you use samples on generator
level, that should be pointed out here.

We opted to state that all event samples are on generator level and that only two are simulated on detector level as well since they are used to correct for the detector effects.

L 150 has become:
These tunes are obtained from fitting predictions to data. Off all the generated samples, two have been passed through the detector simulation program \GEANTfour \cite{AGOSTINELLI2003250}. These two samples will be used to correct the data for detector effects by means of an unfolding procedure.

L 154 reads :
The simulation of the detector effects has been applied to these generated events since they will be used to correct the date through an unfolding procedure.

L238: "K AT IE" => "KaTie [66]" and please correct the name in the citation.

We use the same \KATIE command throughout the paper for all instances where the MC generator is mentioned.
The name in the citation has been altered to the same command now as well.

L251: "four single-jet triggers" is a bit confusing as one could think of the searched four jets in
different events. It could help if "selection of events" => "selection of multi-jet events".

Changed accordingly.

L288: It would be much simpler to say here " Whenever at least four jets are found in the combined
final state originating from the same process, the combined event is labeled as a SPS process,
otherwise the event is labeled as a DPS process."
In the paragraph after L291 the formulae are probably cross sections, so "processes A and B"
=> "the cross sections of Processes A and B"

Changed accordingly.

L408: "dominated by effects of the jet cone size" => "determined by the jet cone size"

Changed accordingly.

Figs. 2-14: Please align the vertical labels (e.g. via using phantom indices)
Captions: "while" => "whereas"

The vertical labels have been aligned.

"while" has been replaced with "and"

L416: "Deviations between ... data of 10-20% were found in" => "Deviations of 10-20% were
found between ... data in"

Changed accordingly.

L450: "The on-shell KATIE predictions agree with the data in the first bin of each of the pT spectra,
but are above the data at higher pT. " Could this agreement be due to underflow?

The on-shell KaTie predictions are similar to madgraph5 predictions that employ only a 2->4 matrix element. The latter sample showed the same behavior over the pT spectra, therefore w believe that it is not related to any underflow problems.

Table 4: Please align the cross section values in the last two columns. Add "ME stands for matrix
element" to the caption.

Changed accordingly and in table 3 and 5 as well.

L545: "models" => "algorithms" or "generators"

Changed accordingly.

LL603: "lower bound on σ_eff = 8.3 mb": a lower bound needs "σ_eff > 8.3 mb", and that is what
the cited UA2 paper says, please correct it.

Changed accordingly.

Fig. 15: Balance the character sizes in the legends

Changed accordingly.

Type A comments: language, style
General: We should use capitals in expressions like Region I and Process A (just like Ref. 28,
Figure 3, Table 3, etc.). This is a frequent error in the text. Also there is a surplus space within I I in
Region II, please delete it everywhere. Table is not nice to abbreviate as Tab.

All changed accordingly.

Abstract:
Do we really need the hyphens in "first-, second-, third-, and fourth-leading jet"? It is the first, etc.
among the leading jets.

In previous iteration it was asked of us to do so, we would opt to leave it like that for clarity's sake.

Main text
L14: "in a SPS" => "in SPS"
L18: "parton shower" => "parton-shower"
L32: "protons momentum" => "proton momentum" or "proton's momentum"
LL41, 64, 145, 190, 213: "while" => "whereas"
L106: " in two endcap regions" => " in both endcap regions" or " in the two endcap regions"
LL131, 134, 136: "Jet energy" => "Jet-energy"
L132: "particle level" => "particle-level"
L145: "while an angular ordered" => "whereas an angular-ordered"
L191: "matrix element" => "matrix-element"
L193: "jet rate" => "jet-rate"
L215: "hadron level jets" => "hadron jets" or "hadron-level jets"
L257: "minimum bias" => "minimum-bias"
L262: "region I" => "Region I"
L266: "region I I" => "Region II" (capital and space within the Roman II.)
L287: "type A" => "Type A"
L289: "process A" => "Process A"
L290: "process B" => "Process B"
In the paragraph after L291: "region I" => "Region I", "region I I" => "Region II",
"processes A and B" => "Processes A and B"
L301: "region I I" => "Region II"
LL320-322: Capitalize "Process(es)"
L330: "Tab." => "Tables"
L331: "low pileup, low jet pT" => "low-pileup, low jet-pT"
L353: "least squares" => "least-squares"
L372: "no JER or model" => "neither JER nor model"
L377: "reweighing in function" => "reweighting as a function"
L384: "region I" => "Region I", "region I I" => "Region II"
L390: "as is" => "as it is" or "as"
LL399, 405, 446, 448, 501: "region I" => "Region I"
LL404, 449, 502: "region I I" => "Region II"
Tables 3, 4, 5: "region I" => "Region I", "region I I" => "Region II"
L425: " angular ordered" => " angular-ordered"
L426: "while" => "whereas"
L483: "Angular ordered" => "Angular-ordered"
L499: "Tab." => "Table"
L503: "jets" => "jets,"
L518: "SPS only" => "SPS-only"
L524: "higher order" => "higher-order"
LL588, 627, 633, 640: "parton shower" => "parton-shower"
L628: "angular ordered" => "angular-ordered"
L644: "While" => "However,"
Fig. 15: "generatred" => "generated"
"(Hard MPI rem.)" ?? You could delete it...
References:
Please remove the (partial) capitalization from the titles of Refs. 44, 61, and 69.
In Ref. 52 "Geant4" => "GEANT4"

All changed accrodingly.

General comments
- My biggest concern is the following.
In section 4 we give three categories of models in the three subsections. where subsection
4.3 contains "SPS+DPS" samples, and pointing out that PYTHIA and KATIE allow
for two 2->2 events.
Ok, but all MC generators we include in the hadronization step use PYTHIA or
HERWIG (or cascade). The tunes used in these models, and very explicitly in PYTHIA, include
MPIs, ie we include in fact for all of these generators MPIs (and thus DPS). Or is the
DPS defined to be a different/additional process than what is covered by MPI effects?
If so, I missed that...

For example if you combine MadGraph with PYTHIA CP5 tune you always have MPIs in this
combined result. The inclusion of MPI is controlled by (the input of) Pythia via this
parameter:flag PartonLevel:MPI (default = on)
i. e. this is the Master switch for multiparton interactions; on/off = true/false
IFF we did switch that off in these generator configurations, and thus the MPIs, then of
course we should write that explicitly in eg section 4.2 or 4.1... But I don't think
we switched MPIs off.

What it means is that combining the DPS templates from data with eg MadGraph or
PYTHIA MCs is not a combination of DPS with pure SPS, as the "SPS" is contaminated
by MPIs through these tunes used. I think that is diluting the message from Fig14.
as these intrinsic MPI effects in the "SPS" MCs have an effect on all these
sigma_eff extractions. We do not discuss that.

So, my major comment, assuming that you did as I described above is correct, is not
explained well at all into paper, and should be done better.
Below I will come back in the details to a number of instances were we say something
in the paper on the DPS extraction. Of course if I missed something important in my
discussion above then these comments further down may loose their importance.

Yes you are correct. We assumed wrongfully that the MPI contribution from the UE tunes was small. What we found in the paper v17 is the amount of DPS necessary on top of the tune. In order to make qualitative statements about the total amount of DPS, we have performed the same extraction of the DPS cross section, but with a criterion in place that should leave out events with hard MPI jets originating from the UE description. Our criterion omits events from the sample if a parton originating from an additional interaction (status code 33) has a transverse momentum exceeding 20 GeV. This way we remove all events that could bias out results. We have updated the section on the extraction and the results in section 8.4 accordingly.

- Just two week ago we released another paper on DPS studies, which is in CWR, but we make no
contact with that analysis. Are these analyses fully disjunct and the other recent one
has no bearing at all on the discussion in this paper?

Yes you are correct. We assumed wrongfully that the MPI contribution
from the UE tunes was small. What we found in the paper v17 is the
amount of DPS necessary on top of the tune. In order to make qualitative
statements about the total amount of DPS, we have performed the same
extraction of the DPS cross section, but with a criterion in place that
should leave out events with hard MPI jets originating from the UE
description. Our criterion omits events from the sample if a parton
originating from an additional interaction (status code 33) has a
transverse momentum exceeding 20 GeV. This way we remove all events that
could bias out results. We have updated the section on the extraction
and the results in section 8.4 accordingly."

While I added these lines starting at line 419 in the paper:

"Similar results have been obtained for the predictions of the \DptS
observable from the DPS tune in \cite{CMS:2021wfx}. Deviations between a
model employing a similar DPS tune (CDPSTP8S1 -WJ) and the data of
10-20\% were found in the Z+jets final state."

- The DPS cross section formula looks in eqn(1) looks rather simple, perhaps even a bit
naive, at Born level, as been the form we have been using for many years... Was there no
further phenomenological thought in the community if this is not too simplistic a view,
and has some better/evolved suggestions? (no worry if the answer is 'no' )
Of course using what we did allows us to put different results on the
same plot (Fig 14) but I am doubtful it is very meaningful, which is supported by
the spread go he extracted numbers.

We have to do with the current version of thepocket formula. Strides are being made towards more complex models, but nothing that is developed up to a point that it is applicable in our study.

Details

- line 12: "..call for further studies of the strong interaction "
Does this really follow from the discussion above, they way it is written?

Replaced with "..., are not as well understood and, therefore, require and suggest further studies of the strong interaction."

- line 16: "A second approach"? -> "A different process"?

Changed accordingly.

- line 32 ".. resulting in a strong increase of the gluon density.." actually
it is the other way around: we have more interactions because the gluon density
is increasing strongly at low-x (as measured in e.g. electron-proton scattering )

Rewritten as "...: at higher energy, smaller values of the proton’s momentum fraction carried by the partons can be probed, because of a strong increase of the gluon density, resulting in a larger
probability for DPS."

- line 49: do we see visible effects/changes by using this custom made calibration,
instead of the out of the box CMS numbers? Have these new calibrations been vetted by
the JETMET group?

Yes, large differences were observed, i.e. when examining the resolution of the jet pT ( (pT,gen - pT,det)/pT,gen ), the distribution was not centered around 1 but deviated from it, indicating that there was something wrong with the standard JEC. The JEC we derived have all been checked and approved by the JETMET group.

- line 57: What I found confusing first time around is the use of "inclusive
single-jet events as mentioned here. In fact I think what you mean is that these
are events triggered by a single jet trigger, but in fact they are (mostly) di-jet events,
so that when you merge two of these events you get (in most cases) a 4-jet event,
whereby the two 2-jet events come from a separate production vertex.
I guess when you know the analysis it is obvious but for a fresh reader it
takes a moment to get used to. :). I recommend therefore that one should spell
this out explicitly front he start.

We left the comment there as is, however, we tried to give a more extended and clear picture in section 6 where we detail the procedure more explicitly.

- line 138: ...rather old PDFs here from 2002 :(
But I guess we take here what we have...?

Yes, that is the case.

- line 170: Are you using a merging scheme here of the PS and ME for NLO programs,
like FXFX? With the LO programs we explicitly gave that we used the MLM scheme so if we
use one here as well, we should also mention that for the NLO and give a reference.

It is an MLM matching scheme as well. It is now stated explixitly in the paper.

- line 171: "2019.02.25" is that really the version number? So I guess the
version and the date are the same ?

Yes, it is like this.

- line 173:" for which the real phase space is build afterwards"
looks like slang. What does it mean exactly?

In the papers on dijet and trijet production in Powheg the same language is used. They calculate the Born cross section and afterwards apply a "multi-channel" technique to approach the singular regions. We felt that if we would mention the multichannel technique that we would have to explain it as well, making the text even more technical when it is besides the main point of the paper.

- line 188: what are the most important expected consequences for having on- and off-shell
incoming partons? For the off-shell ones we have to use TMD, and for the
on-shell we use kT integrated PDFs. But do we expect effects on the hadronic final state
phenomenology? If so this would be of interest for the reader to spell that out.

We added "In the case of the latter, the initial partons are generated with a non-zero intrinsic k T which can alter the momentum balance of the jets, yielding different topologies and correlations between the jets compared to on-shell production."

- line 188: "two-off shell" -> "two off-shell"
Also check on- and off-shell in the paper. I saw sometimes we forget to put the
"hyphen".

Changed accordingly.

- line 193: "For two the" -> "For the two"

Changed accordingly.

- KATIE: the program is combined with HERWIG or PYTHIA.
In lines 190-196: do we combine it such that NO DPS interactions are generated
or do they have the default ones from the CP5 and CH3 tunes? (see general comment)

We have the default ones from the CP5 and CH3 tune.

- line 205: Do I understand that this is the same sample as discussed in line 133, or is
there a difference? I believe it is the same but the repetition is somewhat confusing
(particularly since you mention for the next item only that this one has already
been mentioned before)

It is the same, we clarified it to: "A PYTHIA 8.240 sample is generated with the CP5 tune and the NNPDF2.3 NNLO PDFs. It is the aforementioned PYTHIA 8 sample to which a DPS contribution is added"

- line 222: So here we are using pre-scales for the low threshold triggers used?
What are these prescale factors?

We added the scentence: "...| η | < 4.7. As the triggers have been prescaled, they are used in disjoint p T ranges. Offline requirements..."
The prescales range from 1 to 25 depemnding on the trigger.

- line 235-236: This was not clear to me reading that here but I guess the
explanation of for is given explicitly in line 262, correct?

Yes, it is given later, we had to mention it here but explaining it fully in seciton 5 did not seem optimal to us.

- line 300: Recommend you say here how far down in pT the out of the box CMS calibration
works well, and that we need to bring it down to 20 GeV in this analysis.

Changed the scentence to: "The low pileup, low jet \PT data sample used in this analysis necessitates a dedicated JES calibration since the out of the box JES corrections factors did not hold up below $~50$ GeV."

- line 307: despite this effort on the calibration of the JER effects the uncertainty of the
cross sections seems still very large... Wouldn't we have expected to do better?

It is in line with previous results, the uncertainty rises rather sharp below ~40 GeV.

- line 358: "DPS pocket formula" you just mean eqn (2)? Or did you have something
special in mind with "pocket"?

Changed accordingly.

- line 398: "...possible need for DPS" (see my large general comment and assuming
I did not overlook something in that comment): do we mean here "more DPS"? These models
all have MPI'sat some level, no?

Yes, they need more DPS, changed accordingly.

- line 411: also all the KATIE cross sections are too high in region II. We don't
seem to mention that in the text.

Changed to "The \MGvATNLO LO samples and all the NLO samples predict cross sections that are roughly in agreement with the cross sections obtained from data in \regioni, but are larger than those from data in \regionii, as are all the \KATIE cross sections in the same region."

- Figure 6 legend: "PBMTD" is not directly related to the section 4.2 PDF terminology used.
(one can make a good guess of course, but just for definiteness). To be fixed.

We have defined MRW and PBTMD in section 4.2 now.

- line 436: "leave room for a DPS.."? They all have some DPS (see my general comment)
Perhaps "leave room for more DPS.."?

Changed accordingly througout the section.

- line 440: We don't study that here but the large DY cross sections could also be the
result of the fact that we the QCD approximation we are working with is not the right
one for such a measurement, (eg BFKL vs DGLAP although this may go in the wrong
direction here..)

Yes, but discussing it would take us too far of topic.

- line 448-450 The DeltaS distribution in figure 9 shows in the bottom two plots
shows that many the models have essentially ratios with values below 1. Given that
we normalise on the high deltaS bin and the fact that the cross section in
table 3 predicted by the models are all larger than the data, this means that the
shape of the DeltaS distribution for these 5 models is very different from the
data. Maybe something to take note of in the text?

We have extended to "The \DS distributions again show a more robust behavior with respect to the parton shower implementation. The LO \MGvATNLO samples leave less room for an additional DPS contribution compared to the NLO models. Due to the sole use of a $2\to 4$ matrix element, the \PT spectra are far too hard, resulting in a large overestimation of the slope for all the \KATIE models."

- line 460: This seems to be a pattern: all models we have discussed in this paper
give a too high cross section. This is a strong message. Are we sure nothing could have
gone wrong extracting this data cross section number for this region?
I imagine that has been given some thought?
Or do we have a hypothesis what it can mean?

When comparing litterature, similar studies all find the same. It is tied to the collinear factorization that most models use. It is known to fail at low pT. If one looks at KaTie off-shell (based on kt-factorization), one sees that it performs better all over the line. We made sure te check multiple times and perform cross checks.

- line 462: are these really pure SPS samples? (in view of my general comment above)

We removed the "pure".

- line 480: actually the P8 models don't seem to do to badly for the DeltaS distribution...

This is already discussed in the text.

- line 490: I somehow failed to see why we drop that PYTHIA tune here at this stage...
what do I miss?

These are old tunes, we have the more recent ones CP5 and CH3, they are tuned to 13 TeV data as the CUET tunes are not. We only had to use these tunes in the analaysis to perform the unfolding. Both tunes did not yield any meaningfull results, therefore we opted to leave them out.

- line 493: not only this CDPSTP8S1 tune contains DPS (see my general comment above)

The whole section has been overhauled.

- line 502 and caption of Fig 13: "DPS component in..." what do we mean exactly with that?
We did not define that before. Did we 'extract' the pure DPS part with this model
With KATIE this is probably straight forward, but how did we extract it from
PYTHIA? I recommend that we explain that in the text.

In section 4 we explain that Pythia8 and Katie can generate pure DPS samples. Explaining it agiain seems redundant. However, we did change "the DPS component" to "the pure DPS sample"

- line 513: "15" -> "Fig. 15"

Changed accordingly.

- line 523 and figure 14: the extracted values sigma_eff cover a very large range with
relative small uncertainties, casting somehow doubt that we are measuring a well
defined quantity.

The section has been overhauled.

- line 539: does this not show that indeed these 2 ->2 models have MPIs included
already due to their tunes, as I mentioned in the general comments?

The section has been overhauled.

- fig 14: A few observations here:
The spread or unknown systematics seems very large. It shows that we cannot extract
a model independent measurement of sigma_eff with this technique as applied
here, from the 4 jet event data.

The section has been overhauled.

The former measurements in the 90's: can we really put these on the same plot? I assume
that the analyses at the time were much more naive and used different methods and tools
(certainly no MC programs of the same level as we have now)

We went through thepapers and yes, there are differences, expecially in the technicality of the programs, but we did not encounter anything that would clearly suggest that the results are not comparible.

Secondly and more provocatively (if I were a referee of this paper): what did we learn
in these last 30 years? You are not obliged to answer that (to me, but dono if the
referee will question it )

In the case of the inclusive four-jet scenario, we have found that our models do still need to improved further before any more significant conclusions can be drawn. For example, the NLO 2->2 models form outliers compared to the other models (see updated results) or we need a matrix element mixing/matching in the off-shell samples, which look the most promising when looking at the correlations between the jets.

- line 555: The J/psi measurement is a very precise measurement with a systematic
error of ~1mb. In view of the high level discussion in this paper and the reported
results: Is the J/psi method really much more accurate/reliable?
We should comment on that.

Yes, the error is different, which is due to the machinery that they have available at the LHCb.

- Depending on which of my comments survive this may affect a rewording of some
of the conclusions in the summary.

Conclusions have been revised accordingly.

- ref [51] this is submitted to the arXiv now as 2011.03422 . add the arXiv number.

Changed accordingly.

1) Page 1, the lines 25-26
"For fully uncorrelated production of A and B, sigma_eff tends to the
total inelastic cross section, while ..."
The total inelastic section of what?

Changed line 25-26 to: "... the total inelastic cross section of proton-proton interactions..."

2) Page 1, the lines 28-30
The phrase "Indeed, the jets resulting from DPS are more often than not
produced in two independent pairs in a back-to-back configuration."
looks too cumbersome.
We propose to simplify it:
"Indeed, the jets resulting from DPS are more often occur in a back-to-back
configuration than SPS jets."

The reformulation looks double to us, "the jets resulting from DPS" are the four jets of interest of which only the jets created in the same interaction are more likely to be in a back-to-back configuration.

3) Page 1, the lines 42-43
"The CMS collaboration additionally obtained a measurement for ..."
==>
"The CMS collaboration additionally performed a measurement for ..."

Line 42-43 had been changed accordingly.

4) Page 3, "Observables"
It is well known, that "2- > 4" process is described by 7 independent
variables. However, the authors provide the detailed description
of only SIX ones.

In the paper six observables are used that are not necessarily completely independent of each other. The main goal is to look at certain DPS sensitive observables by studying the balance between the jets. Our onset is not to give a complete description of the process.

5) Pages 5 and 6, Multijet models section.
The authors have used MadGraph5 and POWHEGBOX generators.
However, it is not clear why they need to consider KATIE predictions.
We recommend to provide additional motivation for inclusion KATIE model.

KaTie allows for off-shell production in the kT factorization scheme, which allows the initial partons to be off their respective mass-shell. Therefore, the created jet pairs can deviate from the typical back-to-back configurations obtained in on-shell production. A second argument is that collinear factorization is expected to fail at low pT, whereas kT factorzation is expected to show an improvement in this region of the phase space.

When introducing KaTie we have added the following to line 182: "... based on kT-factorization [61-63], allowing for on-shell and off-shell production. In the case of the latter, the initial partons are generated with a non-zero intrinsic $k_\mathrm{T}$ which can alter the momentum balance of the jets, yielding different topologies compared to on-shell production."

6) Page 8, the lines 269-271
"A combined event is discarded whenever two or more jet axes spatially
coincide. This veto condition is ..., where the indices i and j indicate
jets belonging to an event from the first and second data sample,
respectively."
It is not clear should we apply this criteria for jets belonging
to one sample ?

The criteria is only between jets coming from event A compared the jets coming from event B, per definition no overlapping jets within the same event are found as they are reconstructed as one. The veto condition aims to remove coincidentally overlapping jets coming from the two seperate interactions, will clarify in the text.

We have reformulated it to: "..., where the indices $i$ and $j$ indicate jets of an event originating from the datasets of the processes A and B, respectively."

7) Page 8, line 274
The value epsilon_{4j} = 0.324... has used in several places.
It make sense to put it makes sense to put it on a separate line

Has been implemented in the paper. The four jet efficiency has been replaced with references to that equation.

l. 309 full stop missing at end of sentence. At the end of eq.2 you need \,, and start with "where" the next line.

Text has been altered accordingly.

8) Page 11.
It would be useful to briefly consider the nature of the very large
difference between cross section values sigma_A and sigma_B.

Added "A large increase in cross section is observed when lowering the $p_\mathrm{T}$ threshold from 50 GeV to 30 GeV, as is expected." right after the cross sections sigma_A and sigma_B.

9) Page 11, the line 362
"... labelled P8, ..." ==> "... labelled as P8, ..."

Changed accordingly.

10) Page 24, the 499
two commas are missed
"... from data as described in Sec. 6 is used ..."
==> "... from data, as described in Sec. 6, is used ..."

Changed accordingly.

11) Page 24, the 521
The value of sigma_eff = 32 +- 1 ... is appeared at this place only.
One can not find this value in Fig. 14 or Tab. 5

The effective cross section that we give here is not extracted from from the data through the template fit. It is rather a separate calculation: it is observed that the CDPSTP8S1 -4j tune predicts an excess of DPS events given the effective cross section of 21.3 fm that it uses, therefore, we tried to calculate the optimal effective cross section for the tune and do not think the number belongs with the extraction results.

Type A
L88+1: jet pair -> jet pairs

Implimented accordingly.

L140: dipole antenna shower -> dipole-antenna shower

Implimented accordingly.

L193: For two the off-shell samples -> For two off-shell samples

Implimented accordingly.

L194: transverse momentum dependent (TMD) PDFs -> transverse-momentum-dependent (TMD) PDFs

Implimented accordingly.

L326: between 1 and 16% -> between 1% and 16%

Implimented accordingly.

Table1: What is eta_A and eta_B? Is delta S_DPS referring to data-driven template? But not defined

DS_DPS is now defined right above the four-jet efficiency in section 6 and eta_A and eta_B defined in the text right after eq.(2) in section 6.

L358: As I understand, sigma A and B is not used for DPS cross section, but to extract sigma_eff with it

Yes, this is true.

L370 : low p_T jets -> low-p_T jets (to be consistent with the expression in L368) Implimented accordingly.

Table3: NNPDF2.1 LO -> NNPDF2.3 LO according to the main text

Implimented accordingly.

L444: Angular ordered parton showers -> Angular-ordered parton showers

Implimented accordingly.

L513: in 15 -> in Fig. 15

Implimented accordingly.

L561: four jet -> four jets

Implimented accordingly.

L572: angular ordered/dipole antenna -> angular-ordered/dipole-antenna

Implimented accordingly.

Figure15: The expression "the fitted MC model" is confusing. Maybe my misunderstand, but isn't it just bin-normalized PW sample?

It is the bin-normalized PW sample, however, it has been scaled in the fitting procedure, we have changed it to "the scaled MC model".

Figures: delta phi_Soft -> delta phi soft (to be consistent with the main text)
Figures: delta pt_Soft -> delta pt soft (to be consistent with the main text)

Both have been made consistent.

TypeB
L97: Since your analysis uses jets with high pseudo-rapidity, it would be good if you specify more information about forward calorimeter.

In order to be consistent we have added a description of all calorimeters (ECAL, HCAL and HF).
Add also sentence for JER as suggested in PubCom text.

L119: Can we say the parton shower is a next-to-leading-log precision? It is between LL and NLL depending on an observable, and it would be safe just saying LL.

We checked the Pythia 8 manual, you are correct, the scentence has been changed accordingly.

You are still writing NLL. Is that correct now or not?

It has been changed to LL.

>
L221: Why do you use 4 single-jet triggers with different pt threshold? Are some of them prescaled or having additional filter, or cut? It would be better describe some details.

The triggers are prescaled indeed, using them in disjunct pT ranges yield more statistics, we have added the following scentence: "... a \PT above 30, 50, 80, or 100 GeV, and within $|\eta| < 4.7$. As the triggers have been prescaled, they are used in disjunct \PT ranges. Offline cuts ..."

"disjunct" means "discontinuous, not connected" while "disjoint" means "non-overlapping" so they are are not really synonyms. You certainly want to have "disjoint" (no common element) datasets and not "disjunct" datasets. Please check which word is more appropriate here and in many other places. See also further below.

Disjoint is the more correct one here, the text has been changed accordingly.

L267: How the events are divided into two disjunct datasets, is it randomly divided? I think it would be good to briefly mention how it is done.

Changed "Events that meet the online and offline analysis criteria are stored in two disjunct datasets, respectively."
to "Events that meet the online and offline analysis criteria for the processes A and B are stored in two disjunct datasets, respectively."

L337: Is luminosity uncertainty 2.5% like other analysis although your analysis is based on low PU run? And for further information, LUM-17-003 (ReadyForFR) shows lumi uncertainty in 2016 is 1.3%.

We had taken the number of LUM-17-001, however, if the paper is published before ours, we will make the necessary changes.

LUM-17-003 has been submitted. Please apply the proper changes to the integrated lumi and its uncertainty as announced to CMS.

The reference has been updated to the correct one for LUM-17-003.

Figure12: Unlike other observable, the effect of adding DPS contribution for phi_ij is not noticeable in DPS-sensitive region. Rather, it looks like there are more DPS contribution in high delta phi_ij in KT offshell sample when comparing with Figure8. It would be great if there is some comment on it

In the first section of the results we show the large (!) dependence of the observable on the type of parton shower, we conclude there that the obserable is more apropriate to study parton shower effects rather than DPS.

L549: Is sigma_eff universal even for different sqrt(s)? Maybe other 13 TeV results are more suitable for comparison.

It is still open if the sigma_eff deducted from the pocket formula is a propber observable and whats its dependencies exactly are, naivley one does expect it to rise with sqrt(s), however, we think it is indicateve to compare it with older measurements to see the evolution of the measurements.

- abstract: 1 line before the end : what do you mean by earlier measurements ? do you mean measurements already published in a previous paper ? or do you mean the measurements you discussed previously in the abstract ?

Replaced by "previous measurements" as we want to compare by measurements performed in the same topology but at different center-of-mass energies.

- section 1:

- line 30 : The DPS cross section rises with increasing center-of-mass energy at fixed pT : Isn’t it something true for all processes ?

Not necessarily, since e.g. the accessed range of momentum fractions x in PDFs is changed.

- line 55-57 : For a reader external to SMP, this part is not sufficient to understand what is done… why do you need to mix the pure DPS signal with inclusive single-jet events ?
After reading section 6, maybe you wanted to say something like this :
a pure DPS signal sample is created from data by randomly mixing together two inclusive single-jet events.

Altered the scentence to "a pure DPS signal sample is reconstructed from data by randomly mixing two inclusive single-jet events into an inclusive DPS four-jet event, whereafter it is fitted along with several SPS-only background MC models to the distributions obtained from four-jet data." in order to make the statement more clear

- section 2: Do I understand correctly that the hard jet pair is produced by one interaction (leading to the jets a and b on Fig 1 right), while the soft jet part is produced by the other (= c and d on Fig 1 right) ? If yes, could you write it somewhere in the paper. If not, how can I see that the two softest jets are more often back-to-back in DPS if they don’t come from the same interaction ?

Fig. 1 is the ideal case, in reality the pT of the jets of the different interactions can mix whenever the two created jet pairs are close in pT to each other. On average the two softest jets come from the same interaction and will flatten the curve, we have checked the different jet configuraions in a MC study.

- section 4:

- line 141 : what is the default tune ? same comment in Table 2.

The default parameter values of Pythia 8.3 are used, we have changed "default tune" to "default parameter values of \PYTHIA~8.301"

- section 4.3:

- line 205-208 : For the second PYTHIA8 sample, you write that it has already been mentioned before, what I understand as lines 135-138. But the first sample with CP5 seems also to have been mentioned before, as in lines 133-134. Is it correct ? If yes, please rephrase it saying that it too has been mentioned before. If not, then explain briefly why it is not the same sample as before…

The SPS part has been mentioned before, however, the sample now includes an explicit DPS part to the original part described at 133-134, we changed it to: "A \PYTHIA~8.240 sample is generated with the \texttt{CP5} tune and the \texttt{NNPDF3.1 nnlo} PDF. It is the aforementioned \PYTHIA~8 sample to which a DPS contribution is added."

In fact, when reading the full section 4.3, it is very confusing that some sample of PYTHIA8 was already mentioned before, while at least for the KATIE samples it seems that the samples with DPS contribution are different from the multijets one. For a non-expert reader, could you specify what is the difference between multijets and SPS+DPS ? Do the multijets include only the SPS case ? And what about the PYTHIA samples ? Do they include both SPS+DPS in all cases ? or sometimes only SPS ?

All samples in the first two paragraphs (Pythia and Herwig samples, multijet models) obtain a small DPS contribution from the tune they are interfaced with, however, this DPS contribution is much softer as what is needed for these DPS sensitive models. Therefore, models with an explicit DPS contribution (in addition to the much softer MPI from the tune) is added in order to obtain the SPS+DPS models.

- section 6: It is not clear which data you are using to extract the DPS cross section.

What I understand is the following :
- sigma_A (sigma_B) is extracted from events with at least 1 jet with pT > 50 (30).
- you construct a combined sample : mixing one event with at least 1 jet with pT > 50 GeV and one event with at least 1 jet with pT > 30 GeV.
- you want to extract sigma_eff which is computed knowing sigma_A, sigma_B, sigma_DPS, efficiency_4jets.

The fact that you mention that ∆S observable is chosen for the extraction of the DPS cross section in this part of the description (line 282) is very confusing for later. Because at this step of the explanation, I am still within the combined sample.

-> why do you need the Delta S observable to compute sigma_eff (line 282) ?

We need to calculate the fraction of DPS events which gives us sigma_DPS by integrating it. This is done by examining the discrepancy between the data and SPS models for the DeltaS observable. The difference in shape is a measure for the amount of DPS events.

-> In which sample is determined the efficiency_4jets ? In simulation ? or in the combined sample ?

We have changed "data" to "combined sample" in order to clarify, we now write "... ($\epsilon_{\rm 4j}$) has been obtained from the combined sample as detailed below." in what was line 265.

-> which events are used to extract sigma_DPS ? After the full reading of the section, I suppose it is from the real four-jet data events (as indicated on line 290). But I am not sure as you spend a big part of section 6 to describe the combined sample without introducing any change of samples in the text…

The combined sample is used to obtain a signal template in order to estimate the amount of DPS events (needed for the calculation of the sigma_DPS). Together with the SPS models, the DPS template is fitted to the data in this regard.

-> on line 284, you mention the DPS fraction. In which sample do you extract it ? As you derive the signal template from the combined sample, I suppose that the DPS fraction is equal to 1 in that sample. So I suppose you are back in the four-jet data sample for the estimate of fDPS… If yes, it should be specified on line 284.

It has been stated explicitly now, what was line 284 now reads "The fraction of DPS events, $f_\mathrm{DPS}$, is extracted by performing a template fit to the unfolded \DS distribution, obtained from the original inclusive four-jet sample."

-> on line 287, why do you write “which is used to extract the ∆S distribution in exactly the same manner as before” ?

We have stated it more clearly, line 287 now reads "The signal template is taken from the combined data sample, which is used to extract the \DS distribution. The \DS signal template is corrected to generator level in exactly the same manner as before, including the correction for detector effects by means of unfolding."


- section 8

- lines 358 and 359 : I don’t understand this sentence. Why do we need the cross sections (5) to (8) to determine the DPS cross section ? I understood that the the DPS cross section is determined from the fit of the Delta S observable (fit of eq (3)) which allows to evaluate f_DPS, then used in eq (4)) ?

Cross section (5) is of interest because it allows for the comparison of the cross section for the other observable appart from the DeltaS observable. Cross section (6) is needed in order to obtain the DPS cross section, as from the template method only a fraction of DPS events is obtained, e.g. 3.5% and thus the DPS cross section = 3.5% x (6). The cross sections (7) and (8) are needed in the pocket formula as they are the two independent processes A and B in the pocket formula.

- Fig 15 : I don’t understand what are the two ratios in the bottom panel : one is PW/data and the other is (PW+DPS)/data ? If so, the color code of the hatched zones should be better chosen. With a yellow hatched zone, it provides the feeling that we look at DPS/data (and anyhow yellow is usually hardly visible...)

We have added a legend to the ratio plots in order to clarify what is represented by which curves.

- lines 514-521 : I miss why you need to know sigma_eff to derive f_DPS. This probably related to the fact that I don’t understand in which order you extract which information (in section 6).
I find very bizarre to obtain sigma_eff with a procedure where you impose an input value of sigma_eff. Why don’t you then perform a second iteration,
i.e. estimate a new value of f_true with sigma_eff = 32 mb as input ? Would in that case the value of sigma_eff = 32 mb be stable ?

The model with the CDPSTP8S1 -4j tune already has an explicit DPS contribution, however, it has been derived for 7 TeV data. With its fraction of DPS events, a fraction of DPS events is related. We also see that the model overestimates the data, thus the input value of sigma_eff of the model is wrong in the 13 TeV case with our cuts, therefore we invert the calculation in order to obtain the value of the sigma_eff that would be optimal for the model for our phase space.

Comments on the text :
———————————

-->> Text will be adapted accordingly.

- line 29 : Indeed, the jets resulting from DPS are more often than not produced in two independent pairs in a back-to-back configuration. —> Indeed, the jets resulting from DPS are more often than from SPS, produced in two independent pairs in a back-to-back configuration.

Implemented accordingly.

- line 119: give a reference for DGLAP

Included the necessary references.

- line 229-230: defined by cuts -> defined by selection cuts

Implemented accordingly.

- blank line after 263: can be removed

Implemented accordingly.

- line 274: write in parentheses that the systematics on eps_4j are explained in the next section

The 4-jet efficiency has been given its own equation, followed by "Where the statistical uncertainty was found to be negligible and the systematic error is detailed in the next section."

- blank line after 357: can be removed

Implemented accordingly.

- line 358 what is a: DPS pocket formula ?

Has been replaced by Eq.(1), a reference to the formula.

- line 359 : A dot is missing at the end of the sentence.

Implemented accordingly.

- section 8.2 : missing reference to Table 3 in the text of this section

Reference included in the first paragraph "Table~\ref{tab:MultiJet} gives a complete overview of all models, tunes, and PDF sets, along with their respective cross sections."

- line 413 : Fig 6 shows only the pT distributions. Need to add Fig 7 in the sentence to cover also the eta distributions. Or remove the reference to eta in case you want to point only to Fig 6.

Removed the "eta spectra".

- line 513 : shown in 15 -> shown in Fig. 15

Implemented accordingly.

- line 516: attach mb to 21.3 (21.3~mb)

Implemented accordingly.

- Fig.15 caption line 2: In the ratio -> The ratio
Line 3: are show -> are shown

Implemented accordingly.

- line 586: used to the describe -> used to describe

Implemented accordingly.

Type A comments:

L 33: extend --> extent.

Changed accordingly.

L 48: We suggest removing the word "such".

The text reads now reads "... allows not only to mitigate pileup effects, but also ..."

L 193: For two the --> For the two.

Changed accordingly.

L 428: and models --> models.

Changed accordingly.

L 463-464: These lines are easy to miss, please consider moving Fig. 10
to the top of the page.

L 559: While the --> The.

Changed accordingly.

Figures: In many figures in this paper, the yellow bands obscure the
vertical axis ticks. It would be better to draw the ticks
on top of these bands. It is often difficult to see the
point/band locations given that the ticks are hidden.
"RedrawAxis" would help.

All figures have been updated with the RedrawAxis " function.

References:

All references have been updated accordingly.

L 598: R=0.5 --> R = 0.5

L 605: protonproton --> proton–proton

L 634: DOI link is broken

L 639: tevatron --> Tevatron

L 646: DOI link is broken

L 648: cern --> CERN

L 649: Physics Letters B --> Phys. Lett. B

L 651: four jet --> four-jet

L 670: sigma(eff) --> σeff

L 681: Production of four-jets --> Four-jet production

L 685: kt --> kT

L 688: cms --> CMS

L 713: Physics Letters B --> Phys. Lett. B

L 723: DOI not found

L 727: Nuclear Instruments and Methods in Physics Research Section A:
Accelerators, Spectrometers, Detectors and Associated Equipment
--> Nucl. Instrum. Meth. A

L 735: Vincia --> VINCIA

L 743: Accepted for publication. --> arXiv link?, this paper has been
submitted to the arXiv now as 2011.03422. Note that the paper
title has been changed.

L 768: small x --> small-x

L 782: TUnfold: --> TUnfold,

L 787: Svd --> SVD,
Nuclear Instruments and Methods in Physics Research Section A:
Accelerators, Spectrometers, Detectors and Associated Equipment
--> Nucl. Instrum. Meth. A

L 793: ROOT: --> ROOT –


Type B comments:

L 30-31: It is not clear p_T of what is referred to in this statement.

Concerns the pT of the jets, has been altered to "... at fixed \PT of the jets. ..."

L 108: It is unclear what the words "in situ" mean in this context,
as the citation at the end of this statement is for a different
data sample.

"In situ" is the text from the PubCom which we do not wish to alter.

L 226: What type of reference trigger is used?

We use a minimum bias trigger, has been made clear in the text which now reads "The trigger efficiency has been determined by comparing the performance of the jet trigger with a minimum bias trigger serving as an unbiased reference."

L 227: "...exactly one vertex...". Do you mean exactly one primary vertex?

Yes, exactly one primary vertex, text has been altered accordingly.

L 239-242: Naively, if you use the bin width which is twice larger than
the resolution, you expect that 68% of events do not migrate
outside the bin. This results in a diagonally dominant detector
response matrix which is easily invertible without any
additional regularization. Why, then, do you need to regularize
your unfolding procedure instead of simply inverting the
response matrix? As all regularization procedures are based on
some kind of a bias-variance trade-off, this leads to increase
in the bias and potentially underestimates statistical
uncertainties of the results.

We have checked the condition numbers of all distributions. It was not small for all distributions as they span multiple orders over the y-axis, therefore, we opted to use the regularization as it is recommended by the statistics committee.

Section 6: In general, it is not obvious why the jet reconstruction
inefficiencies (e.g., due to parton direction overlap) in
your signal template constructed from data would be similar
to those produced in a genuine DPS process which might
exhibit directional correlations. Perhaps, the relevant
assumptions should be spelled out explicitly.

We have added ""In the derivation of the formula above, the two parton pairs of the processes A and B are assumed to be fully uncorrelated. In Fig.~\ref{fig:SPSvsDPS} this would mean that the partons $i$ and $j$ as well as the partons $k$ and $l$ are fully uncorrelated."" to the introduction, right below where the DPS pocket formula is introduced. As the incoming partons are assumed to be uncorrelated, the jets produced in the processes A and B will be uncorrelated. It seems more indicative to us to place the main assumption there.

Type B.
13-34 No references in this part ?

Included a reference to a general DPS paper where the pocket formula as well as many different aspects of DPS are detailed.

Eq.(1) this equation is referred later (page 8 line 358) to as
'DPS pocket formula'; if you do want to use this jargon language define eq.(1)
as 'DPS pocket formula' here.
See also comment at 358.

We have replaced "DPS pocket formula" with eq.(1).

29-30 back-to-back
Should we specify "in the transverse plane" ?

We have rewritten it as "Indeed, the jets resulting from DPS are more often produced in two independent pairs, each in a back-to-back configuration in the transverse plane."

45-47
The pileup definition (or better description) looks unnecessarily
complicated... which is the reason to refer to 'nearby BX'?
Is nearby a proper term ? Nearby with respect to what?

It is supposed to differentiate between in-time and out-of-time pile up. In order to clarify, "nearby" has been replaced by "neighbouring".

48 but also to reach down to low jet pT
Why ?

Less pileup results in a less convoluted or complicated events. In addition, the offset corrections (to correct for pileup) are not needed, yielding a smaller uncertainty on the JEC factors (which is still the largest in our study).

49 albeit necessitating 49 a custom calibration of the jet energy scale....
Why?

No appropriated/correct JES factors were available for low pT low pileup data, if we were to apply the standard corrections provided by JETMET, our results would have been vastly different due to the systematic and wrongfull scaling of all jets. All corrections have been presented to and were approved by the JETMET group. The standard JEC is not adapted to low PU, but optimised for high PU conditions. It would be a mistake to apply the standard JEC as it is. All corrections ...

55-59 The DPS cross section is extracted with
a template method: a pure DPS signal sample is reconstructed from data by
randomly mixing inclusive single-jet events and is fitted along with several
SPS-only background MC models to the distributions obtained from four-jet
data. Finally, the effective cross section is computed
using Eq. (1), with σ A and σ B measured from data.

This part could be better explained.
It is absolutely not clear...

The lines have been rephrased as: "The DPS cross section is extracted with a template method: a pure DPS signal sample is reconstructed from data by randomly mixing two inclusive single-jet events into one inclusive DPS four-jet event, whereafter it is fitted along with several SPS-only background MC models to the distributions obtained from inclusive four-jet data."

Page 3 bullet after 82 The math expression does represent the variable only
if i, j are the jets giving the largest DeltaY; should it be re-written in a
better way ? is the notation φ_[ij] misleading ?; what
about φ_[Delta eta_[max]] or similar?

The notation has been used in multiple papers before, it was opted to use the same notation in order to stay conform with previous studies.

Section 3 The section contains the reference [35], used for the definition
of the coordinate system and the kinematic variables; should we
swap Section 2 (where eta, phi are used) and Section 3.

In our perspective it is better to discuss the observables after introducing the concept of DPS where we talk about the differences in correlations. Together with Fig.1 we think that the sections flow more naturelly in this way.

114-115 Could this sentence be moved immediately after 100 ?

We have added a description for ECAL,HCAL and HF and put the scentence with ref. [35] right after it, but before the discussion on the global event reconstruction algorithm.

119 DGLAP: this acronym (though standard in QCD) is not defined;
would be a reference or definition suitable ?

The approptiate references have been included.

127-141 Here you refer to PYHTHIA8, PYHTHIA8.240 and PYTHIA8.301...
we find this somehow confusing.

Pythia 8 is the main program, however, we use two different versions as PYTHIA 8.240 is the standard used in CMS, while only from PYTHIA 8.301 and newer versions allow for it to be interfaced with VINCIA.

152: “A second group of models” is too generic. Section 4.1
describes pure PS MC, while this section deals with ME+PS generators.
It would be useful to clarify for the reader how they are going to be used:
are they alternative models for the same SPS template? For different regions?
What is the aim of introducing such very different models?
We do not t expect in general that PS MC will describe correctly SPS 4
jets events.

There are no "pure PS MC". Also the LO versions have a ME+PS, however, we have tried to clarify the text further:

"A second group of models uses higher..." has been altered to "A second group of models, hence on forth refered to as the multijet models, uses higher ..."

We include many models as different types of PS, the order of the calculation, the MEs used in the calculation, ... all have influence on the correlations between the four final state jets. Most studies only included a couple of models, however, in comparing the many different models we are able to learn much more about what works and what works not in this complex topology.

From 187 till the end of the section: can you elaborate a bit more on
the difference between on- and off-shell incoming partons? Why have off-shell
partons to be showered and hadronized with CASCADE and TMD PDFs? What are we
expected to learn from this alternative approach?

Off-shell partons can not be handled by the standard PDFs or MC event generators, CASCADE in combination with TMDs need to be used, ellaborating on this seems a bit redundant.

We have dropped this is general knowledge ... to us." and we did add a scentence to clarify the difference between on- and off-shell as to motivate the use of KaTie, we added "... allowing for on-shell and off-shell production. In the case of the latter, the initial partons are generated with a non-zero intrinsic $k_\mathrm{T}$ which can alter the momentum balance of the jets, yielding different topologies and correlations between the jets compared to on-shell production. A $2\to 4$ matrix element ..."

215-217
What is the size of these non-perturbative corrections?

The non-perturbative corrections range generally between 1-4%, only for the DeltaS observable they range from 1-11% with 11% in the first bin (the most left bin). They are now specified in the text, which reads "The nonperturbative corrections range from 1--4\% for all observables, except for the \DS observable for which corrections up to 11\% were found. "

224 "In this last case a correction as a function of the jet pT is
applied"
A correction to what?!?

"In this last case, a correction as a function of the jet \PT is applied the selected event, effectively altering the weight of the event." should clarify the statement.

230-231 'to meet PT cuts' We guess you mean that PT must exceed the cut
value; in this case it is better to write it esplicitely

"to meet the PT cuts" has been replaced with "to exceed the PT cuts".

231 'Asymmetric cuts have been chosen over symmetric'
What does (a)symmetric mean in this context?

Symmetrical cuts have the same values on the two leading/sub-leading jets, e.g. 50-50-30-30 GeV cuts. Asymmetric cuts are expected to boost the DPS signature.

267 "Events that meet the online and offline analysis criteria are
stored in two disjunct datasets"
How can 'online and offline' be disjunct?
If this was the case the veto criterion would not be needed,
isn't it?

Events that meet both online and offline analysis cuts for process A are stored in one set, and events that pass all criteria for process B are stored in the second disjunct data set. The scentence has been altered to "Events that meet the online and offline analysis criteria for the processes A and B are stored in two disjunct datasets, respectively." in order to clarify for any possible confusion.

Eq (4): should not the DPS contribution to the total cross-section in the data
be the integral of the first term on the right hand side of eq (3)?

Both are equivalent. The template method determined the optimal fraction of DPS that should be added relative to the total cross section of the Delta S observable obtained from data. This is in turn equal to the righ hand side of eq.(3). However, as eq.(3) is rather complex, the statistical errors obtained from eq.(5) were slightly smaller and more stable compared to the errors obtained by performing an error propagation on eq.(3). We did perform cross checks and the results of both methods showed agreement with each other, therefore, we opted to use the simpler method of the two.

Sections 8 and 9: there are a lot of results but it is really difficult
to understand the strategy and to draw any conclusion out of them.
Besides the fitted values in tables 2-5 is there any conclusion about
which models fits the data better? What should the reader conclude
looking at figure 14? The spread of the results seems to be much larger
than the uncertainty on previous results.

From the tables it should become clear that collinear factorization, especially in the Pythia and Herwig case fails at low pT, improvements are found when mixing matrix elements or including NLO corrections, while the off-shell samples show the most promise of all samples. A clear need for a DPS contribution is also observed, however, it seems to be model dependent (fig 14). Indicating that either sigma_eff is rather a model dependent parameter due to the interplay between the ME and the PS and/or that the current models need to be improved (e.g. take correlations between different partons into account) in order to obtain a proper description of DPS events in the four jet topology.


358 "DPS pocket formula"; 'pocket' is jargon and never used
before in the paper; we understood this refers to Eq. (1); refer to that
explicitly. See also previous comment about Eq.(1).

Has been changed accordingly.

593-594 These results demonstrate the need for further development of
models in order to accurately describe final states with multiple jets and
the kinematic correlations between the jets.
Should this strong statement be included directly in the abstract ?

It is implicitly included in the abstract. The last scentence reads "... and of their dependence of the models used for the single-parton scattering background" which is also the main message of the paper.

Fig.2-3-4-5-6-7-8-9 Plots are quite confusing;
in each of them the two (three) bottom panels with the ratios show a total
of 7 (9) models, while the top panel shows only 4 models. Why? We
understand the plots would get too busy otherwise. But, if this is the case,
you need to have more plots. We do not see the point of having ratios for
profiles not shown.

The models shown in the Pythia and Herwig plots are the ones for the most recent tunes, while in the multijet case we have opted to use on of each of the four model groups (KaTie on-shell, KaTie off-shell, MadGraph leading order and Powheg). The ratios for profiles that we do not show allow for an indirect comparement which do yield important conclusions, eg the dependence on the PS of the phi_ij observable for instance. We would argue to keep the plots this way and if needed we could supply the additional curves as supplementary material as adding all curves would extend the paper with another 5-8 pages.


Type A.

2-3 provides a good de-
scription of jets with large transverse momentum (p T ) produced in
high-energy proton-proton (pp) collisions.
--> provides a good de-
scription of production of jets with large transverse momentum (p T )
in high-energy proton-proton (pp) collisions.

Changed accordingly.

11-12 However, final states with multiple jets, and their
kinematic correlations, call for further studies of the strong interaction.
--> However, final states with multiple jets, and their
kinematic correlations, require and suggest further studies of the strong
interaction properties.

Changed accordingly.

14-15 Radi-
ations before and/or after the interaction
--> Radi-
ation before and/or after the interaction

Changed accordingly.

23-24 one --> 1
two --> 2

Changed accordingly.

29 are more often than not produced
--> are often produced

Changed accordingly.

31 smaller values of the proton’s momentum fraction
--> smaller values of the proton momentum fraction

Changed accordingly

32 and accordingly a larger
--> and, accordingly, a larger

Changed accordingly.

33 extend --> extent

Changed accordingly.


141 and l149; in the former LO is uppercase; in the latter nnlo is
lowercase. See also 165, 175.

We have replaced nnlo with NNLO throughout the paper.

172-173 built is used twice

First "built" is replaced by "constructed".

188-189 Two on-shell and two-off
shell samples are produced.

--> Two on-shell and two off
-shell samples are produced.

Changed accordingly.

193 For two the off-shell samples
--> For the two off-shell samples

Changed accordingly.


221-222 "The online selection of events was based on four single-jet
triggers requiring the presence of at least one jet with a pT above 30, 50,
80, or 100 GeV " ==> "The online selection of events was based on four
single-jet triggers each requiring the presence of at least one jet with a
pT above 30, 50, 80, or 100 GeV respectively"

Changed accordingly.

277 the difference of the cross sections
--> the difference between the cross sections

Changed accordingly.

314 can not -->cannot

Changed accordingly.


341 a possible difference with the true efficiency
--> a possible difference with respect to the true efficiency

Changed accordingly.

Table 1 caption
between generator and detector level is estimated at 2%
--> between generator and detector level is estimated to be 2%

Changed accordingly.

In figure captions we see two statements about the way error bars
are displayed.
'The error bars represent the statistical
uncertainty, while the yellow band indicates the total (statistical+
systematic) uncertainty on the measurement.'
'The error bars and
bands are shown similarly to Fig. 2.'.
A uniform statement would be welcome...

We have been advised to use the reference to one of the earlier figures in order to shorten the captions for claritys sake.


428 and models, even the on shell K A T IE models,
--> and models, even the on-shell K A T IE models,

Changed accordingly.

465 the ∆S observables exhibits
--> the ∆S observables exhibit

Changed to "The DeltaS observable exhibits", only one DeltaS observable exists/is defined.

499 The mixed event sample obtained from data as described in Sec. 6
is used to obtain
--> The mixed event sample obtained from data as described in Sec. 6
is used to get

Changed accordingly.

522 The DPS fractions obtained for the other models ranges
--> The DPS fractions obtained for the other models range

Changed accordingly.

Figure 15 caption
systematical--> systematic

Changed accordingly.


Tab.5 In label of column #5 "(.syst.)" ==> "(syst.)"

Changed accordingly.


561 four jet
--> four-jet events ?

Changed accordingly.

Also in some other parts..

Has been changed where appropropriate.

Figure 15 caption
In the ratio of the fitted MC model
and of the total fitted result over the data are show in the bottom plot.
--> The ratio of the fitted MC model
and of the total fitted result over the data are shown in the bottom plot.

Changed accordingly.


586 A strong dependence of the extracted values of σ eff on the model used
to the describe the SPS
--> A strong dependence of the extracted values of σ eff on the model used
to describe the SPS

Changed accordingly.

Type B (physics) comments

- It is not made clear why four different trigger thresholds are used. It would seem simpler to just use the 30 GeV trigger since it should include everything that is triggered by the higher thresholds as well. Are they lower pT triggers prescaled, or do they have tighter requirement in some other variable?

We clarified that the triggers are prescaled, the following scentence was added "As the triggers have been prescaled, they are used in disjunct p T ranges."

- It is not clear what is the motivation behind the inclusion of KATIE predictions. Perhaps the authors could provide additional information on why to include KATIE.

We extended the introduction of KaTie by including a scentence on the difference between oon-shell and off-shell production, it now reads "K A T IE version 23April2019 [64] is a LO parton level event generator, based on k T -factorization [65–67], allowing for on-shell and off-shell production. In the case ofthe latter, the initial partons are generated with a non-zero intrinsic k T which can al-
ter the momentum balance of the jets, yielding different topologies and correlations between the jets compared to on-shell production. ..."

- Line 11: missing citation to the referred QCD predictions?

The citation [1-3] is meant for this part, we moved the references to the end of the scentence "Measurements of the cross section for the production of inclusive high-p T jets have been performed by the CMS collaboration at various center-of-mass energies and show good agreement with perturbative QCD predictions at next-to-leading order (NLO) accuracy [1–3]."

Effective cross-section:
- Line 24: effective cross-section is not defined in the text. Perhaps a reference to some external paper with the relevant theoretical definitions would be sufficient. In Line 282 the variable DeltaS is used to extract Sigma_eff. This is not entirely clear to the reader. Perhaps the authors could provide a clear description of the definition of these quantities and their extraction.

Citation to an more extensive paper on DPS and the pocket formula is included.

We also added/ changed the explenation on the template method in the introduction to be more clear, so that line 282 should be better understood. In the itnroduction the text reads "... a pure DPS signal sample is reconstructed from data by randomly mixing two inclusive single-jet events into an inclusive DPS four-jet event, whereafter it is fitted along with several SPS-only background MC models to the distributions obtained from inclusive four-jet data." We do not yet mention DeltaS here yet but it should be clear from the text that an observable is used in the template method.

- L549-552: it is not clear why the same effective cross-section should be expected at a p-pbar vs pp collider, or at different center-of-mass energies?

The effective cross section is expected to lower with an increasing COM energy (more DPS therefore a lower sigma_eff), however, it is indicatve to compare measurements at different COM energies to see if there is an actual rise or not.

It has been shown that in the inclusive case, the effective cross section is expected to be process independent (https://arxiv.org/abs/1308.6749), we will add a scentence on this in what was line 556 where we introduce the same sign WW and J/Psi measurements, it now reads "It has been shown that \sigmaeff is expected to be process independent for inclusive final states \cite{Seymour:2013sya}, therefore, it is noteworthy that the result from the same-sign WW analysis shows agreement with the results from the \MADGRAPH{}5 and NLO samples. The \cPJgy pair production measurement only shows agreement with the NLO models."

- Line 26: Is it not clear what the total inelastic cross-section refers to. Please rephrase for more clarity.

Changed to "total inelastic cross section of proton-proton interactions".

- Line 50-51: Data are corrected for detector effects by means of an unfolding procedure. Comment: It could be useful to elaborate on the unfolding process a bit or at least say "corrected for detector efficiency and resolution"

We changed the scentence to "Data are corrected for detector efficiency and resolution effects by means of an unfolding procedure.".

- L101-113: this seems to be a generic CMS description, which is not appropriate given that dedicated uncertainties have to be derived (systematics section) to account for the lower-pt jets. It would seem more appropriate to remove those generic statements from here (or if not, minimally comment on if these numbers are valid down to 20 GeV as probed in the analysis).

We have used the generic text, but we altered the numbers so that they do correspond the real uncertainties we derived, however, instead of giving the uncertainty for 30 GeV, we updated the text to go down to 20 GeV.

- L119: define or cite DGLAB

Added the appropriate references.

- Line 142-150: could you mention what ME provider did you use within the HERWIG matchbox?

We use the standard 2->2 LO ME which is already specified in what was line 118-119 of v17 of the paper, it reads "The PYTHIA 8 [36, 37], HERWIG ++ [38], and HERWIG 7 [39] MC event generators use 2 → 2 lead-
ing order (LO) matrix elements"

- Line 139, 165: It is surprising to use such an old PDF set (NNPDF 2.3) for Pythia8, is VINCIA with NNPDF3.1 not available?

It is not.

- Line 238: You state that TUnfold uses SVD, it is based on a least squares fit, as stated in the abstract of the TUnfold paper (https://arxiv.org/abs/1205.6201) "TUnfold is a tool for correcting migration and background effects in high energy physics for multi-dimensional distributions. It is based on a least square fit with Tikhonov regularisation and an optional area constraint."

The text now reads: "The measured distributions are corrected for detector effects with the {TUnfold} program \cite{Schmitt:2012kp,Schmitt:2016orm}, which is based on a lest squares fit and Tikhonov regularization \cite{Tikhonov:1963}. "

- Line 267: disjunct -> disjoint? Does this mean all events are in both samples, and then random events from each sample are combined? Are all combinations of all events used for the final combined sample, or is each event used only once per sample. Please explain more fully how the events are combined.

Disjoint has been adopted in the text.

All events that are in A are per definition in B, meaning that B is the larger of the two samples. In order to avoid any possible correlations, each events in A is randomly matched with one event in B, yielding sufficient statistics. We added the scentence "Each event is only used once in the combination procedure."

- L309-318: Dedicated JES scale factors had to be derived, is it correct that this is not needed for JER?

Yes, that is correct.


- L356: It is not clear what process A vs process B is here?

The processes are the inclusive single jet cross sections with pT cuts of 50 and 30 GeV for A and B respecively, as defined on page 7.

- Line 374-380: Could you be clear about when this normalization is occurring and over which bins it is happening? (With TUnfold it is necessary (?) to unfold before you normalize.)

We changed the text to "In order to make qualitative statements about the shape, the fully corrected distributions have been normalized to one or more bins where a much reduced DPS contribution is expected.", where "fully corrected" should be a clear indication that the normalization is performed after the unfolding procedure, ie it is performed on the generator level distributions.



Type A (editorial) comments:

L6: the probability to find a parton -> the probability to find a given parton

Changed accordingly.

L32: of the gluon density -> in gluon density

Changed accordingly.

L33: "extend" -> "extent"

Changed accordingly.

L36: published in [4-8]. -> published [4-8].

Changed accordingly.

L39: suggest “older” -> “earlier”

Changed accordingly.

L45: Suggest to combine this with L49-50 where the luminosity of this data is provided

Changed accordingly.

L47-48: "This allows not only to mitigate effects of such pileup but also to each down to low jet pT..." -> "This allows not only the mitigation of effects of such pileup but also access to low jet pT..."

Changed accordingly.

L70-72: split sentence into two parts. "The four leading jets are ordered with decreasing p_T. Based on the ...."

Changed accordingly.

L89-91: Perhaps there is a more mathematical word than "flatter" but either way it could be useful to switch the order, first describing the distribution peaking and then saying the other is "flatter".

Switched the order and used "less correlated"

L95: If you are going to refer to the pixel and strip as a single tracker then I think you should say " is a " rather than "are a".

The text is taken from the CMS pubcom twiki.

L111-113: The order should really be switched so that immediately following the jet energy scale you mention the jet energy resolution rather than discussing this additional selection criteria in the middle.

The text is taken from the CMS pubcom twiki.

L120: pT-ordered shower -> pT-ordered parton shower

Changed accordingly.

L121: suggest to add at the end of this sentence “(jointly referred to as HERWIG)”

Changed accordingly.

L133: nnlo not defined (furthermore, looks odd with the different usage “nnlo” vs “LO” throughout)

Defined NNLO and capitalized throughout the text.

L177: factorization and renormalization scale -> factorization and renormalization scales

Changed accordingly.

L193: “For two the…” => “For the two…” ?

Changed accordingly.

L214: "can not" -> "cannot"

Changed accordingly.

L224: Add a comma after "In this last case"

Changed accordingly

L246: "allows to calculate" -> "allows the calculation of"

Changed accordingly

Table 1: (i) what is “upper” vs “lower” here? (ii) add for each of the three parts of the table “(%)” to clarify that these are percentages.

Changed accordingly and explicitly stated what upper and lower are.

L358: what is "pocket"? Is this formula the same as eqn 1? (appears to be jargon, suggest to rephrase)

Changed to eq.(1).

L419: Add a comma after "In this case"

Changed accordingly.

L448: write out w.r.t

Changed accordingly.

L453: section => Sec.

Changed accordingly.

L461: “Fig. 10 and Fig. 11” => “Figures 10 and 11” (at start of sentence, should be written out)

Changed accordingly.

L513: “in 15” => do you mean “in Fig. 15” ?

Changed accordingly.

L516, 518, 520: must clarify what the different uncertainties are (stat vs syst)

Changed accordingly.

Figure 13: (i) for consistency should use “KT” rather than “K” in figure text. (ii) in caption add “(P8)” and “(KT)” similarly as for other figures.

Changed accordingly.

L553: “sqrt(s) equal to” => “of”

Changed accordingly.

Table 5: in the last column “(.syst.)” => “(syst.)” (i.e. there’s a random dot before syst)

Changed accordingly.

L561: "four jet" -> "four jets"

Changed accordingly.

L563: remove “sqrt(s) =”

Changed accordingly.

PHYSICS COMMENTS:

- General: while CMS Style does recommend hyphenating double-parton scattering, it is not really correct. This becomes obvious when you use a hyphen in "single-parton scattering" (as you do), which simply makes no sense! It's not the parton that is single, but the scattering! (Any scattering requires two partons, so "single-parton scattering" is simply nonsense.) For consistency, you should not hyphenate double parton scattering either, as "double" modifies "scattering", not "parton". This affects the title, multiple lines in the abstract, and several occurrences in the main body of the text.

Removed the hyphenation throughout the text.

- Eq. (1): is there a need to use the combinatorial factor that is an integer divided by 2 (m/2 )? Why not simply call t C and define it as 1/2 for identical, and 1 for distinguishable processes?

Historically it is defined in this way, all older papers on DPS use the same definition.

- LL30-31: while the statement about the DPS cross section is correct, the SPS cross section at fixed pT
also rises with the c.o.m. energy. I guess what you want to say is that the DPS rises faster than the logarithmically growing SPS. Please, rephrase the sentence to specify exactly what you mean.

Altered accordingly, the text now reads "The DPS cross section rises faster compared to the SPS cross section with increasing center-of-mass energy at fixed \PT of the jets:"

- L37: vector bosons are no final-state particles; please rephrase as "Signatures involving one or two vector bosons ..."

Altered accordingly.

- L112: give a reference to the noise filtering algorithm [an old CMS PAS].

This is the recommended description provided by the CMS pubcom.

- L113: since you work with low-pT jets, the resolution at 1 TeV is all but irrelevant. Suggest saying: "The jet energy resolution (JER) ranges from 15-20\% at 30 GeV to 10\% at 100 GeV."

In our data we do encounter jets up to 800-1000 GeV, although its only a few instances we thought to give it just to be complete.

- Section 3: add a standard sentence about the triggers.

Added a standard paragraph on the L1 and HLT trigger systems.

- L127 and multiple places in the text: all the tunes should be spelled in normal font, e.g. CUETP8M1. The small-caps font is reserved for computer codes, which tunes are not! Please, go through the paper and replace all the dozens of occurrences.

Font of all tunes has been changed accordingly.

- L129 and multiple places in the text: similarly, PDFs should be spelled in normal font as well, e.g., NNPDF31\_nnlo - again there are a couple of dozens of occurrences throughout the paper, so please fix them all.

All occurences have been fixed.

- L156 and further in the text: standalone MadGraph5 has not been used in Run 2. The name of the program is \MGvATNLO; please, replace all MadGraph5 [and one occurrence of simple "MadGraph" on L437] with \MGv ATNO through the paper.

We have replaced all instances of MadGraph and MadGraph5 by the \MGvATNLO.

- LL165-166: ... the MLM scheme [54] is used to match jets produced via matrix element calculations with those from parton shower, using the matching pT scale of 18 GeV, which ...

Changed accordingly.

- L177: ... and factorization scales are set to the pT

Changed accordingly.

- LL225-226: ... with that of an unbiased reference trigger.

The scentence has been changed to "The trigger efficiency has been determined by comparing the performance of the jet trigger with a minimum bias trigger serving as an unbiased reference trigger."

- L274: the systematic uncertainty here comes out of the blue; please put a forward reference to the following section, where it's discussed.

The four-jet efficiency has been given its own line and the text now reads:

"... The four-jet efficiency has been determined to be equal to

\begin{align}
\epsilon_\mathrm{4j} = 0.324^{+0.037}_{-0.065}{}\mathrm{(syst.)} \label{eq:four-jet-eff}
\end{align}

Where the statistical uncertainty was found to be negligible and the systematic error is detailed in the next section. ..."


STYLE COMMENTS:

Title: suggest dropping "the" on the first line;

Changed accordingly.

Abstract:

LL3-4: for the leading, second-, third-, and fourth-leading jets, respectively.

Changed accordingly.

L4: measured as functions of;

Changed accordingly.

Introduction:

L7: information about the proton;
L21: multiple parton interactions (MPI).
L24: nonidentical [CMS Style];
LL26-27: occurrence of processes A and B.
LL29-30: tow independent pairs, each in the back-to-back configuration.
L38: CMS Collaborations [CMS Style];
L42: CMS Collaboration;
Fig. 1 caption, L5: from an SPS event. [See CMS Style guide on an article before an acronym.]
L46: in 2016, during a special data taking period with a low probability ["run" is jargon!];
L52: studied via the;
LL65-66: of the results, followed by a summary of the paper in Section 9.

All changed accordingly.

Observables:

General: ΔY and ΔS should have Y and S in italics, as variables. This affects multiple places in the paper, so please use search-and-replace to fix them all.
LL83-84: subprocesss, a decorrelation in the;

All changed accordingly.

The CMS detector:

LL96.97: ECAL, HCAL are never used in the paper, so delete the definition of the acronyms;

We have included a short description of ECAL and HCAL, now the acronyms need to be defined.

L93 The CMS detector and jet reconstruction;
LL111-112: to remove those potentially dominated;

All changed accordingly.

Monte Carlo event generators:

L119: at next-to-leading logarithmic level;
LL130,133,138,141,143,147,150,165,169,175,192,195,196,212: PDFs [45]. [You defined PDFs as plural on L6!];
L148: by the CMS Collaboration;
L152: higher-order matrix elements;
LL178-179: The generator-level minimal pT
requirement for the underlying ["cut" is bad jargon];
L181: parton-level event generator;
L183: The generator-level requirement on the pT;
L184: add a comma before "and";
L185: As the pT requirements are;
L191: with the CP5 and CH3 tunes, respectively.
L201: is adopted, as in Ref. [43]. Add a comma before "as it is determined";
L203: same kinematic requirements;
L215: nonperturbative;

All changed accordingly.

Data analysis:

LL219-220: collected in 2016 during a data taking period at low instantaneous luminosity, with;
LL222-223: Offline requirements are imposed to ensure;
LL229-230: defined by selections on jet pT;
LL230-231: to meet pT thresholds of 35,30,25, and 20 GeV. Asymmetric thresholds are chosen;
L233: higher-order calculations;
L235: set of selections is needed;

All changed accordingly.

Extraction of the effective cross section:

L252: laid out in Ref. [22].
L260: The lowest pT single-jet trigger [superlative compound modifiers are not hyphenated];
L274: add a space before "(syst)" and drop the period after "syst" [CMS Style];
L280: σ\rmA and σB [subscripts in Roman];
Eqs. (3-4): don't capitalize "data" in the superscripts;

All changed accordingly.

Systematic Uncertainties:

L295: Systematic uncertainties;
L299: dedicated JES calibration;
L300: in Ref. [34];
L305: the dominant contribution;
L307: of 39 (33)\%.
L309: Jet energy resolution uncertainty [JER is now defined earlier];
L337: Integrated luminosity uncertainty: The uncertainty in the integrated luminosity for data;
LL342-343: \PYTHIA 8 and \HERWIG ++ at both the detector and generator levels, after;
Table 1 caption, L2 asymmetric uncertainties, while;
LL2-3: put commas around "as well as the statistical uncertainty"
L4: between the generator- and detector-level selections is estimated to be 2\%.
Table 2 body, header line: since CMS Style prohibits the use of vertical dividers in the tables, please remove those and center "JES" and "Total" in between the "Upper" and "Lower" sub-headings. In the headings separating three parts of the table, capitalize just the first word, but not the subsequent ones. All the ranges in the table should be typeset with an en-dash, not a hyphen, e.g., 11--39. In the first column, the first four entries should be simply pT,1, etc., per earlier definition.

All changed accordingly.

Results:


L350: two phase space regions defined;
L352: leading jet pseudorapidity [compound modifiers of the type gerund-noun do not require hyphenation, as there is no ambiguity];
Eqs. (5-8): drop periods after "stat", "syst" [CMS Style];
L355: in Section 4;
L357: leading jet pseudorapidity;
L366: larger than the one observed in data.
L370: low-pT jets;
Table 2 body: remove all the vertical dividers;
Fig. 2-3 captions, L2: (upper left); (upper right); LL2-3: (lower left); (lower right); L4: uncertainty in the;
L382: uncertainty in the;
L385: and the MC-to-data ratio;
L389: decorrelated;
L400: Multijet models;
L411: larger than those from data;
Fig. 4 caption, LL4-5: uncertainty in the measurement.
Table 3 body: remove all the vertical dividers;
L443: (as shown, e.g., in the \MGvATNLO ... [don't tell a reader what to do!]
L461: Figures 10 and 11 show; add a comma before "respectively";
Figs. 6-7 captions, L2: add a comma before "and"; (upper left); (upper right); L3: (lower left); (lower right);
Table 4 body: remove all the vertical dividers;
L463: low-pT region;
Fig. 10-11 captions, L2: (upper left) (upper right); LL2-3: (lower left); (lower right) jet.
L472: add a comma before "and";
L476: (shown in Figs. 8 and 9);
L478: (taken from Ref. [43]);
LL481-482: pT-ordered parton shower show a too large decorrelation.
L490: Section 6.
L499: in Section 6 is;
L506: karger decorrelation;
L512: Table 5 and Fig. 14.
L513: in Fig. 15.
L536: ATLAS Collaboration;
L555: in Ref. [12]. Add spaces before the two opening parentheses and delete periods after "stat", "syst"; from the J/ψ meson pair;
L556: in Ref. [17].
L569: While the J$/\psi meson pair production;
Table 5 body: remove all the vertical dividers;

All changed accordingly.

L359: drop the period after "syst";

The four-jet efficiency has now itsequation-line in the text, we refere to the equation instead of giving the value again here.

Summary:

L562: from proton-proton collisions;

Changed accordingly.

Acknowledgments:

Add a standard acknowledgment section, appropriate for long papers.

Added the acknowledgment text for long papers.

References:

Ref. [13]: typeset the addendum as a doi reference.
Ref. [15]: capitalize "Tevatron" in the title.
Ref. [18]: Phys. Lett. B;
Refs. [18,40,41,46,68]: delete "https://doi.org/" from the doi reference.
Refs .[36,37]: replace the two reference to obsolete PYTHIA versions with a single standard PYTHIA 8.2 reference.
Ref. [39]: delete "Herwig is available from ...".
Ref. [40]: Phys. Rep. {\bf 97} (1983) 31.
Ref. [41]: Phys. Lett. B {\bf 83} (1979) 87.
Ref. [46]: Nucl. Instrumentavam. Meth. A {\bf 506} (2003) 250.
Ref. [51]: replace the PAS with the submitted arXiv version.
Ref. [68]: Nucl. Instrum. Meth. A {\bf 372} (1996) 469.

All changed accordingly.

Ref. [16]: the second doi reference must be an Erratum; please mention and typeset as such.

We assumne that you meant Ref.[17], we put the second doi refernce as an erratum here.

L203 - In this case how do you take care of the scaling of the cross section ?

The scaling is handled by PYTHIA 8 internally. The effective cross section is calculated from the non-diffractive cross section sigma_ND = 55.51 mb and the impact parameter (which is in turn determined by tune specific parameters), ie sigma_eff = sigma_ND / <f_impact>. More details are found here http://home.thep.lu.se/~torbjorn/pythia82html/ASecondHardProcess.html

L222 - How do you take care of same event passing two or more different triggers ?

The triggers are used in succession, i.e. in seperate/disjunct regions of the phase space. (in different pT intervals) We have altered the text to be more clear on the subject, the paper now reads "... The online selection of events was based on four single-jet triggers each requiring the presence of at least one jet with a \PT above 30, 50, 80, or 100 GeV, and within $|\eta| < 4.7$. As the triggers have been prescaled, they are used in disjunct \PT ranges. ..."

L274 - The word "found" should be replaced by "estimated" or something similar.

The text reads "... The four-jet efficiency has been estimated to be equal to ... \begin{align} 4-jet efficiencty here \end{align} ..."

L322 - How is this fit performed ? How are the uncertainties on fit parameters determined ?

We have altered the text to "... These weights are obtained by fitting the trigger efficiency curve determined in data using a lest-squares minimization. ..."

Table 1 caption - How is the uncertainty due possible differences between generator and detector
level determined ? Has this been discussed in the systematic section ?

It is discussed on lines 339-348 and in more detail in the AN.

L358 - DPS pocket formula is described by equation 2 ? It is better to reference the equation
number here.

Reference to an equation has been used now isntead.

L367 - "...much larger..." which two values are being compared here ? If you mean MC versus data,
it will be good to mention it here.

The text now reads "... show that the much larger cross section of the MC models compared to the data is due to an abundance of low-\PT jets ..."

L389, L482 - How do you conclude about the de-correlation ? It should be clarified further.

The curves for the Pythia 8 2->2 models are significantly flatter compared to the Herwig models when comparing them to the shape of the data. We think that it is clear from the text as we explicitly mention that we perform a bin-normalization in order to compare the shapes of the different models in a more qualitative way.

We decided not to take an average of all models as that would increase the error drastically and it would not show the clear model dependence which we think to be a central message of the paper. Rather we would like to keep all values in as they are.

Q Marco quoted uncertainties include full syst
A Yes Pierre comments that Model dependence on the unfolding is included,
but not on model itself used to extract sigmaeff.

Q: Paolo For model dependency two models are considered?
A: yes CUET tunes. The model unc. is not the dominant contributor to the x-section, see Sl.14.
JES is the dominant unc. source.

Q: Paolo How merging rate with vertex resolution is estimated? (1mm seems to be large )
A: Vertex resolution estimated from data then z-distribution of vertices used for merging rate.

Q Style comments by Guillelmo
- CMS/lumi/energy text shouldn't touch the frame and be consistent (Figs. 2, 3, 4, 5, 6, 7, 8, 11, 12, 14, 15, 16, 17)
-replace (XXX) by [XXX] both X and Y titles (Figs. 2, 4, 5, 6, 7, 8, 9, 11, 12, 14, 16)
- units should be visible, it is not on the right side of several of them or touching X titles (Figs. 2, 6, 14, 16)
- some of the legends are too large (Figs. 2, 3, 4, 5, 6, 14)

Updated all figures which will be added in the next paper draft.

- Don't we need references for Fig. 10?

References added in the caption of Fig. 10.

Additional comments:

Q Discussion on presentation of results sigmaeff
Pierre: Sl.33 It would be good to get to have opinion of group, whether to produce a combined value here.
Klaus: Given the uncertainties, don’t find the averaging NLO and LO as a good idea. you cannot have a unique CMS number, it’s not trivial to average these numbers.

Decision: Follow up offline.
****************************************************

Q AndreaR General comment. It’s hard to understand and to follow in the paper
where are the DPS contributions. Legends and captions can be made more clear.
****************************************************
ARC Chair Report by Klaus
* Preaproval was about three months ago
* A large amount of results and info is produced and presented
* Paper was reorganised and updated couple of times after pre-app.Authors very well responsive.
* Plotting style also improved a lot
* Most difficult part is to conclude from many MC models and predictions.
* If there are suggestions to improve the presentation of results, please do so.
* Agree that results can be presented in a better way for non-experts.

Comments to physics content: ========================

1. Introduction:

2nd paragraph: "An alternative approach" -> "A second approach" In the end SPS with higher-order pQCD AND DPS both contribute to multijet final states. So these two are not alternatives like one or the other. BTW the line numbering is missing here.

Text has been changed to "a second approach".

l 20: rises with increasing cms energy for fixed pT?

Yes at fixed pT, text has been altered accordingly.

l 46: Data events contain everything including potential DPS effects already. So how can mixed data events provide you with a "pure DPS" signal sample? Did you compare such mixed data events against mixed MC SPS events? You have to better explain why you can do this and that DPS already in data does not bias your templates.

Added more information to the text, along with a reference to a four jet DPS measurement performed by ATLAS where a similar approach has been used in order to construct a signal template.
We made a comparison of the mixed data on detector level with mixed MC samples (Pythia8 and Herwig++ interfaced with the CUET tunes) in the AN in Sec. 8.3. Such plots at generator level can be provided as well, although the general trend will be the same as at detector level.
The fraction of DPS events in the inclusive single jet event subsets is expected to be of percent level, therefore a possible bias will not have a significant impact. Additionally, we opted to use a data-driven method for the construction of the signal template. The alternative is to construct the signal template in a MC driven method which introduces an inevittable MC dependence. The approach has been used for other processes for which the calculations are more precise as for jet production, eg DPS in same sign WW pair boson production and DPS in W+jets.

2. Observables:

Figures 2-4: We have the impression that there are way too many plots already and find these ones the least important. We'll see later on anyway how these distributions look like at multiple occasions. We suggest to drop them. This would also avoid to introduce the KaTie model here with a forward reference to section 4.

We have moved the predictions from KaTie of the observables (the six DPS-sensitive observables, not the pT and eta spectra) to the appendices and put the proper reference there. We believe that it is important to show these plots for clarity and for a proper understanding of all observables. It is not only the shape of the distributions that is important here but the ratioplots that show the DPS sensitive areas. The lines 85-93 have been moved to the discussion on the data analysis, ie the paragraph on the event selection. The two phase space regions are now defined there, with the additional information on the asymmetric cuts

l 58: We advocate to use $region~I$ and $region~II$ whenever referring to the two phase space regions. Subscripts are quite unusual in this context and at first look like typos. For quantities in formulae or such subscripts are ok.

Adapted the $region~I$ and $region~II$ notations.

Moreover, the two regions here come out of the blue. Why using region II at all when it dampens (l.90) the DPS contribution? Maybe one could add here something that is explained only later: "To extract sigma_eff from Delta S ... for reasons of trigger efficiency a second set of pT thresholds is used defining region II."

Added

"Due to the \PT thresholds of the available triggers, the second set of cuts is necessary in order to perform the extraction of the \sigmaeff from the \DS distribution, see Sec.~\ref{sec:extraction}."

4. MC event generators:

It is very good to see that the authors try to explain in detail which MC samples have been used and what characterises them. However, it could be presented in a better way. Maybe you could consider a table?

We have looked at the possibility of a table, however, there are to many varying parameters for the different models that in addition to the table additional specifications have to be made for different models.

At least, if numbers and letters instead of bullets would be used, some references would become easier and some duplication could be avoided. For example, labelling lines 161/2 as "LO I.b)" one could refer to this in lines 190-192 and 197-198 without the need to repeat the same description again.

In earlier stages of the paper (before the arc review) we used a structure with more bullets/numbers which we were advised against. We also think that even though the models are repeated often, it is more clear in this way and will allow for an easier reading of the paper. A lot of text is between the introduction of the models and the discussion of the results.

line 434 in 8.2 Multijet models and later "mixed MadGraph5 " is mentioned a lot, but it is never really defined! Also not in 4.2 and we do not aprreciate much the term "mixed".

The "mixed" has been dropped.

line 189: It should read "scalar sum", not "squared sum", right?

Yes, the error has been corrected.

ll 213-214: What does the phrase in () mean? Unclear, please explain.

The 35-30-25-20 GeV cuts have been placed on parton level, after hadronization the corresponding jets generally obtain a pT that is 5-10 GeV lower compared to their corresponding parton.
We rewrote the scentence more explicit:
"as the cuts are introduced at parton level, the effective thresholds on the resulting hadron level jets are typically 5 to 10 GeV lower"

l 237: It is unclear what "same parameters as before" refers to.

It is generated in the exact same manner as the multijet KaTie sample, ie same generation cuts, renormalization and factorization scales, ...
The text has been altered to:
"..., with the exact same generation parameters as the on-shell \KATIE LO sample from the multijet models."

5. Data analysis:

l 262 "to enable the extraction of sigma_eff" For the 2nd set of pT cuts, see also remark above, please clarify more elaborately the role of the two set of pT cuts and their connection to the inclusive single jet cut regions.

Added the following sentence to connect everything more:

"Inclusive single jet events will be used as the two independent processes A and B, of which the lowest possible \PT threshold is limited by the available triggers to 30~GeV."

ll 271ff: In the last sentences it is unclear what the various "rates" refer to. Please improve.

Changed to:
"..., as well as the probabilities of the occurence of migrations into and out of the phase space for all observables."
Is further explenation about these migrations into and out of the phase space needed? Can be extended to errors the reconstruction, ... if needed.

6. Extraction of sigma_eff:

ll 277ff: -> with "at least" 4 jets Here, for the first time you go beyond the DPS picture from Fig. 1 by including the cases, where the "inclusive jets" from PS #1 contribute 3 jets to your 4-jet selection and the "inclusive jets" from PS #2 only 1 (or vice versa). Please explain more elaborately. Otherwise it will be difficult to follow or to understand eqs. 10 & 11.

Extended the explenation beyond the ideal case where both A and B are dijet processes.

linie after 281; --> "lowest pT single jet trigger"

Added the missing pT.

l 301: "least affected by parton shower effects": Maybe bring this argument earlier to motivate Delta S?

Added a similar argument at the end of the definition of DeltaS:
"As the information of all four leading jets enters the defeinition of the DeltaS observable, it is expected to display the most resilient behavior to effects of the parton shower."

7. --> "Systematic effects" or "... uncertainties"

l 341 and Figs. 17/8 in the AN: Are you sure the uncertainties on the trigger efficiency curves are correct? Are they Wilson or Clopper-Pearson intervals or at least "efficiency" uncertainties? If not please correct and check impact on turn-on points if any.

We are certain that the trigger efficiencies are correct, due to the high prescales of the trigger statistical fluctuations in the trigger refernce method can result in bin above 1. for the triggers with the lowest pT thresholds a prescale of ~100 and larger are used and only a handfull of jets are selected at values larger than 50 GeV for the dijet triggers. The error bars on the bins still overlap with unity in all cases.

l 343ff: "by the models" or "As mentioned previously" is too unclear. Please be more specific!

Models used for the unfolding have been specified here for clarity.

l 361: "reweighting" to what exactly? Has second MC event generator been used to extract the efficiency? This is also a model dependence, no?

Samples are rewighted to the jet multiplicity and the leading jet pT spectrum, now specified in the text.
The same check has been done for the Herwig++ sample and the values of 0.4027 and 0.3921 at generator and detector level resp. were found.
As the differences were of the same order, we overestimated the uncertainty with two percent. To be completely formal we could use the average difference between detector and generator level of the two models.

l 368; "as they are in all calculations." ?? What does this first sentence want to state? Unclear.

It was a misconstrued scentence and was meant to indicate that the correlations are taken into account in all calculations of the cross sections, is now removed as it should be clear from the first part of the scentence.

Table 1 caption: Why is the 4jet eff. "defined" on detector level here? Why do you evaluate an uncertainty of 2% from the difference to Gen Level before, which is not even quoted here? What about the model dependence e.g. when using Herwig?

The 4-jet efficiency is on detector level as it is obtained from the data mixing before they are fully corrected.
The 2% uncertainty is added into the caption, along with the statement that it is included in the calculation of the total uncertainty on the 4-jet efficiency.
The same check has been done for the Herwig++ sample and the values of 0.4027 and 0.3921 at generator and detector level resp. were found. Therefore, we decided to work with a more conservative number of 2%. However I have added the numbers from Herwig to the text.
We also did not iclude this number in the table as here the model uncertainty is due to the unfolding of the observables and the 4-jet efficiency is not unfolded. We tought it would be more cofnusing when putting the number there.

8. Results:

How is result (14) obtained here? Trigger efficiency plays a role here in region I. Please explain.

Added that the cross sections are obtained by integrating the measured differential cross sections in function of the pseudorapidity spectrum of the leading jet and the DeltaS observable for the region I and region II respectively.

l 383/4 "in exactly the ... as above": The reference is unclear. Be more specific please.

Has been replaced by: "The measured differential cross sections have been integrated in order to obtain the inclusive single jet cross sections."

l 410 They are normalised to which bin precisely? Please be more specific.

Specified the bin: "... are normalized to their last single bin, as these..."

The design of all these figures improved dramatically. That is great! Nevertheless:

Figures 5, 6, 10, 11, 15, 16: Do we really need ALL pT_i and eta_i distributions although often they resemble each other?

We left Fig 5 and 6 in the text, the other pT and eta spectra have been placed in the appendix. We would like to argue that they all are of interest. Multiple single inclusive jet studies have been performed in the past, however, inclusive four jet spectra have not been studied this extensively in combination with all the models. It is not only the spectrum of the leading jet that is important but of all four of them.

Figures 7,8,9,12,13,14,17,18,19: Do we really need all the "unnormalised" plots? Isn't the difference in total cross section already visible just from the eta_1 distribution?

We have removed the plots in terms of the absolute cross sections and have adapted the text accrodingly, however, we would like to include these figures as supplementary material to the paper or in the appendices, with the possibility of making them too long, let us continue this discussion further of what you think of this idea.

We believe, an effort should be made by the authors to decide on figures to keep in the main part, to move to an appendix or similar, or to drop altogehter.

See above.

Figures and Tables 2,3,4: You discuss the results in the tables before you discuss the figures => suggest to move the tables to appear before the corresponding figures.

Position of tables and figures have been switched.

Comments to text: =============

Abstract:

l 9: data is -> data are next-to-last: -> centre-of-mass energies

Corrected.

3th -> 3rd.

Corrected.

pb -> pb^-1

Corrected.

The common × 10 factor is very odd.

Rewritten in mu barn instead of pb.

1. Introduction:

l 7: information on the -> information of the l 40: In the absctract it is 0.042 / pb. Please use one number with sufficient, but not more, digits throughout the text, tables and figures.

Corrected and 0.042 / pb will be used hence forth.

(missing line number): Radiation before -> Radiations before

Corrected.

l 43: Maybe better here "MC models" -> "MC event generators"?

Corrected.

l 43: remove quotations for DPS-sensitive

Corrected.

2. Observables:

1st and 4th bullet: "azimuthal angle difference" or "azimuthal angular difference"? A change would need to applied to the text as a whole, of course.

Used "azimuthal angular difference" throughout the text.

l 81: "in the tail" is unclear. "towards low Delta S"?

Corrected.

l 93: distributions --> predictions? (but maybe dropped anyway)

Corrected.

3. The CMS detector:

l 138: Is there a ref. for 13 TeV as well instead of only [34]?

Nothing published, there is an inactive cadi entry for the same study at 13 TeV but without any information:

http://cms.cern.ch/iCMS/analysisadmin/cadilines?line=JME-16-001&tp=an&id=1642&ancode=JME-16-001

5. Data analysis:

l 248: 0.04185 ... remove digits and superfluous "t" Also suggest: --> "... consisted of a series of four single-jet triggers ..."

Changes implemented.

l 253: drop "more robust" otherwise it could not be a reference ....

Changes implemented.

l 260ff: Here and throughout, also in figure captions: Please decide for one type of writing "first, second, third ..." or "1st, 2nd, 3rd, ...". Up to now it's a mixture.

Used "first", "second", ... everywehre.

6. Extraction of sigma_eff:

line after 281; --> "lowest pT single jet trigger"; --> "has a pT threshold of"

Changes implemented.

l 293: There is a superfluous "(stat)"

Removed.

7. --> "Systematic effects" or "... uncertainties"

l 395 "uncertainty of is determined" ??

Removed the "of", was superfluous.

l 372 Missing () around Eq. ...

Parenthesis added to eq number.

Table 1 caption: --> "... uncertainties. The statistical uncertainty ..." "is given" --> "are given"

Rewrote scentence to:
"An overview of systematic uncertainties of all the observables, along with the statistical and the total uncertainties.
All uncertainties are given in percentages."

8. Results:

Table 2 caption: --> "... data and all ..." (similar in table 3, 4) 2nd row last col.: +- super/subscripts missing (same error in table 3, 4)

Changes implemented.

Suggest to have abbreviations "KT" and "PW" for KaTie and POWHEG. P too often means Pythia ...

Altered the legends in the plots to this notation.

Fig. 17 caption: Reference with [??]

Added correct label for the reference.

Fig. 21: lowest panel: the template band is missing

The template band has been scaled with a factor -10 in order to show it on the figure, the legend and caption have been altered accordingly.

l 389, 390: P8, H++, H7 abbreviations are not defined. This is the same for figures. Perhaps defined them in caption as well, similar to l 437, 438?

Defined P8, H++ and H7 in text, and added definition in caption of all figures.

1. I don't quite understand the reason for vetoing events with R_ij < 0.4 when mixing two SPS data events to form DPS template. Supposedly, in a true DPS process, nature doesn't care that there is a jet overlapping. I feel like this veto condition will lead to a formation of template events that bias towards larger number of njets. And it would lead to eff_4j that is higher than the true value. What am I missing?

We implemented the veto condition to avoid the overplapping jets in the detector, we encountered the approach in an DPS measurement from ATLAS [1]. You are correct that in the data this does occer. However, we wanted to avoid a formal re-clustering procedure if this would be possible and if this would not affect the resulting DPS distributions, as such a procedure would have complicated things a lot more. In order to justify the veto condition, we did a naive clustering of the overlapping jets, i.e. recombination of the pT vectors of both jets by a vectorial sum and a recombination of the pseudorapidity of the overlapping jets. Afterwards we compared the shape of the DeltaS _DPS distributions with and without reclustering and found no deviations larger than 3%. Note that only the shape is important as the distribution is rescaled in the template fit. A variation of 3% in the shape of the DeltaS _DPS distribution would not affect the extracted values of f_DPS in a significant way. All of this is detailed, along with the shape comparison plot, in the analysis notes in Sec. 7 (pages 36-37). I hope this clarifies why we included the condition if not please let me know.

[1] https://arxiv.org/pdf/1608.01857.pdf

2. What percentage of events were thrown away due to this veto condition?

Approximately 10% of all events are lost (veto passing rate = 0.89408), however not the amount of events is important, but rather the change in shape of the DeltaS _DPS distribution due to the removal of these events. In the AN we have compared the DeltaS distribution with veto and with naive clustering and found a relative difference in shape of at most 3%, see the answer above.

3. What would happen if you don't require the veto condition and let the jet merge into a single jet (post re-clustering) and then recompute eff_4j? (i.e. njets will be slightly smaller, and some portion of events will have higher pt for the merged case) Does/would it change significantly? If so, should this be included as systematics?

The difference between the eps_4j values is of sub percent level. Below you can find the exact amount of events passing the veto and surviving selection, as well as the epsilon_4j values. Note that the events are weighted, the totals are therefore no whole numbers. The values also differ lightly from the one in the paper as I used a smaller sub-sample to check these numbers, as it is quite a long calculation when done on the full sample.

NO veto condition with clustering
Total events = 1545194.36937
Total events passing the veto = 1545194.36937, veto-passing rate = 1.0
Total events of selected DPS events = 498202.640429
epsilon_4j = 0.322420693671

veto condition with NO clustering
Total events = 1546323.65758
Total events passing the veto = 1383170.53586, veto-passing rate = 0.894489668501
Total events of selected DPS events = 440271.956622 ,
epsilon_4j = 0.318306344161

4. Is there some comparison between this data-merged template of DPS vs. simulated DPS samples somewhere?

In the result section 8.4 we compared the distributions for the DeltaS _DPS distribution for the pure DPS Pythia 8 + CP5 and onshell KaTie + CP5 models.

5. I am having a hard time understanding Equation 10. Equation 10 does not seem to reduce to DPS formula Equation 1 in Section 1. I was naively expecting the equation to reduce to Equation 1 with two processes being Eq 8 and Eq 9, but with the factor of eff_4j at the front. I feel like I am missing something there...

The reason for the two terms in eq. (10) is due to the symmetry factor m in eq. (1) and due to the cuts of the inclusive single jet processes A and B. Whenever both processes are identical, the symmetry factor takes a value equal to 1, and 2 otherwise. Therefore the symmetry factor is not unambiguously defined for the processes A and B as they become identical whenever the pT of the leading jet surpasses 50 GeV. Therefore we define two disjunct processes sigma_A and (sigma_B-sigma_A) that cover the whole phase space for which the symmetry factor is clearly defined. Eq.(1) is then rewritten as
sigma_eff = (m/2) * (eps_4j/sigma_eff) * (sigma_A* (Sigma_A + (Sigma_A-Simga_B)))
The m becomes equal to 1 for the first term and equal to 2 for the second term.

6. Why was ∆S chosen for the variable to extract f_DPS and not ∆phi_soft? Based on Figure 3, it seemed DPS is much more pronounced in ∆phi_soft. If I am reading this correctly it's 2:1 between SPS and DPS at high ∆phi_soft And it is 4:1 between SPS and DPS for ∆S.

Historically we used the same 35-30-25-20 GeV cuts for all observables. When studying the migrations for the unfolding procedure we noticed that the matrix of migrations was non-diagonal for the DeltaS observable. We had to increase the cuts to obtain a stable unfolding for the DeltaS observable due to the jet energy resolution at low pT. We then started comparing the data with different models and saw that the DeltaS observable, which has been used in many DPS studies before for the extraction of the sigma_eff, was the most stable. In the sense that it was least affected by effects from different parton showers, the order of the calculations ect. All models, except the KaTie models, undershoot the shape of the slope, leaving room for a DPS contribution. Therefore we decided to continue with the DeltaS observable as it showed to be the most robust variable with regards to DPS effects.

Lastly, I am curious what the answer was to one of the questions from other ARC member

- While double parton scattering would imply at least 4 jets, it may be > discussed in the paper why exactly 4-jet events were studied. In
> principle, one
> of the scatterings could even produce 3 jets probably, so multiple
> combinations
> would be possible altogether. -> DISCUSSED DURING MEETING ALREADY

To clarify: we do not select exactly four jet events, we perform a study on inclusive 4-jet events, which has been adapted in the title and throughout the paper for the next version. We also take into account that 3 or 1 jets instead of 2 can originate from a DPS events by using the inclusive single jets spectra to construct a DPS sample. Generally one is interested in the production of two dijet pairs in DPS as they can exhibit very different correlations compared to SPS. An inclusive 5-jet, ... state would unnecessarily complicate things even more.

General comments:

- From the data vs model comparision plots it seemed that basically all models are away by multiple sigmas from the data, and this is not mentioned in the abstract or in the summary. -

Most models are LO models, based on collinear factorization and the DGLAP equations for the parton evolution. It is expected/known that these models fail at low transverse momentum, where cross sections larger than the data are generally obtained. At high transverse momentum a better agreement is observed, as expected.

- Figure axis are not properly labeled in many cases, especiall w.r.t. units.

Figures will be updated.

- Introduction in general is a little bit short for such an extensive paper (where there are no length constraints).

Going into detail on the DPS cross section would take us to far we believe, however a figure depicting SPS vs DPS events will be included in the introduction with some explenation in the caption.

- Momentum is in GeV throughout the paper, probably its fine, but it could be mentioned that the "/c" is dropped.

We have never encountered this before and do not think it is necessary to explicitly state the use of natural units.

- Repetitive statements in captions could be shortened, e.g. "Error bars and bands are shown similarly to Fig. 4"

Updated the captions as suggested throughout the paper.

- In what sense are multijet models multijet compared to the non-multijet models (as all contain at least 4 jets in the end, I assume)? This is I guess trivial, maybe it is the 2->2 vs 2->3,4 matrix elements (also, later on, these are called multi-leg models, if I understand well), but it could be mentioned to educate some of the readers.

The multijet models have a combination of higher order MEs and/or NLO corrections, which allow for more than 2 jets to come from the ME and thus becoming less dependent on the parton shower in order to create an inclusive 4-jet event. I will add a line to the introduction of the multijet events in section 4.2.

Section 1
- Maybe some more details in paragraphs 1&2 would be useful to explain more about DPS? Also, a figure could be placed here (as from a non-expert point of view).

Going into detail on the DPS cross section would take us to far we believe, however a figure depicting SPS vs DPS events will be included in the introduction with some explenation in the caption.

- While double parton scattering would imply at least 4 jets, it may be discussed in the paper why exactly 4-jet events were studied. In principle, one of the scatterings could even produce 3 jets probably, so multiple combinations would be possible altogether.

It is an inclusive 4-jet measurement, the title has been changed. We take into account that 3 jets can be produced in one event (see the section on the extraction strategy). Generally one is interested in the production of two dijet pairs in DPS as they can exhibit very different correlations compared to SPS. An inclusive 5-jet, ... state would unnecessarily complicate things even more.

Section 2
- Why are [18,19] not referenced in the first paragraph?

Both use a different variable in the study of DPS and none of the observables we have used.

Section 3
- It may not be customary, but if CMS is discussed, a plot of the detector could be considered to be shown, maybe with how particles are detected in it. (Even if ref. [35] gives a lot of details.)

We do not think this is necessary, it is not customary to include a plot of the detector.

- Does jet reconstruction choice (e.g. radius) contribute to systematics?

The anti-kT reconstruction algorithm with a cone radius of 0.4 is the preference of CMS. A different choice would result in differences in the distributions, however this would be a phenomenological study that would take us too far. However, the performance of the different algorithms has been discussed in older papers which are cited.

Section 5
- The online selection of jets (pT thresholds) seems to be more restrictive than the offline cuts. It is not fully clear to me how this is reconciled, i.e. how are the events in the sample e.g. with the leading jet having a pT of 35 or 50 GeV, when the online cut was already 100 GeV?

The triggers are used in disjunct areas of the phase space determined by the leading jet pT (no overlap between the triggers, the details and the trigger ranges are given in the AN). The cuts are not applied individually but on the four jets at a time. and event with transverse momenta of eg 65-50-40-15 will fire the trigger with a pT threshold of 30 GeV, however the set of cuts of 35-30-25-20 GeV are not met.

- This section seems to be very short, one could consider unifying it with the next one?

The section outlines the details of the data analysis while the next gives an overview of the extraction procedure, I fear that merging them would make things more confusing and take the focus away of what is important in each section.

Section 6
- A very minor thing, but mentioning TFractionFitter seems a bit too technical here. I understand it is built upon or mentioned later many times, nevertheless, I would consider discribing it instead of mentioning the object (that may change in the future).

TFractionFitter has implemented a binned likelihood method, and has been described this way in the paper now.

Section 7
- Surely there are other sources of systematic uncertainty, these may be negligible, but this could be mentioned, such as "There are other sources of systematic uncertainty (e.g. jet reconstruction, thresholds, offline cuts), but these are negligible as compared to the listed sources."

We do not view the jet reconstruction as a systematic uncertainty, a different jet cone radius would yield different distributions but should yield the same physics, eg the same value for the sigma_eff parameter. The thresholds and offline cuts are rather phase space restrictions than systematic uncertainties, therefore they are also not mentioned in the section of the systematic uncertainties.

- Are all systematic uncertainties fully correlated in pT, eta, phi, etc? This does not seem to be mentioned.

Added "The different cross sections are extracted from the corresponding rapidity spectra taking correlations into account, as they are in all calculations." to the paper.
If it would be insufficient, I could ellaborate more.

Section 8
- Two missing references (Tab. ??) in 8.1 and 8.3

Proper references inserted.

Abstract
- space missing in "important. Models"

Space added between the words.

- "better description" has to be better than something in line 13

Added "... compared to standard leading-order models."

- centre-of-energies instead of center-of-mass energies

Change included.

Title
- 4-jet production -->> Inclusive 4-jet production

Title updated.

1 Introduction
- page 1 par 2 line 2 : perhaps constant instead of factor

Change included.

- page 1 par 2 line 6 : remove however in An alternative approach, however, ...

Change included.

- page 1 bottem page : INCLUSIVE four jets

Change included.

- page 2 par 2 line 2 : "obtained from data" should be reformulated, not completely clear

Sentence has been reformulated.

2 Observables
- page 2 separation data driven and phenomenological -->> earlier measurements and phenomenological studies

Change included.

- page 2 eq 3 : drop last inequal to i or cross check with other papers

Other papers do not have the last inequality, will be dropped from hence on forth.

- page 3 DeltaS : more flat -->> flatter

Change included.

- page 3 pT selections : define a label for the different cuts to avoid captions shorter and make paper more readable readable

The two sets of cuts are hence on forth denoted as p_T = (35,30,25,20) GeV and p_T^DeltaS = (50,30,30,30) GeV.

- page 3 Mention explicitly the advantage of having asymmetric cuts + elaborate on different sets of cuts for DeltaS

More detailed explenation + references are now included justifying the choice of asymmetric over symmetric pT cuts.

- page 3 Figures : units on the axes, include GeV and pb instead of pb/eta

Change included.

4 Monte Carlo event generators
- page 8 : Pythia 8.240 first point, mention Geant somewhere

Mentionned GEANT 4 with reference.

- page 9 : par 2 : PowHeg factorization and renormalization scale is set at 10GeV is this the way of handling?

Checked in the publication: for PowHeg NLO 2->2 (dijet production) the transverse momentum of the underlying Born configuration is used and not set to 10 GeV. Paper has been updated.

- page 9 : Title 4.3 and title 8.3 not consistent, total is not best description perhaps SPS+DPS

Both titles have been updated and subsequentally all references to the "total" samples have been replaced with "SPS+DPS" samples.

5 data analysis
- page 10 par 2 : correction as function of the jet pT and PS, mention trigger efficiency and how it is derived

Mentionned reference trigger method, i.e. trigger in question is compared to a more robust reference trigger (minimum bias trigger in our study).

- page 10 par 4, line 3 : to avoid instead of combat

Change included.

- page 10 par 4 : rate of misses and fakes, is slang + should be defined, a better description of fake and miss rate is necessary

Now using "into and out of phase space migration". Explicitly mentionned the high rates in the lowest bins of the pT spectra of the four leading jets and that these contributions also are the main reason why the rates of the other observables are this large. The size can also be viewed from the fact that 4 jets are selected simultanously thus the chance of the event being viewed as a fake or miss increases as only one of the four jets needs to be a fake or miss to classify the event this way.

- In the AN : plot eta fig 41 : include only the higher pT part, remove lowest pT bins for AN for a measure of comparison

6 extraction of effective cross section
- page 11 par 2 : add to veto condition that it is for a spatial matching only and not for anti kT as it works with the rapidity instead of pseudorapidity

Added: "... whenever two or more jet axes SPATIALLY coincide.", can reformulate further if this is not clear.

- page 11 epsilon_4j : rewrite systematic error and mention that statistical is negligible

Updated systematical and statistical uncertainty. Also updated the epsilon_4j rates determined at generator and detector level in section 7 to the same amount of number as the efficiency from data.

- formula 10 : merge paragraph above formula together with "Taking into account ..."

Paragraphs merged for clarities sake.

7 Systematics
- page 12 vertex uncertainty : 0.1404 --> 0.14 too many digits

Change included.

- page 12 luminosity uncertainty : 2.5% valid for low PU runs?

Checked the 2016 luminosity measurement results (LUM-17-001), no explicit mention of low pile up conditions and wether or not an exception should/could be made. Can contact the Lumi POG conveners in this regard if needed.

- table : backslash not commenly used, longs dash or N.A., apply to all other tables

All tables updated.

- epsilon_4j : mention that stat unc is negligible, only three/four digits

Change icluded.

8 Results
- page 14 sentence 1 : phase domains should be phase space domains

change included.

- page 14 8.1, line 3 : table question marks

Reference updated.

- page 14 last paragraph : better wording for jet cone bump

A more descriptive formulation has been given: "the tail of the distributions, which is dominated by effects of the jet cone size".

- page 19 line 1 : either well described or not well described, not marginally well

Changed the wording

- table 2 : DeltaS cross section not an good label, should use something that indicates the different pT selection, see previous comment on the different labels for pT cuts

Updated the cross section according to the previously introduced labels for the different sets of pT cuts.

Fig 8 : Find better solution to presenting these results (too many lines), eg ATLAS publications : multiple ratios + only one line per model in the top plotor specify bands for groups of models

The Pythia+Herwig and MultiJet figures have been updated to double and triple ratio plots. Will be included in next version of the paper.


8.4 paragraph 2
instead of jet configuration remark a more general comment

Made the comment more general by stating that the differences originate from the differences in pT specta, which lie at the root of the different jet configuration rates.

Fig 18+19: caption: last word -->> plot instead of pot

Corrected.

Comments from pre-approval (10 July 2020)

Link to pre-approval presentation (can also be found on cadiline):

https://indico.cern.ch/event/937383/contributions/3938177/attachments/2072396/3479320/preAppTalk_DPS.pdf

- Patrick: You are using TFractionFitter. Did you double-check with home-made implementation? TFractionFitter seemed to get uncertainties wrong

Yes also made alternative chi2 implementation and were consistent. TFractionFitter was more stable and seems better.

- Guillelmo: Default generator used to report the unfolding ?

Unfolding with Pythia and Herwig independently, and the average of the two is used. Then the half-difference is used as an uncertainty.

- Guillelmo: Sounds unusual, is that common practice?

yes, used in other papers

- Guillelmo: but this is not dominant uncertainty

No JES is mostly the dominant uncertainty

- Guillelmo: what does <21% syst mean ?

it is the maximum impact among all bins

- Guillelmo: would be better to have a range

Ok will update the syst table

- Gabor: final results plots s26 Could you explain again the model dependence ?

Sigma_eff extracted for different models. Have several different options for ME & PS.

- Gabor: Differences coming from the interplay of the data and the model fit

yes

- Gabor: which was the model used for 7 TeV ?

Custom UE tune using Professor to achieve best Data/MC agreement not exactly used now, but consistent with the PY8 results.

Q Gabor s7 veto condition on overlapping jets (DR 0.4) do you create any complicated acceptance effect with that ?

Cross checked with a different reclustering condition and obtained stable results. Could check stability vs the veto condition requirement cut.

Q Gabor Syst table s11. Very large ranges. Would be better to explain better why the syst change so much and what they depend on

OK Will provide that on the AN

- Gabor: s15 right plot. What are the two deviating models ?

The two blue are PY8 SPS CP5 & SPS CUETP8M1; pt-ordered showers from PYTHIA don’t describe angular correlation between jets. DPS effecs hidden because of large Parton shower dependence

- Gabor: Does this show up in the final sigma_eff ?

No because those effects appear to cancel in DeltaS, more robust for extraction of sigmaeff, while the Phi_ij is more sensititive to parton shower.

- Henning: About KaTie prediction. Generation cuts for the jets pT. Are they the same as the offline cuts ? Is that not a problem ?

It seems to work fine with that setup. Compared parton level and particle level, can have migration effects. Explained a bit also in the paper.

- Ela: uncertainty of 2% of 4jet efficiency ?

Based on a gen/reco level study, difference was 1% but doubled it to be conservative and cover it well.

- Henning: vertex uncertainty ? Very small. It is based on a very old study.

Will check if a more recent study was done. Should not have much impact anyhow.

Comments by Salvatore Rappoccio (April 03, 2020)

Incorporation feedback ongoing as of April 07, 2020.

Paper draft:
=========

- Please find a CCLE ASAP. The grammar needs considerable improvement, and there is a lot of jargon. It's hard to follow the paper for this reason.

Pierre Van Mechelen will be the CCLE

- Title: I suggest "Study of double parton scattering in proton-proton collision events with 4 jets with low transverse momentum at sqrt(s) = 13 TeV " or something similar.

Title has been updated.

- The organization of the paper is a bit disjointed, with material scattered throughout different parts in a nonuniform way. - Material is repeated often, which can be condensed, especially in the Results "section".

The overall order of the paper has been revised and as soon as Pierre (the CCLE) is done reviewing the text, we will upload the new version.

- The abstract has too much technical detail, and needs a better description of the overall procedure. Rules of thumb include that if you mention something in the abstract, the introduction should explain it in more detail (or the conclusions). For example, the "pocket formula" is mentioned and then not explained until page 24.

Abstract has been rewritten, new version of the paper wil be uploaded ASAP.

- The definitions and discussions of the various "sigmas" is inscrutable. They are mentioned in the abstract, so presumably important, but not even mentioned until page 6 in the MC description, and then only obliquely. The other sigmas (sigma_DeltaS, sigma^DS_A,B, sigma_A, sigma_B, etc) are not defined at all. They are defined only in the AN, which needs to be put into the paper.

The definitions have been clarified and are introduced gradually and in a more logic way as before.

- The Results section is 2/3 of the paper. Much of the information is repeated, or should be moved earlier in the paper. The descriptions were rather hard to read at some points, so a CCLE would help a lot also. The descriptions are too detailed. You don't need to describe every bin of every plot, and some more summary information is appropriate here. Try to condense this into a few specific, targeted physics conclusions. On that note, see the next point.

See previous answers.

- The figures could use some more guidance to the readers, like adding (either in the caption or on the figures themselves) where the DPS enhancement is. This is critically important especially since you are normalizing to non-DPS regions.

Added indication of where the DPS sensitive area is.

- Section 8.6 and onward are indecipherable to me. I don't know what you're doing here. The overall picture needs to be explained much better. This is not mentioned at all in the introduction, the methodologies are not discussed, and the connection between the various pieces are unclear. I'm sorry, I cannot review from 8.6 onward in any meaningful way, I'm going to need a clearer paper draft to understand what is done.

The section has been restructured together with the whole paper, will be updoalded in the latest version (v3).

Physics comments:
=======

MC samples: Paper line 179-193: The generator cuts for KaTie are very close to the offline selection. Has this been checked to ensure there are no biases?

This was checked by Hans for KaTie and Madgraph (with a 2 to 4 ME, which we do not have in our final analysis).

Response matrices: AN: Overall Figures 28-34 look extremely non-diagonal. I'm a bit concerned about the matching criteria. If you look at Fig 28, there are several large populations along eta_det = 0 and eta_gen=0. These are almost 10% of the diagonal elements, which seems enormous to me. To me this suggests some mis-association in the matching procedure. Do you have 1-d plots of the delta R between the detector and generator level jets? Also, the "R_pt" in Equation X is set to 0.3, which seems extremely large (especially at high pt) given Figure 18, where the resolutions are much smaller. I'm not convinced the pt matching is appropriate here, and may be allowing in the wrong jets. My suggestion is to adjust the matching criterion to be eta-phi matching for all of the observables. Even if they are technically correlated with the unfolded distribution it may be necessary to ensure you're picking the right jets. The recommended matching is to use half of the jet radius. Incidentally, the color scheme makes it nearly impossible to understand this. You should use a monochromatic scheme so that differences in color are meaningful. As of now, you're seeing 10-40% off-diagonal elements, from what I can tell. The fraction of fakes and misses are also enormous at low pt. (Incidentally we also usually quote this in terms of purity and stability, not fakes and misses).

I will update the migration matrices to a monochromatic scheme for clarity.

In chapter 7 we do match jets with a R_pt of 0.3 in the determination of the resolution. The matching procedure here is not related to the one we perform during the unfolding. Here the mathing is done to determine the resolution and find an optimal binning.

We started the unfolding without any jet matching, the migration matrices in Fig. 28 are those of the rapidity spectra without any matching. The migration matrices show a large background due to the fact that the first leading jet on detector level not always corresponds to the first leading jet on generator level for example. Therefore we have implemented a spatial matching, detailed in section 8.2.2.1 of the AN. The matching ensures that the same jets are compared to each other on detector and generator level and effectively erasing the background in Fig. 28, resulting in Fig. 30 for the rapidity spectra.

The pT spectra show non-diagonal behavior, however the bin values off the diagonal are multiple orders smaller as the diagonal elements. While the pT dependent observables, especially the Delta S observable, show non diagonal behavior due to the bad resolution of low pT jets. The only observable for which a background remains is the difference in azimuthal angle between the most remote jets, which is due to its definition, as explained in the AN.

I can provide purity and stability plots if needed. I have left them out, as they are originally defined for square migration matrices and as the TUnfold package uses different detector and generator level binning.

MC statistical uncertainties: AN Table 5: You're using the flat QCD samples for the measurement, which have relatively low statistics. Are you using the jackknife resampling technique for the MC uncertainties in the response matrices?

We do not use the jackknife resampling techique as these are not the standard flat QCD samples provided by CMS, they have a pTHat range of 15-7000 and a different way of oversampling (FlatP6). We requested these two samples for the derivation of the JEC originally, with a pTHat of 15-1000 and with the Flat oversampling with the goal of have more statistics in the low pT region. Roughly a third of all events (3.5 million) survive the analysis cuts of the samples.

PT distribution: Paper Figure 1: I'm not really understanding why the pt distribution is so far off, given that you reweight the samples via the AN Figure 25 histograms, which show almost perfect agreement, as do AN Figure 42. I'm confused what you're doing here. Also, are you applying nonperturbative corrections? I don't recall the inclusive jet measurement seeing such large differences in the pt spectrum but perhaps I'm wrong (it's hard to tell from their paper, the figures are small).

In the unfolding procedure we have reweighted the samples to obtain a more accurate description of the migrations. When discussing the results, we show the original samples, without the reweighing, compared to the unfolded data. In the inclusive jet measurement paper they indeed obtain smaller differences in the pT spectrum. However the discrepancies between the data and the models grows when going from an inclusive jet selection to 2 ,3 and eventually 4 jets, these distributions at detector level we have checked in the past.

Regularization: AN Table 7: The regularization conditions you're quoting seem enormous. Is this tau? If so, shouldn't this be a number much less than unity?

This is an error on my part, the title of the columns Reg.Conc. and cond(A) seem to have been switched. The numbers are the condition numbers of the migrations matrices, while the column with Size/Der./Curv. is the choice of the regularization condition in the unfolding procedure.