[GEN Check Script] Adding a number of events check for pLHE

[DickyChant]

Recently we noticed that in production there could be requests mistakenly requesting more events than presented private LHE events. See [1] for an example.

Without a check we (GEN and PdmV) could not effectively identify this issue... We should brainstorm on how we could add this check.

1. The most straightforward way would be reading the whole files, though it would be very heavy workload
2. @vlimant proposed to skip events until the final one 1k event with a cmsDriver command.

More ideas are welcomed for sure.

[bbilin]

Hi,

I chose a random folder with many files: WJets sample at 8 TeV mcdbid: 5497.

The simple
[bbilin@lxplus9110 5497]$ grep "</event>" *lhe |wc -l

Returned the nevents =107350000 in a timely manner. This can hence indeed be included in the script to check nreq<= n available.

Cheers,

B.

[DickyChant]

I think one of the issue with this solution is however that we could have ".xz" compressed LHE files :(

[bbilin]

ah, at least vim was working on them, we can find a way out :)

[bbilin]

google helped. xzgrep works.

[DickyChant]

Look forward to a PR to close this then ;)

[agrohsje]

Just a reminder that vim created spurious copies in the past and screwed the system. Keep also in mind when running locally.

[bbilin]

yes, it causes problems in the event generation. We had to remove those swp files. (I am not using vim by default :) )

[DickyChant]

I forget to have [1]: https://its.cern.ch/jira/browse/HIGHPRIOREQ-825 ...