[WIP] MET finetuning #320
Draft
[WIP] MET finetuning #320
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Update README.md * Update README.md * Update README.md * document running script * md5sum * up * clean
* May 2024 training
* fix validation scripts * dqm script * fccee_cld postprocessing * update submission script * update validation script * finalize dqm part * add data link
* enable onnx export via dynamo with dynamic shapes * added standalone export script * fp16 quantization sort of works also * use sdpa * MultiheadAttention op runs * update timing study * cleanup * model closes * update timing study * onnx is factorized * update onnx script * revert main model code * move to notebook
* update validation for cmssw 14 * it's running * update dqm for cmssw 14 * update with link * update recipe * added runtime plot notebook
…nd vbf to training (jpata#330) * update cmssw plots, add ttbar sample to valid * update validation notebook * disable ray for now * update README [skip ci] * remove DQM part [skip ci]
… add per-particle ispu flag (jpata#332) * generate ttbar nopu events * up * update postprocessing * small sample generation * v3_1 run * updates for CMSSE 14 generation * [skip ci] cleanup postprocessing * [skip ci] update pu gen * update postprocessing with new truth definition based only on caloparticles * remove pdb, switch genjet to energy * [skip ci] prepare for v3_3 * [skip ci] fix flag * added time and mem limits * pu files from scratch * 20240702_cptruthdef submission * ttbar nopu v2 * up * added genjet, genmet to clic postprocessing * remove delphes * update tests * add postprocessing jobs * update torch * update dataset version * propagate genjets, genmet * shared memory error * training on v2.0.0 for cms * fix occasional root file load bug * add jmenano * fix qq * clic training * up
* CMS training instructions
* remove correct dir * update samples * add samples * fix supervised key * resubmit training * add missing * add mpgun * plt target and gen separately * separate submission scripts * add stats * fix softmax bug * add CLD sample * add CLD sample * add finetuning script * add genjob script
* chore: update raytune search space, utils and startscript * fix: raytune deprecated env var for storage_path Also add num samples to draw in HPO as cmd line arg * chore: update clic config file for jureap57 * feat: script to build python env from scratch * chore: update startscripts for raytrain and raytune * fix CMS model path for ACAT2022 * MLPF datasets v2.0.0: track pythia-level genjets, genmet in datasets; add per-particle ispu flag (jpata#332) * generate ttbar nopu events * up * update postprocessing * small sample generation * v3_1 run * updates for CMSSE 14 generation * [skip ci] cleanup postprocessing * [skip ci] update pu gen * update postprocessing with new truth definition based only on caloparticles * remove pdb, switch genjet to energy * [skip ci] prepare for v3_3 * [skip ci] fix flag * added time and mem limits * pu files from scratch * 20240702_cptruthdef submission * ttbar nopu v2 * up * added genjet, genmet to clic postprocessing * remove delphes * update tests * add postprocessing jobs * update torch * update dataset version * propagate genjets, genmet * shared memory error * training on v2.0.0 for cms * fix occasional root file load bug * add jmenano * fix qq * clic training * up * CMS training instructions (jpata#336) * CMS training instructions * Update pyg-clic.yaml * Update pyg-clic.yaml * fix: black formatting * Enable CI/CD test of HPO workflow * fix: typo in test script --------- Co-authored-by: Joosep Pata <[email protected]>
… classifier (jpata#340) * dataset relabeling
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implements a training script that performs MET finetuning using a pre-trained backbone MLPF.