diff --git a/.buildinfo b/.buildinfo new file mode 100644 index 0000000..0a8dd87 --- /dev/null +++ b/.buildinfo @@ -0,0 +1,4 @@ +# Sphinx build info version 1 +# This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done. +config: 0c2971fc4724fb237d485a5cbc2526b1 +tags: 645f666f9bcd5a90fca523b33c5a78b7 diff --git a/.github/scripts/yamlheader.html b/.github/scripts/yamlheader.html new file mode 100644 index 0000000..bc1df6e --- /dev/null +++ b/.github/scripts/yamlheader.html @@ -0,0 +1,269 @@ + + + + + + + <no title> — Key4hep documentation + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ + + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/CONDUCT.html b/CONDUCT.html new file mode 100644 index 0000000..8eb0db9 --- /dev/null +++ b/CONDUCT.html @@ -0,0 +1,284 @@ + + + + + + + Contributor Code of Conduct — Key4hep documentation + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Contributor Code of Conduct

+

As contributors and maintainers of this project, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities.

+

We are committed to making participation in this project a harassment-free experience for everyone, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, or religion.

+

Examples of unacceptable behaviour by participants include the use of sexual language or imagery, derogatory comments or personal attacks, trolling, public or private harassment, insults, or other unprofessional conduct.

+

Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct. Project maintainers who do not follow the Code of Conduct may be removed from the project team.

+

Instances of abusive, harassing, or otherwise unacceptable behaviour may be reported by opening an issue or contacting one or more of the project maintainers.

+

This Code of Conduct is adapted from the Contributor Covenant, version 1.0.0, available at https://contributor-covenant.org/version/1/0/0/.

+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/CONTRIBUTING.html b/CONTRIBUTING.html new file mode 100644 index 0000000..03f8a3f --- /dev/null +++ b/CONTRIBUTING.html @@ -0,0 +1,318 @@ + + + + + + + Contributing — Key4hep documentation + + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Contributing

+

starterkit-lessons is an open source project, and we welcome contributions of all kinds:

+
    +

  • New lessons;

    • +

  • Fixes to existing material;

    • +

  • Bug reports; and

    • +

  • Reviews of proposed changes.

    • +
+

By contributing, you are agreeing that we may redistribute your work under these licenses. +You also agree to abide by our contributor code of conduct.

+
+

Getting Started

+
    +

  1. We use the fork and pull model to manage changes. +More information about forking a repository and making a Pull Request.

    2. +

  2. To build the lessons please install the dependencies.

    3. +

  3. For our lessons, you should branch from and submit pull requests against the master branch.

    4. +

  4. When editing lesson pages, you need only commit changes to the Markdown source files.

    5. +

  5. If you’re looking for things to work on, please see the list of issues for this repository. +Comments on issues and reviews of pull requests are equally welcome.

    6. +
+
+
+

Dependencies

+

To build the lessons locally, install the following:

+
    +

  1. starterkit-ci

    2. +
+

Then build the pages:

+
$ starterkit_ci build --allow-warnings
+$ starterkit_ci check --allow-warnings
+
+
+

and start a web server to host them:

+
$ cd build
+$ python -m http.server 8000
+
+
+

You can see your local version by using a web-browser to navigate to http://localhost:8000 or wherever it says it’s serving the book.

+
+
+
+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/LICENSE.html b/LICENSE.html new file mode 100644 index 0000000..9a68e6c --- /dev/null +++ b/LICENSE.html @@ -0,0 +1,328 @@ + + + + + + + Instructional Material — Key4hep documentation + + + + + + + + + + + + + + + + + + + + +
+ + +
+ +
+
+
+ +
+
+
+
+ +
+

Instructional Material

+

All instructional material is made available under the Creative Commons +Attribution license. The following is a human-readable summary of +(and not a substitute for) the full legal text of the CC BY 4.0 +license.

+

You are free to:

+
    +

  • to Share — copy and redistribute the material in any medium or format

    • +

  • to Adapt — remix, transform, and build upon the material for any +purpose, even commercially.

    • +
+

The licensor cannot revoke these freedoms as long as you follow the license +terms.

+

Under the following terms:

+
    +

  • Attribution — You must give appropriate credit, provide a link to the +license, and indicate if changes were made. You may do so in +any reasonable manner, but not in any way that suggests the licensor endorses +you or your use.

    • +

  • No additional restrictions — You may not apply legal terms or +technological measures that legally restrict others from doing anything the +license permits.

    • +
+

Notices:

+
    +

  • You do not have to comply with the license for elements of the material in +the public domain or where your use is permitted by an applicable exception +or limitation.

    • +

  • No warranties are given. The license may not give you all of the permissions +necessary for your intended use. For example, other rights such as publicity, +privacy, or moral rights may limit how you use the material.

    • +
+
+
+

Software

+

Except where otherwise noted, the example programs and other software provided +by Software Carpentry are made available under the OSI-approved MIT +license.

+

Permission is hereby granted, free of charge, to any person obtaining a copy of +this software and associated documentation files (the “Software”), to deal in +the Software without restriction, including without limitation the rights to +use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies +of the Software, and to permit persons to whom the Software is furnished to do +so, subject to the following conditions:

+

The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software.

+

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE.

+
+ + +
+
+ +
+
+
+
+ + + + \ No newline at end of file diff --git a/_downloads/37854d19817c792316d481f5beb93cc7/LICENSE.md b/_downloads/37854d19817c792316d481f5beb93cc7/LICENSE.md new file mode 100644 index 0000000..1998afa --- /dev/null +++ b/_downloads/37854d19817c792316d481f5beb93cc7/LICENSE.md @@ -0,0 +1,64 @@ +# Instructional Material + +All instructional material is made available under the [Creative Commons +Attribution license][cc-by-human]. The following is a human-readable summary of +(and not a substitute for) the [full legal text of the CC BY 4.0 +license][cc-by-legal]. + +You are free to: + +* to **Share** --- copy and redistribute the material in any medium or format +* to **Adapt** --- remix, transform, and build upon the material for any + purpose, even commercially. + +The licensor cannot revoke these freedoms as long as you follow the license +terms. + +Under the following terms: + +* **Attribution** --- You must give appropriate credit, provide a [link to the + license][cc-by-human], and indicate if changes were made. You may do so in + any reasonable manner, but not in any way that suggests the licensor endorses + you or your use. + +* **No additional restrictions** --- You may not apply legal terms or + technological measures that legally restrict others from doing anything the + license permits. + +Notices: + +* You do not have to comply with the license for elements of the material in + the public domain or where your use is permitted by an applicable exception + or limitation. +* No warranties are given. The license may not give you all of the permissions + necessary for your intended use. For example, other rights such as publicity, + privacy, or moral rights may limit how you use the material. + +# Software + +Except where otherwise noted, the example programs and other software provided +by Software Carpentry are made available under the [OSI][osi]-approved [MIT +license][mit-license]. + +Permission is hereby granted, free of charge, to any person obtaining a copy of +this software and associated documentation files (the "Software"), to deal in +the Software without restriction, including without limitation the rights to +use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies +of the Software, and to permit persons to whom the Software is furnished to do +so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + +[cc-by-human]: https://creativecommons.org/licenses/by/4.0/ +[cc-by-legal]: https://creativecommons.org/licenses/by/4.0/legalcode +[mit-license]: https://opensource.org/licenses/mit-license.html +[osi]: https://opensource.org diff --git a/_downloads/46b8dc461fb4104140b72e4043aaeefe/CONDUCT.md b/_downloads/46b8dc461fb4104140b72e4043aaeefe/CONDUCT.md new file mode 100644 index 0000000..922ba82 --- /dev/null +++ b/_downloads/46b8dc461fb4104140b72e4043aaeefe/CONDUCT.md @@ -0,0 +1,13 @@ +# Contributor Code of Conduct + +As contributors and maintainers of this project, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities. + +We are committed to making participation in this project a harassment-free experience for everyone, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, or religion. + +Examples of unacceptable behaviour by participants include the use of sexual language or imagery, derogatory comments or personal attacks, trolling, public or private harassment, insults, or other unprofessional conduct. + +Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct. Project maintainers who do not follow the Code of Conduct may be removed from the project team. + +Instances of abusive, harassing, or otherwise unacceptable behaviour may be reported by opening an issue or contacting one or more of the project maintainers. + +This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org/), version 1.0.0, available at [https://contributor-covenant.org/version/1/0/0/](https://www.contributor-covenant.org/version/1/0/0/). diff --git a/_images/key4hep_logo1.svg b/_images/key4hep_logo1.svg new file mode 100644 index 0000000..52093c9 --- /dev/null +++ b/_images/key4hep_logo1.svg @@ -0,0 +1,167 @@ + + + + + + + + + + image/svg+xml + + + + + + + K + + + + + Y + E + HEP + 4 + + + + diff --git a/_images/key4hep_logo2.svg b/_images/key4hep_logo2.svg new file mode 100644 index 0000000..956c304 --- /dev/null +++ b/_images/key4hep_logo2.svg @@ -0,0 +1,607 @@ + + + + + + + + + + + + + + + + + + + + + + + image/svg+xml + + + + + + + + + + Key4HEP + + + + diff --git a/_images/key4hep_logo3.png b/_images/key4hep_logo3.png new file mode 100644 index 0000000..3194eb6 Binary files /dev/null and b/_images/key4hep_logo3.png differ diff --git a/_images/key4hep_logo6.svg b/_images/key4hep_logo6.svg new file mode 100644 index 0000000..819f427 --- /dev/null +++ b/_images/key4hep_logo6.svg @@ -0,0 +1,125 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 4hep + Key + + diff --git a/_images/key4hep_logo6p1.svg b/_images/key4hep_logo6p1.svg new file mode 100644 index 0000000..95fd45c --- /dev/null +++ b/_images/key4hep_logo6p1.svg @@ -0,0 +1,164 @@ + + + + + + + + + + + + + + Key + + + + + + + + + + + + + + + + 4hep + + + + + + + + + + + + + diff --git a/_sources/.github/scripts/yamlheader.md.txt b/_sources/.github/scripts/yamlheader.md.txt new file mode 100644 index 0000000..a37817e --- /dev/null +++ b/_sources/.github/scripts/yamlheader.md.txt @@ -0,0 +1,8 @@ +--- +jupyter: + kernelspec: + display_name: Python 3 + language: python + name: python3 +--- + diff --git a/_sources/CONDUCT.md.txt b/_sources/CONDUCT.md.txt new file mode 100644 index 0000000..922ba82 --- /dev/null +++ b/_sources/CONDUCT.md.txt @@ -0,0 +1,13 @@ +# Contributor Code of Conduct + +As contributors and maintainers of this project, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities. + +We are committed to making participation in this project a harassment-free experience for everyone, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, or religion. + +Examples of unacceptable behaviour by participants include the use of sexual language or imagery, derogatory comments or personal attacks, trolling, public or private harassment, insults, or other unprofessional conduct. + +Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct. Project maintainers who do not follow the Code of Conduct may be removed from the project team. + +Instances of abusive, harassing, or otherwise unacceptable behaviour may be reported by opening an issue or contacting one or more of the project maintainers. + +This Code of Conduct is adapted from the [Contributor Covenant](https://www.contributor-covenant.org/), version 1.0.0, available at [https://contributor-covenant.org/version/1/0/0/](https://www.contributor-covenant.org/version/1/0/0/). diff --git a/_sources/CONTRIBUTING.md.txt b/_sources/CONTRIBUTING.md.txt new file mode 100644 index 0000000..12cd184 --- /dev/null +++ b/_sources/CONTRIBUTING.md.txt @@ -0,0 +1,64 @@ +# Contributing + +[starterkit-lessons][repo] is an open source project, and we welcome contributions of all kinds: + +* New lessons; +* Fixes to existing material; +* Bug reports; and +* Reviews of proposed changes. + +By contributing, you are agreeing that we may redistribute your work under [these licenses][license]. +You also agree to abide by our [contributor code of conduct][conduct]. + +## Getting Started + +1. We use the [fork and pull][gh-fork-pull] model to manage changes. + More information about [forking a repository][gh-fork] and [making a Pull Request][gh-pull]. + +2. To build the lessons please install the [dependencies](#dependencies). + +2. For our lessons, you should branch from and submit pull requests against the `master` branch. + +3. When editing lesson pages, you need only commit changes to the Markdown source files. + +4. If you're looking for things to work on, please see [the list of issues for this repository][issues]. + Comments on issues and reviews of pull requests are equally welcome. + +## Dependencies + +To build the lessons locally, install the following: + +1. [starterkit-ci](https://pypi.org/project/starterkit-ci/) + +Then build the pages: + +```shell +$ starterkit_ci build --allow-warnings +$ starterkit_ci check --allow-warnings +``` + +and start a web server to host them: + +```shell +$ cd build +$ python -m http.server 8000 +``` +You can see your local version by using a web-browser to navigate to `http://localhost:8000` or wherever it says it's serving the book. + +[conduct]: CONDUCT.md +[repo]: https://github.com/lhcb/starterkit-lessons +[issues]: https://github.com/lhcb/starterkit-lessons/issues +[license]: LICENSE.md +[pro-git-chapter]: http://git-scm.com/book/en/v2/GitHub-Contributing-to-a-Project +[gh-fork]: https://help.github.com/en/articles/fork-a-repo +[gh-pull]: https://help.github.com/en/articles/about-pull-requests +[gh-fork-pull]: https://reflectoring.io/github-fork-and-pull/ + + +```eval_rst +.. toctree:: + :hidden: + + CONDUCT.md + LICENSE.md +``` diff --git a/_sources/LICENSE.md.txt b/_sources/LICENSE.md.txt new file mode 100644 index 0000000..1998afa --- /dev/null +++ b/_sources/LICENSE.md.txt @@ -0,0 +1,64 @@ +# Instructional Material + +All instructional material is made available under the [Creative Commons +Attribution license][cc-by-human]. The following is a human-readable summary of +(and not a substitute for) the [full legal text of the CC BY 4.0 +license][cc-by-legal]. + +You are free to: + +* to **Share** --- copy and redistribute the material in any medium or format +* to **Adapt** --- remix, transform, and build upon the material for any + purpose, even commercially. + +The licensor cannot revoke these freedoms as long as you follow the license +terms. + +Under the following terms: + +* **Attribution** --- You must give appropriate credit, provide a [link to the + license][cc-by-human], and indicate if changes were made. You may do so in + any reasonable manner, but not in any way that suggests the licensor endorses + you or your use. + +* **No additional restrictions** --- You may not apply legal terms or + technological measures that legally restrict others from doing anything the + license permits. + +Notices: + +* You do not have to comply with the license for elements of the material in + the public domain or where your use is permitted by an applicable exception + or limitation. +* No warranties are given. The license may not give you all of the permissions + necessary for your intended use. For example, other rights such as publicity, + privacy, or moral rights may limit how you use the material. + +# Software + +Except where otherwise noted, the example programs and other software provided +by Software Carpentry are made available under the [OSI][osi]-approved [MIT +license][mit-license]. + +Permission is hereby granted, free of charge, to any person obtaining a copy of +this software and associated documentation files (the "Software"), to deal in +the Software without restriction, including without limitation the rights to +use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies +of the Software, and to permit persons to whom the Software is furnished to do +so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + +[cc-by-human]: https://creativecommons.org/licenses/by/4.0/ +[cc-by-legal]: https://creativecommons.org/licenses/by/4.0/legalcode +[mit-license]: https://opensource.org/licenses/mit-license.html +[osi]: https://opensource.org diff --git a/_sources/call-for-logos/README.md.txt b/_sources/call-for-logos/README.md.txt new file mode 100644 index 0000000..55fb526 --- /dev/null +++ b/_sources/call-for-logos/README.md.txt @@ -0,0 +1,26 @@ +# Call for logos + +Logo 1 + +![](key4hep_logo1.svg) + +Logo 2 + +![](key4hep_logo2.svg) + +Logo 3 + +![](key4hep_logo3.png) + +Logo 4 + + + +Logo 5 + + + +Logo 6 + +![](key4hep_logo6.svg) +![](key4hep_logo6p1.svg) diff --git a/_sources/contents.md.txt b/_sources/contents.md.txt new file mode 100644 index 0000000..60e58c0 --- /dev/null +++ b/_sources/contents.md.txt @@ -0,0 +1,32 @@ +# Key4hep + + +```eval_rst +.. toctree:: + :maxdepth: 3 + :includehidden: + :caption: Contents: + + setup-and-getting-started/README.md + k4marlinwrapper/doc/starterkit/k4MarlinWrapperCLIC/Readme.md + k4marlinwrapper/doc/starterkit/k4MarlinWrapperCLIC/edmConverters.md + k4marlinwrapper/doc/starterkit/k4MarlinWrapperCLIC/howtoMultithread.md + k4marlinwrapper/doc/starterkit/k4MarlinWrapperCLIC/CEDViaWrapper.md + k4simdelphes/doc/starterkit/k4SimDelphes/Readme.md + developing-key4hep-software/README.md + spack-build-instructions-for-librarians/README.md + talks-and-presentations/README.md + call-for-logos/README.md + CONTRIBUTING.md + +.. toctree:: + :maxdepth: 2 + :includehidden: + :caption: External links: + + FCC software + CLIC software + ILC software + CEPC software +``` + diff --git a/_sources/developing-key4hep-software/CMakeBuild.md.txt b/_sources/developing-key4hep-software/CMakeBuild.md.txt new file mode 100644 index 0000000..c2a2993 --- /dev/null +++ b/_sources/developing-key4hep-software/CMakeBuild.md.txt @@ -0,0 +1,97 @@ +# Building Key4hep using CMake: For Developers + +A common problem when developing is needing to build multiple packages on top of +the stack to test changes in several repositories. One way of doing this is +using Spack but there is an experimental way using cmake. + +First, source the releases or the nightlies as usual. Then, run the following +command: + +``` +/cvmfs/sw-nightlies.hsf.org/key4hep/experimental/setup.py pkg1 pkg2 +``` + +Where `pkg1` and `pkg2` are the packages that will be cloned (can be empty). On +top of that, `setup.py` checks for folders (or symbolic links) in the current +directory that contain `CMakeLists.txt`. This can be useful if we want to use a +local version and don't need to clone some repositories. After running the +command, a CMakeLists.txt file will appear. We may have to edit it manually if +we are using packages that are not recognized or are in a different organization +than the predefined ones. The information that we'll need to edit is most likely +one of the first lines at the top: + +``` +set(pkgs EDM4hep k4FWCore) +``` +that sets the packages that will be built in their right build order. +and the individual `FetchContent_Declare` entries for each package. + +After we are happy with the CMakeLists.txt file, it's important that we set the +new environment variables: + +``` bash +mkdir install +cd install +export PATH=$PWD/bin:$PATH +export LD_LIBRARY_PATH=$PWD/lib:$PWD/lib64:$LD_LIBRARY_PATH +export ROOT_INCLUDE_PATH=$PWD/include:$ROOT_INCLUDE_PATH +export PYTHONPATH=$PWD/python:$PYTHONPATH +export CMAKE_PREFIX_PATH=$PWD:$CMAKE_PREFIX_PATH +cd .. +``` + +Then, we can run the usual commands for building: + +``` +mkdir build +cd build +cmake .. -DCMAKE_INSTALL_PREFIX=../install +make -j N install +``` + +and it will build all the packages at the same time, so it may take some time. Symbolic links are +created in the build directory for each package that allow us to run `ctest` like: + +``` bash +cd k4FWCore +ctest -j 2 +``` + +in case we are building k4FWCore. + +For the cloned directories, in case we want to clone a different commit or +branch from the master/main branch, this can be done by editing the +CMakeLists.txt + + +## Possible issues + +Not everything has been tested so it's likely some things won't work. For +many packages there are recent changes that enable these builds so using older +versions probably won't work. + +## Packages that have been tested + +The following packages have been built together successfully so it should be +possible to build any combination of them: +- podio +- EDM4hep +- k4FWCore +- LCIO +- ILCUTIL +- LCCD +- GEAR +- Marlin +- k4EDM4hep2LcioConv +- k4MarlinWrapper + +## Changing to a different commit or branch + +If we don't want to build the master or main branch of the repositories we are +cloning, we have two options: + +- Set the commit or branch we want in the `CMakeLists.txt` in the corresponding + `FetchContent_Declare` entry by changing `GIT_TAG` to whatever commit or branch + we want to checkout. +- Go to the source directory, which can be found in the build directory under + `_deps/-src` and checkout manually whatever commit or branch you want. diff --git a/_sources/developing-key4hep-software/Key4hepCMakeGuide.md.txt b/_sources/developing-key4hep-software/Key4hepCMakeGuide.md.txt new file mode 100644 index 0000000..bbfdb70 --- /dev/null +++ b/_sources/developing-key4hep-software/Key4hepCMakeGuide.md.txt @@ -0,0 +1,131 @@ +# CMake guide for Key4hep software + +## Overview + +CMake is a tool for building software, which has become the de-facto +standard outside HEP. In HEP, it is for example used by the ILC/CLIC +communities and by the LHCb collaboration. For CMS people, CMake is the +equivalent of scram. + +## Quick start into building Key4hep Software + +### Set up the environment + +The first step is adding all dependencies to the bash environment. +In case you are unsure, it is best to use the default init script, provided on CVMFS for Centos7: + +``` +source /cvmfs/sw.hsf.org/key4hep/setup.sh +``` + +Note that mixing setup scripts (from another package, for example) may or may not work as intended - more likely not. +For any requests to changes in the environment, feel free to contact the software team on the mailing list or any other channels. +Developers may also look into `spack` to have more fine-grained control over the build dependencies. + + +### Running CMake + +* Create a build directory: `mkdir build; cd build` +* Run CMake in the build directory: `cmake .. ` +* Change any cmake options by rerunning cmake.\ + For example: `cmake .. -DCMAKE_INSTALL_PREFIX=../install -DCMAKE_BUILD_TYPE=Debug`.\ + Tools like ccmake may also be useful: `ccmake ..` +* Compile the software, using all the cpus available: ```make -j `getconf _NPROCESSORS_ONLN` ``` +* Install by running `make install` +* In case any dependency is changed, most likely you need to remove all the contents of the build folder and rerun cmake and the compilation. + +### Using your local installation + +In order to run the code you just installed, there are a few environment variables to set up (assuming the installation directory is the working directory): + +``` +mkdir build; cd build +cmake .. -DCMAKE_INSTALL_PREFIX=../InstallArea +make install +cd ../InstallArea +``` + +* `export CMAKE_PREFIX_PATH=$PWD/:$CMAKE_PREFIX_PATH` + - in order to use this installation as a dependency for other packages +* `export PATH=$PWD/bin/:$PATH` + - in order to make any executables available on the command line +* `export LD_LIBRARY_PATH=$PWD/lib:$PWD/lib64:$LD_LIBRARY_PATH` + - in order to make libraries available for linking and for the plugin systems in Gaudi/DD4hep +* `export PYTHONPATH=$PWD/python:$PYTHONPATH` + - in order to make libraries available for linking and for the plugin systems in Gaudi/DD4hep +* `export ROOT_INCLUDE_PATH=$PWD/include:$ROOT_INCLUDE_PATH` + - in case the package builds ROOT dictionaries +* `export =$PWD/share/` + - some packages distribute data files that are found with a special environment variable, usually this is the package name in all caps. + - e.g. `export K4SIMDELPHES=$PWD/share/k4SimDelphes/` + +## CMake example packages + +Colin provides [a few simple CMake +examples](https://github.com/cbernet/cmake-examples). They are helpful to understand the basics of +CMake. + +Get these packages: + + git clone https://github.com/cbernet/cmake-examples.git + cd cmake-examples + +Follow the instructions in +[README.md](https://github.com/cbernet/cmake-examples/blob/master/README.md). + +## Changing the CMake Configuration + + +When adding new source files to a package, the CMake build system needs +to be made aware of them. Usually `CMakeLists.txt` contains a wildcard +expression that adds all implementation files in a subfolder, e.g. +`src/*.cpp` , so there is no need to explicitly add the names of the +new files. To update the list of files, it is fastest to run +`make configure` . + +Note that when changing the name of a property of an algorithm or a +tool, `make` (and not only `make packagename` ) needs to be run for +Gaudi to be aware of the change. + + +### Runtime Environment + +Key4hep packages, in particular the gaudi framework components, consist of executables, headers, scripts, dynamic libraries, xmls and special files describing gaudi components. +In order to use these, some environment variables need to be set. + +Gaudi also offers the possibility to set up the environment via the `xenv` command. This is done by simply prefixing the command you want to run with the `run` script in the top level directory of FCCSW, or directly in the build directory. + +```bash +./build/run key4run Examples/options/pythia.py +``` + +Sometimes it is convenient to run FCCSW directly from the binaries in the build directory without installing them. +This can be done by using the `run` script in the build directory, or setting the environment variables as in `setup.sh` for the build folder. +Note that the directories in the build folder differ a bit. Mostly it is important the the LD_LIBRARY_PATH is pre-fixed with the library directories. The fccrun command should pick up the components from the build folder then. + + + +## CTest in Key4hep + +Key4hep also uses CMake for integration tests. +They are added with `add_test()` and can be run with `make test` in the build folder. For Gaudi packages, the environment should be set so they can be run also in a build environments, see https://github.com/HEP-FCC/k4Gen/blob/main/k4Gen/CMakeLists.txt + + + +### Customizing how CMake is run + +An environment variable is used to forward command line arguments to the cmake command, for example to run cmake with the `trace` option: + +``` +CMAKEFLAGS='--trace' make +``` + +{% callout "How do I check compilation flags?" %} + +Instead of running ` make` , run: + + ```bash + make VERBOSE=1 + ``` + +{% endcallout %} diff --git a/_sources/developing-key4hep-software/Key4hepSoftwareGit.md.txt b/_sources/developing-key4hep-software/Key4hepSoftwareGit.md.txt new file mode 100644 index 0000000..db9baae --- /dev/null +++ b/_sources/developing-key4hep-software/Key4hepSoftwareGit.md.txt @@ -0,0 +1,193 @@ +# Github workflow and contribution guide + +## Overview + +This page should allow users that are new to Git to get started with Key4hep software, and describes the workflow for accessing and contributing code. + +For a general introduction to git, have a look at these tutorials: + +- [Atlassian tutorial](https://www.atlassian.com/git/tutorials/) +- [Interactive tutorial](http://pcottle.github.io/learnGitBranching/) +- [The git book](https://git-scm.com/book/en/v2) + +## First time setup of git + +Please refer to [this tutorial](https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup) and the [GitHub help](https://help.github.com/articles/set-up-git/). + +## Generate and set up ssh keys for github + +We recommend to clone github repositories via SSH, especially if you want to +contribute code. For this to work, you need to generate ssh keys for +authentication. See the corresponding github +[help-page](https://help.github.com/articles/generating-an-ssh-key/). + +{% callout "Generate and set up ssh keys for github" %} + + If you only want to use the software it may be easier to use https. In that case you don't need to generate the keys but have to replace `git@github:` with `https://github.com/` in all the instructions. Note that you'll not be able to push to your repository when you are on lxplus. You can also start using https for now and later re-add your repository with ssh authentication, see the [trouble shooting section](#trouble-shooting). + +{% endcallout %} + + +## Improving your git experience + +It may be useful to install [Git integration tools](https://github.com/git/git/tree/master/contrib/completion) for your shell that allow tab-completion of most git commands and also can show you in your prompt on which branch you currently are, what changes you have pending, etc. + +## Development workflow + +For any key4hep repository (taking `k4SimDelphes` as an example), you will be using (at least) 3 sources: + +1. The official [Key4hep repository on github](https://github.com/key4hep/k4SimDelphes) +2. Your fork of the repository (see [github help](https://help.github.com/articles/fork-a-repo/) on what that means) +3. Your local repository in your work area (e.g. on AFS) + +The repositories 1 and 2 are added as remote to the repository 3: + +```bash +git clone git@github.com:[YOUR_GITHUB_USER]/k4SimDelphes.git # create a local copy (3) of your fork (2) +cd k4SimDelphes +git remote add upstream git@github.com:key4hep/k4SimDelphes.git # add official repo (1) as additional remote +``` + +### Keeping your local repository up to date + +- fetch all changes from the official repository (1) + + ```bash + git fetch --all --prune + ``` + +- rebase your development area to the master branch from the official repository (1), **please read [this](https://www.atlassian.com/git/tutorials/rewriting-history) to avoid loss of work** + + ```bash + git rebase -i upstream/main + ``` + + in this process you can also fix any commits that need touching up, **be aware that deleting commits in the list will result also in the deletion of the corresponding changes** (more info in the [GitHub help](https://help.github.com/articles/about-git-rebase/) and the [Atlassian tutorial](https://www.atlassian.com/git/tutorials/rewriting-history)) + +- push your local changes to your fork (2), see [below](#contributing-code) how to create a local branch + + ``` + git push origin [NAME_OF_LOCAL_BRANCH] + ``` + +### Contributing code + +- if you are fixing a bug, first create an issue in the github issue tracker. +- develop your feature in your local copy (3) on a local branch of your choice, to create a branch do: + + ``` + git branch -b [NAME_OF_LOCAL_BRANCH] + ``` + +- refer to [this tutorial](https://www.atlassian.com/git/tutorials/saving-changes) to see how to commit changes +- occasionally, get new code from the official repository (1) as explained above and merge it in this branch +- test: + - that the code compiles and all tests succeed (`make && make test`) + - that your code runs (even better: [add an automatic test] + - that it produces the expected results +- push your local branch to your fork (2) (see [above](#keeping-your-local-repository-up-to-date)) +- create a pull request from your fork (2) to the offical repository's (1) master branch (see github [help-page](https://help.github.com/articles/creating-a-pull-request/)) + - also see the [recommendations for pull requests](#pull-requests) + +## Recommendations + +Please always follow the recommendations below. + +### General recommendations + +- if you're working on a given topic, always create a branch for + it, e.g. `pythia_interface`. You may commit many times to this branch + in your local repository. When you have something solid create a + pull request to the official repository. + +### Commit comments + +- feel free to commit often to your local repository, make a pull request once the topic you are working on is finished + - if the feature you are working on is large, consider making a work in progress-pull request (see [below](#pull-requests)) + - git commits represent a snapshot of the software as a whole, and not only the difference to a previous commit (although that as well, in practice). It is recommended that each commit compiles and passes the tests. Take a look at the [commit history of a key4hep repository](https://github.com/key4hep/k4simdelphes/commits/main) and the histories of some individual files to find both good and bad examples. + +- always provide a meaningful comment for each commit + - if you are working on an issue, refer to that issue by adding "refs. #[issue id]", see also + [GitHub help](https://help.github.com/articles/closing-issues-via-commit-messages/) +- commit comments should look like the one below, so that they show up + correctly in git printouts. + + ``` + first version of a pythia interface # this line should be a short 1 liner + + Here, you may write a few more lines if needed + ``` + +### Cleaning history + +- before opening a pull request it may be a good idea to check that your history makes sense (commit messages explain what you did, no unnecessary commits, etc.), check with: + + ``` + git log + ``` +- if you see commits that you'd like to change, there are several ways of doing that, the most commonly used is `git rebase`: + - with the interactive version you can rebase your development branch to the official master and fix the history at the same time + + ``` + git fetch upstream # get changes from the official repo + git rebase -i upstream/main # do the actual rebase + ``` + + - git will guide you through the steps, where you can delete entire commits (and the corresponding changes), merge commits and change commit messages + - more information can be found in [this tutorial](https://www.atlassian.com/git/tutorials/rewriting-history#git-rebase-i) + +### Pull requests + +- Give a meaningful title that appropriately describes what you did (e.g. Add new calorimeter clustering) + - Pull requests of work in progress (to make people aware that you are working on a feature) create a PR starting with "[WIP]" +- In the description, give a short bullet-point list of what was done +- If your pull request fixes issues tracked in the issue tracker: + - Make sure you added a test that shows they are actually fixed + - In the description mention that you fixed it by referring to the issue: "fixes #" (this will automatically close the issue, see also [GitHub help](https://help.github.com/articles/closing-issues-via-commit-messages/)) + +## Trouble-shooting + +### When I try to push to the repository, I get an authentication error + +Check with `git remote -v` which remote repositories you have added to your local copy. You should see something like: + +``` +upstream git@github.com:key4hep/k4SimDelphes.git (fetch) +upstream git@github.com:key4hep/k4SimDelphes.git (push) +origin git@github.com:[your git user name]/k4SimDelphes.git (fetch) +origin git@github.com:[your git user name]/k4SimDelphes.git (push) +``` + +If you see something similar but all the addresses start with `https`, see [below](#i-have-cloned-with-https-and-now-i-cant-push-my-changes-what-do-i-do). + +If you only see `origin git@github.com:key4hep/k4SimDelphes.git`, you need to add your own repository, push to that one and do a pull request, as described above. To add your own repository do: + +``` +git remote rename origin hep-fcc +git remote add myfccsw git@github.com:[your git user name]/FCCSW.git +``` + + +### I have cloned with https and now I can't push my changes, what do I do? + +You only need to change the URL of your remote pointing to your repository to one that uses SSH instead: + +``` +git remote set-url [the remote name] git@github.com:[your git user name]/k4SimDelphes.git +``` + +Now you can push to that remote with: + +``` +git push [the remote name] [the branch you want to push] +``` + +--- + +## Need help? + +In case you have any questions on this guide, or need help to sort out +an issue with a repository, feel free to drop a mail to +key4hep-sw at CERN, and we'll be happy to help you. +Alternatively create an issue in the [bug tracker](https://github.com/key4hep) +![smile](https://twiki.cern.ch/twiki/pub/TWiki/SmiliesPlugin/smile.gif "smile"). diff --git a/_sources/developing-key4hep-software/README.md.txt b/_sources/developing-key4hep-software/README.md.txt new file mode 100644 index 0000000..b7ac22b --- /dev/null +++ b/_sources/developing-key4hep-software/README.md.txt @@ -0,0 +1,18 @@ +# Developing Key4hep + +Sooner rather than later you will find it necessary write code for Key4hep. These pages cover some software development topics to remove any friction. + + + +```eval_rst +.. toctree:: + :caption: Contents: + + Key4hepSoftwareGit.md + Key4hepCMakeGuide.md + WritingAlgorithms.md + SpackForDevelopers.md + +CMakeBuild.md + +``` diff --git a/_sources/developing-key4hep-software/SpackForDevelopers.md.txt b/_sources/developing-key4hep-software/SpackForDevelopers.md.txt new file mode 100644 index 0000000..bf0fdea --- /dev/null +++ b/_sources/developing-key4hep-software/SpackForDevelopers.md.txt @@ -0,0 +1,266 @@ +# Building Key4hep using Spack: For Developers + +Using spack to develop software is somewhat pushing its intended usage to its limits. +However, it is not impossible and this is an area of spack that is currently under active development. +Unfortunately, this also means that the spack documentation might not be fully up-to-date on these topics. +Hence, this page tries to collect some of the experiences the Key4hep developers have made. + +```{tip} +To obtain and setup `spack` take a look at {doc}`/spack-build-instructions-for-librarians/spack-setup`. +``` + +## Developing a single package + +When only developing on a single package it is possible to use the [`dev-build`](https://spack.readthedocs.io/en/latest/command_index.html#spack-dev-build) command of spack. +A brief tutorial can be found in [the spack documentation](https://spack-tutorial.readthedocs.io/en/latest/tutorial_developer_workflows.html). +There is also a dedicated channel on slack [spackpm.slack.com](https://spackpm.slack.com) (to get an invitation, visit [slack.spack.io](https://slack.spack.io)) where questions regarding the development workflow can be discussed. +It allows to build a given package directly from local sources in the same way as spack does it, and even makes this package available to other packages in the same way it does packages that have been installed by spack directly. +Here we will use [LCIO](https://github.com/iLCSoft/LCIO) as an example since it can be installed without (or with only one) dependency. + +As a first step let's have a look at what installing `lcio` with spack would entail. +Note that we explicitly disable the ROOT dictionaries in order to limit the number of dependencies +```bash +spack spec -Il lcio ~rootdict +``` +``` +Input spec +-------------------------------- + - lcio~rootdict + +Concretized +-------------------------------- + + - vdwx2aq lcio@2.16%gcc@9.3.0~examples~ipo~jar~rootdict build_type=RelWithDebInfo cxxstd=17 arch=linux-ubuntu20.04-skylake +[+] utzbuq7 ^cmake@3.16.3%gcc@9.3.0~doc+ncurses+openssl+ownlibs~qt patches=1c540040c7e203dd8e27aa20345ecb07fe06570d56410a24a266ae570b1c4c39,bf695e3febb222da2ed94b3beea600650e4318975da90e4a71d6f31a6d5d8c3d arch=linux-ubuntu20.04-skylake +[+] pljbs5a ^sio@0.0.4%gcc@9.3.0+builtin_zlib~ipo build_type=RelWithDebInfo cxxstd=17 arch=linux-ubuntu20.04-skylake + +``` + +In this configuration `lcio` has only two dependencies, `sio` and `cmake`, which are both already installed in this case. +If these dependencies are not yet installed, spack will automatically install them for you when using the `dev-build` command. + +### Installing a local version with `dev-build` + +In order to install a local version of LCIO with spack, first we have to clone it into a local directory +```bash +git clone https://github.com/iLCSoft/LCIO +``` + +Now we can install this local version via +```bash +cd LCIO +spack dev-build lcio@master ~rootdict +``` +This should install `lcio` and all dependencies that are not yet fulfilled, giving you the full output of all the build stages ending on something like the following +``` +... +==> lcio: Successfully installed lcio-master-7dovpqn3kscbg672ham5wcqro7lg45gh + Fetch: 0.00s. Build: 1.62s. Total: 1.62s. +[+] /home/tmadlener/work/spack/opt/spack/linux-ubuntu20.04-skylake/gcc-9.3.0/lcio-master-7dovpqn3kscbg672ham5wcqro7lg45gh +``` + +Note, that it is necessary to specify a single concrete version here for `lcio`. We use `@master`. +This version has to be one that is already available for the package (use `spack info` to find out which ones are available) and cannot be an arbitrary version. +It also does not necessarily have to correspond to the actual version of the source code you are currently installing. +However, it is of course encouraged to use a meaningful version specifier, since this package should also be useable as desired by dependent packages. + +### Using the local version as dependency + +Now that the local version has been installed, it would of course be nice to be able to use it in downstream packages as well. +As far as spack is concerned, a package that has been built from local sources is not really different from one that has been built from automatically downloaded sources. +The main difference is that the fact that it has been built from local sources manifests in the spec +```bash +spack find -lv lcio +``` +will yield something like +``` +==> 1 installed package +-- linux-ubuntu20.04-skylake / gcc@9.3.0 ------------------------ +7dovpqn lcio@master~examples~ipo~jar~rootdict build_type=RelWithDebInfo cxxstd=17 dev_path=/home/tmadlener/work/ILCSoft/LCIO +``` +As you can see the local path from which this version was installed has become part of the spec for the installed package (the `dev_path=...` part of the spec above). +Hence, also the hash is affected by the fact that it has been built from a local source. + +To use this specific version as a dependency the usual spack syntax can be used, e.g. +```bash +spack install marlin ^lcio/7dovpqn +``` +will install `marlin` but use the version of `lcio` that we have just built locally. + +### More advanced usage +Note: If you have installed lcio following the description above you might have to uninstall it again first to follow these instructions, because spack will not overwrite an already installed package. + +The above instructions only dealt with installing a package from a local source, but not how to easily get a development environment allowing for a quick edit, compile cycle. +This can be achieved by using the `--drop-in` and the `--before`/`--until` arguments of the `dev-build` command: + +```bash +spack dev-build --drop-in bash --until cmake lcio@master ~rootdict +``` + +This command will first install all necessary dependencies, then run the install process for `lcio` until **after** the `cmake` stage and then drop you into a new bash shell with the proper build environment setup. +It will also setup a build folder, which follows the naming scheme `spack-build-${hash}`, in this case: `spack-build-7dovpqn`. +To compile `lcio` simply go into this directory and run `make` in there +```bash +cd spack-build-7dovpqn +make -j4 +``` +You are now in an environment where you can easily edit the local source code and re-compile afterwards. + +Once all the development is done, it is still necessary to install everything to the appropriate location. +This installation has to be registered in the spack database as well. Hence, simply calling `make install` in the build directory will not do the trick. +Another call to `dev-build` is necessary. +```bash +cd .. # go back to the source directory where you started +spack dev-build lcio@master ~rootdict +``` +This will run the whole chain again, but it will not overwrite the build directory. +Hence, it will not recompile everything again, but simply install all the build artifacts to the appropriate location. +`spack find -lv lcio` can be used to check if the installation was successful. + +**NOTE: You are probably still in the build environment at this stage. To return back to you original shell simply type `exit`.** + +## Developing multiple packages or dependencies of other packages + +Developing on many packages simultaneously using the `dev-build` command can become cumbersome and doesn't really scale. + +### Using an environment to setup all dependencies +One way to develop on multiple packages simultaneously can be to setup an [environment](https://spack.readthedocs.io/en/latest/environments.html) that contains the dependencies of all packages. + +As an example the following definition of an environment has been used to develop on +[`podio`](https://github.com/AIDASoft/podio), [`EDM4HEP`](https://github.com/key4hep/EDM4HEP) and some other packages. +```yaml +spack: + specs: + - python + - root@6.20.04 +davix+gsl+math~memstat+minuit+mlp~mysql+opengl~postgres~pythia6+pythia8+python~qt4+r+root7+rootfit+rpath~shadow+sqlite+ssl~table+tbb+threads+tmva+unuran+vc+vdt+vmc+x+xml+xrootd build_type=RelWithDebInfo cxxstd=17 patches=22af3471f3fd87c0fe8917bf9c811c6d806de6c8b9867d30a1e3d383a1b929d7 + - dd4hep +geant4 + - geant4 + - heppdt + - hepmc@2.06.10 + - tricktrack + - py-pyyaml + - py-jinja2 + - cmake + - pythia8 + - evtgen + concretization: together + view: true + packages: + all: + compiler: [gcc@9.3.0] + variants: cxxstd=17 +``` + +Assuming this is the content of `edm4hep_devel.yaml` an environment can be created, activated, concretized and installed with the following commands: +```bash +spack env create edm4hep-devel edm4hep_devel.yaml +spack env activate -p edm4hep-devel +spack concretize +spack install +``` + +After an environment has been installed, it can easily be activated via +```bash +spack env activate -p edm4hep-devel +``` +which immediately drops you in an environment with all the packages stated in the environment file above available and properly set up. +Developing packages that depend on these should then be straight forward, especially for properly setup CMake based projects that can automatically find and configure their dependencies. + +The disadvantage of this approach is that the packages you want to develop on have to be on the top of the stack and if they depend on each other, you still have to properly handle these dependencies on your own. + +### Environments and a development workflow + +Recently spack gained the ability to setup environments and specify multiple packages that you would like to develop on (See [spack/spack#256](https://github.com/spack/spack/pull/15256)). +It is not yet really documented and it is not yet fully optimized, but it allows for a decent development experience if your package is not too deep down in the stack. +It is not impossible to develop on packages deep down the software stack, but this can imply frequently recompiling large parts of the software stack, since spack does not yet handle this in the best way, but instead builds all packages that you are not developing on from scratch. Hence, even if a simple relinking would have done the trick, spack will still build a lot of packages again. +Nevertheless, the feature is in a usable state and this section briefly describes how to use it. +Especially if you mainly develop on one package but sometimes want to check whether the rest of the stack, that depends on this package still compiles with the latest version, this can be a very useful workflow. + +As an example we will be using the `k4simdelphes` package that depends on `edm4hep`, which in turn depens again on `podio`. +Suppose we want to change `podio` and `edm4hep` and see if `k4simdelphes` still compiles and works. +We would then use an environment definition file similar to the usual environments. For this example it has the following content +```yaml +spack: + spec: + - k4simdelphes + concretization: together + view: false + packages: + all: + compiler: [gcc@9.3.0] + variants: cxxstd=17 + develop: + podio: + spec: podio@master +sio + path: ../../../../../podio + edm4hep: + spec: edm4hep@master + path: ../../../../../EDM4hep +``` + +The first part is the same as previously, but a new `develop` section containing information about the packages that should be developed on has been added. +For each package there is a `spec` and a `path` field. The `spec` field tells spack which spec to build, while the `path` field tells spack where the source files are located. +**The path is relative to the `$(prefix)/var/spack/environments/${environment-name}` directory or an absolute path.** + +Assuming that you are currently in the directory that contains local `spack` installation, the following steps are necessary to create the development environment +```bash +git clone https://github.com/AIDASoft/podio +git clone https://github.com/key4hep/EDM4hep +spack env create my-development-env development_env_packages.yaml +``` +where `development_env_packages.yaml` is the yaml file with the contents just described above. + +It is now possible to activate this environment via +```bash +spack env activate -p my-development-env +``` + +To install all the packages, including the local versions of `podio` and `edm4hep` it is now enough to simply do `spack install`**in the activated environment**. +This will build your local copies of `podio` and `edm4hep` and use these versions as dependencies for the `k4simdelphes` package. +Changes can also be made to either of the two packages. +To compile only one package without installing it yet, it is easiest to simply go to the directory where the sources are. +There should now be a few spack related files and a spack build folder among the other source files +``` +[...] +spack-configure-args.txt +spack-build-env.txt +spack-build-out.txt +spack-build-${hash} +``` +Here `${hash}` is the same that you get from `spack find -l package`. +After you have done all the necessary changes you can simply change into this build directory and run `make` to compile the package again. +Once all your development is done and you want to install the package `spack install` will run the whole build chain again. +This means that all the (local) development packages in your environment will only be recompiled as far as necessary, while all other packages that depend on the development packages will be re-built from scratch. + +Once you are done developing, this environment can be used like any other environment simply by running `spack env activate my-development-env` to activate it. + +#### Adding another package to develop +If you now realize that your changes to `podio` or `edm4hep` broke `k4simdelphes` and you need to also implement some changes there, you do not have to define a new environment. +Instead it is possible to add `k4simdelphes` to the `develop` section via `spack develop` (assuming you are still in the activated environment and in the same directory where also the `podio` and `edm4hep` sources live) +```bash +git clone https://github.com/key4hep/k4SimDelphes +spack develop --no-clone --path ../../../../../k4SimDelphes k4simdelphes@main +``` +Here, the `--path` is again either relative to the environment directory inside spack. It could also be an absolute path. +You now have to concretize the environment again before you can install the packages. +```bash +spack concretize -f +spack install +``` + +Now you can work on `k4simdelphes` in the same way as you can for `podio` or `edm4hep`. +You can also check that the environment now indeed uses your local version of `k4simdelphes` via +``` +spack find -lv k4simdelphes +``` +which should now yield something along the lines of +``` +==> In environment my-development-env +==> Root specs +------- k4simdelphes@main + +==> 1 installed package +-- linux-ubuntu20.04-broadwell / gcc@9.3.0 ---------------------- +m5khm2w k4simdelphes@main~ipo build_type=RelWithDebInfo dev_path=/home/tmadlener/work/spack/var/spack/environments/test-devel-env/../../../../../k4SimDelphes +``` +where the path to the local source files has now again become part of the spec as can be seen by the `dev_path=...` part of the spec. diff --git a/_sources/developing-key4hep-software/WritingAlgorithms.md.txt b/_sources/developing-key4hep-software/WritingAlgorithms.md.txt new file mode 100644 index 0000000..6855e4a --- /dev/null +++ b/_sources/developing-key4hep-software/WritingAlgorithms.md.txt @@ -0,0 +1,77 @@ +# Writing Gaudi Algorithms + + +{% objectives "Learning Objectives" %} + +This tutorial will teach you how to: + +* write an algorithm for Key4hep +* interact with the cmake based build system +* use other Gaudi components in the algorithms + +{% endobjectives %} + + +## Getting Started + +Writing Gaudi components requires a bit of boilerplate code. +Often it is easiest to start from existing files and modify them as needed. +For this tutorial, there is a dedicated repository that contains an example. +Start by cloning it locally: + +```bash +git clone https://github.com/key4hep/k4-project-template +``` + +It contains a CMake configuration (as described in more detail in the previous tutorial) so it can be built with: + +```bash +cd k4-project-template +mkdir build install +cd build +cmake .. -DCMAKE_INSTALL_PREFIX=../install +make -j 4 +``` + +To run the algorithms contained in this repository, it is not necesary to run + +``` +make install +``` + +you can use the `run` script in the `build` directory, like: + +```bash +./run k4run ../K4TestFWCore/options/createExampleEventData.py + +``` + + +## Exercise: Adding an Algorithm + +The repository contains an `EmptyAlg` in `K4TestFWCore/src/components`. + + +* As a first exercise, copy and modify this algorithm to print out the current event number. + +* Second step: If you used `std::cout` in the first step, try to use the gaudi logging service instead. + +* Third Step: Print out a string before the event number that should be configurable at runtime. + +* Finally: use the Gaudi Random Number Generator Service to approximate pi with a [Monte Carlo Integration](https://en.wikipedia.org/wiki/Monte_Carlo_integration) + + +## Debugging: How to use GDB + +[The GNU Project Debugger](https://www.sourceware.org/gdb/) is supported by +Gaudi and can be invoked by passing additional `--gdb` parameter to the `k4run`. +For example: +```bash +k4run ../K4TestFWCore/options/createExampleEventData.py --gdb +``` +This will start the GDB and attaches it to the Gaudi steering. After initial +loading, user can start running of the steering by typing `continue` into the +GDB console. To interrupt running of the Gaudi steering use `CTRL+C`. + +More details how to run GDB with Gaudi can be found in +[LHCb Code Analysis Tools](https://twiki.cern.ch/twiki/bin/view/LHCb/CodeAnalysisTools#Debugging_gaudirun_py_on_Linux_w). diff --git a/_sources/index.md.txt b/_sources/index.md.txt new file mode 100644 index 0000000..60e58c0 --- /dev/null +++ b/_sources/index.md.txt @@ -0,0 +1,32 @@ +# Key4hep + + +```eval_rst +.. toctree:: + :maxdepth: 3 + :includehidden: + :caption: Contents: + + setup-and-getting-started/README.md + k4marlinwrapper/doc/starterkit/k4MarlinWrapperCLIC/Readme.md + k4marlinwrapper/doc/starterkit/k4MarlinWrapperCLIC/edmConverters.md + k4marlinwrapper/doc/starterkit/k4MarlinWrapperCLIC/howtoMultithread.md + k4marlinwrapper/doc/starterkit/k4MarlinWrapperCLIC/CEDViaWrapper.md + k4simdelphes/doc/starterkit/k4SimDelphes/Readme.md + developing-key4hep-software/README.md + spack-build-instructions-for-librarians/README.md + talks-and-presentations/README.md + call-for-logos/README.md + CONTRIBUTING.md + +.. toctree:: + :maxdepth: 2 + :includehidden: + :caption: External links: + + FCC software + CLIC software + ILC software + CEPC software +``` + diff --git a/_sources/k4marlinwrapper/doc/starterkit/k4MarlinWrapperCLIC/CEDViaWrapper.md.txt b/_sources/k4marlinwrapper/doc/starterkit/k4MarlinWrapperCLIC/CEDViaWrapper.md.txt new file mode 100644 index 0000000..f890e4d --- /dev/null +++ b/_sources/k4marlinwrapper/doc/starterkit/k4MarlinWrapperCLIC/CEDViaWrapper.md.txt @@ -0,0 +1,139 @@ + +# Running an Event Display with EDM4hep input + +It is possible to run the [C Event Display (CED)](https://github.com/iLCSoft/CED) via a wrapped [CEDViewer](https://github.com/iLCSoft/CEDViewer) Marlin Processor. This makes it possible to run the Event Dispaly with EDM4hep input files using an on the fly conversion to LCIO for CED. This introduction shows the basic concepts and also provides a options file that should work for most use cases. This example will be using the CLIC detector but should also work for other DD4hep detector models. The example is fully self contained, if you already have everything set up you can jump directly to [running the event display](#running-the-event-display). + +## Setting up an environment + +The following steps have been tested with the Key4hep nightly builds release which can be setup using +```bash +source /cvmfs/sw-nightlies.hsf.org/key4hep/setup.sh +``` + +To get the CLIC detecor description we clone the `CLICPerformance` repository +```bash +git clone https://github.com/iLCSoft/CLICPerformance +``` + +All the following steps assume that the environment is setup like above and that the detector description is in the `CLICPerformance` directory. All commands start from the diretory from which `git clone` has been executed. + +## Creating an input file + +To create an input file for the event display we run a simple detector simulation using `ddsim` and a particle gun that shoots photons. The input file that we create here for illustration purposes has only 10 events, which also means that the creation should only take a few minutes. The steps to create this file are the following + +```bash +ddsim --steeringFile CLICPerformance/clicConfig/clic_steer.py \ + --compactFile $K4GEO/CLIC/compact/CLIC_o3_v14/CLIC_o3_v14.xml \ + --enableGun \ + --gun.distribution uniform \ + --gun.particle gamma \ + --gun.energy 10*GeV \ + --outputFile gamma_10GeV_edm4hep.root \ + --numberOfEvents 10 +``` + +You should now have a `gamma_10GeV_edm4hep.root` file containing 10 events. + +## Runing the event display + +In order to run the event display via the `DDCEDViewer` we use the Marlin wrapper and attach an EDM4hep to LCIO converter to the wrapped processor. In the following we will build the complete Gaudi options file step by step. Here we simply present the most important steps, but do not go over all details of the `DDCEDViewer` configuration, for that it is probably best to directly look at the [CEDViewer repository](https://github.com/iLCSoft/CEDViewer) directly. The complete Gaudi configuration can be found in [`k4MarlinWrapper/examples/event_display.py`](https://github.com/key4hep/k4MarlinWrapper/blob/master/k4MarlinWrapper/examples/event_display.py) which is also installed at `$K4MARLINWRAPPER/examples/event_display.py` + +To read EDM4hep input we use the `PodioInput` module and the `k4DataSvc` +```python +from Gaudi.Configuration import * +from Configurables import k4DataSvc, PodioInput + +algList = [] + +evtsvc = k4DataSvc('EventDataSvc') +evtsvc.input = '' + +inp = PodioInput('InputReader') +inp.collections = [ + # ... all collections that should be read ... +] + +algList.append(inp) +``` + +The `DDCEDViewer` also requires setting up a DD4hep geometry, which can simply be done by wrapping the `InitializeDD4hep` Marlin Processor +```python +from Configurables import MarlinProcessorWrapper + +MyInitializeDD4hep = MarlinProcessorWrapper("MyInitializeDD4hep") +MyInitializeDD4hep.OutputLevel = INFO +MyInitializeDD4hep.ProcessorType = "InitializeDD4hep" +MyInitializeDD4hep.Parameters = { + "DD4hepXMLFile": ["CLICPerformance/Visualisation/CLIC_o3_v06_CED/CLIC_o3_v06_CED.xml"] + } + +algList.append(MyInitializeDD4hep) +``` +Note that in this case we already have passed in the geometry that we want to use for the event display. + +Finally, the main work is done by the `DDCEDViewer`, which we again use via the `MarlinProcessorWrapper` (omitting a lot of the detailed configuration here). In order to convert the EDM4hep inputs to LCIO we attach an `EDM4hep2LcioTool` to this algorithm +```python +from Configurables import EDM4hep2LcioTool + +MyCEDViewer = MarlinProcessorWrapper("MyCEDViewer") +MyCEDViewer.OutputLevel = INFO +MyCEDViewer.ProcessorType = "DDCEDViewer" +MyCEDViewer.Parameters = { + # ... lots of CEDViewer configuration ... + } + +# EDM4hep to LCIO converter +edmConvTool = EDM4hep2LcioTool("EDM4hep2lcio") +edmConvTool.convertAll = True +edmConvTool.collNameMapping = {'MCParticles': 'MCParticle'} +edmConvTool.OutputLevel = DEBUG +MyCEDViewer.EDM4hep2LcioTool = edmConvTool + +algList.append(MyCEDViewer) +``` + +The only thing that is left to do now is to put everything into the `ApplicationManager` in order to run things +```python +from Configurables import ApplicationMgr +ApplicationMgr( TopAlg = algList, + EvtSel = 'NONE', + EvtMax = 10, + ExtSvc = [evtsvc], + OutputLevel=INFO + ) +``` + +With all these pieces put together (as in `examples/event_display.py`) it is now possible to run the event display. In order to run the event display we first have to start the `glced` server program to which the wrapped `CEDViewer` processor will then connect. Starting the server and running the wrapped processor can be done via +```bash +glced & + +k4run $K4MARLINWRAPPER/examples/event_display.py --EventDataSvc.input=gamma_10GeV_edm4hep.root +``` + +## Creating a Gaudi options file + +The `event_display.py` options file that is used above and that is present in the examples has been produced following these steps: +- Using an `.slcio` input file and the desired geometry, run `ced2go` with the desired arguments. + - This produces a Marlin xml steering file on the fly and stores it at `/tmp/ced2go_${USER}_steering.xml` +- Using the `convertMarlinSteeringToGaudi.py` converter script convert this into a gaudi options file +- Exchange the LCIO input reading by the podio input reading (see above) +- Attach the `EDM4hep2LcioTool` to the wrapped `CEDViewer` processor + +This should allow one to arrive at a similar steering file even for slightly different configurations. One potential pitfall is the slightly different naming of the `ddsim` outputs between LCIO and EDM4hep (see [this issue](https://github.com/AIDASoft/DD4hep/issues/921)). This can be addressed by configuring the EDM4hep to LCIO converter (`edmConvTool` in the code above) to map the names accordingly. diff --git a/_sources/k4marlinwrapper/doc/starterkit/k4MarlinWrapperCLIC/Readme.md.txt b/_sources/k4marlinwrapper/doc/starterkit/k4MarlinWrapperCLIC/Readme.md.txt new file mode 100644 index 0000000..a3af152 --- /dev/null +++ b/_sources/k4marlinwrapper/doc/starterkit/k4MarlinWrapperCLIC/Readme.md.txt @@ -0,0 +1,178 @@ + +# Using the Key4hep-Stack for CLIC Simulation and Reconstruction + + +This assumes that you have access to an installation of the Key4hep-stack, either via ``CVMFS`` or ``spack install``. +To setup the installation on cvmfs, do: + +```bash +source /cvmfs/sw.hsf.org/key4hep/setup.sh +``` + +These commands will explain how one can run the CLIC detector simulation and reconstruction using the Key4hep-Stack. +First we will obtain all the necessary steering and input files for CLIC, simulate a few events and run the +reconstruction both with ``Marlin`` and ``k4run``. These steps can be adapted to simulate or run other ``Marlin`` +processors as well. + +The ``CLICPerformance`` repository contains the steering and input files. + +```bash +git clone https://github.com/iLCSoft/CLICPerformance +``` + +## Simulation + +Simulating a few events with `ddsim` can produce output in EDM4hep or LCIO format. + +- To produce events in **EDM4hep** format one can run indicating `--outputFile _edm4hep.root` to produce the +output in such format: + + +```bash +cd CLICPerformance/clicConfig + +ddsim --compactFile $K4GEO/CLIC/compact/CLIC_o3_v14/CLIC_o3_v14.xml \ + --outputFile ttbar_edm4hep.root \ + --steeringFile clic_steer.py \ + --inputFiles ../Tests/yyxyev_000.stdhep \ + --numberOfEvents 3 +``` + +- To produce events in **LCIO** format one can run indicating `--outputFile .slcio` to produce the output file +in such format: + +```bash +cd CLICPerformance/clicConfig + +ddsim --compactFile $K4GEO/CLIC/compact/CLIC_o3_v14/CLIC_o3_v14.xml \ + --outputFile ttbar.slcio \ + --steeringFile clic_steer.py \ + --inputFiles ../Tests/yyxyev_000.stdhep \ + --numberOfEvents 3 +``` + + +## Reconstruction + +### Reconstruction with Marlin + +To run the reconstruction with ``Marlin``: + +```bash +cd CLICPerformance/clicConfig + +Marlin clicReconstruction.xml \ + --InitDD4hep.DD4hepXMLFile=$K4GEO/CLIC/compact/CLIC_o3_v14/CLIC_o3_v14.xml \ + --global.LCIOInputFiles=ttbar.slcio \ + --global.MaxRecordNumber=3 +``` + +### Reconstruction with Gaudi through k4MarlinWrapper + +We can convert the ``xml`` steering file to a Gaudi steering file (python): + +```bash +cd CLICPerformance/clicConfig + +convertMarlinSteeringToGaudi.py clicReconstruction.xml clicReconstruction.py +``` + +Reconstruction can be performed with LCIO or EDM4hep input, depending on the output format of the events produced +during [Simulation](#simulation). + +#### Reconstruction with LCIO input + +- When using **LCIO** format for the input events to be used in reconstruction: + + Modify the ``clicReconstruction.py`` file to point to the ``ttbar.slcio`` input file, and change the + ``DD4hepXMLFile`` parameter for the ``InitDD4hep`` algorithm. In addition the two processors with the comment ``# + Config.OverlayFalse`` and ``# Config.TrackingConformal`` should be enabled by uncommenting their line in the ``algList`` + at the end of the file. + +```bash +cd CLICPerformance/clicConfig + +sed -i '1s/^/import os\n/' clicReconstruction.py +sed -i 's;read.Files = \[".*"\];read.Files = \["ttbar.slcio"\];' clicReconstruction.py +sed -i 's;EvtMax = 10,;EvtMax = 3,;' clicReconstruction.py +sed -i 's;"MaxRecordNumber": ["10"],;"MaxRecordNumber": ["3"],;' clicReconstruction.py +sed -i 's;# algList.append(OverlayFalse);algList.append(OverlayFalse);' clicReconstruction.py +sed -i 's;# algList.append(MyConformalTracking);algList.append(MyConformalTracking);' clicReconstruction.py +sed -i 's;# algList.append(ClonesAndSplitTracksFinder);algList.append(ClonesAndSplitTracksFinder);' clicReconstruction.py +sed -i 's;# algList.append(RenameCollection);algList.append(RenameCollection);' clicReconstruction.py +sed -i 's;"DD4hepXMLFile": \[".*"\],; "DD4hepXMLFile": \[os.environ["LCGEO"]+"/CLIC/compact/CLIC_o3_v14/CLIC_o3_v14.xml"\],;' clicReconstruction.py + +``` + +Then the reconstruction using the k4MarlinWrapper can be run with + +```bash +cd CLICPerformance/clicConfig + +k4run clicReconstruction.py +``` + + +#### Reconstruction with EDM4hep input + +- When using **EDM4hep** format for the input events to be used in reconstruction, refer to the [**EDM converters**](https://github.com/key4hep/k4MarlinWrapper/blob/master/doc/starterkit/k4MarlinWrapperCLIC/edmConverters.md) +included with k4MarlinWrapper. Note that: + + *MarlinProcessorWrappers* need input in LCIO format: EDM4hep collections need to be converted to LCIO + + The output collections of *MarlinProcessorWrappers* may be used later by other algorithms: + * Output collections of *MarlinProcessorWrappers* will be in LCIO format unless these are explicitly converted + * Some *MarlinProcessorWrappers* may modify collections instead of producing new ones: the original EDM4hep collection wont be updated in this case and would need conversion from LCIO to EDM4hep. + +- To run *clicReconstruction* with EDM4hep format, use the steering file found in the `examples` folder of k4MarlinWrapper: +`k4MarlinWrapper/examples/clicRec_e4h_input.py` (this also gets installed to `$K4MARLINWRAPPER/examples` in Key4hep releases) + + Change the line where `evtsvc.input` is defined to point to the location of your input file. + + At the bottom of the file, in the `ApplicationMgr` parameters, change `EvtMax = 3,` to the number of events to run. + +This can be run in the following way. +```bash +cd CLICPerformance/clicConfig + +cp $K4MARLINWRAPPER/examples/clicRec_e4h_input.py . + +k4run clicRec_e4h_input.py --EventDataSvc.input ttbar_edm4hep.root +``` + +### DD4hep Geometry Information + +The ``MarlinDD4hep::InitializeDD4hep`` processor can be replaced by the ``k4SimGeant4::GeoSvc`` and the +``TrackingCellIDEncodingSvc`` the latter of which is part of the k4MarlinWrapper repository. + +For example: + +```python +import os +from Gaudi.Configuration import INFO +from Configurables import GeoSvc, TrackingCellIDEncodingSvc +svcList = [] +geoservice = GeoSvc("GeoSvc") +geoservice.detectors = [os.environ["K4GEO"]+"/CLIC/compact/CLIC_o3_v15/CLIC_o3_v15.xml"] +geoservice.OutputLevel = INFO +geoservice.EnableGeant4Geo = False +svcList.append(geoservice) + +cellIDSvc = TrackingCellIDEncodingSvc("CellIDSvc") +cellIDSvc.EncodingStringParameterName = "GlobalTrackerReadoutID" +cellIDSvc.GeoSvcName = geoservice.name() +cellIDSvc.OutputLevel = INFO +svcList.append(cellIDSvc) +``` diff --git a/_sources/k4marlinwrapper/doc/starterkit/k4MarlinWrapperCLIC/edmConverters.md.txt b/_sources/k4marlinwrapper/doc/starterkit/k4MarlinWrapperCLIC/edmConverters.md.txt new file mode 100644 index 0000000..ff81839 --- /dev/null +++ b/_sources/k4marlinwrapper/doc/starterkit/k4MarlinWrapperCLIC/edmConverters.md.txt @@ -0,0 +1,205 @@ + +# Event Data Model converters + +The converters are implemented as Gaudi Tools: these can be attached to any Gaudi Algorithm defined in the options file. + +- [Reading events](#reading-events) +- [Writing events](#writing-events) +- [Event Data Model converters](#Event-Data-Model-converters) + + [EDM4hep to LCIO converter](#EDM4hep-to-LCIO-converter) + + [LCIO to EDM4hep converter](#LCIO-to-EDM4hep-converter) + +## Reading events + +Read **EDM4hep** events: + +1. Instantiate the Event Service from k4FWCore: `k4DataSvc` +2. Indicate the event file to read. +3. Then use the algorithm `PodioInput` to select the collections to read by their name. +4. Add the algorithm `PodioInput` to the algorithm list. + +```python +from Configurables import k4DataSvc, PodioInput + +# 1 +evtsvc = k4DataSvc('EventDataSvc') +# 2 +evtsvc.input = 'path/to/file.root' + +# 3 +inp = PodioInput('InputReader') +inp.collections = [ + 'ReconstructedParticles', + 'EFlowTrack' +] +inp.OutputLevel = DEBUG + +# 4 +algList.append(inp) +``` + +=== + +Read **LCIO** events: + +1. Instantiate the Event Data Service. +2. Use `LcioEvent` and indicate the event file to read. +3. Add the algorithm `LcioEvent` to the algorithm list. + +```python +from Configurables import EventDataSvc, LcioEvent + +# 1 +evtsvc = EventDataSvc() + +# 2 +read = LcioEvent() +read.OutputLevel = DEBUG +read.Files = ["path/to/file.slcio"] + +# 3 +algList.append(read) +``` + +## Writing events + +Write **EDM4hep** events: + +1. Instantiate `PodioOutput` and indicate event file to write. +2. Select command for `PodioOutput`. +3. Add the `PodioOutput` to the algorithm list. + +```python +from Configurables import PodioOutput + +# 1 +out = PodioOutput("PodioOutput", filename = "my_output.root") +# 2 +out.outputCommands = ["keep *"] + +# 3 +algList.append(out) +``` + +=== + +Write **LCIO** events: + +1. Instantiate a `MarlinProcessorWrapper` with a relevant name. +2. Indicate the `ProcessorType` to be `LCIOOutputProcessor`. +3. Indicate the relevant parameters for the Marlin Processor. +4. Add the `MarlinProcessorWrapper` to the algorithm list. + +```python +from Configurables import MarlinProcessorWrapper + +# 1 +Output_DST = MarlinProcessorWrapper("Output_DST") +Output_DST.OutputLevel = WARNING +# 2 +Output_DST.ProcessorType = "LCIOOutputProcessor" +# 3 +Output_DST.Parameters = { + "DropCollectionNames": [], + "DropCollectionTypes": ["MCParticle", "LCRelation", "SimCalorimeterHit"], + ... + } + +# 4 +algList.append(Output_DST) +``` + + +## Event Data Model converters + +Note and review the following when using the converters: + +- Collection names must be unique. +- If using the `PodioInput` to read events: collection names must be indicated both in the `PodioInput` and the `EDM4hep2LcioTool`. +- Collections not indicated to be converted **will not** be converted even if its a dependency from an indicated collection to be converted. +- If a converted collection is used later by a Gaudi Algorithm, and this Gaudi Algorithm indicates the use of that collection in the `Parameters`, the converted collection name must match the name indicated in the Gaudi Algorithm `Parameters`. + + For example: A collection may be converted with the following parameters: `"ReconstructedParticles": "ReconstructedParticleLCIO"` + + A Gaudi Algorithm may indicate in their `Parameters`: `"PFOCollection": ["ReconstructedParticleLCIO"]` + +### EDM4hep to LCIO converter + +Collections from events that are already read, or are produced by a Gaudi Algorithm can be converted from EDM4hep to LCIO format: + +1. Instantiate the `EDM4hep2LcioTool` Gaudi Tool, e.g. as `edmConvTool = EDM4hep2LcioTool("EDM4hep2lcio")` +2. Indicate the collections to convert using the options of the tool. + + To simply convert all available collections use `edmConvTool.convertAll = True` (this is also the default!) + + If you want to convert all available collections but want to rename some of them, e.g. because an algorithm expects a different name, use the `collNameMapping` option. This maps the input name to the output name; `edmConvTool.collNameMapping = {'MCParticles': 'MCParticle'}` will convert the input `'MCParticles'` into the `'MCParticle'` collection. + + To convert only a subset of all available collections, set the `convertAll` option to `False` and indicate the collections to convert via the `collNameMapping` option. +3. Select the Gaudi Algorithm that will convert the indicated collections. +4. Add the Tool to the Gaudi Algorithm. + +```python +from Configurables import ToolSvc, EDM4hep2LcioTool + +# 1 +edmConvTool = EDM4hep2LcioTool("EDM4hep2lcio") +# 2 +edmConvToll.convertAll = False +edmConvTool.collNameMapping = { + "EFlowTrack": "EFlowTrack_LCIO", + "ReconstructedParticles": "ReconstructedParticle_LCIO" +} +edmConvTool.OutputLevel = DEBUG + +# 3 +InitDD4hep = MarlinProcessorWrapper("InitDD4hep") + +# 4 +InitDD4hep.EDM4hep2LcioTool=edmConvTool +``` + +### LCIO to EDM4hep converter + +Collections from events that are already read, or are produced by a gaudi Algorithm can be converted from LCIO to EDM4hep format: + +1. Instantiate the `Lcio2EDM4hepTool` Gaudi Tool, e.g. as `lcioConvTool = Lcio2EDM4hepTool("LCIO2EDM4hep")`. +2. Indicate the collections to convert in the options of the tool. + + To simply convert all available collections use `lcioConvTool.convertAll = True` (this is also the default!) + + If you want to convert all available collections but want to rename some of them, e.g. because an algorithm expects a different name, use the `collNameMapping` option. This maps the input name to the output name; `lcioConvTool.collNameMapping = {'MCParticles': 'MCParticle'}` will convert the input `'MCParticles'` into the `'MCParticle'` collection. + + To convert only a subset of all available collections, set the `convertAll` option to `False` and indicate the collections to convert via the `collNameMapping` option. +3. Select the Gaudi Algorithm that will convert the indicated collections. +4. Add the Tool to the Gaudi Algorithm. + +```python +from Configurables import ToolSvc, Lcio2EDM4hepTool + +# 1 +lcioConvTool = Lcio2EDM4hepTool("LCIO2EDM4hep") +# 2 +lcioConvTool.convertAll = False +lcioConvTool.collNameMapping = { + "EFlowTrackConv": "EFlowTrackEDM4hep", + "ReconstructedParticle": "ReconstructedParticlesEDM4hep", + "BuildUpVertices": "BuildUpVerticesEDM4hep", + "PrimaryVertices": "PrimaryVerticesEDM4hep" +} +lcioConvTool.OutputLevel = DEBUG + +# 3 +JetClusteringAndRefiner = MarlinProcessorWrapper("JetClusteringAndRefiner") + +# 4 +JetClusteringAndRefiner.Lcio2EDM4hepTool=lcioConvTool +``` diff --git a/_sources/k4marlinwrapper/doc/starterkit/k4MarlinWrapperCLIC/howtoMultithread.md.txt b/_sources/k4marlinwrapper/doc/starterkit/k4MarlinWrapperCLIC/howtoMultithread.md.txt new file mode 100644 index 0000000..b7e6cf0 --- /dev/null +++ b/_sources/k4marlinwrapper/doc/starterkit/k4MarlinWrapperCLIC/howtoMultithread.md.txt @@ -0,0 +1,142 @@ + +# How to run multithreading with k4MarlinWrapper (Gaudi) + +**Supported** +- Reading LCIO events with `LcioEvent()` +- Writing LCIO events with `LcioEventOutput()` +- Running `MarlinProcessorWrapper`s with **no** converters +- Using `whiteboard` as `ExtSvc` (no k4DataSvc or EventDataSvc) + +**Not supported** +- Using EDM converters `EDM4hep2LcioTool` and `Lcio2EDM4hepTool()` +- Reading EDM4hep events with `PodioInput()` +- Writing EDM4hep events with `PodioOutput()` +- Writing EDM4hep events with `MarlinProcessorWrapper` of type `LCIOOutputProcessor` +- Running non-thread algorithms/processors in parallel +- Running wrapped Marlin processors that make use of the `isFirstEvent` method in their `processEvent` method to do some setup only in the first event. There is no way to make this thread safe. If you want your processor to be usable on in a multi threaded environment, you have to move this setup to `init`. + +## Running Gaudi with multithreading support + +Gaudi uses [Intel TBB](https://oneapi-src.github.io/oneTBB/) under the hood for multithreading. + +Gaudi exposes two main levels of parallelism: +- Inter-event parallelism: running multiple events in parallel +- Intra-event parallelism: running multiple algorithms in parallel, within an event + +The two levels of parallelims can be combined: events can run in parallel, and algorithms within the events can run in parallel. + +### How to run with inter-event parallelism + +The following components are used to achieve parallelism: + +```python +from Configurables import (HiveWhiteBoard, HiveSlimEventLoopMgr, AvalancheSchedulerSvc) +``` + +These 3 components need to be configued to adapt the level of parallelism to the sequence, algorithms and hardware to be used. + +- Event Data Service: `HiveWhiteBoard` + + *EventSlots*: Number of events that may run in parallel, each with its own EventStore + + This is the Event Data Service, which needs the number of EventSlots +- Event Loop Manager: `HiveSlimEventLoopMgr` + + Event Loop Manager with parallelism support +- Thread Scheduling config: `AvalancheSchedulerSvc` + + Scheduler to indicate the number of threads to use + + In needs the total number of threads to use: this determines how many events and algorithms can be in flight (run in parallel) + + Default value is `-1` which indicate TBB to take over the machine with what it decides to be the optimal configuration + + +All these components can be set as follows in the options file: + +```python +evtslots = 4 +threads = 4 + +whiteboard = HiveWhiteBoard("EventDataSvc", EventSlots=evtslots) +slimeventloopmgr = HiveSlimEventLoopMgr(SchedulerName="AvalancheSchedulerSvc", OutputLevel=DEBUG) +scheduler = AvalancheSchedulerSvc(ThreadPoolSize=threads, OutputLevel=WARNING) + +from Configurables import ApplicationMgr +ApplicationMgr( TopAlg = [seq], + EvtSel = 'NONE', + EvtMax = 10, + ExtSvc = [whiteboard], + EventLoop=slimeventloopmgr, + MessageSvcType="InertMessageSvc" + ) +``` + +To only run events in parallel, with no parallelism between the algorithms within each event, the cardinality of the algorithms must be set: + +- Given a list of algorithms, set the cardinality of all to be 1 +- Create a `GaudiSequencer` with all the algorithms as `Members` +- Set the `GaudiSequencer` sequential property to true +- Pass the created `GaudiSequencer` to the Application Manager in the `TopAlg`: `TopAlg = [seq]` + +```python +from Configurables import MarlinProcessorWrapper +from Configurables import (GaudiSequencer) + +cardinality = 1 + +alg1 = MarlinProcessorWrapper("alg1") +alg2 = MarlinProcessorWrapper("alg2") +alg3 = MarlinProcessorWrapper("alg3") +alg4 = MarlinProcessorWrapper("alg4") + +algList = [] + +algList.append(alg1) +algList.append(alg2) +algList.append(alg3) +algList.append(alg4) + +for algo in algList: + algo.Cardinality = cardinality + algo.OutputLevel = DEBUG + +seq = GaudiSequencer( + "createViewSeq", + Members=algList, + Sequential=True, + OutputLevel=VERBOSE) + +from Configurables import ApplicationMgr +ApplicationMgr( TopAlg = [seq], + EvtSel = 'NONE', + EvtMax = 10, + ExtSvc = [whiteboard], + EventLoop=slimeventloopmgr, + MessageSvcType="InertMessageSvc" + ) +``` + +## Running example + +A multi-threaded CLIC Reconstruction can be run in multi-threaded mode, for LCIO input and output. +After successful compilation, from the build location: + +```sh +# Check available tests +ctest -N + +# Run multi-threaded clicReconstruction test +ctest -R clicRec_lcio_mt +``` diff --git a/_sources/k4simdelphes/doc/starterkit/k4SimDelphes/Readme.md.txt b/_sources/k4simdelphes/doc/starterkit/k4SimDelphes/Readme.md.txt new file mode 100644 index 0000000..2e5c8e8 --- /dev/null +++ b/_sources/k4simdelphes/doc/starterkit/k4SimDelphes/Readme.md.txt @@ -0,0 +1,64 @@ +# Running Delphes fast simulation with EDM4hep output + +The [`k4SimDelphes`](https://github.com/key4hep/k4SimDelphes) package provides utilities to convert output from the [Delphes fast simulation framework](https://cp3.irmp.ucl.ac.be/projects/delphes) into the [EDM4hep](https://github.com/key4hep/EDM4hep) format. It offers standalone executables, similar to the ones Delphes offers, as well as integration into the Key4hep framework. Here we will provide examples of how to run the standalone executables as well as the usage in the Key4hep framework. + +## Setup and prerequisites +The following examples assume that you have access to an existing installation of the Key4hep software stack. The easiest way to achieve this to use an existing installation on `cvmfs` +```bash +source /cvmfs/sw.hsf.org/key4hep/setup.sh +``` +Alternatively it is possible to build the complete stack via `spack`, see the [instructions](https://key4hep.github.io/key4hep-doc/spack-build-instructions-for-librarians/README.html) for how to do this. + +In order to run the examples below it is necessary to get some inputs that are used for the generation of the physics events. In the following we will be using pythia generator files that are also used by the FCCee. In particular we will be generating the following process + +e+e- -> ZH -> mu+mu- X (i.e. the Z decays into a pair of oppositely charged muons and the H to anything) + +To start we download the pythia cards for this process +```bash +wget https://raw.githubusercontent.com/HEP-FCC/FCC-config/spring2021/FCCee/Generator/Pythia8/p8_noBES_ee_ZH_ecm240.cmd +``` + +All the other resources are available from within a Key4hep release. + +## Standalone executables + +For this example we will be using the `DelphesPythia8_EDM4HEP` standalone executable, which is essentially the same as the `DelphesPythia8` executable that is provided by Delphes. Contrary to the Delphes executable the one provided by k4SimDelphes will produce output in the EDM4hep format. + +To run the generation and fast detector simulation in one go run +```bash +DelphesPythia8_EDM4HEP ${DELPHES_DIR}/cards/delphes_card_IDEA.tcl \ + ${K4SIMDELPHES}/edm4hep_output_config.tcl \ + p8_noBES_ee_ZH_ecm240.cmd \ + delphes_events_edm4hep.root +``` + +The arguments in this case are (in the order they are passed) +- The delphes card that describes the (parameterized) detector. In this case we use one that is shipped with the Delphes installation +- The output configuration for the Delphes to EDM4hep converter. Here we use the defaults that are provided by `k4SimDelphes`, which should cover the majority of use cases. It is however possible to [configure these to your needs](https://github.com/key4hep/k4SimDelphes/blob/main/doc/output_config.md) if necessary. +- The pythia card that describes the physics process [as described above](#setup-and-prerequisites) +- The name of the output file that will be created and that will contain the generated and simulated events in EDM4hep format + +### Other standalone executables +k4SimDelphes provides other standalone executables that can read different inputs. They offer the same functionality as the ones available from Delphes: +- `DelphesSTDHEP_EDM4HEP` - for reading STDHEP inputs +- `DelphesROOT_EDM4HEP` - for reading ROOT files in the Delphes format +- `DelphesPythia8_EDM4HEP` - for running Pythia8 as part of the simulation + +For all executables it is possible to at least get the order of the input arguments via `--help` or `-h`, e.g. +```bash +DelphesSTDHEP_EDM4HEP --help +``` +will print +```console +Usage: DelphesHepMC config_file output_config_file output_file [input_file(s)] +config_file - configuration file in Tcl format, +output_config_file - configuration file steering the content of the edm4hep output in Tcl format, +output_file - output file in ROOT format, +input_file(s) - input file(s) in STDHEP format, +with no input_file, or when input_file is -, read standard input. + +``` + +## Usage in the Key4hep framework + +- [ ] TODO diff --git a/_sources/setup-and-getting-started/README.md.txt b/_sources/setup-and-getting-started/README.md.txt new file mode 100644 index 0000000..9681fe7 --- /dev/null +++ b/_sources/setup-and-getting-started/README.md.txt @@ -0,0 +1,87 @@ +# Getting started with Key4hep software + +## Setting up the Key4hep Software Stack + +### Using a central installation on cvmfs + +Two builds with the key4hep stack are distributed on cvmfs. The releases happen +every few months on demand (for example, if there is a new important feature or +a breaking change) and at the moment only CentOS 7 is supported (support for +AlmaLinux 9 and Ubuntu is coming soon). Run the following to set up the stack: + +```bash +source /cvmfs/sw.hsf.org/key4hep/setup.sh +``` + +In addition, nightly builds for CentOS 7, AlmaLinux 9 and Ubuntu 22.04 with the +latest version of many packages are available: + +```bash +source /cvmfs/sw-nightlies.hsf.org/key4hep/setup.sh +``` + +The `setup.sh` script always points to the latest build and it will change +without warning. However, after sourcing the script some information will be +displayed with instructions on how to reproduce the current environment. Nightly +builds are intended for development and testing and they will be deleted after +some time from `/cvmfs`. + +### Using Virtual Machines or Docker containers + +The instructions above should work on any virtual machine or Docker container +with `cvmfs` available. We give in the following one example for each of the two +cases. + +#### CernVM Virtual Appliance + +The CernVM project provides a convenient tool to start VMs, [cernvm-launch](https://cernvm.cern.ch/portal/launch), and a [public repository](https://github.com/cernvm/public-contexts) of contexts to be used with `cernvm-launch` to configure the VM at your needs. A context dedicated to Key4hep is available in the repository. The [cernvm-launch](https://cernvm.cern.ch/portal/launch) works with [VirtualBox](https://www.virtualbox.org/), virtualization manager available for free for all platforms. + +To create and use a CernVM virtual machine for Key4hep please follow the following steps: + + * Make sure [VirtualBox](https://www.virtualbox.org/) is installed (details installing instructions from the product web page). + * Download the `cernvm-launch` binary for your platform either from the [dedicated download page](https://ecsft.cern.ch/dist/cernvm/launch/bin/) or from the following links: + * [Linux](https://fccsw.web.cern.ch/fccsw/utils/vm/cernvm/launch/linux/cernvm-launch) + * [Mac](https://fccsw.web.cern.ch/fccsw/utils/vm/cernvm/launch/mac/cernvm-launch) + + Make sure is visible in your $PATH. + * Get the [k4h-dev.context](https://raw.githubusercontent.com/cernvm/public-contexts/master/k4h-dev.context) (use wget or curl) + +Once you have all this you can create the VM with this command: +``` +$ cernvm-launch create --name k4h-dev --cpus 4 --memory 8000 --disk 20000 k4h-dev.context +``` +You an choose how many CPU cores to use, the memory and the disk space. Good rules of thumb are to use half the cores of your machine, at least 2 GB memory per core, and enough disk for your job. The above command should oepn a window with VirtualBox and produce on the screen an output like this +``` +Using user data file: k4h-dev.context +Parameters used for the machine creation: + name: k4h-dev + cpus: 4 + memory: 8000 + disk: 20000 + cernvmVersion: 2019.06-1 + sharedFolder: /mnt/shared/k4h-dev-vs +``` +You see in partcular that your `$HOME` area is shared with the VM, so you can exchange files between the VM and the host machine very conveniently. +From now on you can either work in the VirtualBox window or ssh to the machine with +``` +cernvm-launch ssh k4h-dev +``` +In either case you need a user name and password, which by default are `k4huser` and `pass`; these can be changed in the `k4h-dev.context` file. + +To enable graphics you need to find out the port on which the VM responds and use `ssh -Y -P fccuser@localhost`. For example +``` +$ cernvm-launch list +k4h-dev`: CVM: 2019.06-1 port: 36998 +... +$ ssh -Y -P 36998 k4huser@localhost +The authenticity of host '[localhost]:36998 ([127.0.0.1]:36998)' can't be established. +ECDSA key fingerprint is SHA256:JXjpOzSu7vIwgEDxc8s/fdDJv4gQs2SUjnbMnEZsaYI. +Are you sure you want to continue connecting (yes/no)? yes +Warning: Permanently added '[localhost]:36998' (ECDSA) to the list of known hosts. +k4huser@localhost's password: +[k4huser@localhost ~]$ +``` +(you can safely ignore warnings about setting LC_CTYPE). +Graphics should of course work well if you choose to work in the VirtualBox window. + +The `cernvm-launch` also supports listing, stopping, starting virtual machines. Please run `cernvm-launch -h` for all the available options. diff --git a/_sources/spack-build-instructions-for-librarians/README.md.txt b/_sources/spack-build-instructions-for-librarians/README.md.txt new file mode 100644 index 0000000..240d431 --- /dev/null +++ b/_sources/spack-build-instructions-for-librarians/README.md.txt @@ -0,0 +1,21 @@ +# Building Key4hep: For Librarians + +Key4hep comprises a fairly large number of software and depends on even more externals, so some tooling is needed to efficiently build the whole software stack. The [spack](https://spack.io) package manager can be used to build scientific software at scale, and is part of the Key4hep software R&D program. + + +A spack install of Key4hep is regularly deployed to `/cvmfs/sw.hsf.org/`, and can be used on lxplus/centos7 just by sourcing the following setup script: + +```bash +source /cvmfs/sw.hsf.org/key4hep/setup.sh +``` + +In this page, the workflow to create this installation is documented. + +```eval_rst +.. toctree:: + :caption: Contents: + + spack-setup.md +``` + + diff --git a/_sources/spack-build-instructions-for-librarians/spack-advanced.md.txt b/_sources/spack-build-instructions-for-librarians/spack-advanced.md.txt new file mode 100644 index 0000000..d04c0d3 --- /dev/null +++ b/_sources/spack-build-instructions-for-librarians/spack-advanced.md.txt @@ -0,0 +1,168 @@ + +# Spack Usage and Further Technical Topics + +This page collects a few known workarounds for issues and areas of development in spack. +Check also the issues in [key4hep-spack](https://github.com/key4hep/key4hep-spack/issues) for up-to-date information. +Additionally, we also provide a few more advanced invocations of `spack` commands that allow a certain degree of debugging of the decisions spack has made when installing a given package and its dependencies. + +## Checking which packages will be newly installed + +When installing a package it might be interesting to estimate how long it will take to do so. +An important proxy for this is how many and which dependencies spack will install in order to build a package. +This can be done with the `spack solve` command, which invokes the concretizer and then spits out the solution that would be installed if the same arguments were passed to `spack install`, e.g. + +```bash +spack solve -I +``` +```console +==> Best of 12 considered solutions. +==> Optimization Criteria: + Priority Criterion Installed ToBuild + 1 number of input specs not concretized - 0 + 2 number of packages to build (vs. reuse) - 4 + 3 requirement weight 0 0 + 4 deprecated versions used 1 0 + 5 version weight 0 0 + 6 number of non-default variants (roots) 0 0 + 7 preferred providers for roots 0 0 + 8 default values of variants not being used (roots) 0 0 + 9 number of non-default variants (non-roots) 3 0 + 10 preferred providers (non-roots) 0 0 + 11 compiler mismatches 0 0 + 12 OS mismatches 0 0 + 13 non-preferred OS's 0 0 + 14 version badness 156 0 + 15 default values of variants not being used (non-roots) 1 0 + 16 non-preferred compilers 0 0 + 17 target mismatches 0 0 + 18 non-preferred targets 165 44 + + - whizard@3.0.3%gcc@10.3.0~fastjet~latex~lcio~lhapdf~openloops~openmp+pythia8 hepmc=3 arch=linux-ubuntu20.04-x86_64 +[+] ^hepmc3@3.2.4%gcc@10.3.0~interfaces~ipo~python~rootio build_type=RelWithDebInfo arch=linux-ubuntu20.04-x86_64 +[+] ^cmake@3.16.3%gcc@10.3.0~doc+ncurses+ownlibs~qt build_type=RelWithDebInfo patches=1c54004,bf695e3 arch=linux-ubuntu20.04-x86_64 + - ^libtirpc@1.2.6%gcc@10.3.0 arch=linux-ubuntu20.04-x86_64 + - ^krb5@1.19.3%gcc@10.3.0+shared arch=linux-ubuntu20.04-x86_64 +[+] ^bison@3.8.2%gcc@10.3.0 arch=linux-ubuntu20.04-x86_64 +[+] ^diffutils@3.8%gcc@10.3.0 arch=linux-ubuntu20.04-x86_64 +[+] ^libiconv@1.16%gcc@10.3.0 libs=shared,static arch=linux-ubuntu20.04-x86_64 +[+] ^m4@1.4.18%gcc@10.3.0+sigsegv patches=3877ab5,fc9b616 arch=linux-ubuntu20.04-x86_64 +[+] ^perl@5.30.0%gcc@10.3.0~cpanm+shared+threads arch=linux-ubuntu20.04-x86_64 +[+] ^gettext@0.21%gcc@10.3.0+bzip2+curses+git~libunistring+libxml2+tar+xz arch=linux-ubuntu20.04-x86_64 +[+] ^bzip2@1.0.8%gcc@10.3.0~debug~pic+shared arch=linux-ubuntu20.04-x86_64 +[+] ^libxml2@2.9.13%gcc@10.3.0~python arch=linux-ubuntu20.04-x86_64 +[+] ^pkgconf@1.8.0%gcc@10.3.0 arch=linux-ubuntu20.04-x86_64 +[+] ^xz@5.2.5%gcc@10.3.0~pic libs=shared,static arch=linux-ubuntu20.04-x86_64 +[+] ^zlib@1.2.12%gcc@10.3.0+optimize+pic+shared patches=0d38234 arch=linux-ubuntu20.04-x86_64 +[+] ^ncurses@6.2%gcc@10.3.0~symlinks+termlib abi=none arch=linux-ubuntu20.04-x86_64 +[+] ^tar@1.30%gcc@10.3.0 zip=pigz arch=linux-ubuntu20.04-x86_64 +[+] ^openssl@1.1.1f%gcc@10.3.0~docs~shared certs=mozilla arch=linux-ubuntu20.04-x86_64 + - ^ocaml@4.13.1%gcc@10.3.0+force-safe-string arch=linux-ubuntu20.04-x86_64 +[+] ^pythia8@8.306%gcc@10.3.0~evtgen~fastjet~hdf5+hepmc+hepmc3~lhapdf~madgraph5amc~mpich~openmpi~python~rivet~root+shared arch=linux-ubuntu20.04-x86_64 +[+] ^hepmc@2.06.11%gcc@10.3.0~ipo build_type=RelWithDebInfo length=MM momentum=GEV arch=linux-ubuntu20.04-x86_64 +[+] ^rsync@3.1.3%gcc@10.3.0 arch=linux-ubuntu20.04-x86_64 +``` + +The `-I` flag shows which packages are alrady installed. +`spack solve` also shows some information about all the things that were considered during concretization. It can also be used to dump more information on the concretization process. +Unfortunately, this information is rather hard to parse, and still a work in progress from the spack developers. + + +## Requiring certain variants globally + +spack can be configured using some [configuration +files](https://spack.readthedocs.io/en/latest/configuration.html). Specifically +using `packages.yaml` which is read from the user directory, i.e. `~/.spack` (or +`/.spack/linux`) can be used to enforce the value of certain default variants +globally. To solve the above problem it is enough to put the following into +`packages.yaml`: + +```yaml +packages: + all: + variants: cxxstd=17 + ``` + +It is still possible to override this for certain packages either by +individually configuring them in `packages.yaml` or via the command line which +take precedence over all configuration files. + +In the Key4hep software stack build recipes for releases, we use the same mechanism, as this configuration is also available from spack environments. + + +## System Dependencies + +Some spack packages have *external find* support. For these packages it is possible to let spack detect the variants and versions for system (or otherwise) installed packages. +For such cases use the `spack external find` command. It has to be noted that detecting external packages and using them does not always work perfectly. + + +## Target Architectures + +Since HEP software is usually deployed on a variety of machines via cvmfs, installations need to pick a target architecture. `broadwell` is for now the default choice, and can be set with: + +``` +packages: + all: + target: [broadwell] +``` + +in `$HOME/.spack/linux/packages.yaml` + + + + +## Bundle Packages and Environments + +Right now, key4hep is installed via a `BundlePackage`, that depends on all other relevant packages. +An alternative would be to use [spack environments](https://spack-tutorial.readthedocs.io/en/latest/tutorial_environments.html#creating-and-activating-environments). This alternative is still under investigation. + + +## Setting Up Runtime Environments + +The simplest way to set the environment to use spack installed packages is to use the `spack load` command. +In order for users not to have to set up spack when it is not needed, the `key4hep-stack` bundle package includes a script that will automatically generate a bash setup script and install it into its prefix. + +Spack can also create "filesystem views" of several packages, resulting in a directory structure similar what you would find in `/usr/local`. +This simplifies library and include paths, but the setup generation for views still has to be developed. + +## Compiler Dependencies and Data Packages + +Some HEP packages (like `geant4-data`) consist only of data files, and can thus be used on any platform. +Spack cannot yet handle this gracefully, but an ongoing development tries to treat compilers as dependencies, which would help with re-using data packages. + + +## Duplicating Recipes in Downstream Repositories + +Although it is possible to "patch" spack build recipes by overriding them in another repository (key4hep-spack, for example), this is discouraged. +The central repo is one of the strenghts of spack, with many contributors ensuring that packages build smoothly. +Also, packages are installed in different namespaces, so it is not possible to deprecate changed recipes and use the upstream ones without re-installing the packages. + + +## CVMFS Installation Workflow + +The distribution on cvmfs is an exact copy of the spack installation on the build machine, just copied with this rsync command on the publisher: + +``` +rsync -axv --inplace --delete --verbose -e "ssh -T -o Compression=no -o StrictHostKeyChecking=no -o GSSAPIAuthentication=yes -o GSSAPITrustDNS=yes" user@build-machine:/cvmfs/sw.hsf.org/spackages/ /cvmfs/sw.hsf.org/spackages/ +``` + +The `--delete` option can be omitted in order to preserve already installed packages, regardless of the state of the build machine. + + + +## Compiler Wrappers + +Spack uses compiler wrappers instead of exposing the actual compilers during the build. +For packages like whizard, which register the compiler path to use during runtime, this will not work, as the wrappers are not available at runtime. +For these packages, the current workaround is to force spack to use the actual compilers during build (see the build recipe of `whizard`). + + +## Spack-Installed LCG releases + +A spack installation that contains all packages in the LCG releases is work in progress, see https://gitlab.cern.ch/sft/sft-spack-repo. + +## Using Spack-installed GCC + +When installing gcc with spack, it is necessary to add a `cc` symlink to `$PATH`, in order to avoid errors with cling, see https://github.com/spack/spack/issues/17488. + + + diff --git a/_sources/spack-build-instructions-for-librarians/spack-buildcache.md.txt b/_sources/spack-build-instructions-for-librarians/spack-buildcache.md.txt new file mode 100644 index 0000000..7868606 --- /dev/null +++ b/_sources/spack-build-instructions-for-librarians/spack-buildcache.md.txt @@ -0,0 +1,49 @@ + +# Spack Buildcaches + +{% callout "Spack documentation" %} + +Buildcaches were investigated, but are no longer supported for Key4hep software, due to the difficulty of relocating some packages. + +For more information refer to the [spack documentation](https://spack.readthedocs.io/en/latest/binary_caches.html). + +{% endcallout %} + +It is possible to relocate and re-use binaries with the so-called buildcache. +Some central buildcaches are on the key4hep `eos` space under: + +``` +/eos/project/k/key4hep/www/key4hep/spack_build/mirror/spackages +``` +`spackages` is intended as the central buildcache for key4hep packages built from scratch, +but there exist other buildcaches for packages built against the LCG releases, and rolling builds that are using the branchname `master` or other moving targets. +All packages versioned `package@master` are treated by spack as the same version, even if master points to different commits, thus they must be installed to separate directories, ideally indicating the date. +A buildcache called `contrib` is intended for build tools such as compilers. +Spack automatically creates subdirectory for different platforms and compiler versions. + +For write permissions to this space, subscribe to the egroup `cernbox-project-key4hep-writers`. + +The following command can be used to put packages into the buildcache: + +```bash +# key4hep-stack was already installed with 'spack install key4hep-stack' +spack mirror add key4hep /eos/project/k/key4hep/www/key4hep/spack_build/mirror/spackages +spack buildcache create -m key4hep -u -a -f key4hep-stack +``` + +Since the packages need to be relocated as well as copied, this might take up to an hour. + + +In order to install packages from the buildcache, use: + +```bash +spack buildcache install -u -a key4hep-stack +``` + +Spack will then search all added mirrors for `key4hep-stack`. +For read-only access on machines without `eos`, these files are served also over http: + + +```bash +spack mirror add key4hep-web http://key4hep.web.cern.ch/key4hep/spack_build/mirror/spackages/ +``` diff --git a/_sources/spack-build-instructions-for-librarians/spack-nightlies.md.txt b/_sources/spack-build-instructions-for-librarians/spack-nightlies.md.txt new file mode 100644 index 0000000..46b1d0a --- /dev/null +++ b/_sources/spack-build-instructions-for-librarians/spack-nightlies.md.txt @@ -0,0 +1,21 @@ +# Nightly Builds with Spack + + +## Usage of the nightly builds on CVMFS + +For Centos7, the latest nightly build can be set up by running: + +``` +source /cvmfs/sw-nightlies.hsf.org/key4hep/setup.sh +``` + +Spack can also be configured to use `/cvmfs/sw-nightlies.hsf.org/spackages` as an upstream installation. + +## Technical Information + +Nightly builds can be a very useful tool, both to test code correctness and to quickly and automatically deploy the latest developments. +In contrast to the release builds, which use the latest stable version of the individual packages, nightly builds typically use the HEAD of the main development branch. + +It is not very efficient to completely rebuild the stack every day, as some packages change fairly infrequently. +The key4hep-spack repository includes some scripts in order to use commit hashes as versions. + diff --git a/_sources/spack-build-instructions-for-librarians/spack-setup.md.txt b/_sources/spack-build-instructions-for-librarians/spack-setup.md.txt new file mode 100644 index 0000000..47d2726 --- /dev/null +++ b/_sources/spack-build-instructions-for-librarians/spack-setup.md.txt @@ -0,0 +1,110 @@ + +## Setting up Spack + + +### Downloading a spack instance + +Spack is easy to set up: simply clone the key4hep fork, and use one of the provided spack "environments", that is spack configuration that is created automatically from the current key4hep/key4hep-spack repository. + +```bash +git clone https://github.com/key4hep/spack +git clone https://github.com/key4hep/key4hep-spack +source spack/share/spack/setup-env.sh +source key4hep-spack/environments/key4hep-release-user/setup_clingo_centos7.sh # NOTE: only needed on centos7 +spack env activate key4hep-spack/environments/key4hep-release-user # or other environment, see below +``` + +These instructions assume that a Centos7 machine with a running CVMFS client is used (for other OSs see below). Since Spack cannot bootstrap with the system compiler on Centos7, a setup script for Clingo ( a spack dependency ) is provided. + +### Using Spack Environments + +The [spack environments](https://spack.readthedocs.io/en/latest/environments.html) available in `key4hep-spack/environments` bundle the spack configuration, setting up a suitable compiler from cvmfs, the key4hep package recipes, whether to create a view, etc. It is recommended to always use spack in an environment. New environments can be easily created by copying and modifying existing ones, but some use relative links to common configuration files in `key4hep-spack/environments/key4hep-common`, so they should be kept in the `key4hep/environments` directory. + +The basic environment is `key4hep-release`, which is used for the central installations and therefore uses `/cvmfs` as install area. `key4hep-debug` is a variation for debug builds. The default compiler is gcc, but an environment that uses clang is provided under `key4hep-release-clang`. + For local builds that use cvmfs read-only, `key4hep-release-user` can be used. + +### Using the key4hep-release-user environment + +The key4hep user environment has the `key4hep-stack` bundle package in its spec list. By concretizing it, spack selects the latest compatible versions, re-using installations from cvmfs + +```bash +spack concretize -f +spack find # lists the available concretized packages +``` + +The environment can be installed as is, although this will just install the bundle packages. However, this will create a setup script that can be used to load the software. + +``` +spack install +``` + +Custom builds can now be realized, by adding specs to the environment and concretizing together. For example, to build the stack with a local version of EDM4hep: + +``` +spack add edm4hep@master +git clone https://github.com/key4hep/edm4hep +# make some local changes to edm4hep +spack develop -p $PWD/edm4hep edm4hep@master +spack concretize -f +spack install + +``` + + +### Configuring Spack + +Alternatively, and for other platforms, spack can be configured in a few steps. These steps are essentially what is used to create the pre-configured spack instance in this script: https://github.com/key4hep/key4hep-spack/blob/master/scripts/ci_setup_spack.sh + +While this still puts the configuration files in the global scope of spack, it is recommended to use them in an environment, as provided by key4hep-spack. + +#### Installing Spack +Spack itself is very easy to install - simply clone the repository with git. + +```bash +git clone https://github.com/key4hep/spack.git +source spack/share/spack/setup-env.sh +``` + +#### Installing the key4hep package recipes + + The spack repository for key4hep packages is installed the same way: + +``` +git clone https://github.com/key4hep/key4hep-spack.git +spack repo add key4hep-spack +``` + +### Configuring `packages.yaml` + +In order to choose the right package versions and build options, spack sometimes needs a few [hints and nudges](https://spack.readthedocs.io/en/latest/build_settings.html). With the new concretizer (default as of spack version 0.17) this should be mostly obsolete. +key4hep-spack ships a spack config file that should give a good build customization out of the box, but can also be customized further. It just needs to be copied to the configuration where spack searches for configurations: + +``` +cp key4hep-spack/environments/key4hep-common/packages.yaml spack/etc/spack/ +``` + + + +#### Configuring `upstreams.yaml` + +The cvmfs installation can be used as an "upstream installation", by adding the following configuration: + +```bash +cat <> spack/etc/spack/upstreams.yaml +upstreams: + spack-instance-1: + install_tree: /cvmfs/sw.hsf.org/spackages6/ +EOT +``` + + +#### Setting up additional compilers + +Often it is practical to use a compiler already installed upstream. Spack provides the `spack compiler find` command for this, but the compiler needs to be loaded into the PATH: + +```bash +# loading the compiler from upstream +source /cvmfs/sft.cern.ch/lcg/contrib/gcc/11.2.0/x86_64-centos7/setup.sh +spack compiler find --scope site +``` + diff --git a/_sources/talks-and-presentations/README.md.txt b/_sources/talks-and-presentations/README.md.txt new file mode 100644 index 0000000..c2ec9a6 --- /dev/null +++ b/_sources/talks-and-presentations/README.md.txt @@ -0,0 +1,43 @@ +# Talks and Presentations + +A Zotero group with the items listed below can be found [here](https://www.zotero.org/groups/2447672/key4hep-talksandpublications). + +* *Carceller, Juan Miguel*, **Spack in Key4hep** presented at the HSF Software + Developer Tools & Packaging Working Group July 11, 20 + + +* *Carceller, Juan Miguel*, **key4hep Update** presented at the CERN EP R&D: Software meeting, June + 21, 2023, + + +* *Volkl, Valentin*, **Status of the key4hep Software Development** presented at + IAS Program on High Energy Physics (HEP 2021), January 20, 2021, + + +* *Madlener, Thomas*. **Key4HEP - Turnkey Software for Future Colliders** + presented at the XXVII Cracow EPIPHANY Conference on Future of particle + physics, January 10, 2021. + + +* *Helsens, Clement*, **FCC software for physics and detector studies** + presented at the XXVII Cracow EPIPHANY Conference on Future of particle + physics, January 10, 2021. + + +* *Ganis, Gerardo*. **“The Turnkey Software Stack: Where Are We and Where We + Want to Go.”** presented at the IAS HEP 2020, January 17, 2020. + . + +* *Sailer, André*. **“Key4HEP: Turnkey Software for Future Collider + Experiments.”** presented at the Joint Workshop on Future Charm–Tau Factory, + September 24, 2019. + . + +* *Sailer, André*. **“Towards a Turnkey Software Stack for HEP Experiments.”** + presented at the International Conference on Computing in High Energy & + Nuclear Physics 2019, September 5, 2019. + . + +* *Mato, Pere*. **“Key4HEP: The Common Turnkey Software Stack.”** presented at + the CLICdp Collaboration Meeting, August 28, 2019. + . diff --git a/_static/basic.css b/_static/basic.css new file mode 100644 index 0000000..bf18350 --- /dev/null +++ b/_static/basic.css @@ -0,0 +1,906 @@ +/* + * basic.css + * ~~~~~~~~~ + * + * Sphinx stylesheet -- basic theme. + * + * :copyright: Copyright 2007-2022 by the Sphinx team, see AUTHORS. + * :license: BSD, see LICENSE for details. + * + */ + +/* -- main layout ----------------------------------------------------------- */ + +div.clearer { + clear: both; +} + +div.section::after { + display: block; + content: ''; + clear: left; +} + +/* -- relbar ---------------------------------------------------------------- */ + +div.related { + width: 100%; + font-size: 90%; +} + +div.related h3 { + display: none; +} + +div.related ul { + margin: 0; + padding: 0 0 0 10px; + list-style: none; +} + +div.related li { + display: inline; +} + +div.related li.right { + float: right; + margin-right: 5px; +} + +/* -- sidebar --------------------------------------------------------------- */ + +div.sphinxsidebarwrapper { + padding: 10px 5px 0 10px; +} + +div.sphinxsidebar { + float: left; + width: 230px; + margin-left: -100%; + font-size: 90%; + word-wrap: break-word; + overflow-wrap : break-word; +} + +div.sphinxsidebar ul { + list-style: none; +} + +div.sphinxsidebar ul ul, +div.sphinxsidebar ul.want-points { + margin-left: 20px; + list-style: square; +} + +div.sphinxsidebar ul ul { + margin-top: 0; + margin-bottom: 0; +} + +div.sphinxsidebar form { + margin-top: 10px; +} + +div.sphinxsidebar input { + border: 1px solid #98dbcc; + font-family: sans-serif; + font-size: 1em; +} + +div.sphinxsidebar #searchbox form.search { + overflow: hidden; +} + +div.sphinxsidebar #searchbox input[type="text"] { + float: left; + width: 80%; + padding: 0.25em; + box-sizing: border-box; +} + +div.sphinxsidebar #searchbox input[type="submit"] { + float: left; + width: 20%; + border-left: none; + padding: 0.25em; + box-sizing: border-box; +} + + +img { + border: 0; + max-width: 100%; +} + +/* -- search page ----------------------------------------------------------- */ + +ul.search { + margin: 10px 0 0 20px; + padding: 0; +} + +ul.search li { + padding: 5px 0 5px 20px; + background-image: url(file.png); + background-repeat: no-repeat; + background-position: 0 7px; +} + +ul.search li a { + font-weight: bold; +} + +ul.search li p.context { + color: #888; + margin: 2px 0 0 30px; + text-align: left; +} + +ul.keywordmatches li.goodmatch a { + font-weight: bold; +} + +/* -- index page ------------------------------------------------------------ */ + +table.contentstable { + width: 90%; + margin-left: auto; + margin-right: auto; +} + +table.contentstable p.biglink { + line-height: 150%; +} + +a.biglink { + font-size: 1.3em; +} + +span.linkdescr { + font-style: italic; + padding-top: 5px; + font-size: 90%; +} + +/* -- general index --------------------------------------------------------- */ + +table.indextable { + width: 100%; +} + +table.indextable td { + text-align: left; + vertical-align: top; +} + +table.indextable ul { + margin-top: 0; + margin-bottom: 0; + list-style-type: none; +} + +table.indextable > tbody > tr > td > ul { + padding-left: 0em; +} + +table.indextable tr.pcap { + height: 10px; +} + +table.indextable tr.cap { + margin-top: 10px; + background-color: #f2f2f2; +} + +img.toggler { + margin-right: 3px; + margin-top: 3px; + cursor: pointer; +} + +div.modindex-jumpbox { + border-top: 1px solid #ddd; + border-bottom: 1px solid #ddd; + margin: 1em 0 1em 0; + padding: 0.4em; +} + +div.genindex-jumpbox { + border-top: 1px solid #ddd; + border-bottom: 1px solid #ddd; + margin: 1em 0 1em 0; + padding: 0.4em; +} + +/* -- domain module index --------------------------------------------------- */ + +table.modindextable td { + padding: 2px; + border-collapse: collapse; +} + +/* -- general body styles --------------------------------------------------- */ + +div.body { + min-width: 450px; + max-width: 800px; +} + +div.body p, div.body dd, div.body li, div.body blockquote { + -moz-hyphens: auto; + -ms-hyphens: auto; + -webkit-hyphens: auto; + hyphens: auto; +} + +a.headerlink { + visibility: hidden; +} + +a.brackets:before, +span.brackets > a:before{ + content: "["; +} + +a.brackets:after, +span.brackets > a:after { + content: "]"; +} + +h1:hover > a.headerlink, +h2:hover > a.headerlink, +h3:hover > a.headerlink, +h4:hover > a.headerlink, +h5:hover > a.headerlink, +h6:hover > a.headerlink, +dt:hover > a.headerlink, +caption:hover > a.headerlink, +p.caption:hover > a.headerlink, +div.code-block-caption:hover > a.headerlink { + visibility: visible; +} + +div.body p.caption { + text-align: inherit; +} + +div.body td { + text-align: left; +} + +.first { + margin-top: 0 !important; +} + +p.rubric { + margin-top: 30px; + font-weight: bold; +} + +img.align-left, figure.align-left, .figure.align-left, object.align-left { + clear: left; + float: left; + margin-right: 1em; +} + +img.align-right, figure.align-right, .figure.align-right, object.align-right { + clear: right; + float: right; + margin-left: 1em; +} + +img.align-center, figure.align-center, .figure.align-center, object.align-center { + display: block; + margin-left: auto; + margin-right: auto; +} + +img.align-default, figure.align-default, .figure.align-default { + display: block; + margin-left: auto; + margin-right: auto; +} + +.align-left { + text-align: left; +} + +.align-center { + text-align: center; +} + +.align-default { + text-align: center; +} + +.align-right { + text-align: right; +} + +/* -- sidebars -------------------------------------------------------------- */ + +div.sidebar, +aside.sidebar { + margin: 0 0 0.5em 1em; + border: 1px solid #ddb; + padding: 7px; + background-color: #ffe; + width: 40%; + float: right; + clear: right; + overflow-x: auto; +} + +p.sidebar-title { + font-weight: bold; +} + +div.admonition, div.topic, blockquote { + clear: left; +} + +/* -- topics ---------------------------------------------------------------- */ + +div.topic { + border: 1px solid #ccc; + padding: 7px; + margin: 10px 0 10px 0; +} + +p.topic-title { + font-size: 1.1em; + font-weight: bold; + margin-top: 10px; +} + +/* -- admonitions ----------------------------------------------------------- */ + +div.admonition { + margin-top: 10px; + margin-bottom: 10px; + padding: 7px; +} + +div.admonition dt { + font-weight: bold; +} + +p.admonition-title { + margin: 0px 10px 5px 0px; + font-weight: bold; +} + +div.body p.centered { + text-align: center; + margin-top: 25px; +} + +/* -- content of sidebars/topics/admonitions -------------------------------- */ + +div.sidebar > :last-child, +aside.sidebar > :last-child, +div.topic > :last-child, +div.admonition > :last-child { + margin-bottom: 0; +} + +div.sidebar::after, +aside.sidebar::after, +div.topic::after, +div.admonition::after, +blockquote::after { + display: block; + content: ''; + clear: both; +} + +/* -- tables ---------------------------------------------------------------- */ + +table.docutils { + margin-top: 10px; + margin-bottom: 10px; + border: 0; + border-collapse: collapse; +} + +table.align-center { + margin-left: auto; + margin-right: auto; +} + +table.align-default { + margin-left: auto; + margin-right: auto; +} + +table caption span.caption-number { + font-style: italic; +} + +table caption span.caption-text { +} + +table.docutils td, table.docutils th { + padding: 1px 8px 1px 5px; + border-top: 0; + border-left: 0; + border-right: 0; + border-bottom: 1px solid #aaa; +} + +table.footnote td, table.footnote th { + border: 0 !important; +} + +th { + text-align: left; + padding-right: 5px; +} + +table.citation { + border-left: solid 1px gray; + margin-left: 1px; +} + +table.citation td { + border-bottom: none; +} + +th > :first-child, +td > :first-child { + margin-top: 0px; +} + +th > :last-child, +td > :last-child { + margin-bottom: 0px; +} + +/* -- figures --------------------------------------------------------------- */ + +div.figure, figure { + margin: 0.5em; + padding: 0.5em; +} + +div.figure p.caption, figcaption { + padding: 0.3em; +} + +div.figure p.caption span.caption-number, +figcaption span.caption-number { + font-style: italic; +} + +div.figure p.caption span.caption-text, +figcaption span.caption-text { +} + +/* -- field list styles ----------------------------------------------------- */ + +table.field-list td, table.field-list th { + border: 0 !important; +} + +.field-list ul { + margin: 0; + padding-left: 1em; +} + +.field-list p { + margin: 0; +} + +.field-name { + -moz-hyphens: manual; + -ms-hyphens: manual; + -webkit-hyphens: manual; + hyphens: manual; +} + +/* -- hlist styles ---------------------------------------------------------- */ + +table.hlist { + margin: 1em 0; +} + +table.hlist td { + vertical-align: top; +} + +/* -- object description styles --------------------------------------------- */ + +.sig { + font-family: 'Consolas', 'Menlo', 'DejaVu Sans Mono', 'Bitstream Vera Sans Mono', monospace; +} + +.sig-name, code.descname { + background-color: transparent; + font-weight: bold; +} + +.sig-name { + font-size: 1.1em; +} + +code.descname { + font-size: 1.2em; +} + +.sig-prename, code.descclassname { + background-color: transparent; +} + +.optional { + font-size: 1.3em; +} + +.sig-paren { + font-size: larger; +} + +.sig-param.n { + font-style: italic; +} + +/* C++ specific styling */ + +.sig-inline.c-texpr, +.sig-inline.cpp-texpr { + font-family: unset; +} + +.sig.c .k, .sig.c .kt, +.sig.cpp .k, .sig.cpp .kt { + color: #0033B3; +} + +.sig.c .m, +.sig.cpp .m { + color: #1750EB; +} + +.sig.c .s, .sig.c .sc, +.sig.cpp .s, .sig.cpp .sc { + color: #067D17; +} + + +/* -- other body styles ----------------------------------------------------- */ + +ol.arabic { + list-style: decimal; +} + +ol.loweralpha { + list-style: lower-alpha; +} + +ol.upperalpha { + list-style: upper-alpha; +} + +ol.lowerroman { + list-style: lower-roman; +} + +ol.upperroman { + list-style: upper-roman; +} + +:not(li) > ol > li:first-child > :first-child, +:not(li) > ul > li:first-child > :first-child { + margin-top: 0px; +} + +:not(li) > ol > li:last-child > :last-child, +:not(li) > ul > li:last-child > :last-child { + margin-bottom: 0px; +} + +ol.simple ol p, +ol.simple ul p, +ul.simple ol p, +ul.simple ul p { + margin-top: 0; +} + +ol.simple > li:not(:first-child) > p, +ul.simple > li:not(:first-child) > p { + margin-top: 0; +} + +ol.simple p, +ul.simple p { + margin-bottom: 0; +} + +dl.footnote > dt, +dl.citation > dt { + float: left; + margin-right: 0.5em; +} + +dl.footnote > dd, +dl.citation > dd { + margin-bottom: 0em; +} + +dl.footnote > dd:after, +dl.citation > dd:after { + content: ""; + clear: both; +} + +dl.field-list { + display: grid; + grid-template-columns: fit-content(30%) auto; +} + +dl.field-list > dt { + font-weight: bold; + word-break: break-word; + padding-left: 0.5em; + padding-right: 5px; +} + +dl.field-list > dt:after { + content: ":"; +} + +dl.field-list > dd { + padding-left: 0.5em; + margin-top: 0em; + margin-left: 0em; + margin-bottom: 0em; +} + +dl { + margin-bottom: 15px; +} + +dd > :first-child { + margin-top: 0px; +} + +dd ul, dd table { + margin-bottom: 10px; +} + +dd { + margin-top: 3px; + margin-bottom: 10px; + margin-left: 30px; +} + +dl > dd:last-child, +dl > dd:last-child > :last-child { + margin-bottom: 0; +} + +dt:target, span.highlighted { + background-color: #fbe54e; +} + +rect.highlighted { + fill: #fbe54e; +} + +dl.glossary dt { + font-weight: bold; + font-size: 1.1em; +} + +.versionmodified { + font-style: italic; +} + +.system-message { + background-color: #fda; + padding: 5px; + border: 3px solid red; +} + +.footnote:target { + background-color: #ffa; +} + +.line-block { + display: block; + margin-top: 1em; + margin-bottom: 1em; +} + +.line-block .line-block { + margin-top: 0; + margin-bottom: 0; + margin-left: 1.5em; +} + +.guilabel, .menuselection { + font-family: sans-serif; +} + +.accelerator { + text-decoration: underline; +} + +.classifier { + font-style: oblique; +} + +.classifier:before { + font-style: normal; + margin: 0 0.5em; + content: ":"; + display: inline-block; +} + +abbr, acronym { + border-bottom: dotted 1px; + cursor: help; +} + +/* -- code displays --------------------------------------------------------- */ + +pre { + overflow: auto; + overflow-y: hidden; /* fixes display issues on Chrome browsers */ +} + +pre, div[class*="highlight-"] { + clear: both; +} + +span.pre { + -moz-hyphens: none; + -ms-hyphens: none; + -webkit-hyphens: none; + hyphens: none; + white-space: nowrap; +} + +div[class*="highlight-"] { + margin: 1em 0; +} + +td.linenos pre { + border: 0; + background-color: transparent; + color: #aaa; +} + +table.highlighttable { + display: block; +} + +table.highlighttable tbody { + display: block; +} + +table.highlighttable tr { + display: flex; +} + +table.highlighttable td { + margin: 0; + padding: 0; +} + +table.highlighttable td.linenos { + padding-right: 0.5em; +} + +table.highlighttable td.code { + flex: 1; + overflow: hidden; +} + +.highlight .hll { + display: block; +} + +div.highlight pre, +table.highlighttable pre { + margin: 0; +} + +div.code-block-caption + div { + margin-top: 0; +} + +div.code-block-caption { + margin-top: 1em; + padding: 2px 5px; + font-size: small; +} + +div.code-block-caption code { + background-color: transparent; +} + +table.highlighttable td.linenos, +span.linenos, +div.highlight span.gp { /* gp: Generic.Prompt */ + user-select: none; + -webkit-user-select: text; /* Safari fallback only */ + -webkit-user-select: none; /* Chrome/Safari */ + -moz-user-select: none; /* Firefox */ + -ms-user-select: none; /* IE10+ */ +} + +div.code-block-caption span.caption-number { + padding: 0.1em 0.3em; + font-style: italic; +} + +div.code-block-caption span.caption-text { +} + +div.literal-block-wrapper { + margin: 1em 0; +} + +code.xref, a code { + background-color: transparent; + font-weight: bold; +} + +h1 code, h2 code, h3 code, h4 code, h5 code, h6 code { + background-color: transparent; +} + +.viewcode-link { + float: right; +} + +.viewcode-back { + float: right; + font-family: sans-serif; +} + +div.viewcode-block:target { + margin: -1px -10px; + padding: 0 10px; +} + +/* -- math display ---------------------------------------------------------- */ + +img.math { + vertical-align: middle; +} + +div.body div.math p { + text-align: center; +} + +span.eqno { + float: right; +} + +span.eqno a.headerlink { + position: absolute; + z-index: 1; +} + +div.math:hover a.headerlink { + visibility: visible; +} + +/* -- printout stylesheet --------------------------------------------------- */ + +@media print { + div.document, + div.documentwrapper, + div.bodywrapper { + margin: 0 !important; + width: 100%; + } + + div.sphinxsidebar, + div.related, + div.footer, + #top-link { + display: none; + } +} \ No newline at end of file diff --git a/_static/clipboard.min.js b/_static/clipboard.min.js new file mode 100644 index 0000000..02c549e --- /dev/null +++ b/_static/clipboard.min.js @@ -0,0 +1,7 @@ +/*! + * clipboard.js v2.0.4 + * https://zenorocha.github.io/clipboard.js + * + * Licensed MIT © Zeno Rocha + */ +!function(t,e){"object"==typeof exports&&"object"==typeof module?module.exports=e():"function"==typeof define&&define.amd?define([],e):"object"==typeof exports?exports.ClipboardJS=e():t.ClipboardJS=e()}(this,function(){return function(n){var o={};function r(t){if(o[t])return o[t].exports;var e=o[t]={i:t,l:!1,exports:{}};return n[t].call(e.exports,e,e.exports,r),e.l=!0,e.exports}return r.m=n,r.c=o,r.d=function(t,e,n){r.o(t,e)||Object.defineProperty(t,e,{enumerable:!0,get:n})},r.r=function(t){"undefined"!=typeof Symbol&&Symbol.toStringTag&&Object.defineProperty(t,Symbol.toStringTag,{value:"Module"}),Object.defineProperty(t,"__esModule",{value:!0})},r.t=function(e,t){if(1&t&&(e=r(e)),8&t)return e;if(4&t&&"object"==typeof e&&e&&e.__esModule)return e;var n=Object.create(null);if(r.r(n),Object.defineProperty(n,"default",{enumerable:!0,value:e}),2&t&&"string"!=typeof e)for(var o in e)r.d(n,o,function(t){return e[t]}.bind(null,o));return n},r.n=function(t){var e=t&&t.__esModule?function(){return t.default}:function(){return t};return r.d(e,"a",e),e},r.o=function(t,e){return Object.prototype.hasOwnProperty.call(t,e)},r.p="",r(r.s=0)}([function(t,e,n){"use strict";var r="function"==typeof Symbol&&"symbol"==typeof Symbol.iterator?function(t){return typeof t}:function(t){return t&&"function"==typeof Symbol&&t.constructor===Symbol&&t!==Symbol.prototype?"symbol":typeof t},i=function(){function o(t,e){for(var n=0;n