-
Notifications
You must be signed in to change notification settings - Fork 5
Meeting Notes
Carsten Fortmann-Grote edited this page Nov 11, 2022
·
36 revisions
- generally very good and readable.
- refer to tasks and deliverables that each contribution belongs to
- add references and cross links (to code repos, datasets, etc.)
- summarize main outcome (bullet points ok)
- use Mads document as starting point where everyone can enter their contributions
- all documents should go into this repo under documents/reports
- important: refer to Ts and Ds
- Grenoble, Nov. 29-30
- suggest shervin gives the speach.
- as shervin not present, final dec. still pending
- checked all PRs for lpv, will make new release
- prepares for releasing our developments via conda (conda-forge)
- dynamic reading and writing is now public and documented and tested
- worked on putting NMX instrument on InstrumentDB. This will become the template for how to add ESS instruments to the IDB
- OASYS-SimEx interaction widget upgrade is complete (accessible as
v0.3.2on PyPI) - A JSON file is used to pass parameters to SimEx (can be easily extended in the future)
Still sick. Working example of EuXFEL SPB instrument: https://github.com/PaNOSC-ViNYL/instrument_database/pull/1
Added unit tests for latest feature branch of McStasScript and merged with master. Fixed some issues that had crept in from merging a few pull requests on the master branch. Will make a pip release shortly, bringing the new feature of cutting the instrument into pieces online, along with documentation updates.
https://github.com/orgs/PaNOSC-ViNYL/projects/5/views/1
- iterative workflow: DA part (xwiz) is not a notebook tool -> PMC (Carsten)
- date for 1week sprint: end september/early october: sept. 26-30
- when is the final panosc meeting?
- please update items on https://github.com/orgs/PaNOSC-ViNYL/projects/5/views/1
- continued DA+Sim use case
- Generating data to be analyzed
- Will generate dedicated tool if/after pilot works
- ctd on McStasScript:
- output parser (sorting, highlighting)
- awarded EANS award
- preparing for summer school at ELI-ALPS
- prepared PRs for libpyvinyl, iDB, IDB-API, to be reviewed
- preparing for summer school(s)
- gapd calculator is interfaced through oasys
- will implement smooth dependency check to avoid having to take care of simex & gapd installation on oasys site
- summer student developed a video on oasys (on panosc website)
- paper with singapore group, required changes in shadow3
- new paper on wofry
- worked on shadow3 <-> lpv coupling, mostly working, some tunings required. will provide issues on github to clarify some roughnesses
- juan contributing to summer school
- contribution to simex-in-oasys calculator, can be discussed next meeting
- met andy and jordi at esrf
- new eosc project may be in the planning
- performing sfx + data analysis workflow, works with alexey (xwise): can read sim data and analyse
- ask alexey to present in this meeting in ~3weeks
- improved visual workflow in mcstas
- mcstas workshop, remade all material into notebooks
- featuring quizzes with answer checking
- -> new users
- preparing for ICNS conference in Argentina on McStasScript,
- workshop on good data practise in september
- gives short summary about wavefront4ML project
- need to fix layout -> Juncheng
- add top table -> Juncheng
- Carsten provide examples (D1.8)
- plan to release simex-lite
- improving McStasScript
- merged pr to make more compatible with lpv
- added geometry visualizer to debug geometries (coordinate system references)
- Shervin and Gergo provide comment
- will work more on WP8
- Finishing the IDB-API with documentation for users and for developers
- populate IDB with ILL components
- suggests to have sprint and/or in-person meeting
- maybe at panosc end meeting
- summerschool and preparing for workshops
- visar running on openstack, requiring substantial resources
- wp5 shows that different solutions are possible (jupyter + gui components), better maintainable through containers
- shervin sees visars main point in AAI provision to services at facilities
- will use sfx example for deliverable D5.4
- IO performance may deteriorate but this is not a major issue
- Carsten supports the plan, Jun will go ahead with it
- needs review (CFG, SN, MB)
- looking for benchmark data, hard to get
- Carsten explains his ideas behind this Deliverable
- Juncheng and Carsten and Fabio will meet next Wednesday to discuss SFX application
- Every institution needs to think about sustainability plans
- Possible funding schemes were presented by Andy, due in 2023 including maintenance money
- put 3axis spectrometer instrument into instrumentDB
- demonstrates scanning of energy distributions and plotting for validation
- suggests adding mcstas-openpmd-IO module to next mcstasscript release
- mads provides some comments and pointers to solutions for coding problems
- Aljosa added material to report
- currently only with accounts (McStas, Simex, Oasys)
- Aljosa added material on simex-shadow interface to report
- documentation is becoming more mature
- layout still needs some work.
- work is in docs-shervin branch
- carsten to provide feedback after monday/tuesday
- docs finalization is highest prio
- instrumente database ok -> shervin will add links to report
- instrument database api still needs work
- reports: work on github version, not overleaf
Mads - Instrument database in the notebook possible? - How complicated should the notebooks be?
Shervin
- Define Master parameters in the instrument script could be tedious.
- Proposed dedicated meeting about the instrument database next week.
- Should not rely on the default parameters for the instrument script in the database.
- Current master after squashing lost all the contributing information, which is not good.
- Populating the database
- SimEx-Lite example was valuable
- Sphinx plugin to extract type hint for variable type documentation.
- Added SimEx-Lite and ILL examples to the instrument database.
Juncheng
- Shared some thing learnt from the collaborative workshop
Gergely
- Summer school
2022-03-25 Attendees: Juncheng, Manuel, Mads, Aljosa, Juan, Shervin
Shervin: Github CI
- Tigger documentation generation
Database API
- Documentation for that
- libpyvinyl
- Set the documentation
- The features should be available
- Jun will send out the SimEx-Lite instrument exmaple
- Next week DEMO of the API
Jun
- SimEx-Lite PR addressing
Mads
- New release McStatsScript based libpyvinyl has been published
Manuel
- OASYS wofry, submit a paper
Juan
- Ressearch and paper
Aljosa
- WISER
- Deliverable
- tasks are in issue #1 in istrumentdatabaseAPI repo (see project)
- shervin has a prototype instrument for D22@ILL, can maybe adopt it in simex-lite
- reviewed by Shervin -> remove unneeded files (ignore)
- Carsten presents the sprint board and edits according to comments
- Please all review the board and add stuff as needed
- see issues in instrumentdatabaseapi repo and here
- branch "design" in instrumentdatabaseapi repo
- sprint: get one simex and one mcstas instrument
- Improve introduction
- Design guide, how is lpv intended to be used
- Add DataClass to Reference Manual
- need suggestions for MD based laser-matter interaction
- A design testing was created: https://github.com/PaNOSC-ViNYL/instrumentDataBaseAPI/blob/design/tests/test_instrumentdatabaseapi.py
- Compatibility of parameter classes in libpyvinyl: Shervin will open an issue discussing that.
-
instrument.set_sampleandinstrument.set_sample_env()functions are needed. They are wrappers for setting sample-involved simulation tools. - Tracking changes of instruments is hard
- It can be expected that 80% of the users would only download the instrument, only 20% will contribute/modify the instrument database
- Some changes are required in
instrument:- required functions to be implemented
- The overall license to choose. MIT? BSD? GPL? Maybe Carsten can give some suggestions. What is the requirement of PaNOSC/EU? -> not GPL, should be a license that allows linking and packaging in commercial software (could be LGPL) ->
- Worked on PRs, need green light for #36 ([ ] TODO Carsten),
- thoughts on instrument db apis
- need requirements before sprint
- implementation during sprint
- need repo and open issue(s) to start discussion, initial PR with prototype API like for dataclass
- should start from tests,
- discussion scheduled for next meeting
- worked on documentation, ready for libpyvinyl release
- reviewed dataclass in lpv
- needs to implement data classes for mcstas data
- received commenst on data class by gergo and mads, addresses them by early next week
- noticed that PR misses updates of base calculator, will create new PR
- plan to finish by wednesday
- release then needs to be postponed by one wook
- partial testing with simex done
- got review from openpmd for raytracing
- need to make clear it's also for protons etc.
- fix index -> +1
- commented on reviewsubmitted notebook to stella
- BaseData etc.
- Various smaller ones
- Merge in given order
- Review tasks
- new release beginning of February -> version ?
- documentation will be taken care of after release, API docs will be accurate.
- Currently docs are on readthedocs -> move to github? don't touch now, maybe later
- generate examples
- write down thoughts about how it is going to be used
- carsten, mads, shervin, gergo to review PR #37.
- looking into running oasys directly from jupyter hub
- re deliverable: oasys wspace repo fine? will improve a bit, add more beamlines. in workspace add more information
- oasys course on pan-learning is in the planning
- oasys tutorial in hercules course this year (march)
- start creating videos
Instrument database: -> library of examples. should contain instrument descriptions, can then be used for benchmark studies
- thales instrument description (shervin)
- spb-sfx -> juncheng
- pr from sprint 80% done, will need review by carsten
- finished jupyter notebook
- finished most work to use lpv in mcstasscript
- had to move from travis to github actions
- needed to update documentation -> now in sphinx, not yet the pip released version (?)
- documentation on github pages, not rtd. should lpv follow the same pattern?
- https://mads-bertelsen.github.io/
- should be structured along the same way as elearning (wp8)
- juncheng working on comments from sprint
- use lysozyme dataset for benchmark study, will be used in wp4 demonstrator
The meeting takes place in zoom. Please contact the repo's maintainer if you want to participate.
- Carsten: quite useful since so generic
- Aljosa: probably still quite premature, may not be ripe for us right now, maybe CWL better solution
- Mads: prefers scripting the entire simulation workflow
- Manuel: generic system, still quite young stage, orange much more mature. intermediate solution: use workflow to create py script, then continue editing the script manually
- juncheng: simex lacks mpi script "creation", maybe this is an opportunity
- starts: Dec. 1st, until Dec. 14
- tasks organized in github project at https://github.com/PaNOSC-ViNYL/ViNYL-project/projects/10
- openpmd alignment with instrument database
- workflow = directed graph (cyclic/acyclic)
- abstract, not another wms
- pypushflow supports cyclic graphs
- mongodb backend
- ? can import ows?
- ? how data stored? object storage? -> can pass data in memory, final data goes to disk.
- ? can we execute oasys workflows on the web (with ewoksweb)?
- ? setup on existing infrastructure?
- ?(juncheng) dask execution on dask? -> not sure
- ?(juncheng) task dependencies: automatically taken care of by the wms
- ?(mads) how can infinite loops be avoided?
- logic of the graph is up to the user
- needs a fork at one of the looped nodes
- dependencies: e.g. conda suffices to organize dependencies, install ewoks into that conda environment
WPG simDB working points
- SASE1
- SPB-SFX
- KB microfocus and nanofocus
- CRL also possible
- best for hard x-ray beamlines and
- sase2: monochromator for MID and HED
- delay line (SASE3)
- photon Energy: 4.96, 9.0, 12.0
- at focus: full wpg script
- at KB aperture: provide data
- distortion in optical tunnel
- liuba provides script from RB's paper in gh repo
- has not misalignment option
- Adrian's student could do the hard work (comes from optics background in Paganin's gang)
- next meeting at 2 on nov. 19
- Fabio in charge since April
- h5web visualization of h5 files, also as jlab plugin
- fid nmr viewer
- available for public? is one use case and still under dev but alpha release available
- extra-data, extra-geom
- extra-xwiz: crystfel wrapper and workflow manager: from euxfel-h5 data to protein structure
- euxfel2nexus (wip)
- reformat or wrapped -> no transformation, virtual datasets (since hdf5-1.10), metadata added which makes it nexus
- pan portal:
- historical idea: unified service with search api, jupyter, VMs -> RDP or jupyter
- microservices
- production service in operation at ill: visa
- modification to port for general audience:
- visa kept as is, deployed at RIs
- for pan users: redirection to existing services after search api result
- how cameo is accessed: can be through visa or remote desktop
- karabo: allows remote read-only access
- how can we get simulation data into data analysis
- deployment of visa: desy openstack instance
- data provided via nfs mount
- software into visa:
- ill has one big visa image
- cvnfs mounts
- don't have one general platform but site specific installations, run locally at site
- additionally: federated instance hosted by egi (future) -> have to deploy simulations into RI's site installation of VISA
- maybe common simulation data api to produce the right simulation file
- at euxfel: metadata sources spread, standardization is now being addressed
- how to overcome nexus differences, have to simulate data in exactly the nexus format used at RI
- want to strength activities in WP5, esrf so far underrepresented
- juan: runs oasys simulations
- align activities with panosc
- contributions:
- include current oasys calculations (upload workspaces and scripts to workspace repo)
- wofry 1d (wave propagation in wiser style but 1D and specialized for lenses and undulator sources)
- support "any" crystal structure
- shadow4 -> move all raytracing code to python
- use libpyvinyl
- containerize comsyl (complex dependencies)
- oasys school
- Presentation by Mousumi (and Shervin?)
- No dedicated WP sessions -> should instead schedule a sprint. Mads available this year -> likely in Dec.
- Shervin's gantt chart: https://panosc-vinyl.github.io/tools/
- Added issues for Cloud deployment, nomad-connector
- TODO: All contributors review resource allocation graph
- carsten send docker image id to stella
- preparing for ikon py workshop (inkind contributions to ess)
- merged instrument class into master branch of pyvinyl, can make release on pypi
- asks shervin to review gh issues re instrument db
- needs feedback for calculator classes -> carsten, shervin
- busy with tender process for ocre cloud resource, tending towards amazon or exascale
- oasys will be deployed on these resources, access similar to visa
- elettra (ceric-eric) will deploy invinio based zenodo like data repo
- looked into paper by mousumi and provided feedback
- writing sim-vs-sim section
- improving openpmd c++ api doc and CI (travis)
- now testing
- component not compatible with mcstas3, looking into that
- open day at eli, preparing poster
- panosc, use case
- had issues with software, figured these out
- would be great to have presentation from Fabio and/or Andy at next meeting
- Linked to deliverable D5.?
- Panosc toolbox in Oasys:
- openpmd IO
- remote ows repository
- Repo can be github, ftp, http. Link should be to a json file containing ows files
- currently ows cannot be opened directly, instead opens from locally downloaded file
- json file in github repo
- -> metadata should contain some info on purpose, typical scientific use cases
- paths in json are absolute, makes transfer of repo hard
- questions:
- version compatibility with binary ows files
- updates in gui will lead to breaking ows files
- committing to repo -> upload
- ows is xml with a pickle inside, so can extract information. could store version
- How are parameters used in simex, consistency checking could be done
- suggestion: calculator knows about needed parameters, inform user about these
- can then move parameters checking into calculator
- Juncheng will start an issue on github and start discussion there
- new issue on nomad integration, carsten to take a look
- mads will be on reduced working hours for a couple of months (until dec.)
- adaption to new pyvinyl version will have to wait, not mature enough
- libpyvinyl development: needed for simex lite. pending PR for instrument class -> carsten
- travis not running -> maybe switch to github
- needs simex docker image
- existing image has no jupyter https://hub.docker.com/r/trnielsen/jhub_py38_ikon21_nb
- need image that already has jupyter
Jun, Shervin, Mads, Mousumi, Gergely, Aljosa, Fabio, Doro
Shervin asked Fabio questions about WP4:
- Fabio explained WP4 is doing.
- Analyze software available in EuXFEL
- Data access
- Software catalog
- Integrate simulation software there
- Simulation and experiment comparison
Mousumi
- Summer school was successful.
- Will discuss using SimEx for material science next week
Aljosa
- He is going to present the OASYS gadget for reading and saving instrument setup.
Juncheng
- Instrument class pull request
- Pan-learning script development
Shervin
- API for openPMD raytracing
- Maybe Aljosa can submit an openPMD standard pull request?
Mads
- Master parameters pull request
- NOMAD integration
- asks Mads to code-review the libpyvinyl parameters class
- then shervin can finalize
- paper published, suggest to also publish the code material
- behind schedule implementing nomad into mcstassscript
- looked into parameter class, ok with overall architecture
- anything to prep for annual meeting?
- working on bug reports on mcstasscript
- docker image for simex starts in own jupyter service
- have to modifiy this so jupyter containers can pull in the simex
- Mousumi will have presentation in ELI summerschool
- first jupyter notebook for e-learning project: diffraction from a wire
- asked for feedback
- paper reviewing
- try help solve an issue with wpg beamlines for juncheng
- creating jupyter notebooks and scripts for e-learning platform and for instrument database (shervin)
- propagated wf looks wrong, beamline and prop. parameters have to be adjusted for
- opened PR for abstract parameters class for mcstasscritp, needs review
- on vacation
- thought about input/importing instrument description, use 'import' keyword, pushed to branch
- demo on downloading oasys workspaces from repository
- directly opening dl'ed workspace would overwrite current workspace, also not possible to do from within widget
- solution:
- widget acts as a repo browser
- provides workspace links
- select dir where to save it
- then open via File->Open
- could not contribute too much to ongoing discussion on instrument database
- .ows files contain discription. can also split into two, this way user can preload. this is what the widget does
- ELI summerschool on computing w
- Presentation on e-learning platform
- howto add jupyter notebook/containers to the platform
- can contribute own docker containers
- needs manual intervention by stella and ess staff (platform learning export and service provider)á
- should try with simex container and simple diffraction notebook(s)
Attendees: Juncheng, Shervin, Aljosa and Stella
- Looking at the instrument parameter object
- Serialize .json
- Full representation of the instrument class
- Save the actual values
Conclusion: - Jun and Shervin agree on saving actual values - To track the changes of the internal parameters when comparing them. - Most data in neutron and photon facilities are in 2D.
What to do next: - load: to execute the python file. - diff: track the changes of the internal variables - A diff function to do the job. - Have a set_parameters() method in libpyvinyl - Reuse the unchanged data between Calculators
Consider an openPMD data API dealing with the data object. - save->data->format
Attendees: Juncheng, Shervin, Mads, Mousumi, Gergo, Aljosa.
- Shervin discussing the instrument repository
- Mousuimi: troubleshooting pages are important for users.
- Jun: That goes into the API repository
- CI workflow to check if the repository is updated
- Can be run under with a container
- Split the issues into different categories
- We agree that this task is important for the success of this project since it's where things get converged.
- Mousuimi: troubleshooting pages are important for users.
- Gergo wants to have fewer constraints for the parameters.
- The constraints are controlled by the list of limits of parameters.
- The recommended way to do it is to have a copy of the new instrument and change the parameter.
- nexus format
- yuri will leave euxfel
- exporting will only support agipd
- plan was to support arbitrary detectors using crystfel format detector geom through extra-geom
- also thomas kluyver leaving wp4
- digital twins:
- one wp for engineering
- accelerator, propagation
- experiments -> pmi, signal generation, detector response
- scientific problems -> use platform to solve problems
- scalability
- digital twins call different from panosc scope (extend tools instead of exposing them)
- link to experiment
- material science:
- can we do 4d scattering (kk, w) -> not doable at the momenet
- centralized file format (cif files)
- preparing machine for demo
- can give it in next meeting, using nomad for daq and simulations
- new release of cameo (communitation between server, gui, mcstas sim.)
- worked with stella on data comparison protocol -> needs review and data/sim as written example
- document is on https://www.overleaf.com/project/60a7a2277f13716b96b17d55
- WP5 review on Sept 1st:
- need summary slides from each WP5 funded participant
- highlight how activities address our workplan (
- call for eosc future participants: https://eosc-portal.eu/call-researchers-shape-eosc-future-solutions?utm_source=Email&utm_medium=Email&utm_campaign=Consortium-mail&utm_content=User-Group-1
- collaborating with wp8, developing new courses
- made issue in vinyl gh -> needs action by carsten and/or juncheng:
- saxs doable
- others need more work
- reflectometry: maybe reuse mohammad's thesis project
- review of widgets and cryostat builder in progress
- procure commercial cloud resources
- who needs resources and how many
- use usecases as bottom line estimates
- not sure if any hpc resources will be used (shervin)
- initially 15000 euros -> already substantial
- mousumi expresses interest -> amsterdam modeling suite -> tddft
- hdf5 workshop ->
- Mads will add gh issues to our central project, Carsten will then assign
- Juncheng on night shifts so not much done
- exporting simex data to nexus (collab. with yuri from wp3)
- final (observable) data is going to nexus format
- continued on diamond project
- description of potential in simex
- not clear what the potentials are needed for but mousumi will make that clearer
- involved in experiment
- tool section in mcstasscript (setting up,...)
- cryostat builder
- talk at st. petersburg neutron source
- testing for gui
- instrument repo:
- let's use the vinyl gh project
- updated panlearning website
- remote oasys workspace repo
- currently chooser is a widget in oasys, could also be moved to greeter or tools
- do we have consensus to use json (ok for simex or mcstas?)
- suggested to make a dedicated "mini workshop" in the following weeks.
- will shift internal milestone to end october and have a workshop after summer break
- preparing for demo
- working on document for sim-exp comparison
- would be great to have a concrete case
- juncheng will provide a notebook once nexus writer is ready
- openpmd: needs testing feedback
- Shervin presents a draft document:
- use NLL minimization principle with Poissonian distribution
- this is more robust than chi-squared (which corresponds to normal distribution of counts)
- need a few concrete examples
- noise can be added post
- sim data should have ~10x more events to mimic a continuous distribution (compared to the noisy exp. data)
- can we subtract simulation from data -> amounts to chi-square
- discuss some use cases -> work in a jupyter notebook
- submitted use case on using e-learning platform about presenting software in jupyter notebook (not online yet)
- presented in internal talk at ESS
- busy with coordination at ceric (dan webster left)
- expect to be more productive in 2 weeks again
- submitted use cases
- analysed data and simulations
- 2nd use case in preparation
- implement extra-geom in simex-lite
- digital twin proposal ongoing
- discussion on cpu resources for elearning
- test compute setup (students to run simulations)
- cloud resources (any parallelism including gpu)
- e.g. make workshop, run as pilot project
- progress in e-learning
- notebook with widgets to launch simulation, would be great for simex, too
- notebook launched from moodle, runs in jupyter hub using federated moodle login.
- oasys school material: contacted r. celeste, may get involved in dev. oasys learning content.
- working on remote workspaces (documented instrument "database"). make findable: text file holds metadata
- will then populate workspace repositories
- envolved more and more partners, won ocre reward for cloud computing resources, budapest neutron center wants to run mcstas
- 1 use case ready to be submitted
- will share manuscript in following days
- analysis protocol as another use case
- use case for serial crystallography and jungfrau detector
- combines simex with extragreom package
- simulations are for xfel, not for the partially coherent esrf beam:
- can maybe use comsyl webservice
- to consider if qualitative differences are to be expected
- simex-lite:
- IO: use ASE as example/template
- have to specialize IO for each data type (diffr, beam, target)
- Thomas suggests contacting ESS people to align IO efforts
- exporting to nexus is also ongoing (with yuri from wp3).
- github: panosc-nexus/demo
- IO: use ASE as example/template
- seems not well coordinated among centers and with panosc
- Juncheng will represent simex at leaps meeting
- THR made use case involving WP5: https://www.panosc.eu/use-cases/panosc-use-case-17-reusing-data-for-validation-of-force-fields/
- Little returns on keyword search
- most for 1d, 2d data
- higher dimensional datasets ofter refer to ML, mathematically quite involved
- case MD simulation: correlation analysis
- pearson correlation coefficient (in N dimensions)
- discrepancy metric
- R factor
- cosine similarity = pearson coeff
- paper Loudin (2003): discriminant function
- how to treat noise: noise is a parameter in the simulation, simple metrics like chi2 may be used
- process the sim data through the same pipeline as experimental data, are differences relevant or not
- sim data must be in nexus file to allow applying the same pipeline
- compile relevant list of variables of distributions, examples, how to do it in python.
- noise: need multiple cases and do ML analysis
- juncheng will generate data for SPI case (until meeting in 4 weeks) and upload the presentation
- implement new pysingfel
- simex wrapper
- turn presentation into paper form and share in group within days
- use cases in prep
- can we use use case material as a course? -> Gergely: yes, can be used in course material.
- looking at making a sample for mcstas (skyrmions)
- training for oasys: oasys courses are available online (workspace + pdf document)
-
converting mcstas output in ML understandable output (numpy)
- mcstas for beamline sim, train DNN as virtual source to yield beam at a given point
- input: neutron at starting point
-
simex-lite: diffraction data apis, calculator for detector noise to answer effect on orientation and phase retrieval
- Carsten assigned people to issues. Please review and provide comments and ideas.
- Database discussion: make 2 databases, Aljosa takes the lead on instrument database
- So far 4 use cases submitted: 3x CERIC, 1x EuXFEL
- ELI/ESS use case?
- CFEL use case?
- https://github.com/panosc-eu/panosc/wiki/Work-Packages-dashboards#wp5--virtual-neutron-and-x-ray-laboratory-vinyl
- Very sparse at the moment.
- Comsyl: ESRF is working on it (-> Manolo update?)
- Oasys: Can it be accessed as a web / remote desktop service? (-> Aljosa)
- no comments in the gh issues, need more time?
- most immanent: literature research on best-practice sim-exp comparison and benchmarking
- jc will take care of the literature research
- to be submitted at https://www.panosc.eu/submit-your-use-case/
- so far one UC from WP5 (simex)
- candidates:
- mousumi/gergo/mads neutron target simulation+mcstas:
- 1-2 ucs
- use case from the simex user interview
- ibrahim said he needs more time, jc will concact him again
- McStas on GPU for complex scattering geometries
- mousumi/gergo/mads neutron target simulation+mcstas:
- need to identify HPC resources needed to execute the usecases
- mousumi/gergo case:
- mcstas capable of gpu processing
- worked on application cases:
- detector noise and requirements for "good reconstruction"
- anomalous diffraction at spb-sfx
- new version of pysingfel (skopy)
- had presentation on mcstas and mcstasscript in garching (
- McStas 3 / GPU talk: https://www.youtube.com/watch?v=ZQHiLHeI4Tk
- Union talk: https://www.youtube.com/watch?v=xDwZ_ryPl5M
- McStasScript talk: https://www.youtube.com/watch?v=jeM2S4ht9hM
- teaching material in collab with wp8:
- for small angle scattering, execute with n and ph scattering (simex saxs mode needede)
- refractometry with oasys
- what do we need from wp8?
- can elearning platform fire up docker containers?
- working on target simulations and flunshing on the mcstas sims
- next meeting in 14 days
Carsten breaks down tasks into the details
- Tasks into small projects
- Task 5.3
- The database can be anything storing data, can be a repository
- The databse inclues the instrumental information
- Add metadata about instrumentation in openpmd format
- Carsten will PaNOSC hdf server
- Shervin and Mads concered about how detail the description will be
- Instrument simulation database
- The record of the database is the h5 file of the simulation database
- Shervin suggested to have a seperate simulation database and instrumentation database
- Task 5.4
- Protocaol for comparison of raw simulated to raw experiemtal data
- Literature research
- NOMAD Interface
- Jupyter notebooks collections
- How to pull data from the simulation database
- Protocaol for comparison of raw simulated to raw experiemtal data
- Timeline
- A screenshot of the gunshot timeline would be enough
- Task 5.3
- By the next meeting, everyone checks the issues to make it clear and leave comments.
- new postdoc at ess
- need input from carsten on -> carsten will go through the tasks and present next meeting
- clarify databases
- role out libpyvinyl version of mcstasscript
- collab. with wp4:
- use nexus format to store diffraction data
- needs metadata option in simex calculato
- can use virtual dataset function in hdf5
- scientific projects:
- hydrated protein scattering
- make videos, can make videos on simulation tools
- completed python api for cameo package at ILL
- tested in Jan.
- bugfixes going on
- could be used in jupyter notebook
- notebooks are so far run locally
- finished high level API based on openpmd-api for raytracing, could be used by aljosa replacingi his IO widget
- working on improving simulations of instruments, virtual experiments
- scattering component goes at 150kHz, instrument runs faster but wants to avoid MPI. Merging simulations in McStas buggy/misbehaving/unmaintained(?)
- need re-use simulation workflow and store results in database to use for similar sims
- comment from mads: mcstas can only join data for which the splitting is "known".
- using simex and mcstas for scientific case studies
- generate notebooks
- mcstas core library was reviewed, revisions now taking plate
- 3D detectors
- requests from ornl, course with dedicated mcstasscript session + course at garching on mcstasscript
- using nexus by embedding info from other DA packages instead of mantid
- next meeting in a week (Feb. 26)
- use libpyvinyl as umbrella in coming release
- Working on exporting SimEx detector format in NeXus format with WP4.
- Review McStasScript. It's close to finishing.
- Teaching people how to use it within WP8.
- Detector simulations.
- Helping Aljosa with OASYS development
- Preparing WP8 Oasys teaching material
- Compile comsyl code in newer HPC cluster
- Interested in NeXus format
- DFT simulations coupling with McStas
- Package implementation onto cloud service.
- Get connections to neutron people in WP5
- Get in touch with Carsten
- Get in touch with Manuel about plans related to task 5.3 and 5.4 and connections with WP4 for D5.4
- Raytracing API can be installed and tested
- https://github.com/PaNOSC-ViNYL/openPMD_raytrace_API/tree/devel
- Catching up the MacStats sprint
- Clean up
- Collaborate with Mousume to use McStasscript to simulate the neutron scattering from DFT results
- Using SimEx in the hydrated molecular SPI simulation as an example
- Updated the message from PMC to provide user stories to WP9
- Has been working on gromacs calculator for SimEx
- McStas Workshop
- GPU development for McStas
- Will provide user feedback
- User stories were sent to Nicoletta
- Wiser software and widget for PaNOSC are under development
- Python binding of the raytracing data API.
- camel cpp API is done. python API will come out soon.
- No news
Carsten was absent. Juncheng hosted the meeting.
- Updated the demo section of SimEx and docker in the M5.2 report.
- Will create the SimEx-0.5 release on zenodo.
- Will fix the SimEx readthedoc issue next week.
- Developing Raytracing openPMD CPP API. openMPD API was forced to follow CPP11, which causes incompatibility with CPP 17/18.
- It would be nice if Aljosa would test it.
- Presentation preparation
- McStats script code is under review.
- Mads has shared the code developing review policy in his team with us. All the codes should be reviewed before release, and everyone in the team is obligated to review the code. A developer reference documentation attached in the code repository would be helpful for the reviewer who just newly jumps without prior knowledge to catch up.
- The affiliation of Dan can be written as CERIC-ERIC
- Must add demo material, ideally videos to our simulation services. ok if services are limited access
- Mads will give the presentation, please be ready to support him
- got review on mcstas script and created prototype for mcstasscript-libpyvinyl
- developed plans to access notebooks through oasys, showed quick demo
- added simex content to ms5.2
- merged several pull requests in simex
- will test container with dan
- Dev on openpmd_raytrace_API, subclasses for photon and neutrons
- working with juncheng on simex containers
- need user testimonials, references, statements
- starting mid november, juncheng will participate in the weekly pmc meeting
- Carsten scrolls through current status and points out assignments
- TODO: publish releases on zenodo, summarize all DOIs in a short document to be submitted with the Deliverable.
- aljosa needs permission to create new repo
- will contain finalized openpmd c++ api
- on top of openpmd api
- uses backend from openpmd-api
- py binding of cameo
- currently testing (expected to end next week)
- user connects to visa, choose experiment, have nomad program (interface to instrument), can switch to simulation
- will use mads' prototype
- in touch with Luca, now working in his requests
- started on openpmd
- out sick, now catching up
- will work with dan on jupyter hub
- documenting detectors in mcstas-script
- mcstas script is to be obtained seperately from mcstas.
- currently few users, training materials under review, will create more users (also through wp8)
- following developments
- Axel could work on workflow engine to be used with simulation libs (besides oasys)
- catalogue of beamlines at esrf? potential issues when libs need update
- aljosa comments that rafec container is rebuild on daily basis
- Carsten presents the tasks for next sprint: https://github.com/PaNOSC-ViNYL/ViNYL-project/projects/9
- The sprint starts on Sept. 14
- Daily scrum meeting at 9 on zoom
- working on the SimEx documentation
- working on the propagation notebook
- next week will prepare the sprint and present the tasks.
- works on the notebook for WP8
- Check SimEx examples
- Prepare DFT simulation for the sprint
- his organization changed, thus needed discussions with Carsten about his work arrangement.
- considering about sprint tasks
- OASYS WISER will be released soon.
- Data acquisition server for McStats from simulation
- Thinking about how to allow the user to select sample though the server interface
- working on openPMD McStats
- MongoDB to store McStats data
- shared links for jupyter HDF viewer http://h5.apps.okd1.ceric.fedcloud.eu/#
- Juncheng figured out issue with WPG in Simex
- fixed openmpd read issue, now need parallel IO
- works on tutorial notebook for wp8, 2nd notebook needed: carsten suggests wpg notebook, wip.
- openshift cluster up
- met aljosa to discuss various
- Juncheng figured out issue with WPG in Simex
- fixed openmpd read issue, now need parallel IO
- works on tutorial notebook for wp8, 2nd notebook needed: carsten suggests wpg notebook, wip.
- openshift cluster up
- met aljosa to discuss various
- would like to see simex in e-learning platform, jupyter could work.
- security issues: arbitrary c code executable?
- wiser release in spie conference. very good feedback. list of bugs received
- chat with andrea about what to do in sprint
- manolo started on openmpd widgets, copys from aljosa's last sprint devs
- validator
- viewer for h5nuvola for openpmd (?)
- issue with VE
- recuperated openpmd module for mcstas, almost finished.
- MPI feature not obvious, maybe sth. for the sprint.
- File size doubles (hdf5 backend), mpcl still needed for temporary results. maybe adios better
- probably conversion tool more suitable
- Ideas for tasks and stories for the next sprint, please collect on https://github.com/PaNOSC-ViNYL/ViNYL-project/projects/9
- no presentation since content mostly known
- documentation errors
- openshift running on desy egi instance
- can deploy to portal, add apps so partner can use them
- openpmd i/o component
- can write
- expanded shervin's work to read into mcstas
- issue in writing: only 1st chunk is written, debugging
- wiser release, talked to manolo about shadow i/o with openpmd. widget exists
- most votes for 14-25 september, shervin's vote still missing
* Example notebook how to use wpg calculators
* Checked API refman
* RTD look for simex manual
* separated manual into sections
* problems to build refman -> need to have full simex library importable
* will make a doodle regarding 2nd sprint in september
* working on installing oasys ond desy's egi cloud server, WIP. Offers help for installing simex on the same platform
* review meeting, generally positive feedback, suggestion to integrate more
with other eosc cluster projects (eosc-life, ...)
* on vacation from july 6.
* detector systems: prototype running, will demo next week
* requests to prioritize the gh issues and provide more details.
* list of documentation needed modules
- needs help building docs locally, carsten answered
* spie meeting abstract -> will be online, talks available online
* wiser development, went deep into code
* ? contact to manolo:
* discussion on prt openpmd: beamphysics extension close but not the same
- will prepare presentation and then meet with lume people to discuss
steps fw
* nothing to report
* will think about possible involvement with repo2docker tasks.
- busy with mcpl to openpmd-api
- openpmd component in mcstas: c++ vs. c ?
- wednesday meeting on cameo py api.
* Preparing for review meeting.
* DFT simulation for neutron.
* Check the DFT simulation parameters.
* Listed the modules needed to be documented for SimEx last week.
* Will work on SimEx documentations this week.
* Read through the GitHub issue, it's not detailed enough.
* Working on testing system for MacStas script.
* Next project: 3D detectors in MacStas.
* Could we have a Prioritizing list for the issue?
* Went through the github issue, to include openPMD in neutron scattering.
* Failed to understand openPMD API, try to work on it slowly.
* Has been following community development of HL7 standard, a standard used in medical records. If I learn anything is that standards are usually chosen by few, accepted by some, forced on many. In HL7 case study, "few" = hardware sellers.
* feedback from the 18M review rehearsal: very positive, need to work a bit
on the tutorial (w/ juncheng)
* close to enable installation of simex in conda environment
* worked on demo for wp4/wp5
* will continue on list of modules for documentation
* type checking progressing
* example library has many type errors
* add integration test
* proper review going on
* wiser: v1.0 postponed by some weeks
* works on example work spaces for elettra
* busy with ceric internal stuff due to politics
* office reopened, will return to panosc work soon.
* carsten needs to add dataset to zenodo
* change input file in notebook
* recommends to have another sprint
* eli alps hired another person -> mostly for wp8
- finished master thesis
* worked out first version for simulation server
* wanted to include mongo cpp api despite insufficient documentation
* started to look at wavefront, needs information on openpmd-wavefront
extension
* working on demo on simulation service
- worked on 18M report
- worked on demo for review meeting
- error in travis build from tnsa ion interaction test
- intensity_dump function ready to be merged -> carsten
- work on installation instructions for simex
- can look into simex installation with egi service on desy cloud infrastructure
- carsten to send links to andrea
- Can use prototype eosc-pan-jhub.desy.de
- need egi aai account: aai.egi.eu
- Carsten presented new tasks: https://github.com/PaNOSC-ViNYL/ViNYL-project/projects/8
- Assignees should review their tasks, write comments/question/ideas into the issue
- Will coordinate next meeting
- No formal sprint at this time, maybe when mads has returned from vacation
- Carsten will need more material for 18M report and review meeting from all sites, stay alert.
- Meetings
- Reporting due soon
- fixed pyviynyl dump issue
- hydrated molecules
- will do planning of assigned tasks and coordination
- WISER development, getting ready for release for SPIE conference (testing)
- Adapting oasys for wavefront objects
- Working on simulation server for ILL
- API linking GUI to server
- Merge with jupyter notebook workflow
- Looking into simex to see how can be used
- Made change as requested per PR
- juncheng will assist in fixing travis CI complaints
Participants
- 200 cores in cloud project, panosc can consume 1e6 CPU hours.
- registration: umbrella for aai. currently: egi check in.
- need to get in touch with guseppe la rocca.
- software provision: max-jhub. in testing: jhub, user prompted for profile. can
- aai.egi.eu
- login to cloud: confluence pages with wp4. can then launch jhub, uses openconnect (?).
- set up VM with jupyter hub or docker.
- on two weeks vacation after tomorrow, implemented type safety in mcstas-script
- added random noise, custom panels for sfx demo, will polish demo this week. tried to run emc, DA hard to run.
- chat with mads about serialization of parameter object of mcstas script. should have database with all metadata that lead to result -> two collections. parameter tagging links to simulation blocks, cakce intermediate results. parameters should be grouped, Identify changes in earlier blocks. serialization into json. tries c++ api for mongo.
- nothing to report
- Need to draft review until June 1st:
- List of dissemination activities
- event title, event type, title/type of contribution, number of participants (not panosc internal meetings).
- List of dissemination activities
- Need a 15 min summary presentation of WP5 activities until June 1st
- every WP5 collaborator to submit 2-3 slides.
Participants - Juncheng E - Carsten FG - Mads Bertelsen - Aljosa Haffner - Mousumi Upadhyay-Kahaly - Paolo Muti - Shervin Nourbaksh - Andrea Lorenzen (?)
- HE Physics
- Detectors, DA (procesing, reducing)
- Higgs research
- broad software skills
- experience in simulations (CMS) on data analysis side
- Postponed due to sound issues
- suggestion on sprint taken up positively
- Paolo: should still postpone the workshop
- Carsten presents tasks for the workshop
- next meeting: April 7, 1pm
- presentation by Shervin
- introduction Andrea
Participants - Juncheng E - Carsten FG - Mads Bertelsen - Aljosa Haffner
* New date is April 23,24
* Can connect remotely if travel issues (corona pandemic)
* ESS: all travel must be authorized
* Aljosa: OK for now
* Carsten: Getting simex back on track
* Juncheng: Example study for lysozyme. SPB data available. Public? Not sure
* ESS may do the same for NScattering off Lysozyme
* Aljosa: Oasys available as webservice (RDesktop like), will demo on
workshop
* Mads: Windows support for McStasScript.
* Carsten will keep fixing simex
* Juncheng will help preparing the workshop (assign tasks to people).
Participants - Carsten Fortmann-Grote (EuXFEL) - Juncheng E (EuXFEL)
- Carsten has conflicting events at MPI, have to find a new date.
- Will try April 23/24 plus some other suggestion in May -> JE: Make new
doodle
Participants - Carsten Fortmann-Grote - Mads Bertelsen - Juncheng E - Zsolt Lech - Mousumi Upadhyay Kahaly
- Mads presents possible strategies for adopting the API requirements in
McStas/McStasScript. Details at https://github.com/PaNOSC-ViNYL/ViNYL-project/blob/master/Docs/Reports/TDR/API/McStas_integration.md
-> Carsten's first reaction is to go with the 3rd alternative. To be confirmed after more careful analysis.
- Question is discussed how API design should be aligned with WP6 (and other
WPs). No conclusion, should be kept in mind.
- Need to provide more detailled task description to Zsolt and Mousumi
- Zsolt works on coupling EPOCH to McStas via openpmd, WIP (openpmd not yet supported by McStas, use ASCII)
- Mousumi can provide simulation data from DFT for SIMEX/ViNYL simulations.
- Need more information on ASE
- The beamphysics standard extension has adopted neutron raytracing in line
with our extension. Need to check for photon raytracing -> Aljosa.
- Carsten presents draft requirements for the APIs. Details at https://github.com/PaNOSC-ViNYL/ViNYL-project/blob/master/Docs/Reports/TDR/API/specifications.md and https://github.com/PaNOSC-ViNYL/ViNYL-project/blob/master/Docs/Reports/TDR/API/CFG_WP5-VTC_20200120.pdf
- Asked Mads to provide a similar presentation from the McStas perspective
in next VTC.
- Date is set for April 2 and 3
- External participants: Robert Rosca (WP4), Erik Knudsen (?)
- Indico site is up and running
- Will make pull requests to upstream, start with MD (Juncheng)
- Tasks created, more details from Carsten will follow, people assigned.
- Currently many installation issues -> TODO Carsten
Participants
- Carsten Fortmann-Grote (EuXFEL)
- Aljosa Haffner (CERIC-ERIC)
- Juncheng E (EuXFEL)
- Andy Goetz (ESRF)
- Aljosa has a draft for photon raytracing, pull request created
- Carsten will review
- Carsten plans to merge wavefront and raytracing standards, prototype exists in simex. Aljosa mentions that in oasys, this is done by XOPPY, taking wavefronts from SRW.
- Juncheng is finalizing MD standard extension.
- Observes performance issues because many particle types to go into separate datagroups in openpmd-hdf5 file.
- Should scale with Zmax^4*Natom, every charge state and electronic configuration define a new particle species.
- not a problem for now, but may become forbidding in large systems simulation
- in that case, must consider in-memory passing of data, anyway.
- Carsten will present standard and IO tools for the wavefront standard next meeting.
- Juncheng will start writing the deliverable report. Template available on panosc google drive wp9. Please derive a latex template and use overleaf for collaborative writing.
- Andy inquires about speakers for WP5 overview talk at the annual meeting. JC will go and give the talk. Micha Bussmann is also invited but has not decided whether presenting or not.
- JC tried to connect to the WP8 online meeting last week but missed. Meeting took place as reported in PMC meeting.
- JC tasked to organize face-to-face Wp5 meeting in Hamburg in Q1 2020. Invite 1 or 2 reps from WP4 and 8.
- Next meeting on Oct.21 1pm.
Participants
- Carsten Fortmann-Grote (EuXFEL)
- Aljosa Haffner (CERIC-ERIC)
- Mads Bertelsen (ESS)
- Thomas Rod (ESS)
- Mousumi Upadhyay Kahaly (ELI)
- Zsolt Lech (ELI)
- Carsten gives feedback to Aljosa regarding possible hierarchy for the
photon ray-tracing domain extension:
data/
1/
particles/
rays/
0/
position/
x
y
z
velocity/
x
y
z
...
1/
...
only one timepoint necessary here since no explicit time dependance
- Mads has drafted a domain extension for neutron raytracing. Here time has
to be taken into account
- Carsten will review the code and provide feedback via github.
- Domain extensions for MD and wavefront propagation are ready for the
report, final touches required to make it work with the openpmd API.
- There is only one API, should recognize the required and optional data
by specifing the domain extension through the API. Not one API per
domain.
- Mousumi has looked into requirements for the NOMAD database
- accepts output from QuantumEspress
- will develop python tools to convert to/from openpmd
- Zsolt is about to run a neutron source PIC simulation and converter to
openpmd that will allow feeding the data to McStas/neutron raytracing.
- Thomas inquires about attendance to the annual meeting. All but Carsten
will be present.
- Thomas suggests to have a separate workpackage meeting (possibly invite relevant other WP representatives). Carsten will discuss with Juncheng.
- Next VTC on October 14
Participants
- Juncheng E (EuXFEL)
- Carsten Fortmann-Grote (EuXFEL)
- Aljosa Haffner (CERIC-ERIC)
- Mads Bertelsen (ESS)
- Thomas Rod (ESS)
- Mousumi Upadhyay Kahaly (ELI)
- Mads reports about his last development sprint, mainly code base review
for McStas
- Thomas reports from the EXPANDS meeting in Hamburg
- there's no simulation workpackage like ours
- strong interest to collaborate and utilize WP5 services in their
e-learning platform
- there is a e-learning platform workpackage, lead mostly by
PR people, not so technically oriented like panosc's WP8.
- Juncheng reports on progress on the openpmd extension for MD:
- A draft is ready, see https://github.com/ejcjason/MDDomainExtension
- next step is to add documentation
- Carsten to provide review comments
- prototype for IO also in progress
- Carsten presents a breakdown of tasks for the upcoming deliverable D5.1.
See https://github.com/PaNOSC-ViNYL/ViNYL-project/issues/19
- Carsten reports on progress on the wavefront domain extensions for
openpmd.
- Mousumi works on a project on laser damage to structured surfaces with DFT
codes.
- Zsolt develops tools to connect PIC output to McStas neutron raytracing,
will study the dependence of the beam quality on the heavy water nozzle.
- Mousumi uses various DFT codes (VASP, QuantumEspresso, Octopus)
- will look into requirements for the NOMAD database and corresponding
conversion tools to develop in WP5
- Oasys is now available to remote users either via rdp or plain ssh and
display forwarding
- Will approach Jamie Hall re. VISA software as RDP solution for HPC
platforms at the Panosc annual meeting.
- Where to host the service is not decided
- Juncheng mentions the LUME simulation platform under development at LCLS
(lead by Paul Fuoss). They focus more on e beam parameters and beam
quality, could join SIMEX as downstream simulator. Also use openpmd as metadata standard
Participants
- Juncheng E (EuXFEL)
- Carsten Fortmann-Grote (EuXFEL)
- Aljosa Haffner (CERIC-ERIC)
- Mads Bertelsen (ESS)
- Thomas Rod (ESS)
- Juncheng reports on recent visit to HZDR:
- talked to Michael Bussmann about involvement in WP5, he would be happy to contribute
- Will suggest to invite him to annual meeting to present openPMD
- works on openPMD extension for MD data, will have draft next week
- Mads reports about being added to a SCRUM team
- work on McStas python API is progressing
- will present work at ICANS conference
- Aljosa is about to merge the GUI version of WISE into the git repo
- will use numba for just-in-time compilation
- It was clarified that we pursue a RDP-like solution to expose OASYS as a web service, not a web (browser) based solution
- this is considered a useful feature at ELETTRA
- Carsten reports on outstanding items as per project management council:
- KPIs: Everybody should draft KPIs for their contributions, to be discussed in the next meetings
- Financial reporting: To be done by one person per partner RI
- Intermediate milestones: We shall define intermediate milestones for the periods of more than 4 months without a project milestone or deliverable. Carsten is working on this.
Participants
- Juncheng E (EuXFEL)
- Carsten Fortmann-Grote (EuXFEL)
- Aljosa Haffner (CERIC-ERIC)
- Andy Goetz (ESRF)
- Mads Bertelsen (ESS)
- Thomas Rod (ESS)
- Juncheng is working on a openpmd domain extension
- Is in contact with core developers, will meet some in Dresden when travelling there next month.
- Work on McStas API is progressing
- now installable from pypi
- Awaiting guidelines and experiences from implementing openpmd in simex before doing so in McStas
- Implementing deformable mirrors in oasys
- Work on wise program for wavefront propagation is progressing
- Openpmd seems good canditate for wise metadata standard
- Currently facing the issue that wise version in use at ELETTRA has diverged from the public github hosted version, need to merge both to enable collaborative development.
- Should contact Jamie Hall at ILL to discuss access to oasys through VISA remote desktop solution.
- A. will attend Oasys workshop in US.
- Announces HDF5 workshop next month at ESRF, organized by hdf working group
- expands workshop next month in Hamburg (Carsten or Juncheng should attend)
- Annual meeting in November (Carsten will try to make it.)
- Name showcase examples to simulate with panosc
- Annual meeting:
- Thomas suggests to avoid parallel sessions so cross-wp meetings are possible
- suggest to have joint wp5/wp8 session
- Nicoletta (Communications WP) asks to check listing of software development projects on website.
Participants:
- Juncheng E (EuXFEL)
- Mads Bertelsen (ESS)
- Zsolt Lech (ELI-ALPS)
- Carsten Fortmann-Grote (EuXFEL)
- Mads reports on the European Neutron Science Conference where he presented
his work on McStas and received many valuable comments which he will take
into account in his work over the next time
- One activity is to make McStasScript installable via pip
- new hire Juncheng E started on July 15th, will continue the dev. and
maintenance of simex
- Carsten reports on efforts to make simex usable as a binder service with
repo2binder. Difficulties are in resolving all backengine's dependencies,
e.g. mkl. mkl via conda leads to runtime exceptions in the wavefront propagation
module
- Carsten invites all to review the wp5 relevant sections of the panosc
website (see Andy's comments from last vtc).
- Zsolt reports on progress in writing a converter for EPOCH output to
McStas input, prototype is ready for application
- Carsten comments that a conversion to/from openpmd would also be
desirable (in view of D5.1, T5.1)
- Next VTC scheduled for August 12 at 1 pm (Carsten not available on Aug. 5)
Participants:
- Carsten (EuXFEL)
- Aljosa (CERIC-ERIC)
- Andy (ESRF)
- Work on WISE progressing
- Optimizing the code after rewriting it in python (previously IDL)
- Had discussion about coupling WISE to SIMEX
- Should coordinate with dev. of openpmd domain extensions (Carsten)
- No work planned on COMSYL so far
- Breakdown of Task 5.1 and Deliverable 5.1 for SIMEX → [github.com/eucall-software/simex_platform/issues/171](https://github.com/eucall-software/simex_platform/issues/171) (WIP)
- Working on a “repo2binder” script to enable simex-as-a-service on binder.
- DESY/XFEL considers installing a binderhub instance, this would make a good test case
- Hiring: Juncheng E will start on July 16th
- Suggestions from Andy:
- Consider making minimal addition to openpmd hdfs which would make them nexus compatible
- Identify and contact potetial users
- Review WP5 section of panosc website
- Contact Manolo in regular intervals to coordinate work on oasys
- Next VTC is scheduled for July 22.
- Development of McStas-Script
- now ready to be used in production simulations
- design principles: SIMEX as template, added more direct access to components
- compatible with mcstas version 2.5 (latest)
- Zsolt and Mousumi introduce themselves as this is their first participation in this meeting.
- Zsolt explains the high power laser based neutron source and how to simulate this with PIC and McStas. He uses the EPOCH code (PIC). EPOCH writes into .stf files which can be converted to hdf.
- This is a suitable use case for adaptation of openpmd as the default metadata standard on hdf5 backend.
- Neutrons have ~2.5 MeV energy and are emitted isotropically.
- At higher energies, more directed beams possible.
- Mousumi has experience in DFT simulations of light-matter (optical laser) interaction.
- Currently, ELI-ALPS is preparing for the ATTO2019 conference, hence not much activities for PaNOSC.
- Aljosa introduces himself
- He has experience in neutron scattering simulations at ILL
- will work on OASYS
- x-ray optics simulation framework
- contains ray-tracing and wavepropagation
- Aljosa’s main task is the development of the wavepropagation (WISE2)
- Ideas about how to turn OASYS into a web service are investigated.
- KNIME is mentioned as a similar canvas based workflow engine, which can possibly be utilized for OASYS.
- Carsten reports on progress in adopting openPMD as common data and metadata standard for simulation data.
- openPMD will be adopted first in SIMEX to define prototype domain extensions.
- This experience shall then be used in other simulation projects
- Carsten asks partners to install and test SIMEX on their HPC systems to gather more use cases and experience and to debug installation and execution issues.
- Mads reports on McStas interface update
- now usable for jupyter
- documentation available
- Carsten is employed at EuXFEL on a 10% part time contract.
- Will coordinate WP5 activities and lead the WP time permits.
Manuel is not able to join the meeting but sent a report by email, quoted here: A software engineer, Thibault Vallois was hired at ESRF with PaNOSC funds. He started on February 4th 2019. He started working on the first item in the list of tasks related to Oasys: the preparation of the Windows version. On February 28th he announcing his intention of not continuing after the completion of the probation period. So he quit on March 1st.
I had to continue this work on my own, and I managed to have the windows version of Oasys in due time (May 2019).
There is no decision taken of how to proceed now, if recruiting another engineer now or wait a bit. In any case the working plan is changed because of lack of manpower and my imminent departure on sabbatical. Unfortunately, all the Oasys developments in the PaNOSC context are put in stand-by.
We have organised the First Oasys School at ESRF (14-16 May 2019) counting with the support of PaNOSC. A report draft will be shared with PaNOSC soon. The material of the course is available here: github.com/oasys-kit/oasys_school
- Andy noted from the discussion that WP5 will indeed produce data and that the data management plan will have to be updated accordingly.
-
• Zsolt, Aljosa, Juncheng, Andy, Thomas Rod, CFG introduction andrea short intro of all participants to meet shervin and andrea esrf task: make oasys a web service, already done in ceric-eric shervin: integrate ill analysis software with mcstass zsolt email thomas: andy: need resources? mousumi got access to cloud resource panosc wp6 meets this week: discuss needs for resources, have 500M Euro mads → get slack channel for the sprint review meeting: who presents? 2pm meeting on wednesay juncheng: in contact with aljosa re oasys check pmi diag scripts from s2e gaussian
introduction Shervin (ILL) • HE Physics • Detectors, DA (procesing, reducing) • Higgs research • broad software skills • experience in simulations (CMS) on data analysis side
discussion on workshop format • suggestion on sprint taken up positively • paolo: should still postpone the workshop
carsten presents tasks for the workshop