Next steps

[bertsky]

• make asynchronous option work (disown -a?)
• separate docker-compose files for Kitodo/Manager vs. Controller, e.g. docker-compose -f docker-compose.yml -f docker-compose-kitodo.yml -f ../ocrd_controller/docker-compose-controller.yml, or native controller docker-compose -f docker-compose.yml -f docker-compose-kitodo.yml -f docker-compose.override.yml – configuration mechanisms...
• generalize error handling for script tasks in Production: differentiate between errors and asynchronous tasks (and thus allow actual error handling)
  or alternatively, keep errors out of Production (task stays open until successful), and offer GUI in Manager for history, processing status, error monitoring and workflow modification+restart)
• refactor for_production.sh (functions, file includes) to make re-usable for different scenarios (for_presentation, ...)
• Secure active mq call Use secure active mq ocrd_manager#14
• ocrd-import / workspace init as part of for_production.sh, not of the workflow itself
• workflow script: ocrd process syntax instead of bare shell script (entails previous)
• workflow script: ocrd validate tasks before running
• versioning and labelling of Docker images (manager → core and controller → ocrd_all)
• decoupling Production process share and Manager data volume (explicit copy to and fro)
• decoupling Manager volume and Controller volume – more options?
  • gzip before sending using option --rsyncable https://beeznest.com/blog/2005/02/03/rsyncable-gzip/
  • explicit scp/rsync to and fro
  • explicit ocrd-make ssh --return --cleanup --transfer etc
  • n/w share
• decoupling Kitodo+Manager services (Docker compose file) and Controller services (Docker compose file)
• smarter and more efficient processing in Controller
  • to prevent oversubscription, use semaphores (via .profile for ocrd user)
  • or delegate to Nextflow as worfklow engine
  • or delegate to workflow server (i.e. ocrd workflow server + ocrd workflow client process instead of ocrd process), at least for preconfigured workflows
  • and/or delegate to processing server (i.e. ocrd-PROCESSOR --server HOST PORT WORKERS + ocrd workflow client process...)
• more flexible workflows:
  • advertise existing (installed) workflows in Production ("drop-down list")
  • inherit workflows from preconfigured (installed or edited) templates, then copy to process directory
  • edit workflows in Production (simple text editor, perhaps syntax highlighting)
• parallelization and job pipeline