Open-Source Release Status

[arthur-flam]

Go!

Welcome to QA-Board! We are working on open-sourcing the project and are targeting an official release for Q3 2020.

What needs to be done before the release

• Pre-release, as all items marked 🌱 are done.

Open-source process

• Authorization to release as OSS 🌱
• Licence (Apache 2) 🌱

Docs

• Docs: Getting Started. (Available at https://samsung.github.io/qaboard/) 🌱
• Website
• Docs with common use-cases
• README
• Server bring-up guide

Technical changes

• qa init creates a functional project
• Merge the qatools and qaboard internal repositories 🌱
• Make sure nothing is hardcoded for our infra : pass secrets at least as ENV 🌱variables or config files, and handle mount naming convention with a centralized config.... 🌱
• Decide on a way to pick a default location where to store the output data. Needs to be mounted by the container and writable easily.
• Ensure 1min server bringup with e.g. docker-compose
• Remove USER arthurf from Dockerfile, maybe make it configurable to avoid NFS+squash_root issues.

API changes:

• qaboard-backend.slamvizapp>backend.backend, qaboard-webapp>webapp
• qatools.yaml->qaboard.yaml
• Import as qaboard not qatools
• Output folder naming conventions with hashes
• Friendlier run() API (ctx.obj->ctx : RunContext)
• sed -i s/qatools/qaboard/g *

Features

• Make it easy to support more async task queues - Wiki
• Support for at least one common task queue beyond what we have internally at Samsung, and explain how to implement support for others.
• Create a public CI
• Support more git servers (e.g. Github). Don't even require Read rights.
• Make it easier to debug visualizations and metrics with per-batch settings

PR

• Public demo server
• All good for a release blog post

[amirfru]

Hey @arthur-flam !
Can I ask which task-runners is it tested for? thanks!

[arthur-flam]

Hi Amir ! Even better, please suggest something you are using!

I planned on adding support for something easy to setup and flexible like celery or python-rq.

For other integrations (unless there is a big demand) I'll write docs, make sure the code is easy to extend, and rely on help from the community.

[arthur-flam]

This week I improved the docs a lot, they are now good enough to be published. I expect to have them online early next week, along with a website.

I also published the CLI wrapper to pypi:

pip install qaboard

As soon as I merge the two repositories composing QAboard today, I'll push the code online

[arthur-flam]

Hi, some updates:

• Merged the two repos that composed qaboard
• Added basic support for "secrets" to pass e.g. gitlab credentials.

We're getting closer to a pre-release. What's missing is creating a "master-open-source" branch with a clean history, replacing a few default / docs, and pushing it..!

[arthur-flam]

Update:

• there is now a fully sanitized version of the code. I'll publish it ASAP now 😌
• I uploaded the website/docs to the gh-pages branch, but GitHub Pages isn't setup correctly so we'll have to wait for it to be live...

[arthur-flam]

The code is online 🎉🎉🎉 !
There is still work to do, but things feel more concrete now. 🎆

[arthur-flam]

The website is live https://samsung.github.io/qaboard !

[arthur-flam]

Hi!
I've been making progress refactoring batches to allow more types of runners. More info at:
https://github.com/Samsung/qaboard/wiki/adding-new-runners

@amirfru hi! Any update about what you'd need?

I expect to push the branch in the next few days, possibly directly merge on master.

[arthur-flam]

✨ There is now support for Celery ! https://samsung.github.io/qaboard/docs/celery-integration/
Tuning from the web application is still hardcoded to use our infra... But that will be fixed soon.

[arthur-flam]

Hi! I quick update to say that we're still working on the open-source release, although we've focused more on quality of life features and small fixes this past month. Stay tuned!

[arthur-flam]

Hello!
Last month we've been busy with other priorities, still some notable improvements landed:

• support for "matrix" batches
• filtering is now much more performant
• lossless image viewer support

[arthur-flam]

Hi!
I am finishing adding support for docker-compose:

• the whole application stack can already be started in 1 command, and there is a structured way to specify custom ENV variables.
• makes it easier to implement a public CI, host a demo server, or just have users try QA-Board.
• will help us provide pre-built container images.
• will provide an easy migration path to more complex orchestration tools (e.g. k8s..).
• Other benefits: the app now has only seconds of downtime during upgrades, is served from a unique port, no need for cron jobs managed separately, and our internal tweaks are much more isolated.

I expect to push the changes and new docs in the coming days. Bye!

[arthur-flam]

We now use docker-compose!

• New guide to start a QA-Board server
• Faster startup/update time, as all services are split
• Everything is served from a single port, we reviewed the whole mess of reverse proxies.
• The docs are available from the application at /docs
• Almost all the code is the same in the OSS and our internal version.
• We will shortly add in docker-compose.yml a sample cron job doing backups

We also started using github actions to create a public CI:

• the website is already updated automatically from the docs
• we're building the docker images and doing a basic smoke test.
• we're also running the unit tests, type checks and linter.
• On master we push images to github's docker registry and on releases we upload the CLI package to pypy

The next task is not requiring read-access to the git repositories of the users' projects - or at least support github in addition to gitlab (currently required...)

[arthur-flam]

Hi!

Here are some updates:

• QA-Board now doesn't require Gitlab anymore, nor read-right to projects' repositories. Using Gitlab enables nice-to-have features like showing user avatars, but we'll work on doing it with Github too.
• Each batch can now have its own config and metrics, making using with qa --share much more pleasant.
• If you've missed it, we now support a new visualization, flame graphs.

We're now in new territory: anyone can start using QA-board! We'll check it's as smooth as promised by the docs, and give updates.

[arthur-flam]

There is now a demo server at demo.qamarks.com!

• It was straightforward to setup just following the guide, so we can now feel at ease looking for users.
• For now there is a single demo project comparing 2 versions of a map-based algorithm. We'll add more demos and polish it...

[arthur-flam]

Some major updates:

1. The API was simplified:

• Instead of the ugly context.obj["configurations"], there is context.configs. And so on.
• Tuning comes for free, no need to worry anymore about context.obj["extra_parameters"]
• For convenience there is now context.params
• The docs were updated and changes are backward compatible.

2. Users can now save artifacts and outputs at different locations, and it's all a bit simpler to specify. Go to the docs to read more.

Those changes close the TODOs to make QA-Board "public-ready" 🚀 !

I'm closing the issue!