ZTF Variable Marshal (ZVM)

A dockerized Variable Marshal for ZTF powered by aiohttp and mongodb.

Search Saved Sources GUI	Source page: Photometry	Source page: Spectroscopy and aux

Python client library zvm

Install the client library zvm.py, with pip into your environment:

pip install git+https://github.com/dmitryduev/ztf-variable-marshal.git

zvm is very lightweight and only depends on pymongo and requests.

A tutorial on how to programmatically interact with the ZVM:

See this jupyter notebook, or

---

Production service

Set-up instructions

Pre-requisites

Clone the repo and cd to the cloned directory:

git clone https://github.com/dmitryduev/ztf-variable-marshal.git
cd ztf-variable-marshal

Create secrets.json with confidential/secret data:

{
  "server" : {
    "admin_username": "ADMIN",
    "admin_password": "PASSWORD"
  },
  "database": {
    "admin": "mongoadmin",
    "admin_pwd": "mongoadminsecret",
    "user": "user",
    "pwd": "pwd"
  },
  "kowalski": {
    "instances": {
      "gloria": {
        "protocol": "https",
        "host": "gloria.caltech.edu",
        "port": 443,
        "token": null // set to your token
      },
      "melman": {
        "protocol": "https",
        "host": "melman.caltech.edu",
        "port": 443,
        "token": null // set to your token
      },
      "kowalski": {
        "protocol": "https",
        "host": "kowalski.caltech.edu",
        "port": 443,
        "token": null // set to your token
      }
    },
    "fritz": {
      "url": "https://fritz.science",
      "token": null // set to your token, optional
    }
  }
}

Using docker-compose (for production)

Change skipper.caltech.edu in docker-compose.yml and in traefik/traefik.toml if necessary.

Run docker-compose to build and start the service (may have to sudo if acme complains):

sudo docker-compose up --build -d

To tear everything down (i.e. stop and remove the containers), run:

docker-compose down

---

Using plain Docker (for dev/testing)

If you want to use docker run instead:

Create a persistent Docker volume for MongoDB and to store data:

docker volume create ztf_variable_marshal_mongodb
docker volume create ztf_variable_marshal_data

Pull, build, and launch the MongoDB container. Feel free to change u/p for the admin, but make sure to change secrets.json and docker-compose.yml correspondingly.

docker run -d --restart always --name ztf_variable_marshal_mongo_1 -p 27025:27017 --expose 27025 \
       -v ztf_variable_marshal_mongodb:/data/db \
       -e MONGO_INITDB_ROOT_USERNAME=mongoadmin -e MONGO_INITDB_ROOT_PASSWORD=mongoadminsecret \
       mongo:latest

To connect to the db:

docker exec -it ztf_variable_marshal_mongo_1 /bin/bash
mongo -u mongoadmin -p mongoadminsecret --authenticationDatabase admin

Build and launch the app container:

docker build --rm -t ztf_variable_marshal:latest -f Dockerfile .
docker run --name ztf_variable_marshal -d --restart always -p 8000:4000 -v ztf_variable_marshal_data:/data --link ztf_variable_marshal_mongo_1:mongo ztf_variable_marshal:latest
# test mode:
docker run -it --rm --name ztf_variable_marshal -p 8000:4000 -v ztf_variable_marshal_data:/data --link ztf_variable_marshal_mongo_1:mongo --entrypoint /bin/bash ztf_variable_marshal:latest

ztf_variable_marshal will be available on port 8000 of the Docker host machine.