Add support to configs via environment variables

[RobertoSchneiders]

To use the docker image, I have to create a config file and copy it to the container (like this). It would be easier (maybe better) to configure this settings by environment variables.

I'm deploying to AWS Elastic Beanstalk and If it were possible to configure by environment variables I will be able to get the app up and running just by configuring the env vars on AWS Console and creating a file like this:

Dockerrun.aws.json

{
  "AWSEBDockerrunVersion": "1",
  "Image": {
    "Name": "edgurgel/poxa",
    "Update": "true"
  },
  "Ports": [
    {
      "ContainerPort": "8080"
    }
  ],
  "Logging": "/var/log/nginx"
}

I'm new to Elixir, so, I'm not familiar with Mix. From what I understand, the config.exs file is already getting some configs from env vars, but, this code is executed only on docker image generation command is executed (MIX_ENV=prod mix edip --prefix edgurgel) and not on the application initialization. So, it doesn't help who is trying to use the docker image. Right?

[arrowcircle]

From readme

You can run and configure these values using these environment variables:

PORT=8080
POXA_APP_KEY=app_key
POXA_SECRET=secret
POXA_APP_ID=app_id

[RobertoSchneiders]

Yeah, but this don't work with docker. It should?

[RobertoSchneiders]

As an example, this do not work:

POXA_SECRET=test docker run --rm -i -p 8080:8080 edgurgel/poxa:0.4.3

[bsedat]

I believe you have to expose the environment variable via the env flag docker run --rm -p 8080:8080 -e POXA_SECRET=test edgurgel/poxa:0.4.3 but that didn't seem to work for me.

[RobertoSchneiders]

You are absolutely right @bsedat. I forgot that. I tested this on ElasticBeastalk, which expose the envs for me and as you said, it didn't work.

[edgurgel]

The docker image uses the release generated by exrm and it's configured by the .conf files. This is important as this will translate to actual Elixir/Erlang terms.

It would be possible to translate ENV variables to these terms today but soon we will support multiple apps (#43) and this will become really complex to handle through ENV variables. I suggest managing the .conf file.

I'm not sure what's the best way to mount a file through ElasticBeanstalk but I will give it a try this weekend and report back here.

[arrowcircle]

@edgurgel Hi!
I tried to deploy poxa via docker and found interesting thing. When I run docker image and pass env vars POXA_APP_ID, POXA_SECRET, POXA_APP_KEY it totally ignores it and use default from config.
Is it possible to fix docker image to use env vars instead of config?
Here is /etc/environment and docker-compose.yml files to reproduction https://gist.github.com/arrowcircle/4e4a93c89f1e7a7cfc4f

PS. Please tag docker image 0.4.3 as latest.

[edgurgel]

My previous comment is still valid and the env variables won't handle the configuration as soon as we move to multiple apps. Then I would need to support both ways :/
I would suggest you to put a conf file and mount as a volume as pointed in the README file.

I will tag the latest as 0.4.3!

[arrowcircle]

@edgurgel I see the problem, its impossible to pass hashes to env vars, but its easy to add one "if" to check if ENV vars are valid and ignore config. It will enable both env and config ways. And env is good way to use according to http://12factor.net/config

Also, if you use shared application settings (for pusher gem), you only need to point it to env var and have no problems with sharing settings from yml (rails) to poxa.
I think its not technical problem and env config is not a problem in terms of code support.
Could you please consider having a chance of keeping env configs?

[edgurgel]

If I keep the env vars as config they will be limited to a single application as already discussed. I would like to keep the priority of configuration like this:

.conf file -> ENV vars -> default values

I'm not sure how to accomplish this with the current schema.

[arrowcircle]

But in this case ENV vars in docker image is unusable.
Problem is not with priority of vars, but with docker image. It always have config, so its impossible to use env vars. Is it possible to fix it?

[arrowcircle]

Ok, but how we can fix docker image? It comes with config and totally ignores ENV vars. Is it possible to fix it?

[arrowcircle]

@edgurgel Could you please share Dockerfile? I can try to build custom poxa image with support on env vars

[edgurgel]

Poxa doesn't have a Dockerfile as it uses edip.

Even if you build your own image it won't load as ENV vars are load only by mix config (https://github.com/edgurgel/poxa/blob/master/config/config.exs#L9) and mix is not available as part of a release. The heroku buildpack uses mix to run it by default and that's why it works with ENV vars there.

You can always just build a docker image using mix to run it if you want to.

You would run with something like this: https://github.com/edgurgel/poxa#deploying-on-heroku

[arrowcircle]

@edgurgel Thats really strange behaviour. So there are no easy way to use ENV vars in docker?

[edgurgel]

There's but we would need to change current code to load them if there's no config available OR you would need to build a docker image using mix to run it (which is not the way exrm releases work)

[edgurgel]

My argument is that it's not a good change as soon these ENV vars won't work for multiple apps soon.

[hendrikhofstadt]

One thing you can do is build your own Dockerfile based on the original edip image like this.

FROM edgurgel/poxa:latest
COPY poxa.conf /app/running-config/

This would result in a preconfigured Docker image that you can easily deploy from your private repository.
It is not best practise but at least this should work perfectly on hosted environments

[arrowcircle]

@SLAMPER Problem is you need private docker repository

[hendrikhofstadt]

You've got a point there. To use env variables in the docker container you only have to change a few lines in the code. But as soon as multi app support will be implemented this will be broken as edgurgel pointed out.

[arrowcircle]

@SLAMPER Actually, I think we need to write a script that generates poxa config from env variables if they are present. We simply run it before running docker container.
By the way, after the release of multiapps, we can think about easy setup for one application with env vars. Or read apps from config and add one from env vars (magic).
What do you think?

[hendrikhofstadt]

@arrowcircle that's a good idea.
We will have to add that script to the docker container and run it before the normal start script which should be no problem.

I just wrote a super simple dirty script which does this. It also uses the $POXA_APP_COUNT env variable to check how many apps we have and also supports multiple apps as far as I could get how they work from the multi-app branch.
It has a few checks but many things are still unchecked.

For the Docker container we will then need a script that calls this script first,
then copies the config file to /app/running-config if the script doesn't fail and then call the normal startscript.

https://gist.github.com/Slamper/b883241e4bd79b88aa7b

[edgurgel]

If we agree with this solution we can add this script to the poxa docker image as a helper script.

[hendrikhofstadt]

@edgurgel I'm currently working on a small buildscript that can easily build the container with this script in the startup. Also this script only works in bash and the edip image only has ash so it will need some changes. I will get back when I've got something that is working

[hendrikhofstadt]

Ok I fixed the script and it works with the container.
Startup: https://gist.github.com/Slamper/2afb09806abd7915e9ab
setup_env: https://gist.github.com/Slamper/b883241e4bd79b88aa7b
Dockerfile: https://gist.github.com/Slamper/eaece9ac0f51664d2be0

I put the simple container on git: https://github.com/Slamper/easy_poxa
and setup an automated build on DockerHub: https://hub.docker.com/r/slamper/easy_poxa/
It is linked to your repo so the build gets updated as soon as you push a new container to the Hub.
Support for your coming multi-app update is also included to support this format
apps: [{"app_id2", "app_key2", "secret2"}]

If you want to add the scripts to the official image feel free to do so. I will then delete my repo :)

[arrowcircle]

@SLAMPER I think we dont need COUNT variable. Just check app_id, key and secret. If they are present - generate config.

@edgurgel What do you think?

PS. Happy New Year!

[hendrikhofstadt]

@arrowcircle Happy New Year too.
I added the POXA_APP_COUNT variable to support multiple apps. I'm not sure whether this is the best way to do it but it works :)

[edgurgel]

Hey guys1 Happy New Year!

Maybe start with a simple script and as soon as the multiple apps branch lands we can move to the one that supports multiple apps? I'm keen to move this forward these next weeks.

[hendrikhofstadt]

I simplified the script in my master branch to only support the single app version.
In the multi-apps branch there is support for the multi-app version.
https://github.com/Slamper/easy_poxa
https://github.com/Slamper/easy_poxa/tree/multi-apps
On Dockerhub the multi-app version is available with the :multi-apps tag

[edgurgel]

We have good news! There's a new option on conform that understands env vars (at least for "default values") when you don't define a .conf file I suppose.

I will give it a try this weekend, also moving to distillery (no more exrm).

More here:
https://github.com/bitwalker/conform/blob/7c78207bea249771e952d474d9a710a575ee7cd1/docs/Getting%20Started.md#mappings