Narrative back end logging

NOTE: These instructions are currently for code in development, not merged with the master branch or deployed. If you want to try them out locally, use the branch 'dang_log_proxy'

Introduction

The narrative Python back-end has the ability to log the start, end, and every well-defined "stage" of the narrative functions to both a file and, via a network proxy, to MongoDB. This page describes the process of configuring and running the narrative server with this capability.

Setup and run

There are only 3 steps to running the narrative with logging enabled:

1. Set the log level with the environment variable KBASE_DEBUG.

   Narrative functions log automatically, and there is not (yet) any configuration file. To control the log level for narrative functions, set the KBASE_DEBUG environment variable to any value for level DEBUG, or unset it for level INFO.

2. Look for logs inside the container will be in /tmp.

   Now you can run the narrative and look at the logs.

   Each narrative instance writes a copy of its logs, inside the Docker container, to a hardcoded path that is in the module biokbase.narrative.common.kblogging called KBASE_TMP_LOGFILE.

3. Set up logging to MongoDB by running kb-log-proxy with a configuration file.

   This is the one thing that is truly additional. The kb-log-proxy is a little script wrapping the biokbase.narrative.common.log_proxy module. It should be installed into the bin/ directory automatically when you run python setup.py install, but you could also run it from the scripts directory.

   The purpose of this script is to provide a local proxy for logs from the narrative container(s) to insertion of these logs into MongoDB. By separating this process out, we can have it read sensitive password information from a protected file in the host filesystem, away from the prying eyes of the IPython narrative backend. It also acts a buffer for the logs from all the narratives, virtually guaranteeing that they will never hang or wait to write their log information.

   The script has a help option (-h) that you should use to see how to format its input configuration file. This file basically just tells the script what host and port to list on, and how to connect to MongoDB to write out the logs.

   Redirect the output of the script, which will be a log file itself, to somewhere convenient and use nohup to run it safely as a daemon. For example:

   nohup kb-log-proxy -f /etc/kb-log-proxy.conf 2>/var/log/kb-log-proxy.log &