diff --git a/docs/README.md b/docs/README.md index 727157b2d..1b133412e 100644 --- a/docs/README.md +++ b/docs/README.md @@ -25,6 +25,20 @@ cd _builds/html python -m http.server ``` +You can also automatically build the docs with `sphinx-autobuild`: + +Install it: + +``` +pip install sphinx-autobuild +``` + +Then run: + +``` +sphinx-autobuild . _build/html/ --port 8001 +``` + ## Hosting the docs Docs are hosted on readthedocs and managed by Dimagi. diff --git a/docs/administration.rst b/docs/administration.rst index 25a7aac0d..6bc4065ca 100644 --- a/docs/administration.rst +++ b/docs/administration.rst @@ -3,6 +3,9 @@ cStock Administration This section outlines common tasks performed by a cStock administrator. +Administrators are users that have special permissions on the cStock site, +and can manage users, access the system data models, and perform other administrative operations. + .. toctree:: administration/admin-site.md diff --git a/docs/administration/admin-site.md b/docs/administration/admin-site.md index 66b9f4bf1..c6364a164 100644 --- a/docs/administration/admin-site.md +++ b/docs/administration/admin-site.md @@ -1,6 +1,7 @@ Admin site access ================= -To access the admin site, go to [http://cstock.jsi.com/admin/](http://cstock.jsi.com/admin). -If you are prompted to login, then you may not have access. +To access the admin site, first [login to cStock](https://cstock.health.gov.mw/accounts/login/), then +go to [https://cstock.health.gov.mw/admin/](https://cstock.health.gov.mw/admin/). +If you are prompted to login again, then you may not have access. You should contact another administrator to grant you access. diff --git a/docs/conf.py b/docs/conf.py index 1eb4f246a..77399ace3 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -28,7 +28,8 @@ # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = [ - 'recommonmark' + 'recommonmark', + 'sphinx_markdown_tables', ] # Add any paths that contain templates here, relative to this directory. diff --git a/docs/images/cstock-architecture.png b/docs/images/cstock-architecture.png new file mode 100644 index 000000000..e54858e21 Binary files /dev/null and b/docs/images/cstock-architecture.png differ diff --git a/docs/index.rst b/docs/index.rst index 1c07b28bf..0e78fdf43 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -3,8 +3,16 @@ You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. -Welcome to cStock's documentation! -================================== +cStock Project Documentation +============================ + +cStock is a website that provides real-time, actionable HSA logistics data for managers and stakeholders to coordinate, +plan, and identify solutions to better meet customer needs in a timely manner. + +cStock is hosted at `https://cstock.health.gov.mw/ `_. + +This documentation is intended for cStock administrators, system administrators, and developers. + .. toctree:: :maxdepth: 3 diff --git a/docs/requirements.in b/docs/requirements.in index 085b3a792..d28745162 100644 --- a/docs/requirements.in +++ b/docs/requirements.in @@ -1,2 +1,3 @@ sphinx recommonmark +sphinx-markdown-tables diff --git a/docs/requirements.txt b/docs/requirements.txt index f2fbaa85a..ac717b505 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -1,6 +1,6 @@ # -# This file is autogenerated by pip-compile with python 3.8 -# To update, run: +# This file is autogenerated by pip-compile with Python 3.9 +# by the following command: # # pip-compile requirements.in # @@ -23,9 +23,13 @@ idna==3.3 imagesize==1.3.0 # via sphinx importlib-metadata==4.11.3 - # via sphinx + # via + # markdown + # sphinx jinja2==3.1.1 # via sphinx +markdown==3.5.1 + # via sphinx-markdown-tables markupsafe==2.1.1 # via jinja2 packaging==21.3 @@ -46,6 +50,8 @@ sphinx==4.5.0 # via # -r requirements.in # recommonmark +sphinx-markdown-tables==0.0.17 + # via -r requirements.in sphinxcontrib-applehelp==1.0.2 # via sphinx sphinxcontrib-devhelp==1.0.2 diff --git a/docs/systems-administration.rst b/docs/systems-administration.rst index 07d2c5844..90fce1772 100644 --- a/docs/systems-administration.rst +++ b/docs/systems-administration.rst @@ -5,7 +5,9 @@ This section outlines common tasks performed by a cStock *system* administrator. .. toctree:: + systems-administration/overview.md + systems-administration/server-access.md systems-administration/common-tasks.md systems-administration/troubleshooting.md systems-administration/server-setup.md - systems-administration/vpn.md + diff --git a/docs/systems-administration/common-tasks.md b/docs/systems-administration/common-tasks.md index 57a17eac2..45e051ee1 100644 --- a/docs/systems-administration/common-tasks.md +++ b/docs/systems-administration/common-tasks.md @@ -1,19 +1,37 @@ Common System Administration Tasks ================================== -## Deploying Code +These are some common tasks required of system administrators. -Deploying code requires installing fabric3 (`pip install -r requrements/deploy/dev-requirements.txt`). +All of these tasks require [server access](./server-access.md). -Then, to deploy you must first connect to the VPN. Then run: +## Deploying Updated Code + +Deploying code requires installing fabric3: + +``` +pip install -r requrements/deploy/dev-requirements.txt +``` + +And then cloning [the code repository](https://github.com/dimagi/logistics/): + +``` +git clone https://github.com/dimagi/logistics.git +``` + +Then, to deploy you must first connect to the VPN. Then run the following command in the repository root: ``` fab malawi deploy ``` -## Restarting processes +This command should be run on *your own machine*. +For the remaining sections, you must run the commands *on the server*. -We use supervisor (`/etc/supervisor.conf`) to manage the running processes. +## Restarting application processes + +We use supervisor (`/etc/supervisor.conf`) to manage the running cStock processes, +including the web application, SMS application, and background task application. To see the list of running processes run: @@ -26,3 +44,74 @@ You can restart individual processes with: ``` sudo supervisorctl restart ``` + +## Restarting other processes + +Sometimes other processes (the web server, database, etc.) need to be restarted. +To manage those you can find the information in the [overview page](./overview). + +## Making configuration changes + +If you need to make any configuration changes, for example, updating passwords or IP addresses, +those happen in the relevant configuration files for whichever service you need to update. + +To find the service's configuration file, look it up on the [overview page](./overview). + +For example, to update the credentials for the TNM or Airtel SMS gateway, you would modify +the SMS gateway (Kannel) configuration file at `/etc/kannel/kannel.conf`. + +Typically, whenever you make changes to a service, you also need to restart the service, +as per the instructions above. + +## Getting an application shell + +Sometimes it can be useful to get an application shell, to run the cStock Python code manually, +for example, to inspect data models or make once-off changes. + +To get an application shell you can enter the virtual environment like this (as the `cstock` user): + +``` +workon cstock +``` + +This should enter the virtual environment and load you in the right directory. +From there you can run: + +``` +python manage.py shell +``` + +To get a Python shell. You can then run code to work with the Django application. +For example, to see how many registered web users there are, you can run: + +``` +>>> from django.contrib.auth.models import User +>>> User.objects.count() +551 +``` + +## Getting a database shell + +The easiest way to get a database shell is to first enter the virtual environment as per above: + +``` +workon cstock +``` + +Then run: + +``` +python manage.py dbshell +``` + +Then to run the equivalent command in MySQL you could run: + +``` +mysql> SELECT count(*) FROM auth_user; ++----------+ +| count(*) | ++----------+ +| 551 | ++----------+ +1 row in set (0.00 sec) +``` diff --git a/docs/systems-administration/overview.md b/docs/systems-administration/overview.md new file mode 100644 index 000000000..a080d0349 --- /dev/null +++ b/docs/systems-administration/overview.md @@ -0,0 +1,130 @@ +cStock System Overview +====================== + +[cStock](https://cstock.health.gov.mw/) is a website built and maintained by the Malawi Ministry of Health. +There is a web component as well as an SMS component. + +## Hosting + +cStock is hosted by the Ministry of Health in a local data center. + +## Code + +The cStock application code is hosted [on Github](https://github.com/dimagi/logistics/). +It is a Python / Django application, based on RapidSMS. + +In general system administrators should not need to modify cStock code, +but it would be necessary to add new features or fix bugs. + +To get up and running with the cStock code, see [the development setup documentation](/dev-setup/). + +## Key Services + +cStock consists of the following key services: + +![cStock Architecture](/images/cstock-architecture.png) + +In the above, green boxes run custom cStock code, blue boxes are 3rd-party applications running +on the cStock server, and grey boxes are external services. + +Here is a high-level description of each cStock service: + +| Service | Description | Technology | +|-------------------------|---------------------------------------------------------------------------------------------------------------------------------|-------------------| +| Web Application Process | The web application is the business logic that powers the cStock web application. | Django | +| SMS Application Process | A process that runs alongside the web application to manage SMS workflows | Django + RapidSMS | +| Background Task Process | A process that runs alongside the web application to manage background tasks and scheduled SMS messages | Django + Celery | +| Database | The database houses all persistent data. | MySQL | +| Cache / Message Broker | The cache is used to improve the performance of the site and pass messages between other processes. | Redis | +| Web Server | The web server sits in front of the web application, providing outside internet access. It also hosts static files like images. | Nginx | +| SMS Gateway | The SMS Gateway connects to third-party SMS providers (TNM and Airtel) and routes the messages to the SMS application process. | Kannel | + +In general, all of these need to be running and functioning properly for cStock to work. + +### Web Application Process + +The web application is the business logic that powers the cStock web application. +Here is the key information for it: + +| Item | Value | +|-------------------------|---------------------------------------------------------------------------------------------------------| +| Process | Django (Gunicorn) | +| Log files | `/home/cstock/www/cstock/log/gunicorn.command.log` and `/home/cstock/www/cstock/log/gunicorn.error.log` | +| Configuration file | `/home/cstock/www/cstock/code_root/logistics_project/localsettings.py` | +| View status | `sudo supervisorctl status` (gunicorn process) | +| Stop / Start / Restart | `sudo supervisorctl stop gunicorn` (or `start`, or `restart`) | + +### SMS Application Process + +The SMS web application runs alongside the web application to manage SMS workflows. +Here is the key information for it: + +| Item | Value | +|------------------------|--------------------------------------------------------------------------| +| Process | Django (RapidSMS) | +| Log files | `/home/cstock/www/cstock/log/rapidsms.log` | +| Configuration file | `/home/cstock/www/cstock/code_root/logistics_project/localsettings.py` | +| View status | `sudo supervisorctl status` (rapidsms-router process) | +| Stop / Start / Restart | `sudo supervisorctl stop rapidsms-router` (or `start`, or `restart`) | + +### Background Task Process + +The background task process runs alongside the web application to manage background tasks and scheduled SMS messages. +Here is the key information for it: + +| Item | Value | +|------------------------|-------------------------------------------------------------------------| +| Process | Django (Celery) | +| Log files | `/home/cstock/www/cstock/log/celery.error.log` | +| Configuration file | `/home/cstock/www/cstock/code_root/logistics_project/localsettings.py` | +| View status | `sudo supervisorctl status` (celery process) | +| Stop / Start / Restart | `sudo supervisorctl stop celery` (or `start`, or `restart`) | + +### Database + +The database houses all persistent data for the application. +Here is the key information for it: + +| Item | Value | +|------------------------|------------------------------------------------------| +| Process | MySQL | +| View status | `sudo service mysql status` | +| Stop / Start / Restart | `sudo service mysql stop` (or `start`, or `restart`) | + +### Cache / Message Broker + +The cache is used to improve the performance of the site and pass messages between other processes. +Here is the key information for it: + +| Item | Value | +|------------------------|------------------------------------------------------| +| Process | Redis | +| View status | `sudo service redis status` | +| Stop / Start / Restart | `sudo service redis stop` (or `start`, or `restart`) | + +### Web Server + +The web server sits in front of the web application, providing outside internet access. It also hosts static files like images. +Here is the key information for it: + +| Item | Value | +|------------------------|------------------------------------------------------| +| Process | Nginx | +| Log files | `/var/log/nginx/` | +| Configuration file | `/etc/nginx/sites-available/cstock` | +| View status | `sudo service nginx status` | +| Stop / Start / Restart | `sudo service nginx stop` (or `start`, or `restart`) | + + +### SMS Gateway + +The SMS Gateway connects to third-party SMS providers (TNM and Airtel) and routes the messages to the SMS application process. +Here is the key information for it: + +| Item | Value | +|------------------------|-------------------------------------------------------| +| Process | Kannel | +| Log files | `/var/log/kannel/` | +| Configuration file | `/etc/kannel/kannel.conf` | +| View status | `sudo service kannel status` | +| Stop / Start / Restart | `sudo service kannel stop` (or `start`, or `restart`) | diff --git a/docs/systems-administration/server-access.md b/docs/systems-administration/server-access.md new file mode 100644 index 000000000..270820af5 --- /dev/null +++ b/docs/systems-administration/server-access.md @@ -0,0 +1,48 @@ +Connecting to the Server +======================== + +Access to the cstock server requires connecting to a VPN. +Follow the steps below to connect to the VPN and access the server. + +## Prerequisites + +Before you will be able to login to the cStock server you will need two accounts: + +1. A VPN account (or access to shared VPN credentials). +2. An account on the cStock server (or access to shared server credentials) + +If you don't have these, ask a system administrator to provide you access. + + +## VPN set up + +To access the VPN you will need to set up your VPN account and two-factor application. +You may need to install an older version of Sophos Authenticator to get the two-factor codes to work. + +Once you've gotten set up, download the provided OpenVPN config file and save it locally. + +## Logging in to the VPN + +On Ubuntu, login to the VPN by running: + +```bash +sudo openvpn --config ~/user__ssl_vpn_config.ovpn +``` + +Enter your credentials, which are you username, followed by your password + 2FA code appended together. +So if your password is `hunter2` and the 2-factor code is `123456`, +you would enter `hunter2123456` for the password field. + +For different operating systems, refer to the VPN documentation provided by the system administrator. + +## Logging in to cstock + +Once on the VPN, you can access cstock using the SSH command by running: + +```bash +ssh cstock@10.10.100.77 +``` + +If you have an individual user account, replace `cstock` with your username. + +For more information on using SSH, see [this page](https://www.ucl.ac.uk/isd/what-ssh-and-how-do-i-use-it). diff --git a/docs/systems-administration/server-setup.md b/docs/systems-administration/server-setup.md index c417f535c..6c4dbaa73 100644 --- a/docs/systems-administration/server-setup.md +++ b/docs/systems-administration/server-setup.md @@ -3,7 +3,7 @@ Server Installation These steps are from a production installation. -# Create a system user +## Create a system user Fill in the prompts after running the command: @@ -13,7 +13,7 @@ adduser cstock This user will be the one to run cstock and other related processes. -## Make the cstock user a sudoer +### Make the cstock user a sudoer This will allow them to run necessary commands as root. @@ -21,7 +21,7 @@ This will allow them to run necessary commands as root. sudo usermod -aG sudo cstock ``` -# Install and configure MySQL +## Install and configure MySQL Follow [this guide](https://www.digitalocean.com/community/tutorials/how-to-install-mysql-on-ubuntu-20-04) to install MySQL. @@ -30,7 +30,7 @@ sudo apt install mysql-server sudo systemctl start mysql.service ``` -## Set a root login/password +### Set a root login/password Replace the password with the one you want to set. @@ -40,13 +40,13 @@ mysql> ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY '* mysql> \q ``` -## Secure installation +### Secure installation ``` sudo mysql_secure_installation ``` -## Restore user-based access +### Restore user-based access ``` $ mysql -u root -p @@ -54,7 +54,7 @@ mysql> ALTER USER 'root'@'localhost' IDENTIFIED WITH auth_socket; mysql> \q ``` -## Create cstock database user +### Create cstock database user Don't forget to change the password! @@ -65,7 +65,7 @@ mysql> GRANT CREATE, ALTER, DROP, INSERT, UPDATE, INDEX, DELETE, SELECT, REFEREN mysql> \q ``` -## Create cstock database +### Create cstock database ``` $ mysql -u cstock -p @@ -73,7 +73,7 @@ mysql> CREATE DATABASE cstock; mysql> \q ``` -# Restore cstock database +## Restore cstock database If you haven't already, take a backup of the database from whatever server it is running on: @@ -100,7 +100,7 @@ sudo mysql cstock < cstock_database.sql Note: the above restore usually has to be run as root due to permissions issues. -# Install Python 3.9, pip, virtualenv, virtualenvwrapper, mysql dependencies +## Install Python, pip, virtualenv, virtualenvwrapper, mysql dependencies **For these steps, be sure you are logged in as the *cstock* user.** @@ -111,7 +111,7 @@ sudo apt install python3-pip python3 -m pip install --user virtualenv virtualenvwrapper ``` -# Set up virtualenvewrapper +### Set up virtualenvewrapper ``` source /home/cstock/.local/bin/virtualenvwrapper.sh @@ -120,7 +120,7 @@ source /home/cstock/.local/bin/virtualenvwrapper.sh If you run into errors, check your environment variables and update your `.profile` as described [here](https://askubuntu.com/a/995130) and [here](https://virtualenvwrapper.readthedocs.io/en/latest/). -# Install Redis +## Install Redis Following [this guide](https://www.digitalocean.com/community/tutorials/how-to-install-and-secure-redis-on-ubuntu-20-04). @@ -144,7 +144,7 @@ sudo systemctl restart redis.service None of the other steps are required. -# Create project directories +## Create project directories As the *cstock* user, set up your project directories: @@ -153,7 +153,7 @@ mkdir -p ~/www/cstock/ mkdir -p ~/www/cstock/log/ ``` -# Set up cstock +## Set up cstock Set up cstock code according to the [Dev Setup Instructions](../dev-setup.md). @@ -168,14 +168,14 @@ setvirtualenvproject pip install -r requirements.txt ``` -## Configure localsettings +### Configure localsettings Copy your localsettings across from the previous production project and edit anything relevant (e.g. database credentials) **If you are able to run `./manage.py runserver` with no issues, things are likely working as expected.** -# Install and configure nginx +## Install and configure nginx By following [this guide](https://www.digitalocean.com/community/tutorials/how-to-install-nginx-on-ubuntu-20-04). @@ -244,7 +244,7 @@ And restart nginx: sudo service nginx reload ``` -## Set up static files +### Set up static files In the cstock code home directory run: @@ -256,7 +256,7 @@ If static files are returning a 403 error, [check the permissions on the path](h If static files are returning a 404 error, check the paths in the nginx configuration above. -## Set up SSL +### Set up SSL Install certbot @@ -273,7 +273,7 @@ sudo certbot --nginx -d cstock.dimagi.com Follow the steps, and you're done. -# Install and configure supervisord +## Install and configure supervisord Follow [these instructions](https://www.digitalocean.com/community/tutorials/how-to-install-and-manage-supervisor-on-ubuntu-and-debian-vps) to set up supervisord. @@ -329,7 +329,7 @@ Once the file is set up correctly, you can add it to supervisor and start all th sudo supervisorctl reload ``` -# Set up and configure Kannel (SMS gateway) +## Set up and configure Kannel (SMS gateway) ``` sudo apt install kannel diff --git a/docs/systems-administration/troubleshooting.md b/docs/systems-administration/troubleshooting.md index b49fd61db..94d4fcf47 100644 --- a/docs/systems-administration/troubleshooting.md +++ b/docs/systems-administration/troubleshooting.md @@ -3,7 +3,16 @@ Troubleshooting Some common issues, and how to resolve them. -# "Too many connections" error +## Monitoring + +The project uses [Sentry](https://sentry.io/) for error monitoring. +Whenever there is an error in the application it will be logged to Sentry, and an alert will be sent to all Sentry users. + +## Common errors + +Some common error messages from Sentry are documented here. + +### "Too many connections" error If you see errors in Sentry saying "too many connections", it is an issue with too many open connections to the MySQL database. @@ -14,3 +23,111 @@ Restarting the router process usually resolves this: ``` sudo supervisorctl restart rapidsms-router ``` + +### "Warehouse already running, will do nothing..." + +This error indicates that a background job that builds the cStock reports every 12 hours failed because +the previous one has not completed. + +1. If it is the first time you have seen the error in a while, check if it is the first of the month. If it is, you can likely ignore it. The warehouse job takes longer than usual on the first and this is expected. +2. If it is not the first, or you see the error multiple times in a row you should check and update the report jobs (see the reporting section below). + +## SMS issues + +The first place to check for troubleshooting SMS issues is the [SMS status page](https://cstock.health.gov.mw/malawi/status/) on the site. + +The following is an example output from that page: + +``` + Kannel bearerbox version `1.4.5'. +Compiler `11.2.0'. +System Linux, release 5.15.0-88-generic, version #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023, machine x86_64. +Hostname cstock, IP 127.0.1.1. +Libxml version 2.9.12. +Using OpenSSL 3.0.0 7 sep 2021. +Compiled with MySQL 8.0.27, using MySQL 8.0.35. +Compiled with PostgreSQL 14.1 (Ubuntu 14.1-1ubuntu1). +Using SQLite 3.36.0. +Using hiredis API 0.14.1 +Using native malloc. + + +Status: running, uptime 32d 22h 13m 28s + +WDP: received 0 (0 queued), sent 0 (0 queued) + +SMS: received 4461 (0 queued), sent 9185 (1178 queued), store size -1 +SMS: inbound (0.02,0.00,0.00) msg/sec, outbound (0.12,0.03,0.00) msg/sec + +DLR: received 0, sent 0 +DLR: inbound (0.00,0.00,0.00) msg/sec, outbound (0.00,0.00,0.00) msg/sec +DLR: 0 queued, using internal storage + +Box connections: + smsbox:(none), IP 127.0.0.1 (0 queued), (on-line 32d 22h 13m 27s) + + +SMSC connections: + airtel-smpp[airtel-smpp] SMPP:messaging.airtel.mw:9001/9001:mnofhlth:VMA (online 178261s, rcvd: sms 2579 (0.02,0.00,0.00) / dlr 0 (0.00,0.00,0.00), sent: sms 5573 (0.12,0.03,0.00) / dlr 0 (0.00,0.00,0.00), failed 0, queued 0 msgs) + tnm-smpp-send[tnm-smpp-send] SMPP:41.78.250.95:5016/5016:CStock:SMPP (re-connecting, rcvd: sms 0 (0.00,0.00,0.00) / dlr 0 (0.00,0.00,0.00), sent: sms 3612 (0.00,0.00,0.00) / dlr 0 (0.00,0.00,0.00), failed 8, queued 0 msgs) + tnm-smpp-receive-1[tnm-smpp-receive-1] SMPP:41.78.250.40:5019/5019:CStock:SMPP (online 178174s, rcvd: sms 1882 (0.03,0.01,0.00) / dlr 0 (0.00,0.00,0.00), sent: sms 0 (0.00,0.00,0.00) / dlr 0 (0.00,0.00,0.00), failed 0, queued 0 msgs) + tnm-smpp-receive-2[tnm-smpp-receive-2] SMPP:41.78.250.42:5019/5019:CStock:SMPP (re-connecting, rcvd: sms 0 (0.00,0.00,0.00) / dlr 0 (0.00,0.00,0.00), sent: sms 0 (0.00,0.00,0.00) / dlr 0 (0.00,0.00,0.00), failed 0, queued 0 msgs) + + + + +Last Celery Heartbeat:2023-12-13 16:17:00.029043 +``` + +The most important part is the bottom section, labeled `SMSC connections`. +This will show the status of the four connections to SMS gateways (one for airtel, and 3 for TNM). +If the connection reports `online` then it is working as expected. +If it says `re-connecting` or anything else, it is not working. + +The most common reason that SMS fails is due to VPN issues between cStock and TNM. +These need to be resolved by the cStock hosting team and TNM networking team. + +If you need to make changes to the SMS gateway configrations, you can edit the Kannel configuration file. + +## Report Issues + +cStock reports are updated by a background task that runs every 12 hours. +If reports are failing to update, the problem is usually related to this task. + +### Viewing report task status + +The easiest way to see the status of the report task is in the Django admin area. + +After logging into cStock, visit this page: [https://cstock.health.gov.mw/admin/warehouse/reportrun/](https://cstock.health.gov.mw/admin/warehouse/reportrun/) +to see the recent report update jobs. + +When things are working well, you will see a green check under the "complete" column of every report update, +and a red "x" under the "has error" column. +If you see any recent reports with errors, there is likely a problem with the job. + +### Checking the logs for a job + +To see the output from a job you can check the celery logs by running: + +``` +tail -f -n 200 /home/cstock/www/cstock/log/celery.error.log +``` + +If the job is still running you'll see something like this: + +``` +[2023-09-25 09:20:55,702: WARNING/PoolWorker-2] processing health facility Ng'onga (6538) (493/641) +[2023-09-25 09:21:48,137: WARNING/PoolWorker-2] processing health facility Chang'ambika (6673) (494/641) +[2023-09-25 09:22:34,373: WARNING/PoolWorker-2] processing health facility Chapananga (6674) (495/641) +[2023-09-25 09:23:20,702: WARNING/PoolWorker-2] processing health facility Chkwawa DHO (6675) (496/641) +[2023-09-25 09:24:06,826: WARNING/PoolWorker-2] processing health facility Chipwaila (6676) (497/641) +[2023-09-25 09:24:53,339: WARNING/PoolWorker-2] processing health facility Dolo (6677) (498/641) +``` + +If the numbers at the end are still counting up then the job has not finished. +You should continue to let it run, and the reports should eventually be updated. + +If the numbers are no longer counting up, or you don't see anything like that in the logs, +you should manually go to the admin page and set the status of the most recent job to "complete". +Unintuitively, you should also set "has_error" to `False`, despite the fact that the job likely errored. +This is due to a legacy issue with how that field is used. diff --git a/docs/systems-administration/vpn.md b/docs/systems-administration/vpn.md deleted file mode 100644 index 328366494..000000000 --- a/docs/systems-administration/vpn.md +++ /dev/null @@ -1,34 +0,0 @@ -Connecting to the Server -======================== - -Access to the cstock server requires connecting to the VPN. -Follow the steps below to connect to the VPN and access the server. - -## VPN set up - -To get set up initially ask a system administrator to set you up with a new VPN account, and two-factor application. -You may need to install an older version of Sophos Authenticator to get the two-factor codes to work. - -Once you've gotten set up, download the OpenVPN config file and save it locally. - -## Logging in to the VPN - -On Ubuntu, login to the VPN by running: - -```bash -sudo openvpn --config ~/user__ssl_vpn_config.ovpn -``` - -Enter your credentials, which are you username, followed by your password + 2FA code appended together. -So if your password is `hunter2` and the 2-factor code is `123456`, -you would enter `hunter2123456` for the password field. - -## Logging in to cstock - -Once on the VPN, you can access cstock by running: - -```bash -ssh user@10.10.100.77 -``` - -Replace `user` with your username, or use the `cstock` user for shared access.