dimagi · czue · Dec 14, 2023 · Dec 13, 2023 · Dec 13, 2023 · Dec 13, 2023
diff --git a/docs/README.md b/docs/README.md
@@ -25,6 +25,20 @@ cd _builds/html
 python -m http.server
 ```
 
+You can also automatically build the docs with `sphinx-autobuild`:
+
+Install it:
+
+```
+pip install sphinx-autobuild
+```
+
+Then run:
+
+```
+sphinx-autobuild . _build/html/ --port 8001
+```
+
 ## Hosting the docs
 
 Docs are hosted on readthedocs and managed by Dimagi.

diff --git a/docs/administration.rst b/docs/administration.rst
@@ -3,6 +3,9 @@ cStock Administration
 
 This section outlines common tasks performed by a cStock administrator.
 
+Administrators are users that have special permissions on the cStock site,
+and can manage users, access the system data models, and perform other administrative operations.
+
 .. toctree::
 
    administration/admin-site.md

diff --git a/docs/administration/admin-site.md b/docs/administration/admin-site.md
@@ -1,6 +1,7 @@
 Admin site access
 =================
 
-To access the admin site, go to [http://cstock.jsi.com/admin/](http://cstock.jsi.com/admin).
-If you are prompted to login, then you may not have access.
+To access the admin site, first [login to cStock](https://cstock.health.gov.mw/accounts/login/), then 
+go to [https://cstock.health.gov.mw/admin/](https://cstock.health.gov.mw/admin/).
+If you are prompted to login again, then you may not have access.
 You should contact another administrator to grant you access.
diff --git a/docs/conf.py b/docs/conf.py
@@ -28,7 +28,8 @@
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
 extensions = [
-    'recommonmark'
+    'recommonmark',
+    'sphinx_markdown_tables',
 ]
 
 # Add any paths that contain templates here, relative to this directory.

diff --git a/docs/images/cstock-architecture.png b/docs/images/cstock-architecture.png
diff --git a/docs/index.rst b/docs/index.rst
@@ -3,8 +3,16 @@
    You can adapt this file completely to your liking, but it should at least
    contain the root `toctree` directive.
 
-Welcome to cStock's documentation!
-==================================
+cStock Project Documentation
+============================
+
+cStock is a website that provides real-time, actionable HSA logistics data for managers and stakeholders to coordinate,
+plan, and identify solutions to better meet customer needs in a timely manner.
+
+cStock is hosted at `https://cstock.health.gov.mw/ <https://cstock.health.gov.mw/>`_.
+
+This documentation is intended for cStock administrators, system administrators, and developers.
+
 
 .. toctree::
    :maxdepth: 3

diff --git a/docs/requirements.in b/docs/requirements.in
@@ -1,2 +1,3 @@
 sphinx
 recommonmark
+sphinx-markdown-tables
diff --git a/docs/requirements.txt b/docs/requirements.txt
@@ -1,6 +1,6 @@
 #
-# This file is autogenerated by pip-compile with python 3.8
-# To update, run:
+# This file is autogenerated by pip-compile with Python 3.9
+# by the following command:
 #
 #    pip-compile requirements.in
 #
@@ -23,9 +23,13 @@ idna==3.3
 imagesize==1.3.0
     # via sphinx
 importlib-metadata==4.11.3
-    # via sphinx
+    # via
+    #   markdown
+    #   sphinx
 jinja2==3.1.1
     # via sphinx
+markdown==3.5.1
+    # via sphinx-markdown-tables
 markupsafe==2.1.1
     # via jinja2
 packaging==21.3
@@ -46,6 +50,8 @@ sphinx==4.5.0
     # via
     #   -r requirements.in
     #   recommonmark
+sphinx-markdown-tables==0.0.17
+    # via -r requirements.in
 sphinxcontrib-applehelp==1.0.2
     # via sphinx
 sphinxcontrib-devhelp==1.0.2

diff --git a/docs/systems-administration.rst b/docs/systems-administration.rst
@@ -5,7 +5,9 @@ This section outlines common tasks performed by a cStock *system* administrator.
 
 .. toctree::
 
+   systems-administration/overview.md
+   systems-administration/server-access.md
    systems-administration/common-tasks.md
    systems-administration/troubleshooting.md
    systems-administration/server-setup.md
-   systems-administration/vpn.md
+
diff --git a/docs/systems-administration/common-tasks.md b/docs/systems-administration/common-tasks.md
@@ -1,19 +1,37 @@
 Common System Administration Tasks
 ==================================
 
-## Deploying Code
+These are some common tasks required of system administrators.
 
-Deploying code requires installing fabric3 (`pip install -r requrements/deploy/dev-requirements.txt`).
+All of these tasks require [server access](./server-access.md).
 
-Then, to deploy you must first connect to the VPN. Then run:
+## Deploying Updated Code
+
+Deploying code requires installing fabric3:
+
+```
+pip install -r requrements/deploy/dev-requirements.txt
+```
+
+And then cloning [the code repository](https://github.com/dimagi/logistics/):
+
+```
+git clone https://github.com/dimagi/logistics.git
+```
+
+Then, to deploy you must first connect to the VPN. Then run the following command in the repository root:
 
 ```
 fab malawi deploy
 ```
 
-## Restarting processes
+This command should be run on *your own machine*.
+For the remaining sections, you must run the commands *on the server*.
 
-We use supervisor (`/etc/supervisor.conf`) to manage the running processes.
+## Restarting application processes
+
+We use supervisor (`/etc/supervisor.conf`) to manage the running cStock processes,
+including the web application, SMS application, and background task application.
 
 To see the list of running processes run:
 
@@ -26,3 +44,74 @@ You can restart individual processes with:
 ```
 sudo supervisorctl restart <process-name>
 ```
+
+## Restarting other processes
+
+Sometimes other processes (the web server, database, etc.) need to be restarted.
+To manage those you can find the information in the [overview page](./overview).
+
+## Making configuration changes
+
+If you need to make any configuration changes, for example, updating passwords or IP addresses,
+those happen in the relevant configuration files for whichever service you need to update.
+
+To find the service's configuration file, look it up on the [overview page](./overview).
+
+For example, to update the credentials for the TNM or Airtel SMS gateway, you would modify
+the SMS gateway (Kannel) configuration file at `/etc/kannel/kannel.conf`.
+
+Typically, whenever you make changes to a service, you also need to restart the service,
+as per the instructions above.
+
+## Getting an application shell
+
+Sometimes it can be useful to get an application shell, to run the cStock Python code manually,
+for example, to inspect data models or make once-off changes.
+
+To get an application shell you can enter the virtual environment like this (as the `cstock` user):
+
+```
+workon cstock
+```
+
+This should enter the virtual environment and load you in the right directory.
+From there you can run:
+
+```
+python manage.py shell
+```
+
+To get a Python shell. You can then run code to work with the Django application.
+For example, to see how many registered web users there are, you can run:
+
+```
+>>> from django.contrib.auth.models import User
+>>> User.objects.count()
+551
+```
+
+## Getting a database shell
+
+The easiest way to get a database shell is to first enter the virtual environment as per above:
+
+```
+workon cstock
+```
+
+Then run:
+
+```
+python manage.py dbshell
+```
+
+Then to run the equivalent command in MySQL you could run:
+
+```
+mysql> SELECT count(*) FROM auth_user;
++----------+
+| count(*) |
++----------+
+|      551 |
++----------+
+1 row in set (0.00 sec)
+```
diff --git a/docs/systems-administration/overview.md b/docs/systems-administration/overview.md
@@ -0,0 +1,130 @@
+cStock System Overview
+======================
+
+[cStock](https://cstock.health.gov.mw/) is a website built and maintained by the Malawi Ministry of Health.
+There is a web component as well as an SMS component.
+
+## Hosting
+
+cStock is hosted by the Ministry of Health in a local data center.
+
+## Code
+
+The cStock application code is hosted [on Github](https://github.com/dimagi/logistics/).
+It is a Python / Django application, based on RapidSMS.
+
+In general system administrators should not need to modify cStock code,
+but it would be necessary to add new features or fix bugs.
+
+To get up and running with the cStock code, see [the development setup documentation](/dev-setup/).
+
+## Key Services
+
+cStock consists of the following key services:
+
+![cStock Architecture](/images/cstock-architecture.png)
+
+In the above, green boxes run custom cStock code, blue boxes are 3rd-party applications running
+on the cStock server, and grey boxes are external services.
+
+Here is a high-level description of each cStock service:
+
+| Service                 | Description                                                                                                                     | Technology        |
+|-------------------------|---------------------------------------------------------------------------------------------------------------------------------|-------------------|
+| Web Application Process | The web application is the business logic that powers the cStock web application.                                               | Django            |
+| SMS Application Process | A process that runs alongside the web application to manage SMS workflows                                                       | Django + RapidSMS |
+| Background Task Process | A process that runs alongside the web application to manage background tasks and scheduled SMS messages                         | Django + Celery   |
+| Database                | The database houses all persistent data.                                                                                        | MySQL             |
+| Cache / Message Broker  | The cache is used to improve the performance of the site and pass messages between other processes.                             | Redis             |
+| Web Server              | The web server sits in front of the web application, providing outside internet access. It also hosts static files like images. | Nginx             |
+| SMS Gateway             | The SMS Gateway connects to third-party SMS providers (TNM and Airtel) and routes the messages to the SMS application process.  | Kannel            |
+
+In general, all of these need to be running and functioning properly for cStock to work.
+
+### Web Application Process
+
+The web application is the business logic that powers the cStock web application.
+Here is the key information for it:
+
+| Item                    | Value                                                                                                   | 
+|-------------------------|---------------------------------------------------------------------------------------------------------|
+| Process                 | Django (Gunicorn)                                                                                       | 
+| Log files               | `/home/cstock/www/cstock/log/gunicorn.command.log` and `/home/cstock/www/cstock/log/gunicorn.error.log` |
+| Configuration file      | `/home/cstock/www/cstock/code_root/logistics_project/localsettings.py`                                  |
+| View status             | `sudo supervisorctl status` (gunicorn process)                                                          |
+| Stop / Start / Restart  | `sudo supervisorctl stop gunicorn` (or `start`, or `restart`)                                           |
+
+### SMS Application Process
+
+The SMS web application runs alongside the web application to manage SMS workflows.
+Here is the key information for it:
+
+| Item                   | Value                                                                    | 
+|------------------------|--------------------------------------------------------------------------|
+| Process                | Django (RapidSMS)                                                        | 
+| Log files              | `/home/cstock/www/cstock/log/rapidsms.log`                               |
+| Configuration file      | `/home/cstock/www/cstock/code_root/logistics_project/localsettings.py`  |
+| View status            | `sudo supervisorctl status` (rapidsms-router process)                    |
+| Stop / Start / Restart | `sudo supervisorctl stop rapidsms-router` (or `start`, or `restart`)     |
+
+### Background Task Process
+
+The background task process runs alongside the web application to manage background tasks and scheduled SMS messages.
+Here is the key information for it:
+
+| Item                   | Value                                                                   | 
+|------------------------|-------------------------------------------------------------------------|
+| Process                | Django (Celery)                                                         | 
+| Log files              | `/home/cstock/www/cstock/log/celery.error.log`                          |
+| Configuration file      | `/home/cstock/www/cstock/code_root/logistics_project/localsettings.py` |
+| View status            | `sudo supervisorctl status` (celery process)                            |
+| Stop / Start / Restart | `sudo supervisorctl stop celery` (or `start`, or `restart`)             |
+
+### Database
+
+The database houses all persistent data for the application.
+Here is the key information for it:
+
+| Item                   | Value                                                | 
+|------------------------|------------------------------------------------------|
+| Process                | MySQL                                                | 
+| View status            | `sudo service mysql status`                          |
+| Stop / Start / Restart | `sudo service mysql stop` (or `start`, or `restart`) |
+
+### Cache / Message Broker
+
+The cache is used to improve the performance of the site and pass messages between other processes.
+Here is the key information for it:
+
+| Item                   | Value                                                | 
+|------------------------|------------------------------------------------------|
+| Process                | Redis                                                | 
+| View status            | `sudo service redis status`                          |
+| Stop / Start / Restart | `sudo service redis stop` (or `start`, or `restart`) |
+
+### Web Server
+
+The web server sits in front of the web application, providing outside internet access. It also hosts static files like images.
+Here is the key information for it:
+
+| Item                   | Value                                                | 
+|------------------------|------------------------------------------------------|
+| Process                | Nginx                                                | 
+| Log files              | `/var/log/nginx/`                                    | 
+| Configuration file     | `/etc/nginx/sites-available/cstock`                  | 
+| View status            | `sudo service nginx status`                          |
+| Stop / Start / Restart | `sudo service nginx stop` (or `start`, or `restart`) |
+
+
+### SMS Gateway
+
+The SMS Gateway connects to third-party SMS providers (TNM and Airtel) and routes the messages to the SMS application process.
+Here is the key information for it:
+
+| Item                   | Value                                                 | 
+|------------------------|-------------------------------------------------------|
+| Process                | Kannel                                                | 
+| Log files              | `/var/log/kannel/`                                    | 
+| Configuration file     | `/etc/kannel/kannel.conf`                             | 
+| View status            | `sudo service kannel status`                          |
+| Stop / Start / Restart | `sudo service kannel stop` (or `start`, or `restart`) |