This is a work in progress project that needs help in a number of ways:
- Importing the data we have collected from councils (See Below)
- If you are a developer (python, django, frontend, etc) or designer, we need help making the site itself.
- If you are interested in helping us gather this data, or if you know a lot about the strange world of the UK geographic system.
If you are interested in helping out in any way at all, please contact [email protected]
UK-Polling-Stations requires python 3.10
UK-Polling-Stations requires Python 3.10, Postgres, PostGIS, libgeos, GDAL, Node JS and NPM.
On Mac OSX, run:
brew install postgresql
brew install postgis
brew install geos
brew install gdal
brew install node
From a clean install of Ubuntu 22.04 (Bionic):
sudo apt-get install postgresql-14 postgresql-server-dev-all python-psycopg2 python3-dev postgis postgresql-14-postgis-3 libxml2-dev libxslt1-dev nodejs npm
sudo npm install -g npm@latest-6
For other linux distributions, see here for details on installing geospatial libraries for use with Django.
- Local Dev without CDK libs
pip install -r requirements/base.txt -r requirements/testing.txt -r requirements/local.txt -r cdk/lambdas/wdiv-s3-trigger/requirements.txt cdk/lambdas/wdiv-s3-trigger/requirements/testing.txt
- Local Dev with CDK libs
pip install -r requirements/base.txt -r requirements/testing.txt -r requirements/local.txt -r requirements/cdk.txt -r cdk/lambdas/wdiv-s3-trigger/requirements.txt
- Just Running Tests
pip install -r requirements/base.txt -r requirements/testing.txt -r cdk/lambdas/wdiv-s3-trigger/requirements.txt
- Update requirements
Bump version in relevant requirements file, e.g.
base.in
then runpython -m invoke requirements --upgrade
.
npm install
pip install -r requirements/testing.txt
We have a suite of end-to-end integration tests. We use Playwright
with pytest
to run these.
To set up playwright, after running pip install -r requirements/testing.txt
,
run playwright install
. This will download the browser biniaries required
for your system.
The Playwright tests will run in a headless browser as part of a normal pytest
run.
You may want to run the Playwright suite in headed mode for debugging purposes. To do that, run the following:
pytest polling_stations/apps/data_finder/tests/playwright/ --headed
cp polling_stations/settings/local.example.py polling_stations/settings/local.py
sudo -u postgres createdb polling_stations
sudo -u postgres createuser dc -P -s
sudo -u postgres psql polling_stations
psql (9.3.6)
Type "help" for help.
polling_stations=# CREATE EXTENSION postgis;
CREATE EXTENSION
polling_stations=#
Be sure that your database user name matches
To check, run:
psql postgres
then:
\l
python manage.py migrate
For development purposes, you can use the ONSPD for geocoding. Grab the latest release from https://geoportal.statistics.gov.uk/search?collection=Dataset&sort=-modified&tags=PRD_ONSPD unzip the data and import it using:
python manage.py import_onspd /path/to/data
N.B. Path is to data directory which contains the csv, not the csv itself.
python manage.py import_councils
For development purposes, you will need to seed your database with some data. We depend on Ordnance Survey Addressbase Plus, which is not publicly available. To allow open source contributions we have prepared the sample data Ordnance Survey provide (https://www.ordnancesurvey.co.uk/business-government/products/addressbase-plus#sample-data) ready for import, and some sample import scripts for testing.
To prepare your database run the following commands:
./manage.py import_cleaned_addresses test_data/addressbase/
./manage.py create_uprn_council_lookup
./manage.py import_uprn_council_lookup uprn-to-councils.csv
You can clean up the look up with:
rm uprn-to-councils.csv
And finally you can import some dummy data with:
./manage.py import_fake_teignbridge
./manage.py import_fake_exeter
Each council that has unimported data has a Github Issue with the Data Import label.
We make a Django manage.py
command in the data_importers app for each council which imports the raw data.
If you are interested in helping the project by writing an import script, see the issues tagged recommended for beginners for more info.
django-admin makemessages -l cy --ignore='env*'
./manage.py compilemessages
If you like you can use the commit hooks defined in .pre-commit-config.yaml
. Run pre-commit install && pre-commit install -t pre-push
.
If a user requests access to the council uploader using a custom domain, the domain should be added to the CUSTOM_DOMAINS
setting.
To remove a custom domain, follow these steps:
- Delete the entry from the CUSTOM_DOMAIN list in
polling_stations/settings/constants/uploads.py
- In the admin panel, delete the UserCouncil object from the bottom of the Council profile page
As part of preparations for upcoming elections, NEXT_CHARISMATIC_ELECTION_DATES
will need to be updated in
settings/constants/elections.py
. This var takes a ["YYYY-MM-DD",] format.
To show messaging about the 2022 GB voter ID legislation, update SHOW_GB_ID_MESSAGING
to True
in settings/constants/elections.py
.
This project has an API, however it's not designed for public consumption.
To authenticate against the API you need to pass a token in the auth_token
GET parameter.
The main API endpoints are postcode
and address
. These endpoints return polling station information for the given location.
In turn, they request the EveryElection API.
sequenceDiagram
AggregatorAPI ->> WDIV: Call from Aggregator API
WDIV ->> EE: WDIV calls EE to see if there are elections
EE ->> WDIV: Election data returned
WDIV ->> AggregatorAPI: Return polling station and election data to AggregatorAPI
This workflow is the same for the postcode
and address
endpoints.
The documentation for the AggregatorAPI should explain the main concepts for this API.
We use drf-spectacular
to document the API.
The documentation is inline with the code, meaning that comments and annotations should be enough
in most case to understand what's going on. drf-spectacular
comes with some views for
Redoc
and Swagger
. To enable these, set ENABLE_API_DOCS=True
in local.py
and visit http://localhost:8000/api/swagger/
or http://localhost:8000/api/docs/
.
There are two types of token:
READ_ONLY_API_AUTH_TOKENS
SUPERUSER_API_AUTH_TOKENS
Both are lists that are defined in settings. To add a token, add a string to the correct list, or set environment variables as comma seperated values.
Superuser API tokens are designed for the Trigger function that posts information about newly updated files back to the uploads app.
Read only tokens are for the workflow outlined above.