web-monitoring-ui

This repository is part of the EDGI Web Monitoring Project.

This component works with web-monitoring-db and web-monitoring-processing to support the next web monitoring workflow.

It’s a React.js-based browser application with a Node.js backend with the following capabilities:

• Consume subset of data from web-monitoring-db as proof of concept, read/write annotations
  • DEMO
  • LIST VIEW shows first page of records from web-monitor-db JSON endpoint
  • PAGE VIEW shows basic info about the latest version of that page: site, URLs and links to Wayback Machine calendar view and page versions
    • updates annotations

Installation

1. Install Node v10.15.3

   • We recommend installing Node Version Manager, then: nvm install 10.15.3
   • If you are using Windows, check out Nodenv or any of these alternatives.

2. Install node dependencies with npm

   npm install

3. Copy .env.example to .env and supply any local configuration info you need (all fields are optional)

4. Start the web server

   npm start

5. (Optional) Set up Google Sheets for saving important changes and repeated, “dictionary” changes. See the section below on Google Sheets.

Running tests

To run all tests once

npm test

while to start the test runner in watch mode

npm run test-watch

Manual view

Access the main view at http://localhost:3001

Screenshot:

Google Sheets (Significant Changes)

The analysis UI keeps some data and runtime configuration separate from the public web monitoring database (web-monitoring-db). This data is kept in 2 Google Docs spreadsheets. You can use the UI without configuring them, but you will be missing some functionality.

First, you’ll need to create a service account the application can use to access the sheets. To do so, follow the first half of this tutorial. During the process, you should have downloaded a .json file with authentication information. Add the client_email and private_key fields from the file to your .env file:

GOOGLE_SERVICE_CLIENT_EMAIL=73874number-example@developer.gserviceaccount.com
GOOGLE_SHEETS_PRIVATE_KEY=-----BEGIN PRIVATE KEY-----\EXAMPLEExampleG9w0BAQEFAASCBKkwggSlAgEAAoIBAQCsBjS7qpN+vUhz\nXRhcL3pwKZtewjZ478rs7FylT+YAJMmy1wOS3ze2FVYaBHelloFromXm9gL82OCSJn1ZThePastuwGA0\npe9oZSAtiB4ujaHhcWCO7ZYZzBvsPRJZR2eo4UokDpmgAS9ExTU7zN+eKTBTFGB4\nKDc7FAxqhk9dBcYFpLU34wuQsS/SZY1j3I/pmqQ7CHnGG+KLhyRiZ6UvlT8KjWejWTFdfMoredksjfGibberishkljfkls+\nkerGibberishll7\n7oU0VVs3xY5nhkjd#r34jkd7vxjknfy3jsdhf5zjkGYfyXFNhVjsl/bJ3AHA/C9Fd5z9JmOCsZE\nyD9Yjy72C50CjOgCp568pse85A==\n-----END PRIVATE KEY-----\n

In the next section, you’ll create the 2 sheets.

Important Changes and Dictionary Sheets

Two Google Docs spreadsheets are used to keep track of changes that users mark as “important” or add to the “dictionary” of repeated, common changes. This information also gets saved to the database, but since analysts’ current workflow is spreadsheet-based, we also send this data to the spreadsheets. To get this working:

1. Make copies of these spreadsheets:

   • Important Changes
   • Dictionary

   Make note the of sheet IDs of your new spreadsheets.

2. Share the spreadsheets with the e-mail address of the service account you created earlier. You must give it write access.

3. Update your .env file with the IDs of the sheets:

GOOGLE_IMPORTANT_CHANGE_SHEET_ID=examplesdf8Za7sdft39a_osnzhJBI2dsftasdf
GOOGLE_DICTIONARY_SHEET_ID=examplesdf8Za7sdft39a_osnzhJBI2dsftasdf

Restart your app server and try clicking on the “add important change” or “add to dictionary” buttons. A new line should be added to the relevant sheet.

Code of Conduct

This repository falls under EDGI's Code of Conduct.

Getting Involved

We need your help! Please read through the Web Monitoring Project project document and see what you can help with and check EDGI’s contribution guidelines for information on how to propose issues or changes.

Docker

You can also run this project via Docker. To build and run (on port 3001, as in the instructions for running directly above):

docker build -t envirodgi/ui .
docker run -p 3001:3001 -e <ENVIRONMENT VARIABLES> envirodgi/ui

Point your browser to http://localhost:3001.

To run tests via Docker:

docker build -t envirodgi/ui:dev --target dev .
docker run envirodgi/ui:dev npm run test

Contributors

This project wouldn’t exist without a lot of amazing people’s help. Thanks to the following for all their contributions!

Contributions	Name
📖 📋 💬 👀	Dan Allan
💡	@allanpichardo
💡	@ArcTanSusan
💡	@AutumnColeman
📋 🔍	Andrew Bergman
💻 📖 💬 👀	Rob Brackett
📖	Patrick Connolly
📖	Manaswini Das
💡	@lh00000000
💻 🎨 📖 💬 👀	Kevin Nguyen
📖 📋 📢	Matt Price
📋 🔍	Toly Rinberg
💻	Ben Sheldon
💡	@StephenAlanBuckley
💡	@stuartlynn
💻	Michelle Truong
📖 📋	Dawn Walker
💻 📖 ⚠️	Sarah Yu
💻 ⚠️	Alberto Zaccagni

(For a key to the contribution emoji or more info on this format, check out “All Contributors.”)

License & Copyright

Copyright (C) <2017> Environmental Data and Governance Initiative (EDGI) This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, version 3.0.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

See the LICENSE file for details.