New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create simple progress visualisation #41

Open

thill-odi opened this issue Apr 5, 2020 · 0 comments

Assignees

Labels

Contributor

thill-odi commented Apr 5, 2020

At the moment, the process for tracking progress between stages 1 and 2 of the harvest is extremely manual.

It would be possible to ameliorate this by:

Adding a field in the harvester-stage-2-state which holds the value of updatedLastSeen as a Date rather than epoch value (e.g. updatedLastSeenAsDate)
Creating an index pattern of the form 'harvester*' in Kibana to allow querying across indices.
Creating visualisation comparing the before and after values of 'updated' in the harvester-raw index to give a rough sense of how much processing remains to be done.

thill-odi self-assigned this

thill-odi added the enhancement label

thill-odi assigned thill-odi and rachelwilson and unassigned thill-odi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment