Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(classifier): set started_at timestamp when annotation starts #7216

Conversation

eatyourgreens
Copy link
Contributor

  • add metadata._inProgress to classifications, and set it to true when a volunteer first updates an annotation.
  • set classification.metadata.started_at to the current time when metadata._inProgress changes from false to true.

Staging branch URL: https://pr-{NUMBER}.pfe-preview.zooniverse.org

Fixes #7215.

This is zooniverse/front-end-monorepo#6449 adapted for PFE's redux store. It updates classification.metadata.started_at the first time that you interact with a subject, eg. by answering a question or drawing a mark from a drawing task.

Required Manual Testing

  • Does the non-logged in home page render correctly?
  • Does the logged in home page render correctly?
  • Does the projects page render correctly?
  • Can you load project home pages?
  • Can you load the classification page?
  • Can you submit a classification?
  • Does talk load correctly?
  • Can you post a talk comment?

Review Checklist

  • Does it work in all major browsers: Firefox, Chrome, Edge, Safari?
  • Does it work on mobile?
  • Can you npm ci and app works as expected?
  • If the component is in coffeescript, is it converted to ES6? Is it free of eslint errors? Is the conversion its own commit?
  • Are the tests passing locally and on GitHub Actions?

Optional

@coveralls
Copy link

coveralls commented Nov 8, 2024

Coverage Status

coverage: 56.656% (-0.03%) from 56.684%
when pulling 40ee27a on eatyourgreens:classification-started-at
into 8ad3b19 on zooniverse:master.

@eatyourgreens eatyourgreens force-pushed the classification-started-at branch from a0ee61f to 89b1650 Compare November 8, 2024 16:24
@@ -51,6 +51,7 @@ function createNewClassification(project, workflow, subject, goldStandardMode, l
// Delete the metadata key because we don't want volunteers to see it.
subject.update({ 'metadata.intervention': undefined });
const newMetadata = {
_inProgress: false,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The underscore here ensures that the Panoptes JS Client will automatically remove this property when it saves the classification to the API.
https://github.com/zooniverse/panoptes-javascript-client/blob/721b696dccecf86a35614244ab29d19d1a62c3b7/lib/json-api-client/model.js#L17-L38

- add `metadata._inProgress` to classifications, and set it to true when a volunteer first updates an annotation.
- set `classification.metadata.started_at` to the current time when `metadata._inProgress` is set to true.
@eatyourgreens eatyourgreens force-pushed the classification-started-at branch from 89b1650 to 40ee27a Compare November 19, 2024 19:03
@goplayoutside3
Copy link
Contributor

The full comment about why this PR is being closed can be found here: zooniverse/front-end-monorepo#6447 (comment)

Some of the info copied here for clarity:
The frontend components that display hours a volunteer spent classifying use session_time from ERAS. If the classification is stored in Panoptes, session_time is calculated by the classification’s finished_at - started_at metadata. When Ouroboros classifications were ported over to Panoptes, steps were taken by the backend team to account for the lack of finished_at in the metadata.

session_time is where a time-limit cap is applied if the two properties were recorded several hours apart or even days apart. The time-limit cap was decided specifically with the new user stats website features in mind. We decided it was better to overestimate from a place of trust than underestimate time spent classifying in the cases where finished_at - started_at is large.

Decision:
started_at will continue to be recorded on subject load. Changing started_at to the time of first annotation misses the volunteer's time spent analyzing the subject before making an annotation, referencing the tutorial, and watching the flipbook or video subjects. For some projects annotation begins with interaction with the subject through drawing while for others it begins only once a simple choice task is completed. started_at recorded on subject load is in line with “better to overestimate from a place of trust than underestimate”.

@eatyourgreens eatyourgreens deleted the classification-started-at branch November 21, 2024 23:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Classification started_at can be off by hours or days
3 participants