Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LTI replace results on every score #1698

Merged
merged 16 commits into from
Nov 23, 2023
Merged

Conversation

kiragrammel
Copy link
Contributor

@kiragrammel kiragrammel commented Jun 1, 2023

Closes #1648

@codecov
Copy link

codecov bot commented Jun 1, 2023

Codecov Report

Attention: 23 lines in your changes are missing coverage. Please review.

Comparison is base (be30666) 66.74% compared to head (3edbb60) 69.77%.

❗ Current head 3edbb60 differs from pull request most recent head 2a0d632. Consider uploading reports for the commit 2a0d632 to get more accurate results

Files Patch % Lines
.../controllers/user_exercise_feedbacks_controller.rb 15.00% 17 Missing ⚠️
app/controllers/community_solutions_controller.rb 0.00% 2 Missing ⚠️
app/controllers/exercises_controller.rb 0.00% 1 Missing ⚠️
app/controllers/submissions_controller.rb 93.75% 1 Missing ⚠️
app/models/user_exercise_feedback.rb 50.00% 1 Missing ⚠️
app/policies/user_exercise_feedback_policy.rb 50.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1698      +/-   ##
==========================================
+ Coverage   66.74%   69.77%   +3.03%     
==========================================
  Files         195      196       +1     
  Lines        6182     6168      -14     
==========================================
+ Hits         4126     4304     +178     
+ Misses       2056     1864     -192     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Dome-GER Dome-GER changed the title Lti replace results on every score LTI replace results on every score Jun 1, 2023
@kiragrammel kiragrammel self-assigned this Jun 19, 2023
Dome-GER

This comment was marked as resolved.

@kiragrammel kiragrammel force-pushed the lti-replace_results_on_every_score branch from 4aadd4c to 2144f69 Compare June 21, 2023 10:21
@MrSerth

This comment was marked as resolved.

MrSerth

This comment was marked as resolved.

config/locales/de.yml Outdated Show resolved Hide resolved
@kiragrammel kiragrammel force-pushed the lti-replace_results_on_every_score branch 2 times, most recently from 0455b3a to 9fc54ad Compare August 21, 2023 09:55
@kiragrammel kiragrammel force-pushed the lti-replace_results_on_every_score branch from 7a0653e to eaf7c45 Compare August 24, 2023 08:11
@MrSerth

This comment was marked as outdated.

MrSerth

This comment was marked as resolved.

@kiragrammel kiragrammel force-pushed the lti-replace_results_on_every_score branch from eaf7c45 to dd32a6c Compare August 25, 2023 12:03
@kiragrammel kiragrammel force-pushed the lti-replace_results_on_every_score branch from 3c7ba0e to 30e4e63 Compare September 28, 2023 14:03
@MrSerth MrSerth linked an issue Oct 23, 2023 that may be closed by this pull request
@kiragrammel kiragrammel force-pushed the lti-replace_results_on_every_score branch 2 times, most recently from af0fcd5 to 66feb70 Compare October 25, 2023 07:00
config/locales/en.yml Outdated Show resolved Hide resolved
app/controllers/submissions_controller.rb Outdated Show resolved Hide resolved
app/controllers/submissions_controller.rb Outdated Show resolved Hide resolved
app/controllers/submissions_controller.rb Outdated Show resolved Hide resolved
app/controllers/submissions_controller.rb Outdated Show resolved Hide resolved
app/controllers/submissions_controller.rb Outdated Show resolved Hide resolved
config/locales/en.yml Outdated Show resolved Hide resolved
@kiragrammel kiragrammel force-pushed the lti-replace_results_on_every_score branch from 824cb3d to 6b168cd Compare October 31, 2023 17:43
MrSerth

This comment was marked as resolved.

@kiragrammel kiragrammel marked this pull request as ready for review November 1, 2023 16:35
MrSerth

This comment was marked as resolved.

@MrSerth MrSerth force-pushed the lti-replace_results_on_every_score branch from a1488b8 to 87113e1 Compare November 1, 2023 23:34
@kiragrammel kiragrammel force-pushed the lti-replace_results_on_every_score branch from b5f5d5f to f62d822 Compare November 8, 2023 11:08
spec/features/score_spec.rb Outdated Show resolved Hide resolved
MrSerth

This comment was marked as resolved.

MrSerth

This comment was marked as resolved.

@MrSerth MrSerth force-pushed the lti-replace_results_on_every_score branch 2 times, most recently from 3b9399d to 5def888 Compare November 22, 2023 21:49
Copy link
Member

@MrSerth MrSerth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, nicely done and many thanks for your perseverance! 👏 🥳 🎉 🥂 👯

I am very that you took the challenge to work on this crucial part of the application while digging deep into everything that's LTI related and covering your changes with so many, extensive tests. As you see, almost each line touched is covered, and a magnitude more lines are newly covered by tests even without any change.

Furthermore, I just finished my review of all code parts and am really satisfied with the result. For a final test, I've deployed the changes to Staging and CodeMoon, allowing us to test-drive the changes in a real-world context.

When doing so myself, I discovered another error with the user exercise feedbacks, not caused by our recent change. Besides that, I was surprised about the 10% rule of the feedbacks, because it was much more frequent in my tests. Regardless of that, the basic workflow was as expected. Nevertheless, it probably makes sense to have a final check 🤓.

MrSerth and others added 16 commits November 23, 2023 14:15
Here, we are only checking the condition based on the URL if both parameters (exercise and programming group) are given. Otherwise, we skip the check.
With this commit, we refactor the overall score handling of CodeOcean. Previously, "Score" and "Submit" were two distinct actions, requiring users to confirm the LTI transmission of their score (after assessing their submission). This yielded many questions and was unnecessary, since LTI parameters are no longer expiring after each use. Therefore, we can now transmit the current grade on each score run with the very same LTI parameters. As a consequence, the LTI consumer gets a more detailed history of the scores, enabling further analytical insights.

For users, the previous "Submit" button got replaced with a notification that is shown as soon as the full score got reached. Then, learners can decide to "finalize" their work on the given exercise, which will initiate a redirect to a follow-up action (as defined in the RedirectBehavior). This RedirectBehavior has also been unified and simplified for better readability.

As part of this refactoring, we rephrased the notifications and UX workflow of a) the LTI transmission, b) the finalization of an exercise (measured by reaching the full score) and c) the deadline handling (on time, within grace period, too late). Those information are now separately shown, potentially resulting in multiple notifications. As a side effect, they are much better maintainable, and the LTI transmission is more decoupled from this notification handling.
@MrSerth MrSerth force-pushed the lti-replace_results_on_every_score branch from 3edbb60 to 2a0d632 Compare November 23, 2023 13:30
@MrSerth MrSerth enabled auto-merge (rebase) November 23, 2023 13:31
@MrSerth MrSerth merged commit 60b0a5b into master Nov 23, 2023
6 checks passed
@MrSerth MrSerth deleted the lti-replace_results_on_every_score branch November 23, 2023 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Prevent Score Button Spamming LTI: Replace result on every "Score"
3 participants