[Question] Help with a plot for an educational video (on The AnKing)

[gauerle]

Do you have a doubt about using FSRS in Anki? If so, please post on https://forums.ankiweb.net/ instead.

This issue tracker is primarily intended for questions and suggestions about the algorithm itself, rather than its usage in Anki. In addition, a larger community can provide support on the Anki Forums.

Before posting, please read the following:

Research

Enter an [x] character to confirm the points below:

• I have checked the FAQ and could not find an answer to my question
• I have read the wiki and still felt confused
• I have searched for similar existing questions here

Question
I’m currently working on a video about the updates introduced in Anki version 24.11, collaborating with @AnKingMed. To dive deeper into the topic of updates to the forgetting curve, I’ve been considering comparing the formulas used by different algorithms. Would you happen to have a combined plot that compares FSRS-3, FSRS-4, FSRS-4.5, DASH, and ACT-R? I found one in the materials provided by @Expertium, but unfortunately, it lacks proper scaling. If you already have something like this available, it would be very helpful. I've been thinking about something like this:

[Expertium]

You mean this?

I don't think that x values matter, it's the shape that matters.
More importantly, I don't think this is worth putting in the video in the first place. Users don't care about theory, they care about which button they need to press.
Btw, I wrote this for Anking: https://docs.google.com/document/d/1YUmXD3Y2wR-_WDMMMfKCLGtlJCe603e2/edit

I'm also not sure if right now is the best time to make a video. In the next major release FSRS will most likely become the default, so it's probably better to make a video then. But then again, this could be 3+ or even 6+ months away from today, so perhaps releasing a video now is good.

[gauerle]

Yes, I was referring to this graph from your blog. However, while the values on the x-axis aren’t particularly important, it doesn’t display the percentage probability of recall. The goal was to better illustrate the changes from FSRS-3 to FSRS-4.5, especially regarding the “changes to the shape of the forgetting curve.” The script is almost ready. @AnKingMed shared this document with me, and it has been incredibly helpful!

Edit: The video is structured into two parts: a fast lane and a slow lane. This in-depth explanation will be part of the slow lane. The goal here is to provide a deeper dive into the reasons behind certain changes and configurations, for those who are interested in exploring the details.

[Expertium]

IMO, the video shouldn't focus on the math too much and only briefly mention anything math-heavy. It's just not what the average user is looking for. And I plan to make a Reddit post and finish the benchmarking article eventually, it's just that there is still some stuff left to benchmark.

If you really want to compare FSRS curves, here: https://www.desmos.com/calculator/au54ecrpiz. I forgot to mention that x is in days, specifically.

Red = FSRS v3
Green = FSRS v4
Purple = FSRS-4.5 and FSRS-5

(notice that the naming convention isn't exactly the same, LMSherlock changed it around the time of FSRS-4.5)

[gauerle]

This will be a huge help, thank you very much! Yes, we know the video needs to be concise and straight to the point. That’s why we’ve divided it into two sections: the fast lane, which will cover everything the average user needs to do, and the slow lane, for those who want to dive a bit deeper into the reasoning behind some of the changes.

Also, would you like to review the script once it’s finished?

[Expertium]

Also, would you like to review the script once it’s finished?

Yep.

[Expertium]

Btw
ankitects/anki#3616 (comment)

I reacted with "thumbs down" for two reasons:

1. Lack of automatic optimization. It's not just to save 10 seconds per month. It's because a new user won't even realize that he needs to click "Optimize" in the first place. Sadly, this is one of the least likely features to be implemented. Dae is fully commited to making FSRS the default instead of working on whatever syncing issues automatic optimization would create.
2. No robust solution to the "Hard Misuse" problem: users using Hard as "fail". Around 10% of them. This results in insanely long intervals since FSRS treats Hard as "pass".
   A lot of solutions were proposed, the best of them is (IMO) this:
   
   Dae wants to make the two-button mode (Again and Good) the default, which is a much worse solution. It will make people ask "Guys, this blog/video shows Anki with 4 buttons but I only have 2, am I using the wrong Anki?", but more importantly, it doesn't teach users. If a user wants to enable 4 buttons, they would go to (probably) Preferences and enable some setting, and then fall into the same pitfall of not understanding how buttons work. A good solution should teach the user and actually make them realize the intended usage of buttons.

Put these two together, and you get a pretty bleak picture: most users never click "Optimize" because nothing in Anki tells them to do that (and Dae doesn't want to add a pop-up notification either), and the "Hard Misuse" problem is just swept under the rug.

[gauerle]

I completely agree with your points. I really liked the separate implementation for "again" and "hard/good/easy." Another approach could be renaming "hard" to something like "not easy," in addition to adjusting the color coding. For instance, using dark blue for "hard" or something similar. There’s research in psychophysics about the emotional responses triggered by colors—it could be interesting to conduct a study on user perception regarding this.
Another intriguing idea would be adding a second step to access the "hard" and "easy" options. For example, the main buttons could be "fail" and "pass," and then users could refine their choice by selecting "hard," "good," or "easy." It’s a risky suggestion, though. The best approach might be consulting users who were mistakenly using "hard," based on your research, to find a balanced solution. What if we implemented a step-by-step guided tutorial for first-time users when they install the program? That could help clarify things right from the start... Just some suggestions here.

Implementing FSRS as the default algorithm could indeed be harmful to many, uninformed users—essentially, the majority. However, I’m not sure how we could convince Dae to reconsider his stance on this. I imagine he might feel it’s not Anki responsibility to prevent users from falling into the trap of misusing the "hard" option.

And then there’s the issue of pattern matching, as you mentioned in one of your posts. While it might seem harmless at first, I think it’s something quite important to consider in future updates.

Also, we made sure to include a section in the video specifically dedicated to raising awareness among users about these issues.

[Expertium]

Another intriguing idea would be adding a second step to access the "hard" and "easy" options.

That's pretty much what will happen, I assume. Hard and Easy will have to be enabled in Tools -> Preferences.

What if we implemented a step-by-step guided tutorial for first-time users when they install the program?

This was also proposed, but it's too much work that Dae doesn't want.

And then there’s the issue of pattern matching, as you mentioned in one of your posts. While it might seem harmless at first, I think it’s something quite important to consider in future updates.

I would appreciate if you made a separate video about these note types: https://expertium.github.io/Avoid_Pattern_Matching.html

[Expertium]

I just updated the Google docs document to include potential changes in the next upcoming release.

[brishtibheja]

@gauerle requesting something be mentioned in the video: a lot of people think, due to FSRS's complexity, that it might be taking into account how much time you're spending per review. that can make some people really self-conscious so if the Anking video can clarify that, it'd be helpful for some I think.

[Expertium]

Another matter: I really hope that you and Anking will clarify that it's ok to use Hard, just as long as it's used as a passing grade.

https://www.reddit.com/r/Mcat/comments/1hukasv/i_made_have_made_a_huge_mistake_with_anki_fsrs/
This is something that I noticed a while ago, but didn't bother to save the post for later. This post reminded me.
There is a communication problem. We say "Don't use Hard as a failing grade, only use it as a passing grade", people hear "Don't use Hard".

[AnKingMed]

1. Lack of automatic optimization. It's not just to save 10 seconds per month. It's because a new user won't even realize that he needs to click "Optimize" in the first place. Sadly, this is one of the least likely features to be implemented. Dae is fully commited to making FSRS the default instead of working on whatever syncing issues automatic optimization would create.

I don't think this is a reason to not implement FSRS automatically because this feature can always be added in later.

2. No robust solution to the "Hard Misuse" problem: users using Hard as "fail". Around 10% of them. This results in insanely long intervals since FSRS treats Hard as "pass".

I don't think this should be a blocker either. Misusing the hard button has led to Ease hell for years now and will continue to do so. I agree something should be done about it, but don't think it should stop us from implementing a better algorithm

[Expertium]

So turns out that Anki 25.01 is coming. I just updated my Google doc.