Looking for explanation on ease factor adjustments/information the addon gives #38
Replies: 6 comments
-
|
Happy to help. These were mainly for internal testing, but I should definitely provide more official info on them. Guide to the values
Say you failed a card a lot in the beginning, but now you've finally got it down. Your reps are: 1, 1, 1, 1, 3, 3, 3, 3. A simple average would put those both at 50% success. The moving average prioritizes what's been happening lately. So it will see the first one as a 59% success rate, and the second one as a 41% success rate. Recent results dominate.
How the algorithm recovers
If the suggested factor is below the last factor, the ease is adjusted down exactly to the suggested factor. This is mostly on Fails, but can happen on some successes too. This can be one of the more surprising aspects of the algorithm. And it's frustrating at first, but there is a reason for this behavior. Basically -- there are many cases where getting one review correct is not enough to make the ease factor go back up immediately, because one more correct review doesn't get you back to the target you were aiming for. Here's an example using the included simulator.py: Missing the 6th review lowered the ease from the cap of 5000 down to 3200. Getting the next answer right still was below target, so it lowered the ease factor again (but much less dramatically, really just tweaking it to 2981). The second right answer starts to really turn things back around though. This effect is amplified on earlier reviews, when you haven't demonstrated knowledge on the card. What this feels like in practice, is that after you miss a card, you sort of go into a "drill" mode for 1-3 reps. Ease factor declines and slowly turns around after you've "proved" to the algorithm you really know it. But once you turn the corner, it will rapidly climb back up to its previous peak, and generally faster than it did the first time. One other way you can see declining ease factors on successes is because of a tight leash and an early miss. Your "true" recommended factor is below the leash each time, so the algorithm keeps reducing the factor step by step to meet it. For example:
This ease factor declines for a long time until it turns the corner, but it's mainly because it's being limited by leash in how much it can swing down early in a card's life. Straight Reward I really like the Straight Reward approach. It's really straightforward and easy to understand, and there's a lot to be said for knowing exactly how your cards are responding to your feedback. It directly and simply solves the ease hell problem. Both Straight Reward and Auto Ease Factor make sure your ease factor grows when you know a card. The big difference is that Auto Ease Factor also takes over the "Again" button, and dynamically adjusts ease factor downward when you miss cards, based on your previous performance. In some cases it will be more aggressive in dropping the ease than hitting an "Again" button would be alone, but the algorithm also recovers to peaks faster. Hope this is helpful, let me know if you have any other questions! |
Beta Was this translation helpful? Give feedback.
-
Just in case you don't know, the term for these collisions in SRS is "negative interference". It's when the knowledge you have from 2 cards mess with each other and make one of both of them more difficult to learn as a result of you mixing the knowledge up. The inverse, when 2 cards have cross-over knowledge that makes one or both easier to learn, is called "positive interference". I remember reading about it on the supermemo wiki a while back, and I thought it was cool that there was a name for these 2 phenomena that I've been dealing with for years. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you very much, brownbat, that is extremely helpful! I understand much better now. Much of my confusion came from the numbers apparently being slightly off, which is explained by the weighting toward more recent reviews that you mentioned. I quite like that approach! Another bit of confusion came from the name "Suggested factor", which I had interpreted as what you later called the "true" recommended factor. I figured the occasional lowering of the factor even after a success might be because of a limit in step size toward that true recommended factor, and it was good to have that confirmed. I hadn't considered the additional source of that phenomenon, i.e. the simple fact that you might still be below the target and therefore see it lowered again. The last remnant of uncertainty I have is whether that really is another effect, or just another way the same effect shows itself. In your first example, is 3200 the true recommended factor after the sixth review, there not being a limit in step size (leash?) in this case? If so, it still surprises me a bit that it would go down again after a subsequent success. But the downward step from 5000 to 3200 being 1800, which is not only divisible by the given leash of 300 but is that leash multiplied by a factor that is the number of reviews up to that point suggests to me a limit in the step size might yet be involved, although without access to the specific equations, it's difficult to say; just an intuition, at this point. If I'm right, then I suppose both your examples are ultimately due to step size limits. Thanks again—I really appreciate you taking the time to answer my questions and explain the inner workings of this addon more fully. |
Beta Was this translation helpful? Give feedback.
-
Exactly right, great catch. Here are the results with the leash set arbitrarily high for comparison:
You can see the ease factors in white at the bottom dropping from 5k to 2637. Then the turnaround is basically immediate after you get the next positive review. (Sidenote, you'll see this pops back to 3083, compared to the leashed version's 2981. Since the leash wasn't slowing down the early reps' initial climb to 5000, the average prior ease factor is higher, so the adjustment off of that makes the rebound slightly steeper too.) One caveat, I've spent a lot of time looking over these small changes between reps, asking myself if something should really be 2900 or 3100. But more and more I'm convinced it's ok for these numbers along the way to be slightly off, as long as there's a path for the ease factor to gradually get more accurate over time. You'll probably find the majority of your cards quickly graduate to your maximum ease factor. The others will bounce around a bit, but once you learn them they'll head to the same schedule as your well known cards. I should mention, if you want to play around with different settings or add ons, but later revert all your ease factors back to sensible ranges, Reset Ease Automatically is a great tool for going back later and setting all your ease factors to a certain value (or even a range): https://ankiweb.net/shared/info/12081346 |
Beta Was this translation helpful? Give feedback.
-
|
The next update will have a clarified tooltip with "New factor" and also "Unleashed," for easier comparisons. |
Beta Was this translation helpful? Give feedback.
-
|
Thank you, I understand now. I also realized that in my last message, when I was speculating whether your first example wasn't just the same effect of leash limits that your second example illustrated, that I did miss something. Since the addon tries to shoot for a certain success rate, at any time you are below that rate, it will presumably seek to lower the ease. So it can indeed be that it remains below that success rate and continue on a downward trend, even if it was not previously bounded by the leash size on its way down, I think. Earlier, I thought that unleashed, a success would always up the factor, since I thought it would seek an intermediate factor between a lower success rate and a higher one (that is still below the target success rate). But instead it would just keep going down forever, right, so long as you remained below the success rate; it's not like it would ever be 'content' with some rate that is just below your target. It just might go down less quickly when the success rate is closer to the target, is all. So those really would be two different effects. I was going to check if I could tell the difference based on the unleashed and leashed simulations you showed, but 6/7 correct hits is just over the 85%, so I could not tell. I might check myself later, to make sure. Although of course I should be looking at the MAvg success rate, in which case some of those first successes might already count less heavily, and you could still land below 0.85 I guess... in which case I was right the first time, since it did go up with that first success after the failure. Oh well, I'll probably play with the simulator later to get a better idea of it, just for the fun of it. I agree, by the way, that the precise path by which the addon gets you to a particular ease is probably not as important as the overall trend of lowering ease when struggling and then raising it again once you get a card right. The comfort that you no longer need to worry about ease hell and that there is some algorithm working to improve the efficiency of the main Anki algorithm is, at least to me, the most important part. I suppose the specific numbers still might slightly improve the efficiency for the user, but overall I think there is some leeway, there. Thanks also for the tip on Reset Ease Automatically. I don't think I will need it anytime soon, but it's good to know about. In the next update, "New factor" I guess will be what is now given as "Suggested factor", and "Unleashed" will be what it would be if there were no leash limit? That sounds more intuitive, yes 👍 |
Beta Was this translation helpful? Give feedback.
-
Hi, this is perhaps not really an issue as such, but I am a bit confused about the feedback the addon gives the user while reviewing, and would be very interested to hear some clarification. Hopefully that's not too much of a hassle to provide.
In the image shown here, I understand the card ID, card queue, and rep list, but I don't know what the rest of it means exactly.
I would think the success rate would be the proportion that you got the card right, but looking at the rep list, that is never quite right.
Is the last factor the ease factor as it was when you were doing the review just then?
Is the MAvg factor the ease factor it has been on average during all reviews of it so far?
Is the suggested factor also the factor the card has now been set to, after the last review? Or is it the factor it would ideally like to get to, but might not get to right away because of the step size limit? And in the latter case, is the factor it is now set to not listed?
If the suggested factor is below the current factor, but you get the last review right, does the ease factor go up or down? (And vice versa.) And if it goes in the opposite direction to the suggested factor, am I right to assume the step would be smaller than if it were to adjust in the direction it wants to go in?
Thanks in advance! I really like the idea of the addon, but whether or not I ultimately stick to it above other options like Straight Reward will depend on the specifics of the calculations used. Regardless, I would really like to understand the information the addon is telling me after each review.
Beta Was this translation helpful? Give feedback.
All reactions