I have been thinking about a couple of new numbers that would be nice to have available from the "progress" page. I think we may have discussed these before, but I don't remember who originally suggested them.

The definition of "learned" is necessarily kind of arbitrary, so it would be very useful if we had a statistic which measured the retention rate for "learned" items (like the one provided by Anki's "Deck Statistics"). That is, if over some span of time I reviewed 100 items which I had already "learned", and I got 80 of them correct, my "learned retention rate" for that period would be 80%.

If it's expensive to compute for the entire period, this wouldn't necessarily have to be calculated over all items reviewed in the period, just over some reasonably large random sample; something like 100 items would give a margin of error of less than a percentage point.

Anki also measures the percent correct for what in our case would be "new items" and "non-learned items", i.e. items which we've never seen before and items which we've seen before but are not "learned". These numbers would mostly be useful for comparison with the "learned retention rate" so we can see what "learned" actually means. For example, if my "new items retention rate" is 10%, "non-learned items retention rate" is 30%, and "learned items retention rate" is 80%, I can be fairly confident that "learned" actually means something significant, and it means that I have a roughly 80% chance of getting a prompt right for a "learned" item.

James

I would kind of like to know my percentage is for words I haven't seen in a week or more. I feel like I've been missing a lot of long-term items lately.

I would be interested in that, too. We currently have a roundabout way of sanity checking retention rates for different intervals of items, but nothing we've built for automatic reports yet. I will think with Scott about what can be done.

I checked writing prompts for both of you guys and your long-term writing scheduling is scraping the bottom of our range, so it's probable that we need to lower the limits for you to hit those retention targets. Then again, an analysis done for us by gregshap seemed to suggest that for Skritter writing prompts, long-term retention tends to fall off very slowly over time, so that you could actually save a lot of review time by accepting lower retention rate for long-term words.

@nick, "I checked writing prompts for both of you guys and your long-term writing scheduling is scraping the bottom of our range..."

Is this your way of telling me I suck? :-)

@nick Awesome! By the way, if you'd like me to do any analysis for you, I do have a background in statistics and have studied some psychometric theory.

@jcdoss: I'm sure that, no matter how bad my Skritter statistics are, they're nowhere near as entertaining as my spoken Chinese. But, as a very wise person once told me, we should embrace negativity. A while ago, Nick noticed that one of my numbers (the delay interval for incorrect items) was at the bottom limit, so he adjusted that bottom limit and it's definitely made studying a lot easier.

Nope; I took a look at mine and mine are low, too. I think in general the long-term interval factor ranges are shifted a bit too high for writing prompts. Interestingly enough, they appear to be fine for definition prompts, maybe even a little low, for the three of us. And on reading prompts, they are fine for jcdoss, too high for jww1066, and too low for me. So there's a lot of individual variance.

James, we may contact with you some statistics questions in the future as there's certainly lots of it around at Skritter; thanks!

Not sure about adjusting the bottom limit on these guys, though--that would mean growing the interval by less than 40% each time, which is going to produce a large long-term review burden.