I strongly disagree with this perception. No one is "lucky" to not have taken a test. I expect uvo may easily have claimed a high spot as he had on the first Nikoli Selection by being 4th.

So, was that test (Nikoli Selection) really an outlier and if so why? Consider these numbers. This is the prorated score (PS) for 2nd, 3rd, 5th, 7th, and 10th on the last 6 puzzle contests in no particular order.

2/3/5/7/10
A - 890, 882, 866, 780, 751
B - 951, 924, 883, 771, 717
C - 911, 795, 749, 735, 716
D - 905, 884, 761, 735, 709
E - 919, 905, 790, 771, 710
F - 909, 906, 884, 813, 742

Does the Nikoli Selection stand out on this list? Yes, if you look exactly at 3rd place. But not as much at 5th and certainly not at 7th or 10th place. There is a range, but nothing that strikes me as really large differences in PS that are outside general performance variation. 4 tests are around 710 at 10th place. Two are higher at 750. The outliers are probably A and F for being so flat.

Why does the Nikoli Selection have a large scoring problem for 3rd - 7th that resolves afterwards? I've proffered my rationales before but I'll do so here again. The best measure of a group of solvers outside of exact time to finish is their points earned/minute rate. No matter what the test length (2 minutes, 2 hours, 2 days, 2 weeks, ...) if you are measuring this rate you will eventually get an accurate measure of their ability removing the puzzle to puzzle variance. When a person "finishes" a test, you have an absolute measure of that solver's points earned/minute rate and hopefully the extra time is rewarded close to this number.

The problem with Nikoli Selection was two-fold:
First, for the top 10 solvers, but not for the larger field, the test had two different parts with very different points/minute rates. The non-traditional "marathon" bonus system was not very granular at the top, and a lot of solvers got much less value for finishing early unless they were the right increment of fifteen-twenty minutes away from total test time for a big bonus jump. Further, it was even more valuable to solve 2 or 3 of these than just 1. Even so, the point/minute rates were grossly compressed for H. Jo and myself. In the first ~45 minutes, he earned 280 points for finishing the main test. In the last 45 minutes, he earned 170 + 42 time bonus points for finishing three marathons, about 75% the original rate on the test everyone else was being compared on. My second half was 105+46 or about 50% of my original rate. I'd suppose those who got only 1 (or 0) marathons out were much lower even than this.

Second, the "main test" was being scored with a time bonus that would, if it were for the whole test, have worked ok except for being (1) slightly undervalued and (2) lost by basically everyone from 5th to 9th (who would have normally filled 3rd to 6th) for making a typo in a main puzzle answer key. Answer keys they did not check as well because there was something else to move onto unlike every other test. So a 20 point puzzle error + 30 time bonus point loss was a more significant penalty (11% max score value) than it usually is. But the performance gap falls aways from 10th and on and the test resembles others in its curve because for everyone else, there was no point/minute drop-off of transitioning from sprint to marathon territory because there was no marathon territory to deal with. The problem is not H.Jo (or on other tests me or uvo or psyho or whomever). The problem this time was the test/scoring structure that depressed the points earned for the top 10 universally and most pronounced for 3rd through 10th compared to what the usual test would look like. In other words: a compressed score range (higher relative median score), but less realized bonus for 3rd to 10th place, equals the scoring crunch which is coming from both the 1000 mark and the 500 mark.

Edited by motris 2011-08-02 5:31 AM

Updated LMI Puzzle Ratings after Magic Cube (July 2011 LMI puzzle test #2) are now available.

The Top Five: 1. motris, 2. deu, 3. uvo, 4. MellowMelon, 5. Nikola

India Top Five: 1. Rohan, 2. Rakesh, 3. Rajesh, 4. Amit, 5. Jaipal

For the goals of keeping a consistent rating, can I make a request that test authors keep control of the base scoring of tests (and twists like Puzzle Jackpot and Twist and such are ok as this is the base value), but that LMI admins set the value of time bonus (which is really just serving a role of standardizing scores from month to month) with a fixed standard for all tests? When set by authors, we get such a wide wide range of these values which are really problematic for comparing tests. The upcoming sudoku test has a 3x value for time bonus compared to puzzle minutes, so finishers will benefit excessively. Past tests have had closer to 0x value for time bonus compared to puzzle minutes, so finishers are punished excessively. Is total value/total time that hard a ratio to fix? Time bonus will never affect the ranks in a given test (which is why 1000 points a minute or .0001 points a minute could be chosen by test authors if they wanted) but it does affect the yearly rankings which is why it should not be a variable in the constructor's mind at all.

Very valid argument, we will do that.

For Sudoku City, Nikola had asked me to review the bonus system, but I had missed it completely. I'll get it rectified. Thanks for bringing it up. And it indeed helps when the anomalies are brought up before the test itself, so that we can get it fixed as needed.

Very nicely put.

Just one question: when you say 70-85% in rank, do you mean those ranked 71st-85th (out of 100) or those ranked 15th-30th? And, I would also like to know the degree (slightly/moderately/heavily) to which they are affected, so that we can think of further improvements at some point of time.

By 70-85% in rank I meant those with a rank score around 700 to 850. So 15th to 30th place in your out of 100 example.

Updated LMI Sudoku Ratings after Sudoku City (August 2011 LMI sudoku test) are now available.

The Top five: 1. motris, 2. nikola, 3. purifire, 4. deu, 5. misko

India Top five: 1. Rishi, 2. Rohan, 3. Rakesh, 4. Gaurav, 5. Sumit

Updated LMI Puzzle Ratings after Japanese Puzzle Land (August 2011 LMI puzzle test) are now available.

xevs is the only new name in the Top 10.

Rohan Rao at #43 is the top ranked puzzler from India.

How to know my LMI rating?

Didn't you register at LMI few minutes back?

Updated LMI Puzzle Ratings after Sprint Test (September 2011 LMI puzzle test) are now available.

MellowMelon sprints to the third place.

Updated LMI Sudoku Ratings after Crazy Arrows (September 2011 LMI sudoku test) are now available.

532 players representing 50 countries are now a part of the ratings. motris and nikola remain #1 and #2 repectively, and deu moved up to #3.

Amongst the heavy gainers, ppeetteerr gained 203 rating points to move to #40, and tarotaro gained 164 rating points to move up to #76.

USA has 6 players in the Top 50, while Japan and India have 5 each. Germany has 4 players in the Top 50.

Updated LMI Sudoku Ratings after A or B (October 2011 LMI sudoku test) are now available.

motris inched closer to the 1000 mark, gaining one point to reach 993. deu and Nikola exchanged places and are within a point of each other now. jaku111 and uvo, both made major gains to enter the Top 10. Four of the top 10 have palindromic ratings (which read the same whether read from left or right).

TiiT (palindrome again) who finished 2nd in A or B, gained 36 rating points to move to #15. Gotroch has a rating of 888 (which is a flush in poker). Hunsudoku, tarotaro, SCORPPROCS (another palindrome), tilansia and spelvin all gained over 100 points and should be rapidly moving towards the top in the coming months.

Among players from India, Rohan moved to within 2 points of 900 and is ranked 8th now. Prasanna gained 27 points.

Ratings are updated after Double Decathlon - Link
I would wait till Rakesh double checks once and make it official.

Ratings updated after ST#2

No changes in top 5, though all top 5 have 900+ ratings.

Sudoku
1. 9 out of the top 10 of WSC are active on LMI.
2. The top 6 players of LMI Sudoku Ratings appeared in the WSC 2011 Playoffs.
3. 8 of the top 10 in WSC 2011 are in the top 15 of LMI Sudoku Ratings.

Puzzle
1. All 10 players of WPC playoffs are active on LMI.
2. The top 4 players of LMI Puzzle Ratings appeared in the WPC 2011 Playoffs.
3. 9 of the top 10 in WPC 2011 are in the top 20 of LMI Puzzle Ratings.

Not bad!

ditto!

That gives a real good feeling.............

Ratings update after Puzzle Fusion available here

We'll spend some time checking it manually before making this as default ratings.

The most significant change at the top is that MellowMelon moves to #2 position ahead of deu. Also, xevs gained 19 points to have ratings of 919.

Sudoku ratings updated after December Test.

motris continues his dominance and retains the top spot throughout the year.

Edited by debmohanty 2011-12-22 12:58 PM

LMI Puzzle ratings are updated now

No changes in top 3, but MellowMelon inches toward motris.
xevs gains lot of ratings and takes the 4th place now

We have a technical problem, and it will take a while to update the ratings this time (and may be in future as well)

Puzzle Ratings are updated after Marathon. No changes in top 5.

Also added Ratings History in profile page.

Sudoku Ratings updated after Diagonal Vision

TiiT moves to top5 after impressive performances.

Among players from India, Prasanna moves into top50 after few good tests.