As some of you have already noticed, the LMI Puzzle Ratings have now been updated after the two recent puzzle tests - LMI Screen Test 1 and Puzzles & Chess.

A total of 223 players from 36 countries find a place in the ratings, including 43 new players. Players who have missed the three most recent puzzle tests have not been included in the rating list.

Apart from India, there are 10+ players from USA, Germany, Japan, UK and Poland on the rating list.

The top 3 are now separated by only 20 points, comfortably ahead of the next 3.

Another Observation....
In top 10 all players have played 5 or more tests out of 8 rated. This shows playing consistently will keep u high in the rating..........

The rating system is really cool, thanks for doing this. One issue though, on both 'change' columns, when you sort by that column, it doesn't quite sort correctly. For instance all numbers starting with 1, whether it's 1 digit, 2 digits, or 3 digits are all grouped together. (it sorta goes 1, 10, 100, 11, 110, 111)...

The sorting works the way it does because of the square brackets around the numbers.
Rakesh, any reason why the numbers have to be bracketed?

The square brackets are not necessary. They can be done away with, if that helps in sorting on the 'Change' column.

The LMI Sudoku Ratings are now updated after the December test. Overall, 258 players from 41 countries are included in the rating list now.

The LMI Puzzle Ratings have now been updated after Puzzle Jackpot.

A total of 236 players from 38 countries find a place in the ratings, including 11 re-entries and 19 new players. Players who have missed the three most recent puzzle tests have not been included in the rating list.

The Top 6 remain the same. motris continues to lead the ratings at 996. The highest gainer has been Alberto who gained 186 points. Amongst the Top 50, pixl has been the highest gainer, gaining 151 points. The highest ranked new player is Valezius at #99.

The LMI Sudoku Ratings have now been updated after the January Monthly Sudoku Test "Prime Exotica".
A total of 268 players are included in the current rating list, including 31 new players and 7 re-entries.

Thomas consolidated his #1 position with a top performance, and remains the only one with a four digit rating. Rishi lost 34 points but remained in 2nd position.

None of the Top 10 gained any rating points. WaterlooMathie was the only new name (re-entry) in the Top 20 at #16. David McNeill gained 17 rating points to move up to #15. Fred76, who tested Prime Exotica, gained 2 ranks as others around him lost a few rating points.

TiiT is the highest ranked new player at #71. Valezius gained 50 ranks to move to #70. In the Top 50, jonm, karin and Psyho were the major gainers.

The LMI Sudoku Ratings have now been updated after the February Monthly Sudoku Test "Double Delight". A total of 231 players are included in the current rating list.

The Top 3:
1. motris - 997
2. purifire - 913
3. jaku111 - 911

WaterlooMathie, the Double Delight winner, continued his rise with a gain of 60 points and 3 ranks. TiiT and bskbri were the two biggest gainers in the Top 100.

Top 3 from India:
1. purifire - 913
2. Rohan rao - 860
3. gauravkorde - 676

Overall, 37 countries are represented in the ratings. A distribution of the countries with the maximum rated participants is shown below:

I notice I'm missing from the rankings now. (I think I've done only one test since November.) How do you determine whether someone is 'active' or not? How long before you get removed?

If you miss three consecutive tests then your name is temporarily dropped from the ratings. You will again enter in the list as soon as you play another test.
Sudoku tests / ratings and Puzzle tests /ratings are totally different entities....and are not mixed together.

Further if you have missed two consecutive tests then you will be shown as N... which is an indication that your name will be dropped in the next list if you dont play the next test.

Hope to see you back in the next ratings update. All your previous scores WILL be considered while calculating the new ratings.

Awesome, thanks. Maybe now I'll have the motivation to do better!

Updated LMI Puzzle Ratings after Puzzle Zoo (February 2011 LMI puzzle test) are now available. Overall, 186 players are included in the ratings, including 14 new players and 8 re-entries.

Not much of topsy-turvy.......except a few changes in Top 10. Psyho climbs up to #7, gaining 54 rating points after Puzzle Zoo. Willwc enters the Top 10, gaining 14 rating points. Uvo sitting just behind deu and battling hard to reach the second spot.

Rob was the biggest gainer from Puzzle Zoo, gaining 221 points and 113 ranks. Amongst the Top 50, Valezius gained 181 points, and Nyuta gained 95 points. Notable re-entries to the rating list include #19 (Tarotaro) and #40 (ByronosaurusRex).

Updated LMI Puzzle Ratings after Puzzle Hybrids (LMI March 2011 Puzzle Test) are now available.

motris (992), deu (984) and uvo (955) occupy the top 3 spots.

The following table shows the highest ranked puzzlers from the countries that are represented in the Top 50:
(The countries are in alphabetical order)

Country	Puzzler	Rank	Rating
Austria	euklid	23	647
Canada	ByronosaurusRex	39	558
Czech Republic	Janka1	13	721
France	godisdead	24	640
Germany	uvo	3	955
Hungary	Valezius	20	674
India	Rohan Rao	34	587
Italy	forcolin	28	617
Japan	deu	2	984
Poland	Psyho	7	808
Romania	rubben	32	591
Serbia	nikola	4	888
Slovakia	pista	16	699
The Netherlands	Para	27	621
Turkey	yureklis	21	673
UK	drsteve	29	601
USA	motris	1	992

Updated LMI Sudoku Ratings after Spring Sudoku Test (LMI March 2011 Sudoku Test) are now available.

motris (998), nikola (892) and jaku111 (881) occupy the top 3 spots.

Sad that India is nowhere near the top.

But we have a better scene in Sudokus....with two persons in top 10.........

I meant to ask this question after Twist, but it certainly comes up again after the April Sudoku test too. For the purposes of the Puzzle Ratings and Sudoku Ratings, how do you account for the different scoring systems in play in different tests. Do you use strict score only, or time, or absolute rank, or some combination of these? Having a fair way to compare very different tests seems really important to keep this world leader board accurate, and I think many of us would like to know more about how the system works. As Nikola mentioned after Twist, it might be time to stop experimenting so much with scoring and start to use a consistent system that will apply to all tests, to keep the Ratings system as fair and even as possible.

Edited by motris 2011-04-25 9:02 PM

At the outset, we would like to thank you for asking this question. Within ourselves, we have been discussing exactly these same things recently. To answer your question, the current system uses strict score only. As long as the score is a function of time, we did not feel the need to include time as a separate factor. This was the case in most tests, where anyone solving the complete (or almost complete) test correctly would get some more points using a factor. But the April sudoku test did not use time as a factor for scoring - it did give 5 points extra but difference between 91 and 101 was considered the same as difference between 116 and 117, for example. We have discussed using rank also but ultimately we had decided against it.

We cannot agree more. Thats our objective too - to keep the rating system accurate and acceptable.

There can be so many different scoring systems. And, unless we try them in a test, we do not know the efficacy of the scoring system. So, I do not necessarily agree that we should stick to one scoring system (it could become boring, in a way). Lets have the flexibility and freedom of having different systems at play in different tests. I think that the rating system should adapt to the scoring system(s)...and not the other way. Neither can the rating system remain static - it also needs to be reviewed from time to time for continuous improvement.

Currently, we are midway through the process of coming up with suitable improvements to the existing rating mechanism. Once finalized, the "how" part will be shared on the site as well.

At the same time, we would like everyone to share any ideas/suggestions/criticisms - small or big - from the ratings point of view on this forum. And, we will address each one of those while finalizing the system.

As Rakesh mentioned, feel free to post ideas / suggestions on ratings, especially on
How to normalize scores in a test?
How to compare scores across tests?
How to deal with 0 scores / no submissions?
How to deal with players joining late or players not playing frequently?
etc...

Should anyone need sample data, please let us know.

Thanks for your reply. I'm sure I'm not the only one who will be interested to learn the methodology behind the system as it is a real "world leader board" these days.

My personal recommendation is to not necessarily lock in scoring systems (what puzzles are worth) as these can be varied by authors on different tests to affect strategy, but after the base score of a test is calculated, the only way for the rankings to use score fairly for complete finishers is to either include time in your formula, or always use N points per minute where N is close to "max value of test/time". I like creative scoring ideas, like in Puzzle Jackpot, or special bonuses as in Evergreens or in my 20+10 Decathlon for completing sets of puzzles, but we should realize scoring of puzzles is a different component of the test than correctly using the time of finishers to measure relative performance.

So on April Sudoku, I would certainly have just used ~5 points per minute (600/120). On Twist I would have used the same staggered "value of minutes" for the bonus, instead of going down to a nominal .5 points per minute. So I would have earned 1 minute at 100%, 10 minutes at 75%, 10 minutes at 60%, and 10 minutes at 50%. In each case, simply making the time worth the value of those minutes for other solvers allows the relative performance of all finishers to be correctly staggered for your ratings, for UKPA rankings, and so on. Unlike a one-time WPC where it doesn't matter as much, LMI has really become one of the big forums for monthly competition and so consistency is absolutely required in the ranking system.

There are some interesting questions here.

I've thought about the 0/no submission score a bit. I think one option is to put a choice on the test upfront, something like "I want to play the test for the official ranking" or "I do not want to play the test for the official ranking". Either gives you a password and locks in a start time for your account. If you just want to see the test but not record a score, then you can now do so. But if you have solvers who are instead starting the test, and only putting in answers if they feel they did well, you can now separate those from each other. There will also always be the potential for technical problems, so I'm guessing that the rating probably does something like drop/minimize the value of the lowest recent test which can account for one time technical problems that led to a 0 result or particularly low score.

I've also always been surprised that the tests have a fixed time for last submission, as opposed to a fixed time for last start. I suppose advertising a 48-hour window is the point, but if that's the case I'd simply always run the tests for 50 hours. Changing to the latter format would eliminate the situation (that is always frustrating for the solver) of realizing they started too late and have a low score. While this won't be common among the frequent players, for people taking one of their first tests you do not want to cause a negative experience or they may not want to come back. A check on test start times is a huge improvement in my opinion.

Edited by motris 2011-04-26 10:03 PM