|
|
Administrator
   
Country : India | Now that we have 5 Sudoku Monthly tests over, we'll soon be publishing a Rating System which will provide a Rating for each player. It will consider several parameters, including
player's performance in each testnumber of tests player has participated or missedplayer's recent performances
A similar Ranking for puzzles will be published after we complete 4/5 puzzle tests. |
|
neerajmehrotra
Posts: 328
Country : India | Thats a grt news.............
We eagerly look forward for the same. |
|
Administrator
Country : India | Little delayed - but better than never.
Here it is LMI Sudoku Ratings
Note that the rating system is based on last 5 monthly Sudoku Tests. We'll re-look at the system around beginning of 2011, when we would have more number of tests concluded. |
|
Administrator
Country : India | Many thanks to Neeraj, Rakesh and Rohan (in alphabetical order :- )
They had spent lot of time discussing about the whole system, many arguments and counter arguments.
 |
|
neerajmehrotra
Posts: 328
Country : India | Administrator - 2010-09-15 1:27 PM
Little delayed - but better than never.
Here it is LMI Sudoku Ratings
Note that the rating system is based on last 5 monthly Sudoku Tests. We'll re-look at the system around beginning of 2011, when we would have more number of tests concluded.
We should have a direct link to the ratings on the home page.......... |
|
vopani
Posts: 739
Country : India | vopani posted @ 2010-09-15 10:38 PM The list is at the extreme left. Its looking a little out-of-place. |
|
Administrator
Country : India | Rohan Rao - 2010-09-15 10:38 PM
The list is at the extreme left. Its looking a little out-of-place. Yes. Does not seem to be aligned properly in IE. |
|
vopani
Posts: 739
Country : India | vopani posted @ 2010-09-16 6:42 PM Also, for authors/testers, I think NumTests should include the tests they authored/tested.
For calculations, it should be ignored.
If its too cumbersome, scrap. |
|
Administrator
Country : India | Rohan Rao - 2010-09-15 10:38 PM
The list is at the extreme left. Its looking a little out-of-place. Fixed. Please check. |
|
vopani
Posts: 739
Country : India | vopani posted @ 2010-09-26 5:43 PM Its fine now. |
|
rakesh_rai
Posts: 774
Country : India | The LMI Sudoku Ratings have now been updated after the very interesting Sudoku Hot Pot. A direct link to the ratings is also included on the main forum page through an icon at the top (next to LMI polls ). A total of 206 players from 32 countries are represented in the current ratings. The top 3 overall are Thomas Snyder, Jason V Zuffranieri and Chen Cen. The top 3 Indians are Rishi Puri, Rohan Rao and Gaurav Korde. Do share your comments on the ratings here. |
|
DreamRose311
Posts: 24
Country : United States | Now that there have been 5 puzzle tests, can we expect a similar LMI Puzzle Ratings soon? |
|
vopani
Posts: 739
Country : India | vopani posted @ 2010-10-21 1:40 PM DreamRose311 - 2010-10-21 1:41 AM
Now that there have been 5 puzzle tests, can we expect a similar LMI Puzzle Ratings soon?
Yes, LMI Puzzle Ratings will be out soon... |
|
neerajmehrotra
Posts: 328
Country : India | LMI Sudoku Ratings have now been updated after the Renban Group Sudokus.... A direct link to the ratings is also included on the main forum page through an icon at the top (next to Authors ). A total of 237 players (an increase of 31 players ) from 38 countries (an increase of 6 countries ) are represented in the current ratings. Thomas Snyder, two time world champion has comfortably retained his number one spot (despite missing the Renban Groups )and Nikola has jumped to the second spot pushing Jason V Zuffranieri down to the third spot by just a single point margin.....really a tough fight. Chen Cen needs to put some more effort and wait till the next game to come back in the top three.......... Coming to the Indian participants.........No change in the top two Indian positions but Rohan slipped to overall 10th slot from his previous 9th while Rishi improved his rating and reached to the 7th from 8th position. Gaurav Korde has been dropped from the ratings as he consecutively missed the third test....and thus Jaipal joins Rishi and Rohan in the top three Indian positions. Both Rishi and Rohan need to work hard for coming in the elite top 5.... Do share your comments on the ratings here. |
|
akash.doulani
Posts: 157
Country : India | just wanted to know how are the rating points calculated? |
|
neerajmehrotra
Posts: 328
Country : India | akash.doulani - 2010-11-18 9:16 PM
just wanted to know how are the rating points calculated?
The rating is calculated based on a scientific algorithm (and very difficult to explain here ), which uses multiple factors (e.g. scores, consistency, # of tests ). The maximum possible rating is 1000 and the minimum is 0.
Best way to come up in the ratings is to keep on playing regularly (dont miss the tests ) and try and do well........
|
|
Gotroch
Posts: 83
Country : Czech Republic | Gotroch posted @ 2010-11-19 12:43 AM It would be nice if you add points change (for ex. +16) in comparison with score from previous leaderboard.
And similar statistic for player position change :)
Edited by Gotroch 2010-11-19 12:46 AM
|
|
Fred76
Posts: 337
Country : Switzerland | Fred76 posted @ 2010-11-19 12:48 AM Just a proposition for the rating:
It would be interesting to indicate (perhaps with arrows next to the ranking ) if the rank of a player is better, worse or equal compared to last ranking (ranking before last test ). 
Fred |
|
Fred76
Posts: 337
Country : Switzerland | Fred76 posted @ 2010-11-19 12:50 AM Gotroch - 2010-11-19 12:43 AM
It would be nice if you add points change (for ex. +16) in comparison with score from previous leaderboard.
And similar statistic for player position change :)
This time, you were faster than me  |
|
Para
Posts: 315
Country : The Netherlands | Para posted @ 2010-11-19 1:05 AM Wasn't exactly aware there was a rating system. Guess doing a 100 min. test in 50 minutes isn't exactly good for that. At least I should be able to go up with the next test? |
|
rakesh_rai
Posts: 774
Country : India | neerajmehrotra - 2010-11-18 9:39 PM akash.doulani - 2010-11-18 9:16 PMjust wanted to know how are the rating points calculated? The rating is calculated based on a scientific algorithm (and very difficult to explain here ), which uses multiple factors (e.g. scores, consistency, # of tests ). The maximum possible rating is 1000 and the minimum is 0.Best way to come up in the ratings is to keep on playing regularly (dont miss the tests ) and try and do well........ Akash: As Neeraj metioned, there is a currently an algorithm in place for ratings. And, we'll re-look at it periodically, to improve it further. The next review should happen around early 2011. So, if you do have any points/concerns, please share them here, so that we can discuss them while reviewing, and include any valid ones. And, Congratulations for improving your rating by 160 points !!! |
|
rakesh_rai
Posts: 774
Country : India | Gotroch - 2010-11-19 12:43 AM
It would be nice if you add points change (for ex. +16) in comparison with score from previous leaderboard.And similar statistic for player position change :) Fred76 - 2010-11-19 12:48 AM Just a proposition for the rating: It would be interesting to indicate (perhaps with arrows next to the ranking ) if the rank of a player is better, worse or equal compared to last ranking (ranking before last test ). Both are valid points. This is very much a part of the plans, and you'll see this happen sooner than later. |
|
rakesh_rai
Posts: 774
Country : India | Para - 2010-11-19 1:05 AM
Wasn't exactly aware there was a rating system. Guess doing a 100 min. test in 50 minutes isn't exactly good for that. Fortunately, the next puzzle test (FLIP ) is for 50 mins duration only : ) At least I should be able to go up with the next test? I think your LMI sudoku rating should defintely improve with a reasonable performance in the next LMI sudoku test. And, the LMI puzzle ratings will also be coming up soon. |
|
debmohanty
Country : India | Para - 2010-11-19 1:05 AM
Wasn't exactly aware there was a rating system. Guess doing a 100 min. test in 50 minutes isn't exactly good for that. At least I should be able to go up with the next test?
50 minutes for a test is an exception.
Most Sudoku tests will be between 100 to 120 minutes (ideally 120 minutes )
Most puzzles tests will be between 80 to 120 minutes (ideally 100+ minutes )
|
|
akash.doulani
Posts: 157
Country : India | it was a pleasant surprise to see my ratings increase by 160 points. wasnt actually expecting it to improve as much as it did. i guess 360 points is not bad for someone with just two monthly tests under his belt. |
|
vopani
Posts: 739
Country : India | vopani posted @ 2010-11-19 5:58 PM Fred76 - 2010-11-19 12:48 AM
Just a proposition for the rating:
It would be interesting to indicate (perhaps with arrows next to the ranking ) if the rank of a player is better, worse or equal compared to last ranking (ranking before last test ).
Fred
I like the idea of an arrow. Maybe a green arrow pointing up and a red arrow pointing down. Lets see what we can do.
Thanks for the idea.
And Thanks to Gotroch too, for initiating the concept :- ) |
|
neerajmehrotra
Posts: 328
Country : India | akash.doulani - 2010-11-19 4:56 PM
it was a pleasant surprise to see my ratings increase by 160 points. wasnt actually expecting it to improve as much as it did. i guess 360 points is not bad for someone with just two monthly tests under his belt.
Yeah Akash, This shows how important is to play consistently in all the tests...............
Congrats for such a gud performance. |
|
rakesh_rai
Posts: 774
Country : India | DreamRose311 - 2010-10-21 1:41 AMNow that there have been 5 puzzle tests, can we expect a similar LMI Puzzle Ratings soon? Finally, the inaugural LMI Puzzle Ratings are up. You can also reach the ratings from the home page, by clicking on Puzzle under Ratings. A total of 359 players from 40 countries are included in the inaugural LMI Puzzle rating list. Apart from India, the top five countries in terms of number of participating players are USA, Germany, Japan, Poland and UK. The LMI puzzle ratings will be updated next in mid-December 2010. Its a neck and neck race for the No 1 spot between Thomas Snyder (motris ) and Hideaki Jo (deu ). Ulrich Voigt (uvo ) is currently ranked third. The top 10 is dominated by the Germans and the Japanese, occupying 8 of the 10 places. Amit Sowani, Rohan Rao and Rajesh Kumar are the top 3 Indians in the list, and the only ones in the Top 50. |
|
rakesh_rai
Posts: 774
Country : India | rakesh_rai - 2010-11-19 9:19 AM Gotroch - 2010-11-19 12:43 AM
It would be nice if you add points change (for ex. +16) in comparison with score from previous leaderboard.And similar statistic for player position change :) Fred76 - 2010-11-19 12:48 AM Just a proposition for the rating: It would be interesting to indicate (perhaps with arrows next to the ranking ) if the rank of a player is better, worse or equal compared to last ranking (ranking before last test ). Both are valid points. This is very much a part of the plans, and you'll see this happen sooner than later. You can check out the LMI Sudoku Ratings page now. Arrows are now included to indicate the change in rank (from previous rating list ). The exact changes in the ranks and ratings are explicitly indicated as well. |
|
debmohanty
Country : India | rakesh_rai - 2010-11-26 2:37 PM
You can check out the LMI Sudoku Ratings page now. Arrows are now included to indicate the change in rank (from previous rating list ). The exact changes in the ranks and ratings are explicitly indicated as well.
Looks good. May be an explanation of what the arrows indicate?
It looks like a jump of 10+ ranks is indicated by Up arrow. |
|
rakesh_rai
Posts: 774
Country : India | debmohanty - 2010-11-26 2:47 PM
May be an explanation of what the arrows indicate? It looks like a jump of 10+ ranks is indicated by Up arrow. You are right. The arrows are largely self explanatory. As of now, we have used five arrows in three colours to indicate the changes in ranks, as below:  - The green arrow indicates that the player is on the up, and has made major gains (10+ ) in the ranks as compared to last ranking list.  - This arrow indicates that the player has made minor gains in ranks (1-9 ).  - The right blue arrows indicate that there is no change in the rank.  - This arrow indiates that the player has dropped 1-9 ranks.  - The red arrow indicates that the player has dropped 10+ ranks. |
|
Fred76
Posts: 337
Country : Switzerland | Fred76 posted @ 2010-11-26 3:18 PM rakesh_rai - 2010-11-26 2:37 PM
You can check out the LMI Sudoku Ratings page now. Arrows are now included to indicate the change in rank (from previous rating list ). The exact changes in the ranks and ratings are explicitly indicated as well.
It's great ! |
|
purifire
Posts: 460
Country : India | rakesh_rai - 2010-11-26 3:10 PM
debmohanty - 2010-11-26 2:47 PM
May be an explanation of what the arrows indicate? It looks like a jump of 10+ ranks is indicated by Up arrow. You are right. The arrows are largely self explanatory. As of now, we have used five arrows in three colours to indicate the changes in ranks, as below: - The green arrow indicates that the player is on the up, and has made major gains (10+ ) in the ranks as compared to last ranking list. - This arrow indicates that the player has made minor gains in ranks (1-9 ). - The right blue arrows indicate that there is no change in the rank. - This arrow indiates that the player has dropped 1-9 ranks. - The red arrow indicates that the player has dropped 10+ ranks.
This is cool Rakesh!  |
|
debmohanty
Country : India | rakesh_rai - 2010-11-26 3:10 PM
You are right. The arrows are largely self explanatory. As of now, we have used five arrows in three colours to indicate the changes in ranks, as below: - The green arrow indicates that the player is on the up, and has made major gains (10+ ) in the ranks as compared to last ranking list. - This arrow indicates that the player has made minor gains in ranks (1-9 ). - The right blue arrows indicate that there is no change in the rank. - This arrow indiates that the player has dropped 1-9 ranks. - The red arrow indicates that the player has dropped 10+ ranks.
Thanks for explaining.
So there is no 'major' jump (in either direction ) in top20.
But I would think that cpickerel and willwc cases are significant jumps.
The point i'm trying to make is a jump of 5 ranks in top20 or jump of 10 ranks in top50 or a jump of 20 ranks in top100 should be considered as major. |
|
purifire
Posts: 460
Country : India | debmohanty - 2010-11-26 4:03 PM
Thanks for explaining.
So there is no 'major' jump (in either direction) in top20.
But I would think that cpickerel and willwc cases are significant jumps.
The point i'm trying to make is a jump of 5 ranks in top20 or jump of 10 ranks in top50 or a jump of 20 ranks in top100 should be considered as major.
Jump or Drop you mean : )  |
|
rakesh_rai
Posts: 774
Country : India | debmohanty - 2010-11-26 4:03 PM
So there is no 'major' jump (in either direction) in top20. Blue is my favourite colour, so I wanted more of blue  . Actually, with the current thumb-rule, it is less likely that we'll see a green or red arrow in Top 20. But I would think that cpickerel and willwc cases are significant jumps.The point i'm trying to make is a jump of 5 ranks in top20 or jump of 10 ranks in top50 or a jump of 20 ranks in top100 should be considered as major. I think you meant a drop for cpickerel. Thanks Rishi. But you have got a very valid point here. Since you want to see a more colourful rating list, we can definitely think of a context based definition of 'major'. Lets have more views on this and we can refine this further. |
|
debmohanty
Country : India | purifire - 2010-11-26 4:05 PM
Jump or Drop you mean : ) rakesh_rai - 2010-11-26 4:28 PM
I think you meant a drop for cpickerel. Thanks Rishi.
Yes - I used jump for both the directions. drop is the correct word. |
|
purifire
Posts: 460
Country : India | debmohanty - 2010-11-26 4:32 PM
purifire - 2010-11-26 4:05 PM
Jump or Drop you mean : ) rakesh_rai - 2010-11-26 4:28 PM
I think you meant a drop for cpickerel. Thanks Rishi.
Yes - I used jump for both the directions. drop is the correct word.
Change would be the best word since drop only signifies a downword trend :D |
|
rakesh_rai
Posts: 774
Country : India | As some of you have already noticed, the LMI Puzzle Ratings have now been updated after the two recent puzzle tests - LMI Screen Test 1 and Puzzles & Chess. A total of 223 players from 36 countries find a place in the ratings, including 43 new players. Players who have missed the three most recent puzzle tests have not been included in the rating list. Apart from India, there are 10+ players from USA, Germany, Japan, UK and Poland on the rating list. The top 3 are now separated by only 20 points, comfortably ahead of the next 3. |
|
neerajmehrotra
Posts: 328
Country : India | Another Observation....
In top 10 all players have played 5 or more tests out of 8 rated. This shows playing consistently will keep u high in the rating.......... |
|
DreamRose311
Posts: 24
Country : United States | The rating system is really cool, thanks for doing this. One issue though, on both 'change' columns, when you sort by that column, it doesn't quite sort correctly. For instance all numbers starting with 1, whether it's 1 digit, 2 digits, or 3 digits are all grouped together. (it sorta goes 1, 10, 100, 11, 110, 111)... |
|
debmohanty
Country : India | DreamRose311 - 2010-12-17 8:56 PM
The rating system is really cool, thanks for doing this. One issue though, on both 'change' columns, when you sort by that column, it doesn't quite sort correctly. For instance all numbers starting with 1, whether it's 1 digit, 2 digits, or 3 digits are all grouped together. (it sorta goes 1, 10, 100, 11, 110, 111)...
The sorting works the way it does because of the square brackets around the numbers.
Rakesh, any reason why the numbers have to be bracketed? |
|
rakesh_rai
Posts: 774
Country : India | debmohanty - 2010-12-28 12:37 PM DreamRose311 - 2010-12-17 8:56 PMThe rating system is really cool, thanks for doing this. One issue though, on both 'change' columns, when you sort by that column, it doesn't quite sort correctly. For instance all numbers starting with 1, whether it's 1 digit, 2 digits, or 3 digits are all grouped together. (it sorta goes 1, 10, 100, 11, 110, 111)... The sorting works the way it does because of the square brackets around the numbers.Rakesh, any reason why the numbers have to be bracketed? The square brackets are not necessary. They can be done away with, if that helps in sorting on the 'Change' column. |
|
rakesh_rai
Posts: 774
Country : India | The LMI Sudoku Ratings are now updated after the December test. Overall, 258 players from 41 countries are included in the rating list now. |
|
rakesh_rai
Posts: 774
Country : India | The LMI Puzzle Ratings have now been updated after Puzzle Jackpot. A total of 236 players from 38 countries find a place in the ratings, including 11 re-entries and 19 new players. Players who have missed the three most recent puzzle tests have not been included in the rating list. The Top 6 remain the same. motris continues to lead the ratings at 996. The highest gainer has been Alberto who gained 186 points. Amongst the Top 50, pixl has been the highest gainer, gaining 151 points. The highest ranked new player is Valezius at #99. |
|
neerajmehrotra
Posts: 328
Country : India | The LMI Sudoku Ratings have now been updated after the January Monthly Sudoku Test "Prime Exotica".
A total of 268 players are included in the current rating list, including 31 new players and 7 re-entries.
Thomas consolidated his #1 position with a top performance, and remains the only one with a four digit rating. Rishi lost 34 points but remained in 2nd position.
None of the Top 10 gained any rating points. WaterlooMathie was the only new name (re-entry ) in the Top 20 at #16. David McNeill gained 17 rating points to move up to #15. Fred76, who tested Prime Exotica, gained 2 ranks as others around him lost a few rating points.
TiiT is the highest ranked new player at #71. Valezius gained 50 ranks to move to #70. In the Top 50, jonm, karin and Psyho were the major gainers. |
|
rakesh_rai
Posts: 774
Country : India | The LMI Sudoku Ratings have now been updated after the February Monthly Sudoku Test "Double Delight". A total of 231 players are included in the current rating list. The Top 3: 1. motris - 997 2. purifire - 913 3. jaku111 - 911 WaterlooMathie, the Double Delight winner, continued his rise with a gain of 60 points and 3 ranks. TiiT and bskbri were the two biggest gainers in the Top 100. Top 3 from India: 1. purifire - 913 2. Rohan rao - 860 3. gauravkorde - 676 Overall, 37 countries are represented in the ratings. A distribution of the countries with the maximum rated participants is shown below:
 |
|
figonometry
Posts: 30
Country : Canada | I notice I'm missing from the rankings now. (I think I've done only one test since November.) How do you determine whether someone is 'active' or not? How long before you get removed? |
|
neerajmehrotra
Posts: 328
Country : India | figonometry - 2011-02-19 5:43 AM
I notice I'm missing from the rankings now. (I think I've done only one test since November.) How do you determine whether someone is 'active' or not? How long before you get removed?
If you miss three consecutive tests then your name is temporarily dropped from the ratings. You will again enter in the list as soon as you play another test.
Sudoku tests / ratings and Puzzle tests /ratings are totally different entities....and are not mixed together. |
|
neerajmehrotra
Posts: 328
Country : India | neerajmehrotra - 2011-02-19 9:22 PM
figonometry - 2011-02-19 5:43 AM
I notice I'm missing from the rankings now. (I think I've done only one test since November.) How do you determine whether someone is 'active' or not? How long before you get removed?
If you miss three consecutive tests then your name is temporarily dropped from the ratings. You will again enter in the list as soon as you play another test.
Sudoku tests / ratings and Puzzle tests /ratings are totally different entities....and are not mixed together.
Further if you have missed two consecutive tests then you will be shown as N... which is an indication that your name will be dropped in the next list if you dont play the next test. |
|
rakesh_rai
Posts: 774
Country : India | figonometry - 2011-02-19 5:43 AM
I notice I'm missing from the rankings now. (I think I've done only one test since November.) How do you determine whether someone is 'active' or not? How long before you get removed? Hope to see you back in the next ratings update. All your previous scores WILL be considered while calculating the new ratings. |
|
figonometry
Posts: 30
Country : Canada | Awesome, thanks. Maybe now I'll have the motivation to do better! |
|
neerajmehrotra
Posts: 328
Country : India | Updated LMI Puzzle Ratings after Puzzle Zoo (February 2011 LMI puzzle test ) are now available. Overall, 186 players are included in the ratings, including 14 new players and 8 re-entries. Not much of topsy-turvy.......except a few changes in Top 10. Psyho climbs up to #7, gaining 54 rating points after Puzzle Zoo. Willwc enters the Top 10, gaining 14 rating points. Uvo sitting just behind deu and battling hard to reach the second spot. Rob was the biggest gainer from Puzzle Zoo, gaining 221 points and 113 ranks. Amongst the Top 50, Valezius gained 181 points, and Nyuta gained 95 points. Notable re-entries to the rating list include #19 (Tarotaro ) and #40 (ByronosaurusRex ). |
|
rakesh_rai
Posts: 774
Country : India | Updated LMI Puzzle Ratings after Puzzle Hybrids (LMI March 2011 Puzzle Test ) are now available. motris (992 ), deu (984 ) and uvo (955 ) occupy the top 3 spots. |
|
rakesh_rai
Posts: 774
Country : India | The following table shows the highest ranked puzzlers from the countries that are represented in the Top 50: (The countries are in alphabetical order ) | Country | Puzzler | Rank | Rating | | Austria | euklid | 23 | 647 | | Canada | ByronosaurusRex | 39 | 558 | | Czech Republic | Janka1 | 13 | 721 | | France | godisdead | 24 | 640 | | Germany | uvo | 3 | 955 | | Hungary | Valezius | 20 | 674 | | India | Rohan Rao | 34 | 587 | | Italy | forcolin | 28 | 617 | | Japan | deu | 2 | 984 | | Poland | Psyho | 7 | 808 | | Romania | rubben | 32 | 591 | | Serbia | nikola | 4 | 888 | | Slovakia | pista | 16 | 699 | | The Netherlands | Para | 27 | 621 | | Turkey | yureklis | 21 | 673 | | UK | drsteve | 29 | 601 | | USA | motris | 1 | 992 | |
|
rakesh_rai
Posts: 774
Country : India | Updated LMI Sudoku Ratings after Spring Sudoku Test (LMI March 2011 Sudoku Test ) are now available. motris (998 ), nikola (892 ) and jaku111 (881 ) occupy the top 3 spots. |
|
vopani
Posts: 739
Country : India | vopani posted @ 2011-03-30 4:46 PM rakesh_rai - 2011-03-23 2:54 PM
The following table shows the highest ranked puzzlers from the countries that are represented in the Top 50: (The countries are in alphabetical order ) | Country | Puzzler | Rank | Rating | | Austria | euklid | 23 | 647 | | Canada | ByronosaurusRex | 39 | 558 | | Czech Republic | Janka1 | 13 | 721 | | France | godisdead | 24 | 640 | | Germany | uvo | 3 | 955 | | Hungary | Valezius | 20 | 674 | | India | Rohan Rao | 34 | 587 | | Italy | forcolin | 28 | 617 | | Japan | deu | 2 | 984 | | Poland | Psyho | 7 | 808 | | Romania | rubben | 32 | 591 | | Serbia | nikola | 4 | 888 | | Slovakia | pista | 16 | 699 | | The Netherlands | Para | 27 | 621 | | Turkey | yureklis | 21 | 673 | | UK | drsteve | 29 | 601 | | USA | motris | 1 | 992 |
Sad that India is nowhere near the top. |
|
neerajmehrotra
Posts: 328
Country : India | But we have a better scene in Sudokus....with two persons in top 10......... |
|
motris
Posts: 199
Country : United States | motris posted @ 2011-04-25 8:56 PM I meant to ask this question after Twist, but it certainly comes up again after the April Sudoku test too. For the purposes of the Puzzle Ratings and Sudoku Ratings, how do you account for the different scoring systems in play in different tests. Do you use strict score only, or time, or absolute rank, or some combination of these? Having a fair way to compare very different tests seems really important to keep this world leader board accurate, and I think many of us would like to know more about how the system works. As Nikola mentioned after Twist, it might be time to stop experimenting so much with scoring and start to use a consistent system that will apply to all tests, to keep the Ratings system as fair and even as possible.
Edited by motris 2011-04-25 9:02 PM
|
|
rakesh_rai
Posts: 774
Country : India | motris - 2011-04-25 8:56 PM
I meant to ask this question after Twist, but it certainly comes up again after the April Sudoku test too. For the purposes of the Puzzle Ratings and Sudoku Ratings, how do you account for the different scoring systems in play in different tests. Do you use strict score only, or time, or absolute rank, or some combination of these? At the outset, we would like to thank you for asking this question. Within ourselves, we have been discussing exactly these same things recently. To answer your question, the current system uses strict score only. As long as the score is a function of time, we did not feel the need to include time as a separate factor. This was the case in most tests, where anyone solving the complete (or almost complete ) test correctly would get some more points using a factor. But the April sudoku test did not use time as a factor for scoring - it did give 5 points extra but difference between 91 and 101 was considered the same as difference between 116 and 117, for example. We have discussed using rank also but ultimately we had decided against it. Having a fair way to compare very different tests seems really important to keep this world leader board accurate We cannot agree more. Thats our objective too - to keep the rating system accurate and acceptable. As Nikola mentioned after Twist, it might be time to stop experimenting so much with scoring and start to use a consistent system that will apply to all tests, to keep the Ratings system as fair and even as possible. There can be so many different scoring systems. And, unless we try them in a test, we do not know the efficacy of the scoring system. So, I do not necessarily agree that we should stick to one scoring system (it could become boring, in a way ). Lets have the flexibility and freedom of having different systems at play in different tests. I think that the rating system should adapt to the scoring system (s )...and not the other way. Neither can the rating system remain static - it also needs to be reviewed from time to time for continuous improvement. and I think many of us would like to know more about how the system works. Currently, we are midway through the process of coming up with suitable improvements to the existing rating mechanism. Once finalized, the "how" part will be shared on the site as well. At the same time, we would like everyone to share any ideas/suggestions/criticisms - small or big - from the ratings point of view on this forum. And, we will address each one of those while finalizing the system. |
|
debmohanty
Country : India | As Rakesh mentioned, feel free to post ideas / suggestions on ratings, especially on
How to normalize scores in a test?
How to compare scores across tests?
How to deal with 0 scores / no submissions?
How to deal with players joining late or players not playing frequently?
etc...
Should anyone need sample data, please let us know. |
|
motris
Posts: 199
Country : United States | motris posted @ 2011-04-26 9:45 PM Thanks for your reply. I'm sure I'm not the only one who will be interested to learn the methodology behind the system as it is a real "world leader board" these days.
My personal recommendation is to not necessarily lock in scoring systems (what puzzles are worth) as these can be varied by authors on different tests to affect strategy, but after the base score of a test is calculated, the only way for the rankings to use score fairly for complete finishers is to either include time in your formula, or always use N points per minute where N is close to "max value of test/time". I like creative scoring ideas, like in Puzzle Jackpot, or special bonuses as in Evergreens or in my 20+10 Decathlon for completing sets of puzzles, but we should realize scoring of puzzles is a different component of the test than correctly using the time of finishers to measure relative performance.
So on April Sudoku, I would certainly have just used ~5 points per minute (600/120). On Twist I would have used the same staggered "value of minutes" for the bonus, instead of going down to a nominal .5 points per minute. So I would have earned 1 minute at 100%, 10 minutes at 75%, 10 minutes at 60%, and 10 minutes at 50%. In each case, simply making the time worth the value of those minutes for other solvers allows the relative performance of all finishers to be correctly staggered for your ratings, for UKPA rankings, and so on. Unlike a one-time WPC where it doesn't matter as much, LMI has really become one of the big forums for monthly competition and so consistency is absolutely required in the ranking system. |
|
motris
Posts: 199
Country : United States | motris posted @ 2011-04-26 9:54 PM debmohanty - 2011-04-26 9:38 PM
As Rakesh mentioned, feel free to post ideas / suggestions on ratings, especially on
How to normalize scores in a test?
How to compare scores across tests?
How to deal with 0 scores / no submissions?
How to deal with players joining late or players not playing frequently?
etc...
Should anyone need sample data, please let us know.
There are some interesting questions here.
I've thought about the 0/no submission score a bit. I think one option is to put a choice on the test upfront, something like "I want to play the test for the official ranking" or "I do not want to play the test for the official ranking". Either gives you a password and locks in a start time for your account. If you just want to see the test but not record a score, then you can now do so. But if you have solvers who are instead starting the test, and only putting in answers if they feel they did well, you can now separate those from each other. There will also always be the potential for technical problems, so I'm guessing that the rating probably does something like drop/minimize the value of the lowest recent test which can account for one time technical problems that led to a 0 result or particularly low score.
I've also always been surprised that the tests have a fixed time for last submission, as opposed to a fixed time for last start. I suppose advertising a 48-hour window is the point, but if that's the case I'd simply always run the tests for 50 hours. Changing to the latter format would eliminate the situation (that is always frustrating for the solver ) of realizing they started too late and have a low score. While this won't be common among the frequent players, for people taking one of their first tests you do not want to cause a negative experience or they may not want to come back. A check on test start times is a huge improvement in my opinion. Edited by motris 2011-04-26 10:03 PM
|
|
debmohanty
Country : India | 0 score/No submission -
I've never thought about that details because I always assumed players don't submit either a ) they never intended to submit or b ) they had some technical issues.
In context of ratings, it may be possible that some players submit only if they think they did well enough. I've no idea if that is a reality, but we can definitely add an option as you describe.
As of now, we simply ignore 0 scores for ratings computations.
Fixed Start Time Vs Fixed End Time -
This has been pending for a long long time even confusing regular players sometimes. It will be fixed in May Puzzle test (and onwards )
I would like to mention that it does not happen frequently because I always push the end time if I believe some serious player started the test later than when they should have. But I've missed it couple of times ( e.g. here & here). So it is better to be clear upfront.
|
|
debmohanty
Country : India | motris - 2011-04-26 9:45 PM
So on April Sudoku, I would certainly have just used ~5 points per minute (600/120). On Twist I would have used the same staggered "value of minutes" for the bonus, instead of going down to a nominal .5 points per minute. So I would have earned 1 minute at 100%, 10 minutes at 75%, 10 minutes at 60%, and 10 minutes at 50%. In each case, simply making the time worth the value of those minutes for other solvers allows the relative performance of all finishers to be correctly staggered for your ratings, for UKPA rankings, and so on. Unlike a one-time WPC where it doesn't matter as much, LMI has really become one of the big forums for monthly competition and so consistency is absolutely required in the ranking system.
I agree on the April Sudoku part.
On Twist, I'm probably missing your point. Since the puzzle points were reduced after 90 minutes, we can't give significant bonus after 90 minutes. Otherwise, it will work as double-penalty for others. Again, I could have misunderstood your point.
[ The idea of .5 per minute is to separate solvers solving at 95 minutes vs 99 minutes. ] |
|
motris
Posts: 199
Country : United States | motris posted @ 2011-04-27 5:32 AM debmohanty - 2011-04-27 4:44 AM
On Twist, I'm probably missing your point. Since the puzzle points were reduced after 90 minutes, we can't give significant bonus after 90 minutes. Otherwise, it will work as double-penalty for others. Again, I could have misunderstood your point.
[ The idea of .5 per minute is to separate solvers solving at 95 minutes vs 99 minutes. ]
I'm not asking for significant bonus (meaning "more than the value of puzzles of that time" ). I'm asking for the time bonus to scale exactly as the expected point-per-minute value other solvers got did for the "extra" time.
If the standard value (as you chose for time bonus ) is ~5 points per minute, then when the rest of the solvers entered 75% value time, and could still effectively earn 3.75 or more points per minute for submitting solutions, the time bonus should also scale to 75% of the value for that time, or 3.75 points per minute. Instead, the time bonus dropped to 10% of its value for the whole extra time and everyone effectively gained in relative score based on extra time despite my large margin of victory by time.
You can view the 90 minute and 120 minute flat results to see that "normal" scoring would have uvo at around 79-80% of my score. The result with only 10% time bonus was uvo at 87% of my score. With a bonus that is instead 3.75 for 10 minutes, 3 for 10 minutes, and 2.5 for 10 minutes, the scores would now be 813.5 versus 641.8 and this would give uvo about 79% of my score. I hope this shows how the balance of results can be preserved with scaling, provided all points scale the same way.
|
|
debmohanty
Country : India | Thanks for explaining, and I agree with what you are saying.
The current ratings system doesn't take time-to-finish into account. So, it is only ideal that we compensate top solvers by adding appropriate time-bonus in each test.
|
|
debmohanty
Country : India | Back to the Rating System, what are the different options to normalizes scores in each test so that we can compare scores across tests?
We've some basic rule which works quite well, but would love to hear independent ideas. |
|
debmohanty
Country : India | And one more - Rakesh mentioned that we don't consider player's Rank for the ratings.
Is it something we need to consider? In a way, I guess it is all related to normalization. |
|
rakesh_rai
Posts: 774
Country : India | Sharing some views and some aspects of the current rating system:
(1) Rank: I personally think we should include test ranks also into our calculation. Even though we are ultimately arriving at ratings and deducing the rank from the ratings. Take this case: In a test, X scores 800 and comes 1st, Y scores 620 for 2nd, Z scores 590 and comes 10th. If we use only the score, Y does not get enough mileage from the test (as compared to Z). But if we do include ranks into our scheme of things, Y will tend to get compensated enough.
(2) 0 scores: To me, this is not a matter of great concern in general. However, the solution suggested by motris seems good to me. There have been a few cases in some tests where X has attempted few puzzles and still got zero. So far, we have treated even such cases as "non-participation" when it is actually a zero score after participation.
(3) Players not playing frequently: We have to ensure that such cases get the "right" rating. This is something which is difficult to do. For example, we do not want someone who played one test to jump into the Top 10. So we have built the current logic such that any player will have to play some tests consistently to be where he/she belongs. Again, this "waiting time" should not be too large. If you have any suggestions around this, please do share.
(4) Scores across different tests: If we do want the "time taken" as a factor, we can build the logic such that the scores in the test can be different from the scores used for calculations (using bonus factor = [total points]/[total time] perhaps). I agree that if we take the scores as-is, (e.g April Sudoku test), the performances are not adequately translated into scores at times. But this is also an issue encountered only once so far - in April sudoku and puzzle tests.
(5) Dependency on top score: This is one area where there are going to be definite changes. So far, we are heavily dependent on the top score for rating calculations. And this leads to certain issues during calculations. For example, an 80% score in an easy test like FLIP should not be treated equally with an 80% score in a Zoo type test.
(6) Others: We are also evaluating certain other factors for any effect on the ratings whatsoever -
No of participants in a test (should performance in a 200-participant test be accorded more weight as compared to performance in a 100-participant test),
Quality/Index of participants in a test (should performance in a test where only 3 out of Top 10 participated be treated equally with a performance in a test where 10 out of top 10 participated),
weights to tests (should recent tests carry more weight),
number of tests (how many tests should be considered for ratings - 6/8/10/12/all, or should it be all tests in last 3/6/9/12 months),
bonus (should I get some bonus if I defeat a Top-3 player , or a Top-10 player)
Please feel free to share your views on any of these factors. And, anything else we have missed out. |
|
debmohanty
Country : India | One more - How do you not penalize authors / testers for 'missing' the test? |
|
purifire
Posts: 460
Country : India | debmohanty - 2011-04-28 8:44 AM
One more - How do you not penalize authors / testers for 'missing' the test?
You mean penalize them if they do not take part in tests by other authors????
If so then I think that is a bit harsh as at times someone can have a genuine reason not to participate... aprior commitment or a family event or any other legitimate reason under the sun : )
Rishi |
|
debmohanty
Country : India | I said "Not" penalize.
I just meant that we shouldn't penalize them because they 'missed' their own test. [They don't figure in the score page ] |
|
purifire
Posts: 460
Country : India | debmohanty - 2011-04-28 8:57 AM
I said "Not" penalize.
I just meant that we shouldn't penalize them because they 'missed' their own test. [They don't figure in the score page ]
Oh that way then I agree with you : ) |
|
Administrator
Country : India | motris - 2011-04-26 9:45 PM
I'm sure I'm not the only one who will be interested to learn the methodology behind the system as it is a real "world leader board" these days.
We had been working over the last couple of months to come up with a new revamped LMI Players Rating System. And, we are happy to share the details (including the rating calculation mechanism ) of the new system with everyone. The details of the rating system have been captured in a pdf. You can either download it or view it. And, feel free to discuss the ratings in this thread. As for the new rating list, it will be published after MAYnipulation, for both Sudoku and Puzzles. |
|
neerajmehrotra
Posts: 328
Country : India | Administrator - 2011-05-11 7:42 PM
motris - 2011-04-26 9:45 PM
I'm sure I'm not the only one who will be interested to learn the methodology behind the system as it is a real "world leader board" these days.
We had been working over the last couple of months to come up with a new revamped LMI Players Rating System. And, we are happy to share the details (including the rating calculation mechanism ) of the new system with everyone. The details of the rating system have been captured in a pdf. You can either download it or view it. And, feel free to discuss the ratings in this thread. As for the new rating list, it will be published after MAYnipulation, for both Sudoku and Puzzles.
The V2.0 of rating system looks interesting but needs thorough discussion. I request all the active players of LMI to please comment to make this system more robust.
Kudos to Rakesh Rai for designing the algorithm. I think it takes care of all the variables required for a proper rating system. Edited by neerajmehrotra 2011-05-11 11:39 PM
|
|
debmohanty
Country : India | As Neeraj mentioned, this system looks like covering all variables, although at the cost of being little complex.
It would help if we can show 3 different cases in action ( players getting advantage, players being penalized ) with some numbers. |
|
rakesh_rai
Posts: 774
Country : India | debmohanty - 2011-05-12 7:34 AM
As Neeraj mentioned, this system looks like covering all variables, although at the cost of being little complex.
It would help if we can show 3 different cases in action ( players getting advantage, players being penalized ) with some numbers. The image shows the working of ratings using a few fictitious players: - Players A, B and E have played very few tests and are, therefore, penalized. The level of penalty depends on the overall weight of tests played. As they play more regularly, the level of penalty will reduce and ultimately go away. - Player D has played average number of tests. The rating takes weighted average of NS from all participated tests. This player does not get any benefit nor is he/she penalized. - Player C is a regular player. This player gets the benefit of only his best performances being considered for ratings.
 |
|
debmohanty
Country : India | Am I correct in saying
For a player who starts playing at LMI
1) he needs to play 4 tests before he gets the ranking he deserves? After 3 tests his penalty will be very less though
2) to get benefit of being a regular player, he has to play more than 7 tests consecutively. (after 7 tests K=6.4), and if he misses some tests, it takes longer
Edited by Rohan Rao 2011-05-13 8:14 PM
|
|
debmohanty
Country : India | Also we discussed few posts back in this thread that we'll add an option for players who would want to consider the results to be rated.
It will be implemented starting MAYnipulation
This is how it will look like
Do we still remove 0-scores after that?
|
|
purifire
Posts: 460
Country : India | debmohanty - 2011-05-13 7:31 PM
Also we discussed few posts back in this thread that we'll add an option for players who would want to consider the results to be rated.
It will be implemented starting MAYnipulation
This is how it will look like
Do we still remove 0-scores after that?
If someone checks the box allowing the score to be considered then I would say even zero scores should be considered.
Rishi |
|
MellowMelon
Country : United States | There's a slight typo in the image by Rakesh Rai: Player E's base rating should read 1000 instead of 775 (although the final calculation is correct). Wonder who he was based off of? :P
I think considering the 0-scores is okay if you clearly say that near the "Start" button and the check box.
I don't know if I like the current weighting system. For one thing, the simulation. Relative to a 738 rating Player C has had a lot of dismal recent performances in that simulation. I suppose the ratings would eventually reflect that if it continued, but perhaps the "penalty" for those performances should kick in sooner. The fact that they never take effect if he picks his game back up is a feature I'm undecided about.
Another related issue is the following case of my own design: two regular players F and G.
-- F gets four scores around 700, then tanks for a bit and gets four scores around 500, then improves and gets four scores around 900.
-- G is consistently improving. He gets four scores around 500, then four scores around 700, then four scores around 900.
If I understand how the weighted average is calculated correctly (all of this may be moot if not), player F gets the higher rating here, because his 500s that are thrown out have a higher weight so the 900s get emphasized more in the calculation. In my opinion G's performance warrants the better rating.
Both of these issues would be fixed if the weighted average divided by the total weights of the most recent U tests, instead of the weights of the highest scoring tests. But this has its own issue in that it is a very harsh penalty on a recent bad performance. You would not want a test of weight 1 thrown out in this method. Not sure what a fix would be. |
|
rakesh_rai
Posts: 774
Country : India | debmohanty - 2011-05-13 7:26 PM
Ami I correct in saying
For a player who starts playing at LMI
1) he needs to play 4 tests before he gets the ranking he deserves? After 3 tests his penalty will be very less though Yes. For normal players, who do not author/test any tests, this would be true. For authors/testers, there can be cases where playing 3 tests may be enough. And, they are justified to get the benefits of a reduced N. 2) to get benefit of being a regular player, he has to play more than 7 tests consecutively. (after 7 tests K=6.4), and if he misses some tests, it takes longer Yes. In order for the player's poor performances to be ignored from rating calculations, about 7 tests out of 12 would be needed. The author/tester logic applies here too. |
|
rakesh_rai
Posts: 774
Country : India | purifire - 2011-05-13 7:33 PM debmohanty - 2011-05-13 7:31 PM
Also we discussed few posts back in this thread that we'll add an option for players who would want to consider the results to be rated.
It will be implemented starting MAYnipulation
Do we still remove 0-scores after that? If someone checks the box allowing the score to be considered then I would say even zero scores should be considered. I agree that 0 scores should be considered ( manipulated) for ratings from now on. But, since this is the first test after the change being implemented, we'd take a call after the test depending on how the players have adapted to the change. Edited by rakesh_rai 2011-05-13 8:19 PM
|
|
rakesh_rai
Posts: 774
Country : India | MellowMelon - 2011-05-13 7:52 PM
There's a slight typo in the image by Rakesh Rai: Player E's base rating should read 1000 instead of 775 (although the final calculation is correct). Yes. Its corrected now. Wonder who he was based off of? :P These are all fictional players. Any resemblances may at best be coincidental.  For one thing, the simulation. Relative to a 738 rating Player C has had a lot of dismal recent performances in that simulation. I suppose the ratings would eventually reflect that if it continued, but perhaps the "penalty" for those performances should kick in sooner. The fact that they never take effect if he picks his game back up is a feature I'm undecided about. We deliberated if we should keep 12 tests or 8 tests. Ultimately we decided for a longer duration. So the ratings will be based on all performances during this period. And, as mentioned earlier too, regular players are entitled to some benefits - they can afford to have a few bad days, for example. And, I would view player C example as the system allowing regular players to recover too. Also, these ratings should reflect the whole 12-month period without being too volatile. One bad or good performance should not shake up the the ratings. Another related issue is the following case of my own design: two regular players F and G.
-- F gets four scores around 700, then tanks for a bit and gets four scores around 500, then improves and gets four scores around 900.
-- G is consistently improving. He gets four scores around 500, then four scores around 700, then four scores around 900.
If I understand how the weighted average is calculated correctly (all of this may be moot if not), player F gets the higher rating here, because his 500s that are thrown out have a higher weight so the 900s get emphasized more in the calculation. In my opinion G's performance warrants the better rating.
Both F and G would get a rating of 822 in this case. But next month, F's 700 and G's 500 go out of the calculations. So G will have a better rating. And so on. Both of these issues would be fixed if the weighted average divided by the total weights of the most recent U tests, instead of the weights of the highest scoring tests. But this has its own issue in that it is a very harsh penalty on a recent bad performance. You would not want a test of weight 1 thrown out in this method. Not sure what a fix would be. I get your point and I agree that the ratings are a little slow to reflect the recent performances, but the weights are still an improvement over what we had so far. Edited by rakesh_rai 2011-05-13 9:40 PM
|
|
debmohanty
Country : India | debmohanty - 2011-05-13 7:31 PM
Also we discussed few posts back in this thread that we'll add an option for players who would want to consider the results to be rated.
It will be implemented starting MAYnipulation
This is how it will look like
Do we still remove 0-scores after that?
This didn't seem to work. Either the check box was too small or the purpose of it was not clear.
Most of the players having zero score still have the check box selected. |
|
rakesh_rai
Posts: 774
Country : India | debmohanty - 2011-05-16 6:13 AM
This didn't seem to work. Either the check box was too small or the purpose of it was not clear.
Most of the players having zero score still have the check box selected. So we'll exclude all zero scores from ratings... |
|
debmohanty
Country : India | rakesh_rai - 2011-05-16 11:06 AM
debmohanty - 2011-05-16 6:13 AM
This didn't seem to work. Either the check box was too small or the purpose of it was not clear.
Most of the players having zero score still have the check box selected. So we'll exclude all zero scores from ratings...
To put in perspective, here are the numbers -
Out of 116 players who started the test, exactly 7 marked that their results shouldn't be considered for ratings. (1 of them got non-zero score )
Of the remaining 109 players, 32 got zero scores. |
|
rakesh_rai
Posts: 774
Country : India | Updated LMI Puzzle Ratings after MAYnipulation (May 2011 LMI puzzle test ), and LMI Sudoku Ratings after the April 2011 LMI sudoku test are now available. The ratings are based on the new logic shared earlier. Four players find a place in the Top 10 in both lists - motris, deu, nikola and misko. Overall 487 players (from 45 countries ) are included in the sudoku ratings and 425 (from 44 countries ) in the puzzle ratings. |
|
kiwijam
Posts: 182
Country : New Zealand | kiwijam posted @ 2011-05-18 4:20 AM Rakesh, good work on the ratings list.
I can see I have a rating number now, and like most of us I'd like to improve it! But I don't know the numbers that were used to calculate it (like my last 12 scores), or the numbers for other puzzlers with similar ratings (these are the people I want to beat in the monthly test to move higher).
Is it possible to have a link to a full table also (like the 'pink' table of examples you published above)? |
|
rakesh_rai
Posts: 774
Country : India | kiwijam - 2011-05-18 4:20 AM
I can see I have a rating number now, and like most of us I'd like to improve it! But I don't know the numbers that were used to calculate it (like my last 12 scores), or the numbers for other puzzlers with similar ratings (these are the people I want to beat in the monthly test to move higher). If you click on your name, you go to your profile page that contains scores in all tests that you have taken part. These are the numbers based on which your ratings are calculated. Note that your scores in TVC are not included for ratings. You can see the history of scores for any player this way. Is it possible to have a link to a full table also (like the 'pink' table of examples you published above)? We have shared the computation logic already. But we feel that a full table of numbers used during ratings calculation can be cumbersome and complex, so it may not be needed online. But, since you have asked, your normalized scores in the three tests are 601, 670 and 723. And you have played only 3 tests so far. Although the weighted average rating (based on these tests ) is 665, you are penalized by a small factor to arrive at your final rating which is 651 currently. Here is the 'pink' table for you:  (kiwijam.png)Attachments ----------------  kiwijam.png (10KB - 1 downloads) |
|
debmohanty
Country : India | May be we should enhance the Profile page to display the pink table and related computations. It needs some amount of work. So no promises right now. |
|
euklid
Posts: 28
Country : Austria | euklid posted @ 2011-05-21 9:11 PM I would love to see "my pink table" in my profile page! And I think I speak for many puzzlers here. :-)
Have fun, Stefan |
|
Administrator
Country : India | euklid - 2011-05-21 9:11 PM
I would love to see "my pink table" in my profile page! And I think I speak for many puzzlers here. :-)
Have fun, Stefan The profile page has the pink table now. |
|
euklid
Posts: 28
Country : Austria | euklid posted @ 2011-05-25 1:06 AM My Puzzle Rating is not correct. The weighted average of my NS should be 707, my rating is 724, though.
Are you sure that you have implemented the weighted average of the NS correctly? I arrive at approx. 724 when I compute a non-weighted(!) average of my results, including the EvergreensI-result which should have weight zero by now.
In the pink table I would love to see the EvergreensI data (NS,...) also. It has weight 0 of course but the data is available and interesting nevertheless.
Thanks,
Stefan |
|
rakesh_rai
Posts: 774
Country : India | euklid - 2011-05-25 1:06 AM
My Puzzle Rating is not correct. The weighted average of my NS should be 707, my rating is 724, though.
Are you sure that you have implemented the weighted average of the NS correctly? I arrive at approx. 724 when I compute a non-weighted(!) average of my results, including the EvergreensI-result which should have weight zero by now. Thanks for pointing out this discrepancy. There are no problems with the implementation; however, we did find that your Twist score was completely missed in the ratings calculation. The ratings will be corrected today. In the pink table I would love to see the EvergreensI data (NS,...) also. It has weight 0 of course but the data is available and interesting nevertheless. OK. We'll include the ratings data (PS/RS/NS ) for zero weight tests as well. Edited by rakesh_rai 2011-05-25 10:37 AM
|
|
rakesh_rai
Posts: 774
Country : India | rakesh_rai - 2011-05-25 10:36 AM In the pink table I would love to see the EvergreensI data (NS,...) also. It has weight 0 of course but the data is available and interesting nevertheless. OK. We'll include the ratings data (PS/RS/NS ) for zero weight tests as well. This is done. Now you can see your normalized scores for even those tests which are not included in current ratings (like EG1 ). |
|
rakesh_rai
Posts: 774
Country : India | rakesh_rai - 2011-05-25 10:36 AM euklid - 2011-05-25 1:06 AM
My Puzzle Rating is not correct. The weighted average of my NS should be 707, my rating is 724, though.
Are you sure that you have implemented the weighted average of the NS correctly? I arrive at approx. 724 when I compute a non-weighted(!) average of my results, including the EvergreensI-result which should have weight zero by now. Thanks for pointing out this discrepancy. There are no problems with the implementation; however, we did find that your Twist score was completely missed in the ratings calculation. The ratings will be corrected today. The Puzzle Ratings have now been updated. Also, the sudoku ratings have been updated to rectify one error in Sampler Platter scores. |
|
euklid
Posts: 28
Country : Austria | euklid posted @ 2011-05-26 1:52 AM One more request of minor importance. Is it possible to mark those players' names in the result tables who have opted not to be included in the LMI rating?
MaYnipulation has 83 participants according to result table but 82 participants according to the LMI rating. If the one player were marked somehow (e.g. asterix at his name), everybody could calculate his LMI rating for himself and "verify" the correctness of the official LMI rating. :-) |
|
debmohanty
Country : India | That is a reasonable request - we should do it in next test.
Thanks for all the suggestions, however minor they may be, they certainly are missing.
Please don't hesitate to suggest any other things! |
|
Administrator
Country : India | euklid - 2011-05-26 1:52 AM
One more request of minor importance. Is it possible to mark those players' names in the result tables who have opted not to be included in the LMI rating?
MaYnipulation has 83 participants according to result table but 82 participants according to the LMI rating. If the one player were marked somehow (e.g. asterix at his name), everybody could calculate his LMI rating for himself and "verify" the correctness of the official LMI rating. :-)
This is done. You should see * next to the name for players who opt not to include their score in ratings. |
|
rakesh_rai
Posts: 774
Country : India | The sudoku ratings are updated after Something is Missing.
motris, deu, misko, nikola and jaku111 are the Top 5.
This was the 13th LMI sudoku test and, hence, Mastermind Twins are excluded from the ratings now.
Edited by rakesh_rai 2011-05-31 8:53 PM
|
|
euklid
Posts: 28
Country : Austria | euklid posted @ 2011-06-02 11:39 AM My Sudoku rating is 623 now, which is incorrect. It should be 681. Please check.
Stefan |
|
debmohanty
Country : India | This looks like a big computation mistake somewhere in the system, hope I'm wrong. |
|
rakesh_rai
Posts: 774
Country : India | euklid - 2011-06-02 11:39 AM
My Sudoku rating is 623 now, which is incorrect. It should be 681. Please check. The rating system, for some reason, likes you. Yes. This was another error which had gone unnoticed somehow. Thanks for spotting it. Those who scored equal to the test median score are affected. For example, you scored 145 in the last sudoku test which happened to be the median score as well. And the system was giving PS as 50 instead of 500. The correct ratings should be up soon. |
|
euklid
Posts: 28
Country : Austria | euklid posted @ 2011-06-03 7:29 PM Thanks for finding the mistake and correcting it. Still there remains a small(?) mistake since I have a sudoku rating of 688 now but should have 681.
My Normalized Scores (NS) given at my profile are correct, i.e. they are the same that I did compute myself.
But then, the rating must be:
(492+837+696+643*.8+721*.8+711*.6)/5.2=681
Since rounding errors are not sufficient to explain the difference of 7 points, there still must be a mistake.
Good luck for finding the error,
Stefan
P.S.: My prorated Score (PS) of the SomethingIsMissing test was always given as 500 at my profile. This proves that you have calculated the PS two times. One time to display it at my profile (PS=500) and one time to calculate my rating (there you used PS=50). Computing two times is always dangerous (if you adapt the rating in the future you will always have to do it twice...) but it might have some practical reasons. A similar mistake based on double computation must still be currently in place. |
|
debmohanty
Country : India | We are having an issue here - we've 2 different systems
1) Rakesh uses excel to compute all ratings and uploads after every test
2) What you see in profile page is directly computed from the score sheet
There are minor differences in computation in these 2 systems because of which we are seeing this difference.
As a first step, we are working on making sure that 1) and 2) are same.
As a future step, we are planning to get rid of 1) so that we've all the computation at one place. This will take a while because of the complexity involved. |
|
rakesh_rai
Posts: 774
Country : India | The LMI Puzzle Ratings have been updated after Fillomino Fillia.
The Top 5: Motris, Deu, Uvo, MelloMelon and Nikola
Edited by rakesh_rai 2011-06-21 9:28 PM
|
|
rakesh_rai
Posts: 774
Country : India | The LMI Sudoku Ratings have been updated after L O G I D O K U.
The Top 5: Motris, Nikola, Jaku111, Misko and Deu. |
|
euklid
Posts: 28
Country : Austria | euklid posted @ 2011-06-23 9:54 PM Just in case that it is a still unknown bug:
My rank score (RS) for "Something is missing" has suddenly changed from 469 to 573. That would mean that the number of participants has jumped from 96 to 119,5 (my rank of #52 has not changed).
My rank score (RS) for "X'mas Special" has suddenly changed from 734 to 770. That would mean that the number of participants has jumped from 139 to 161 (my rank of #38 has not changed).
My prorated scores (PS) have not changed.
Stefan
|
|
rakesh_rai
Posts: 774
Country : India | euklid - 2011-06-23 9:54 PM
My rank score (RS) for "Something is missing" has suddenly changed from 469 to 573. That would mean that the number of participants has jumped from 96 to 119,5 (my rank of #52 has not changed).
My rank score (RS) for "X'mas Special" has suddenly changed from 734 to 770. That would mean that the number of participants has jumped from 139 to 161 (my rank of #38 has not changed).
No bug here. Number of participants stays the same.
debmohanty - 2011-06-03 8:45 PM
There are minor differences in computation in these 2 systems because of which we are seeing this difference.
As a first step, we are working on making sure that 1) and 2) are same.
This step is completed, that is the reason you are seeing these changes.
Probably the calculations are not so obvious from the given formula. For rating calculations, if two players have the same score, they will have the same RS, even if they are actually ranked N and N+1 respectively, in the test. So, the effective formula becomes RS = (1 - [No of scores > your score] / [No of non zero scores])*1000
Interestingly, the significant variations that you see are because 13 other players scored the same as your score in XMas Special, and 12 other players scored equal to you in Something is Missing. |
|
euklid
Posts: 28
Country : Austria | euklid posted @ 2011-06-24 2:17 PM Thanks for the explanation, rakesh. Now I understand.
In the pdf explaining the new rating system (V2.0) it said:
"RS=[1-(No of players ranked above the player)/(Total players with non zero scores)]*1000"
Actually I find this original formula better than the current implementation because the rank score (RS) should depend on the rank only whereas the prorated score (PS) should depend on the points only. As you have implemented the formula now, two players with the same points get the same PS AND the same RS. With the original formula the player that sent his last answer earlier has the higher rank and a small benefit at his RS and thus at his normalized score (NS).
Of course I am well aware that one could argue that the time when one sends his/her last answer does not need to be very significant. But then, consequently, all players with the same score should be awarded the same rank in the result table. My rank in "Something is Missing" should then be 42 instead of 52. If the scoring system uses the rank 42 it should be shown in all statistics.
In my opinion, the current ranking should remain (i.e. rank 52 for my "Something is Missing") and the RS formula should use this rank. I know that this would take away RS (and NS) points from all players, nobody will increase his score.
Stefan
[edit:] Just to make sure that I am not misinterpreted. Please note that I am very happy with the ranking system and can easily accept everything that you want to implement. All my comments are opinions and suggestions only and in no way intended as negative criticism.
Edited by euklid 2011-06-24 2:28 PM
|
|
debmohanty
Country : India | euklid - 2011-06-24 2:17 PM
[edit:] Just to make sure that I am not misinterpreted. Please note that I am very happy with the ranking system and can easily accept everything that you want to implement. All my comments are opinions and suggestions only and in no way intended as negative criticism.
All your comments are really appreciated.
I think we should do the changes that you noticed. The logic in 'View Profile' page is rather easy to change. I guess Rakesh will take some time to update the excel sheets. |
|
rakesh_rai
Posts: 774
Country : India | I also agree with the suggestions. This will be a definite improvement in the implementation part of the rating system. |
|
forcolin
Posts: 172
Country : ITALY | Here I come.
Thank you for all the work you are doing and congratulation for an excellent rating system.
However I feel there is always room for improvement in every system.
I can't express myself in mathematical terms I will try to express my concept qualitatively.
In my opinion the current system can be affected negatively by some individual factors. For example, if a player gets a extremely high score (the case of deu in the Nikoli contest) all the score of the mid-range players will be low. in my case, I scored approximately 50% of his points, so although my score is higher than the median value my PS is relatively low compared to other contests. My RS is higher but this accounts only for 25%. The conclusion is that my NS for this particular contest, in which I believe I played better than my average performance, is in fact lower than my current rating, as opposed to other contests in which perhaps I playes more badly but either there was no such a uncommon performance from a single player or the mechanism of score was less rewarding towards high scorers.
As said I have no particular recipe or formula to express this, just bear this in mind next time you review the system. Of course this calls for a consistent method of scoring and of calculating bonus points, but I fully agree that in certain contest the uncommon way of granting bonuses was the "pepper" in the recipe and the inventiveness of authors must be preserved.
Stefano
|
|
debmohanty
Country : India | This is precisely something Melon had brought up with me after Fillomino test (which had only normal bonus system).
Increasing the weightage for RS is an option, but not sure if that is desirable or if that is the only option. |
|
rakesh_rai
Posts: 774
Country : India | forcolin - 2011-07-12 3:25 AM
In my opinion the current system can be affected negatively by some individual factors. For example, if a player gets a extremely high score (the case of deu in the Nikoli contest) all the score of the mid-range players will be low. in my case, I scored approximately 50% of his points, so although my score is higher than the median value my PS is relatively low compared to other contests. My RS is higher but this accounts only for 25%. The conclusion is that my NS for this particular contest, in which I believe I played better than my average performance, is in fact lower than my current rating, as opposed to other contests in which perhaps I playes more badly but either there was no such a uncommon performance from a single player or the mechanism of score was less rewarding towards high scorers.
I mostly agree with your point.
There are two things that the rating system should do here - (1 ) the normal players should not be heavily affected/penalized, and (2 ) the exceptional performer should also get something substantial out of it.
So, in the last review, we took two steps to address this - we introduced the median, so that all those who performed above the median get at least a 50% score in the PS. So, in the case of Nikoli Selection, a score of 150 (median score ) is getting 500 as PS. The second step was to factor the ranks. If someone is 2nd by a big margin, he still gets a high RS. And when we calculate NS, we initially thought of doing it 50-50 between RS and PS. But after some sample runs, we found that the exceptional performer is not getting enough advantage. So we brought the RS weight down. So, if I take your example, in the earlier system you would have got a NS of about 488. Now you are getting a NS of 672.
We had similar scenarios earlier too - the most recent ones being those of deu in Fillomino Fillia/Puzzle Hybrids, Nikola in Logidoku (and, in the month of June generally ) and motris in Twist/Prime Exotica. So we definitely need a well acceptable solution to this aspect.
Options:
(1 ) Increasing RS weight, as suggested by MellowMelon, is one option. With 50% weight, for example, forcolin will get a NS of ~716 for Nikoli Selection.
(2 ) Another thing which can be done is to introduce a couple of more "median points". So this will fix the PS of the 75th percentile performer at 750, 50th percentile at 500 and 25th percentile at 250. This way, for example, forcolin will get a NS of ~778 for Nikoli Selection. This method can protect a lot of normal players from the volatilities at the top.
What do you think? Edited by rakesh_rai 2011-07-12 10:47 PM
|
|
rakesh_rai
Posts: 774
Country : India | Updated LMI Puzzle Ratings after Nikoli Selection (July 2011 LMI puzzle test #1 ) are now available.
In the current ratings, the RS implementation has been modified as suggested by euklid earlier. |
|
forcolin
Posts: 172
Country : ITALY | Rakesh
what I think is that you have implemented a very good system.
There may be (small) margin of improvement, just bear my opinion in mind alongside with the one of others; and, if there is enough people which say a certain thing and justify a variation, do it.
But I also believe that you cannot change the mechanism after every contest, so once an equilibrium point has been achieved the system should be left running without major perturbations.
I also noticed that after Nikoli 2 my score has gone UP by about 10 points in spite of the score being lower than the previous average. Probably this happened because the new score has replaced old lower scores and consequently the average has gone up. So the compensation mechanisms in place work well after all.
Thank you again
STefano |
|
rakesh_rai
Posts: 774
Country : India | forcolin - 2011-07-12 7:05 PM
I also noticed that after Nikoli 2 my score has gone UP by about 10 points in spite of the score being lower than the previous average. Probably this happened because the new score has replaced old lower scores and consequently the average has gone up. So the compensation mechanisms in place work well after all.
Thats correct - since only the last 12 tests are considered for ratings, and you performed better in Nikoli 2 as compared to the one which went out of ratings calculations (EG2 ). |
|
jhrdina
Posts: 8
Country : Czech Republic | jhrdina posted @ 2011-07-17 3:40 AM rakesh_rai - 2011-07-12 10:56 AM
Options:
(1) Increasing RS weight, as suggested by MellowMelon, is one option. With 50% weight, for example, forcolin will get a NS of ~716 for Nikoli Selection.
(2) Another thing which can be done is to introduce a couple of more "median points". So this will fix the PS of the 75th percentile performer at 750, 50th percentile at 500 and 25th percentile at 250. This way, for example, forcolin will get a NS of ~778 for Nikoli Selection. This method can protect a lot of normal players from the volatilities at the top.
I think there is another option to eliminate negative effects of exceptional performances on the rankings.
(3 ) Base the calculation of NS on linear extrapolation between 0 and median point. It means to use the same calculation for all the players even above median.
E.g. median is 50 points and the top two players have 112,5 and 100 points. In the current system they would get NS 1000 and 900 respectively. So the second player would be heavily penalized. My suggestion would be to use the give them NS 1125 and 1000 instead.
So there would be no upper limit on NS and NS would be always calculated in relation to median performance.
The only condition to keep the system fair is that each competition should have no upper limit on points. So there should always be some time bonus for saved minutes.
But I think that the system is great even as it is and I agree with forcolin that it should not be changed too often. You may let it run for few more months and apply the changes (if desirable ) e.g. from the beginning of new year.
Regards
Jiri
|
|
rakesh_rai
Posts: 774
Country : India | Updated LMI Sudoku Ratings after FiveFold test are now available.
motris is #1 with 993 rating points, followed by deu and jaku111.
Amongst the Top 10, deu, purifire, Kota and Ziti - all gained 3 ranks. In the top 100, ByronosaurusRex and xevs were the two biggest gainers with 100+ rating point gains. |
|
euklid
Posts: 28
Country : Austria | euklid posted @ 2011-07-20 12:55 AM I prefer the current rating system over all the changes (1),(2),(3) proposed above.
If there is a test like Nikoli Selection where the top solvers are exceptionally better than the rest, then this can result in average rating points even for the above-average solvers. No big problem with that.
Only if there were a test where all the top-solvers are absent (i.e. the maximum points are obtained by an average solver) then the results of this test would not be comparable to other tests. The rating system depends on the maximum points, thus it is important that there are competitors that constantly show the others what is theoretically possible. :-)
Stefan
P.S.: Thanks Rakesh that you are updating the rating numbers VERY fast after each test! |
|
rakesh_rai
Posts: 774
Country : India | euklid - 2011-07-20 12:55 AM
P.S.: Thanks Rakesh that you are updating the rating numbers VERY fast after each test!
As mentioned sometime back by Deb, we have managed to automate the ratings calculation process now, as a result of which we are able to generate the ratings faster. |
|
rakesh_rai
Posts: 774
Country : India | jhrdina - 2011-07-17 3:40 AM
(3) Base the calculation of NS on linear extrapolation between 0 and median point. It means to use the same calculation for all the players even above median.
E.g. median is 50 points and the top two players have 112,5 and 100 points. In the current system they would get NS 1000 and 900 respectively. So the second player would be heavily penalized. My suggestion, would be to use the give them NS 1125 and 1000 instead.
So there would be no upper limit on NS and NS would be always calculated in relation to median performance.
The only condition to keep the system fair is that each competition should have no upper limit on points. So there should always be some time bonus for saved minutes.
Thanks for sharing your views, Jiri.
We had thought of a similar system earlier. But, with no upper limit, it does not work out well in terms of consistent results. And, if we use the linear extrapolation, it will come up with randomly high numbers. For example, the median in Nikoli Selection was 150. So, a score of 492 can translate to something like 3000+ on a rating scale of 1000. With an upper limit in place, the results are better. It can also serve as a quantifiable goal/target for the top solvers. And, time bonus is (mostly ) already included in scores. So we should not try to duplicate its effect.
|
|
rakesh_rai
Posts: 774
Country : India | forcolin - 2011-07-12 7:05 PM
what I think is that you have implemented a very good system.
There may be (small) margin of improvement, just bear my opinion in mind alongside with the one of others; and, if there is enough people which say a certain thing and justify a variation, do it.
But I also believe that you cannot change the mechanism after every contest, so once an equilibrium point has been achieved the system should be left running without major perturbations.
jhrdina - 2011-07-17 3:40 AM
But I think that the system is great even as it is and I agree with forcolin that it should not be changed too often. You may let it run for few more months and apply the changes (if desirable) e.g. from the beginning of new year.
euklid - 2011-07-20 12:55 AM
I prefer the current rating system over all the changes (1),(2),(3) proposed above.
As mentioned by all of you, we'll keep it stable for a long enough period. And review for further improvments after that. |
|
jhrdina
Posts: 8
Country : Czech Republic | jhrdina posted @ 2011-07-23 5:06 PM rakesh_rai - 2011-07-20 10:59 AM
jhrdina - 2011-07-17 3:40 AM
(3) Base the calculation of NS on linear extrapolation between 0 and median point. It means to use the same calculation for all the players even above median.
E.g. median is 50 points and the top two players have 112,5 and 100 points. In the current system they would get NS 1000 and 900 respectively. So the second player would be heavily penalized. My suggestion, would be to use the give them NS 1125 and 1000 instead.
So there would be no upper limit on NS and NS would be always calculated in relation to median performance.
The only condition to keep the system fair is that each competition should have no upper limit on points. So there should always be some time bonus for saved minutes.
Thanks for sharing your views, Jiri.
We had thought of a similar system earlier. But, with no upper limit, it does not work out well in terms of consistent results. And, if we use the linear extrapolation, it will come up with randomly high numbers. For example, the median in Nikoli Selection was 150. So, a score of 492 can translate to something like 3000+ on a rating scale of 1000. With an upper limit in place, the results are better. It can also serve as a quantifiable goal/target for the top solvers. And, time bonus is (mostly ) already included in scores. So we should not try to duplicate its effect.
You are right. I had a look at some previous competitions myself and I have to admit that calibrating points on median only is not enough. The top player points would be too volatile. There will always be some objections, but the current rating system looks fair enough.
Thanks
Jiri |
|
debmohanty
Country : India | forcolin - 2011-07-12 3:25 AM
In my opinion the current system can be affected negatively by some individual factors. For example, if a player gets a extremely high score (the case of deu in the Nikoli contest) all the score of the mid-range players will be low. in my case, I scored approximately 50% of his points, so although my score is higher than the median value my PS is relatively low compared to other contests. My RS is higher but this accounts only for 25%. The conclusion is that my NS for this particular contest, in which I believe I played better than my average performance, is in fact lower than my current rating, as opposed to other contests in which perhaps I playes more badly but either there was no such a uncommon performance from a single player or the mechanism of score was less rewarding towards high scorers.
Stefano
Same problem in Magic Cube. motris winning by a huge margin  |
|
euklid
Posts: 28
Country : Austria | euklid posted @ 2011-08-01 11:01 PM Again I am wondering about my rating. Before the Magic Cube test I think I had 717 rating points and 717 as my "Best Rating". Now I still have 717 rating points but 720 as my "Best Rating" (according to my stats page).
According to my calculations I should have 720 rating points right now. Perhaps there is some calculation error because I am a "very regular player" with K>U now. My weakest test (Twist) thus has weight 0 and my second-weakest test (the most recent Magic Cube) has a weight of 0.8 instead of 1.0. But perhaps there is a mis-calculation on my side...
Have fun,
Stefan |
|
debmohanty
Country : India | For once, LMI calculations are not wrong. Your calculations are also not wrong.
Your Ratings after Nikoli2011 is 717. This is the current rating.
Your ratings after Magic Cube will be 720 (as you can see here - http://logicmastersindia.com/forum/lmi/ratings/?test=M201107P2), but it is not made the 'current' rating yet . |
|
motris
Posts: 199
Country : United States | motris posted @ 2011-08-01 11:31 PM debmohanty - 2011-07-31 11:17 PM
Same problem in Magic Cube. motris winning by a huge margin
It's not a bug, it's a feature.
If you look at almost all of the tests here, having the lead score 10% above the rest is not that uncommon (and if you look at puzzle result distributions on a site like croco-puzzles it's not that uncommon there with this same group of competitors ).
Then there are tests like FLIP or Puzzle Jackpot where you can't easily separate the top two, but there is still a rather large gap to the next spot (s ): 25% to 3rd in FLIP, 25% to 5th in Puzzle Jackpot. The truth is whenever H. Jo or uvo or someone else has a really good day, there can be a phenomenal score or two.
I think a consistent rating needs to be fair for solvers at the top and in the middle. The top is suited ok with the system as is with absolute top score contributing a lot more than rank score, rewarding a very good performance. The middle is suited ok by using the median score as the measure of 500, instead of actual score/top score which is where the median might drop to 300 or 400 on an exceptional test day. And the rank score further brings back the front-runner a bit and also separates ties by score (but not time ) in the middle.
I think drastic changes from the current formula will result in a worse ranking system either for the top, or for the middle, based on the test data we have for the last 15 months. At most, I might like to see modeled what would happen if a third inflection point was built into the system, perhaps at the 1st standard deviation above and below the median, equaling another set of fixed score points. My guess is that the middle and top are stable but players at 70-85% in rank are more affected by test to test variation outside of their own performance. Edited by motris 2011-08-01 11:44 PM
|
|
MellowMelon
Country : United States | As far as individual tests are concerned, I think the rating system is fairly well-balanced given what was stated above. Where I think the problem enters is missed/excluded tests (excluded meaning the system for regular players to drop their low scores), because the variability at the top makes it so that an "equivalent" performance on two tests can result in very different ratings for people above the median. The rank score is far more consistent as an indicator across tests. I understand and agree that a good performance should be rewarded, but I think the rank score also plays an important role to the point that it's not getting the weight it should.
The biggest effect of this is probably on the 600-800 range, as you say, since for the top when a test goes your way you're generally contributing to lopsided rankings, and most of the people at the top of the ratings list play regularly. But it still has some effect; I think uvo was lucky to miss the recent Nikoli test, which had the steepest score gradient at the top that I can recall (incidentally he's the one right ahead of me right now; sorry for the personal groaning).
(EDIT: I managed to word this in a way that said something different from my intention. I meant that if one had to miss one test, that would have been the one. Of course, the data below suggests that may have been off anyway. Bit hasty on my part.)
Edited by MellowMelon 2011-08-02 6:42 PM
|
|
motris
Posts: 199
Country : United States | motris posted @ 2011-08-02 4:16 AM MellowMelon - 2011-08-01 11:58 AM
I think uvo was lucky to miss the recent Nikoli test, which had the steepest score gradient at the top that I can recall (incidentally he's the one right ahead of me right now; sorry for the personal groaning).
I strongly disagree with this perception. No one is "lucky" to not have taken a test. I expect uvo may easily have claimed a high spot as he had on the first Nikoli Selection by being 4th.
So, was that test (Nikoli Selection ) really an outlier and if so why? Consider these numbers. This is the prorated score (PS ) for 2nd, 3rd, 5th, 7th, and 10th on the last 6 puzzle contests in no particular order.
2/3/5/7/10
A - 890, 882, 866, 780, 751
B - 951, 924, 883, 771, 717
C - 911, 795, 749, 735, 716
D - 905, 884, 761, 735, 709
E - 919, 905, 790, 771, 710
F - 909, 906, 884, 813, 742
Does the Nikoli Selection stand out on this list? Yes, if you look exactly at 3rd place. But not as much at 5th and certainly not at 7th or 10th place. There is a range, but nothing that strikes me as really large differences in PS that are outside general performance variation. 4 tests are around 710 at 10th place. Two are higher at 750. The outliers are probably A and F for being so flat.
Why does the Nikoli Selection have a large scoring problem for 3rd - 7th that resolves afterwards? I've proffered my rationales before but I'll do so here again. The best measure of a group of solvers outside of exact time to finish is their points earned/minute rate. No matter what the test length (2 minutes, 2 hours, 2 days, 2 weeks, ... ) if you are measuring this rate you will eventually get an accurate measure of their ability removing the puzzle to puzzle variance. When a person "finishes" a test, you have an absolute measure of that solver's points earned/minute rate and hopefully the extra time is rewarded close to this number.
The problem with Nikoli Selection was two-fold:
First, for the top 10 solvers, but not for the larger field, the test had two different parts with very different points/minute rates. The non-traditional "marathon" bonus system was not very granular at the top, and a lot of solvers got much less value for finishing early unless they were the right increment of fifteen-twenty minutes away from total test time for a big bonus jump. Further, it was even more valuable to solve 2 or 3 of these than just 1. Even so, the point/minute rates were grossly compressed for H. Jo and myself. In the first ~45 minutes, he earned 280 points for finishing the main test. In the last 45 minutes, he earned 170 + 42 time bonus points for finishing three marathons, about 75% the original rate on the test everyone else was being compared on. My second half was 105+46 or about 50% of my original rate. I'd suppose those who got only 1 (or 0 ) marathons out were much lower even than this.
Second, the "main test" was being scored with a time bonus that would, if it were for the whole test, have worked ok except for being (1 ) slightly undervalued and (2 ) lost by basically everyone from 5th to 9th (who would have normally filled 3rd to 6th ) for making a typo in a main puzzle answer key. Answer keys they did not check as well because there was something else to move onto unlike every other test. So a 20 point puzzle error + 30 time bonus point loss was a more significant penalty (11% max score value ) than it usually is. But the performance gap falls aways from 10th and on and the test resembles others in its curve because for everyone else, there was no point/minute drop-off of transitioning from sprint to marathon territory because there was no marathon territory to deal with. The problem is not H.Jo (or on other tests me or uvo or psyho or whomever ). The problem this time was the test/scoring structure that depressed the points earned for the top 10 universally and most pronounced for 3rd through 10th compared to what the usual test would look like. In other words: a compressed score range (higher relative median score ), but less realized bonus for 3rd to 10th place, equals the scoring crunch which is coming from both the 1000 mark and the 500 mark. Edited by motris 2011-08-02 5:31 AM
|
|
rakesh_rai
Posts: 774
Country : India | Updated LMI Puzzle Ratings after Magic Cube (July 2011 LMI puzzle test #2 ) are now available.
The Top Five: 1. motris, 2. deu, 3. uvo, 4. MellowMelon, 5. Nikola
India Top Five: 1. Rohan, 2. Rakesh, 3. Rajesh, 4. Amit, 5. Jaipal |
|
motris
Posts: 199
Country : United States | motris posted @ 2011-08-03 12:53 AM For the goals of keeping a consistent rating, can I make a request that test authors keep control of the base scoring of tests (and twists like Puzzle Jackpot and Twist and such are ok as this is the base value), but that LMI admins set the value of time bonus (which is really just serving a role of standardizing scores from month to month) with a fixed standard for all tests? When set by authors, we get such a wide wide range of these values which are really problematic for comparing tests. The upcoming sudoku test has a 3x value for time bonus compared to puzzle minutes, so finishers will benefit excessively. Past tests have had closer to 0x value for time bonus compared to puzzle minutes, so finishers are punished excessively. Is total value/total time that hard a ratio to fix? Time bonus will never affect the ranks in a given test (which is why 1000 points a minute or .0001 points a minute could be chosen by test authors if they wanted) but it does affect the yearly rankings which is why it should not be a variable in the constructor's mind at all. |
|
debmohanty
Country : India | Very valid argument, we will do that.
For Sudoku City, Nikola had asked me to review the bonus system, but I had missed it completely. I'll get it rectified. Thanks for bringing it up. And it indeed helps when the anomalies are brought up before the test itself, so that we can get it fixed as needed.
|
|
rakesh_rai
Posts: 774
Country : India | motris - 2011-08-01 11:31 PM
I think a consistent rating needs to be fair for solvers at the top and in the middle. The top is suited ok with the system as is with absolute top score contributing a lot more than rank score, rewarding a very good performance. The middle is suited ok by using the median score as the measure of 500, instead of actual score/top score which is where the median might drop to 300 or 400 on an exceptional test day. And the rank score further brings back the front-runner a bit and also separates ties by score (but not time) in the middle.
I think drastic changes from the current formula will result in a worse ranking system either for the top, or for the middle, based on the test data we have for the last 15 months. At most, I might like to see modeled what would happen if a third inflection point was built into the system, perhaps at the 1st standard deviation above and below the median, equaling another set of fixed score points. My guess is that the middle and top are stable but players at 70-85% in rank are more affected by test to test variation outside of their own performance. Very nicely put.
Just one question: when you say 70-85% in rank, do you mean those ranked 71st-85th (out of 100 ) or those ranked 15th-30th? And, I would also like to know the degree (slightly/moderately/heavily ) to which they are affected, so that we can think of further improvements at some point of time. |
|
motris
Posts: 199
Country : United States | motris posted @ 2011-08-04 10:53 AM rakesh_rai - 2011-08-02 11:52 PM
motris - 2011-08-01 11:31 PM
I think a consistent rating needs to be fair for solvers at the top and in the middle. The top is suited ok with the system as is with absolute top score contributing a lot more than rank score, rewarding a very good performance. The middle is suited ok by using the median score as the measure of 500, instead of actual score/top score which is where the median might drop to 300 or 400 on an exceptional test day. And the rank score further brings back the front-runner a bit and also separates ties by score (but not time) in the middle.
I think drastic changes from the current formula will result in a worse ranking system either for the top, or for the middle, based on the test data we have for the last 15 months. At most, I might like to see modeled what would happen if a third inflection point was built into the system, perhaps at the 1st standard deviation above and below the median, equaling another set of fixed score points. My guess is that the middle and top are stable but players at 70-85% in rank are more affected by test to test variation outside of their own performance. Very nicely put.
Just one question: when you say 70-85% in rank, do you mean those ranked 71st-85th (out of 100 ) or those ranked 15th-30th? And, I would also like to know the degree (slightly/moderately/heavily ) to which they are affected, so that we can think of further improvements at some point of time.
By 70-85% in rank I meant those with a rank score around 700 to 850. So 15th to 30th place in your out of 100 example. |
|
rakesh_rai
Posts: 774
Country : India | Updated LMI Sudoku Ratings after Sudoku City (August 2011 LMI sudoku test ) are now available.
The Top five: 1. motris, 2. nikola, 3. purifire, 4. deu, 5. misko
India Top five: 1. Rishi, 2. Rohan, 3. Rakesh, 4. Gaurav, 5. Sumit |
|
rakesh_rai
Posts: 774
Country : India | Updated LMI Puzzle Ratings after Japanese Puzzle Land (August 2011 LMI puzzle test ) are now available.
xevs is the only new name in the Top 10.
Rohan Rao at #43 is the top ranked puzzler from India. |
|
anand2100
Posts: 1
| How to know my LMI rating? |
|
debmohanty
Country : India | anand2100 - 2011-08-30 11:50 AM
How to know my LMI rating?
Didn't you register at LMI few minutes back? |
|
rakesh_rai
Posts: 774
Country : India | Updated LMI Puzzle Ratings after Sprint Test (September 2011 LMI puzzle test ) are now available.
MellowMelon sprints to the third place.
|
|
rakesh_rai
Posts: 774
Country : India | Updated LMI Sudoku Ratings after Crazy Arrows (September 2011 LMI sudoku test ) are now available.
532 players representing 50 countries are now a part of the ratings. motris and nikola remain #1 and #2 repectively, and deu moved up to #3.
Amongst the heavy gainers, ppeetteerr gained 203 rating points to move to #40, and tarotaro gained 164 rating points to move up to #76.
USA has 6 players in the Top 50, while Japan and India have 5 each. Germany has 4 players in the Top 50. |
|
rakesh_rai
Posts: 774
Country : India | Updated LMI Sudoku Ratings after A or B (October 2011 LMI sudoku test ) are now available.
motris inched closer to the 1000 mark, gaining one point to reach 993. deu and Nikola exchanged places and are within a point of each other now. jaku111 and uvo, both made major gains to enter the Top 10. Four of the top 10 have palindromic ratings (which read the same whether read from left or right ).
TiiT (palindrome again ) who finished 2nd in A or B, gained 36 rating points to move to #15. Gotroch has a rating of 888 (which is a flush in poker ). Hunsudoku, tarotaro, SCORPPROCS (another palindrome ), tilansia and spelvin all gained over 100 points and should be rapidly moving towards the top in the coming months.
Among players from India, Rohan moved to within 2 points of 900 and is ranked 8th now. Prasanna gained 27 points. |
|
debmohanty
Country : India | Ratings are updated after Double Decathlon - Link
I would wait till Rakesh double checks once and make it official. |
|
debmohanty
Country : India | |
|
vopani
Posts: 739
Country : India | vopani posted @ 2011-11-20 1:13 PM Sudoku
1. 9 out of the top 10 of WSC are active on LMI.
2. The top 6 players of LMI Sudoku Ratings appeared in the WSC 2011 Playoffs.
3. 8 of the top 10 in WSC 2011 are in the top 15 of LMI Sudoku Ratings.
Puzzle
1. All 10 players of WPC playoffs are active on LMI.
2. The top 4 players of LMI Puzzle Ratings appeared in the WPC 2011 Playoffs.
3. 9 of the top 10 in WPC 2011 are in the top 20 of LMI Puzzle Ratings.
Not bad! |
|
debmohanty
Country : India | Rohan Rao - 2011-11-20 1:13 PM
Not bad!
ditto! |
|
neerajmehrotra
Posts: 328
Country : India | That gives a real good feeling............. |
|
debmohanty
Country : India | Ratings update after Puzzle Fusion available here
We'll spend some time checking it manually before making this as default ratings.
The most significant change at the top is that MellowMelon moves to #2 position ahead of deu. Also, xevs gained 19 points to have ratings of 919. |
|
debmohanty
Country : India | Sudoku ratings updated after December Test.
motris continues his dominance and retains the top spot throughout the year. Edited by debmohanty 2011-12-22 12:58 PM
|
|
debmohanty
Country : India | LMI Puzzle ratings are updated now
No changes in top 3, but MellowMelon inches toward motris.
xevs gains lot of ratings and takes the 4th place now |
|
debmohanty
Country : India | We have a technical problem, and it will take a while to update the ratings this time (and may be in future as well) |
|
debmohanty
Country : India | |
|
Administrator
Country : India | Also added Ratings History in profile page. |
|
Administrator
Country : India | Sudoku Ratings updated after Diagonal Vision
TiiT moves to top5 after impressive performances.
Among players from India, Prasanna moves into top50 after few good tests. |
|
zachpuzzle
Posts: 14
Country : United States | what does a little orange bar by the name in the rankings mean? I'll send a picture of what i mean  (Those.PNG)Attachments ---------------- Those.PNG (3KB - 0 downloads) |
|
Fred76
Posts: 337
Country : Switzerland | Fred76 posted @ 2012-03-17 7:54 PM Administrator - 2012-03-13 3:45 PM
Sudoku Ratings updated after Diagonal Vision
TiiT moves to top5 after impressive performances.
Among players from India, Prasanna moves into top50 after few good tests.
And the author is punished for his too hard grids by losing 3 ranks and being kicked out of the top 20 
|
|
MellowMelon
Country : United States | zachpuzzle: If you're checking results multiple times while a test is running (i.e. after you've played), those are the players that have played since the last time you checked. I don't think that orange ever shows up after a test is over. |
|
debmohanty
Country : India | MellowMelon is right. It was not explained anywhere in the forum, and was left as 'another puzzle' for players to figure out.
The colors don't appear after the test is over. |
|
Administrator
Country : India | Sudoku Ratings updated after Killing Time.
There are few swaps at the top. janoslaw and motris swap top 2 positions. Also Kota and TiiT swap 5th and 6th positions.
|
|
Administrator
Country : India | Sudoku Ratings updated after Sudoku 8X8.
No changes in top 5. But lots of swaps after that.
|
|
Administrator
Country : India | |
|
debmohanty
Country : India | |
|
Administrator
Country : India | With just one week till next Sudoku test and most of us busy with our Sudoku championship, ratings will not be updated after SudoClones.
Once Copy Paste is over, ratings will be updated.
|
|
Administrator
Country : India | |
|
Administrator
Country : India | Puzzle ratings updated after Logirace
MellowMelon still 5 points behind motris. |
|
prasanna16391
Posts: 1775
Country : India | Top 100, Yay! |
|
neerajmehrotra
Posts: 328
Country : India | Congrats Prassana!! |
|
Administrator
Country : India | |
|
Administrator
Country : India | Puzzle ratings updated after akil oyunlari magazine competition
No changes in top 5. |
|
Administrator
Country : India | Sudoku Ratings updated after Killer Shootout. motris moves back to top. |
|
nikaash
Posts: 3
Country : India | nikaash posted @ 2012-09-07 7:47 PM hello there!!.. iam a beginner to this sudoku puzzles and i have been solving LMI monthly puzzles since july. How do i get my ratings and where can i check them? is there any prerequisite for entering into the ratings list?
thank you
madhu nikaash |
|
Ours brun
Posts: 148
Country : France | Hi nikaash,
Be careful to distinguish between Monthly sudoku tests and Beginners' contests. As far, you have participated in two Beginner's contests, which results are not merged into a cumulative rating. Only Monthly tests' results do, and since you scored 0 in the only one you attempted, I am afraid you will have to score at least 1 point in the next one if you want to appear in the ratings. Once done, this is the page where you will be able to compare yourself with hundreds of players. Or you could also click on your name to see the details of your past performances; there will be a link to the sudoku ratings page (but, again, only once you have scored at least 1 point on a Monthly test). |
|
Administrator
Country : India | Puzzle Ratings updated after Borders & Beyond.
Interestingly, there are 6 players from Japan in top 9. More importantly, MellowMelon moves to the top position ahead of motris. |
|
Administrator
Country : India | Puzzle Ratings updated after 123 GO.
With 6 perfect scores out of 12 tests included in ratings, MellowMelon is inching toward a perfect 1000. |
|
David McNeill
Posts: 63
Country : United Kingdom | It seems unfair that Rohan's rating should have fallen by 2 points because he was unable to compete in his own contest. Is there a way of ensuring that the puzzle setter's rating is unchanged? |
|
debmohanty
Country : India | Well, I think Rohan's points dropped not because he didn't compete 123, rather because he didn't participate in previous 2 puzzle tests (akil oyunlari, Borders & Beyond). |
|
neerajmehrotra
Posts: 328
Country : India | WSC results
1. Jan Mrozowski
2. Kota Morinishi
3. Hideaki Jo
4. Tiit Vunk
LMI Sudoku ratings
2. Jan Mrozowski
3. Kota Morinishi
4. Hideaki Jo
6. Tiit Vunk
LMI ratings are so realistic and close to WSC results
|
|
neerajmehrotra
Posts: 328
Country : India | WSC results
1. Jan Mrozowski
2. Kota Morinishi
3. Hideaki Jo
4. Tiit Vunk
LMI Sudoku ratings
2. Jan Mrozowski
3. Kota Morinishi
4. Hideaki Jo
6. Tiit Vunk
LMI ratings are so realistic and close to WSC results
|
|
gbansal
Posts: 2
Country : India | gbansal posted @ 2012-10-26 5:39 PM Are monthly LMI Beginner tests included in the ratings?
It seems all tests other than monthly LMI beginner tests and some exceptions like "Di..." are included in Ratings i.e. either LMI Puzzle ratings or LMI sudoku ratings, but the LMI Beginner tests are not included in any ratings. Is it on purpose or I have mistaken ? |
|
Administrator
Country : India | gbansal - 2012-10-26 5:39 PM
Are monthly LMI Beginner tests included in the ratings?
It seems all tests other than monthly LMI beginner tests and some exceptions like "Di..." are included in Ratings i.e. either LMI Puzzle ratings or LMI sudoku ratings, but the LMI Beginner tests are not included in any ratings. Is it on purpose or I have mistaken ? Beginners' contests are not included in Ratings. Di... was an annual contest. By default, annual contests are not included in ratings.
In summary, tests under this sub-forum are the ones which are included in ratings. |
|
kiwijam
Posts: 182
Country : New Zealand | kiwijam posted @ 2012-10-29 2:43 AM Is it possible to have a link to the 'Rating calculations pdf' on the Ratings page? I was trying to remember how ratings worked, and it took me a time to find it in this forum. |
|
Administrator
Country : India | kiwijam - 2012-10-29 2:43 AM
Is it possible to have a link to the 'Rating calculations pdf' on the Ratings page? I was trying to remember how ratings worked, and it took me a time to find it in this forum. Good point.
Added a link in the ratings page. |
|
Administrator
Country : India | Sudoku Ratings updated after Fugitive Warrant. Kota and deu move up one position each. |
|
greenhorn
Posts: 164
Country : Slovakia | Hi, Fillomino Fillia II is considered to be annual or montly competition? Because it is missing within the puzzle ratings.
Thanks |
|
prasanna16391
Posts: 1775
Country : India | |
|
greenhorn
Posts: 164
Country : Slovakia | Thank you. I was confused because there was no other puzzle test in October. Now it is clear. |
|
Administrator
Country : India | Puzzle Ratings updated after Word Show. Para and xevs swap places in the top 5.
Administrator - 2012-09-25 10:07 AM
With 6 perfect scores out of 12 tests included in ratings, MellowMelon is inching toward a perfect 1000. |
|
Administrator
Country : India | |
|
Administrator
Country : India | |
|
Administrator
Country : India | Puzzle Ratings updated. deu moves to 2nd.
Among Indians, Prasanna moves ahead of Rohan.
|
|
Administrator
Country : India | |
|
debmohanty
Country : India | |
|
rakesh_rai
Posts: 774
Country : India | A majority of the top players have gained ratings in this test (I think 44 out of top 50).
Is this on expected lines (perhaps because of the nature of this test)? |
|
Administrator
Country : India | |
|
Administrator
Country : India | |
|
Gabrieleud
Posts: 12
Country : ITALY | Administrator - 2013-05-22 1:10 PM
Puzzle Ratings updated after Deception.
I've been inactive for some months, but I'm coming back solving and taking part in the monthly competitions. After how many games will my name appear again in the LMI Rating system? |
|
rakesh_rai
Posts: 774
Country : India | Gabrieleud - 2013-06-25 5:40 PM
I've been inactive for some months, but I'm coming back solving and taking part in the monthly competitions. After how many games will my name appear again in the LMI Rating system?
It should appear in the next rating update. More details on the rating system are included earlier in this thread. |
|
Administrator
Country : India | Missed to post this : Hideaki moved to top in Sudoku ratings after Turkish round. Congratulations Hideaki. |
|
Administrator
Country : India | |
|
jitendradayma
Posts: 3
Country : India | how i can see my rating points?  |
|
prasanna16391
Posts: 1775
Country : India | jitendradayma - 2013-08-09 11:34 AM
how i can see my rating points?
Once you start participating in the monthly tests, your ratings will appear on the ratings page. As of now, there is no Monthly Test announced because the Indian Sudoku Championship is the next event being held here. But stick around and there'll be a monthly soon. |
|
rajeshk
Posts: 542
Country : India | rajeshk posted @ 2013-08-13 2:35 PM I was just thinking that Is it possible to rate Teams (Countries) as explained below
1. Sum of top 4 four players from a country will be considered their Team score.
2. Score of Players from the same country who are not in top 4 will be ignored for team score.
3. For the countries where less than 4 players participate, sum of their scores will be team score. It will encourage them to get more players from their country to give test to get better Team ranking.
This is just an idea. First thing is that this should be technical implementable easily. If yes, we can discuss further on this or may be start a new thread.
|
|
Administrator
Country : India | |
|
Administrator
Country : India | |
|
chaotic_iak
Posts: 241
Country : Indonesia | I have a suggestion regarding the "pink table" on profile pages. Can it be moved to a new page, that also lists the values of N, L, U, K, Kp, and which tests are considered and how much weight for each (like Player C in the sample)? It helps cleaning the somewhat cluttered profile page, and it can display more information. In fact, if my assumption of how the table is displayed behind the stage is correct, the new page can simply have a copy of the current table, and the current profile page can have the last four columns removed. The main task would be to display a new "weight counted for the rating" column, plus computing N/L/U/K/Kp values.
^ Yeah, mostly because of laziness computing by myself.  |
|
Administrator
Country : India | chaotic_iak - 2013-11-23 4:49 PM
I have a suggestion regarding the "pink table" on profile pages. Can it be moved to a new page, that also lists the values of N, L, U, K, Kp, and which tests are considered and how much weight for each (like Player C in the sample)? It helps cleaning the somewhat cluttered profile page, and it can display more information. In fact, if my assumption of how the table is displayed behind the stage is correct, the new page can simply have a copy of the current table, and the current profile page can have the last four columns removed. The main task would be to display a new "weight counted for the rating" column, plus computing N/L/U/K/Kp values.
^ Yeah, mostly because of laziness computing by myself. Sounds like a good suggestion; so will get it done. It might take a while, because not all information is handy in the database.
In the meantime, ratings updated after ST4 and deu continues his top rank. |
|
Administrator
Country : India | Ratings updated after JPL2 |
|
Administrator
Country : India | |
|
prasanna16391
Posts: 1775
Country : India | Administrator - 2014-01-15 6:07 PM
Ratings updated after Kakuro
I now have the exact same rankings as I did in the WSC and the WPC. |
|
Administrator
Country : India | |
|
Administrator
Country : India | Ratings updated after Marathon
deu now has Rank 1 in both puzzles and sudokus. |
|
sunny yambur
Posts: 1
Country : India | hello everyone....this is great place to use ur talent...thanx LMI |
|
Administrator
Country : India | |
|
Administrator
Country : India | |
|
alisa201127
| alisa201127 posted @ 2014-05-20 2:13 PM Wanted to know how are the rating points calculated? |
|
prasanna16391
Posts: 1775
Country : India | |
|
Bram28
Posts: 35
Country : The Netherlands | Bram28 posted @ 2014-07-04 1:24 AM Are the Sudoku ratings going to be updated after Road to London? |
|
prasanna16391
Posts: 1775
Country : India | Bram28 - 2014-07-04 1:24 AM
Are the Sudoku ratings going to be updated after Road to London?
On the Road to London topic, there's a notification from the Admin that it'll take a few days. |
|
Administrator
Country : India | Bram28 - 2014-07-04 1:24 AM
Are the Sudoku ratings going to be updated after Road to London?
Done |
|
Administrator
Country : India | |
|
kiwijam
Posts: 182
Country : New Zealand | kiwijam posted @ 2014-10-16 2:51 AM New puzzle ratings are up after the Pento Parade test.
Wow, I see William Blatt jumped up 10 places from 24th to 14th. And he didn't even compete!
(Arguably) if he had competed he might have got a lower rating, because this test had a top score well ahead of the main pack, which means everyone gets a lower score than they otherwise would.
As discussed for other competitions, basing the score for the top half of the field on the overall best score gives a wide variety of results, meaning different tests get different distributions of final scores.
|
|
rakesh_rai
Posts: 774
Country : India | Thanks for bringing this point up, James.
Wow, I see William Blatt jumped up 10 places from 24th to 14th. And he didn't even compete!
Yes. That's correct. Ratings can increase if (a ) you do very well in the most recent test, OR (b ) you had a bad performance twelve months ago (which goes out of rating calculations now ) but have since done ok. Generally, we tend to correlate rating improvement with the first of these reasons, which is natural but not a complete view. LMI Rating system (Puzzles ) takes into account performances over the last 12 rated puzzle tests, which roughly means twelve months. William had a (relatively ) lower rating so far because of his (arguably ) inferior show in the Forsmarts Anniversary contest 2013, where he was ranked 37th out of 108 non-zero scorers. In the latest ratings, he got rid of this baggage. And due to relatively stable performances over the last twelve months, his rating has improved and come close to his actual recent performances. I would say, "the rating system waited for a fair number of improved performances before registering an improvement." And, since he has not been a regular participant (yet ), the rating has potential to improve further. I can see some analogy to FIDE ratings (chess ), where it takes time to reach your actual rating levels.
(Arguably) if he had competed he might have got a lower rating, because this test had a top score well ahead of the main pack, which means everyone gets a lower score than they otherwise would.
Your argument is valid since willwc has played less tests in the recent twelve month period. For a more regular player, this argument is not valid. For example, if he had performed as well as you did on Pento Parade, he would have still reached a rating of ~833 to be 17th, a jump of 7. And, if he had got more than ~1140 on Pento Parade, he would have bettered a rating of 845 also. But, in general, yes, NS on this test would be lower because of the exceptional top score.
As discussed for other competitions, basing the score for the top half of the field on the overall best score gives a wide variety of results, meaning different tests get different distributions of final scores.
Agree with your point. This point has been discussed while devising the ratings formula. Also look at this post from motris summarizing the discussions around this. I am not saying we cannot make changes. But "Is an improvement needed for handling such tests with outlier performances?" is the question.
Also, you can refer to this earlier post. In the example, player D resembles willwc. So we take a straightforward weighted average of his NS to get to his rating. |
|
Para
Posts: 315
Country : The Netherlands | Para posted @ 2014-10-16 10:42 AM You can make a similar argument for Neil as he only has 4 tests. But because two of his tests are marathons, where the PS is much higher for the top puzzlers compared to other tests, his rating average is much better. |
|
Administrator
Country : India | Updated ratings after Irregular Variations.
|
|
bilgin
Posts: 2
Country : Turkey | bilgin posted @ 2014-11-25 4:16 AM Hi! Why do you list only the first 300? How can I find myself in the rating lists? |
|
Administrator
Country : India | |
|
Administrator
Country : India | Ratings updated after Sudoku Surprise (sorry about the delay )
After some impressive performances in last 5 sudoku tests, Kota moves to the top position. |
|
Puzlifouk
Posts: 61
Country : France | Hello,
There has been a lot of sudoku tests these times, but few puzzle tests ( only Pentoparade in october and Forsmarts in december ).
Can we know when the next puzzle tests are planned ? |
|
Administrator
Country : India | Sudoku Ratings are updated after Converse,
AND
Puzzle Ratings are updated after Puzzle Marathon. |
|
Administrator
Country : India | Sudoku Ratings are updated after Unlucky 13.
Only three points separate the top 3 - jaku111, Kota and deu. |
|
long123
Posts: 4
| long123 posted @ 2015-07-08 1:19 PM Also, for authors/testers, I think NumTests should include the tests they authored/tested.
For calculations, it should be ignored. |
|
rakesh_rai
Posts: 774
Country : India | |
|
kishy72
Posts: 417
Country : India | kishy72 posted @ 2017-02-21 9:01 PM So,before the ratings get updated after Round 2 of PR , I wish to state here an observation and a rare coincidence . My current LMI Puzzle ranking(133rd) and LMI sudoku ranking(38th) are exactly the same as my position in WPC(133rd) and WSC(38th) 2016 in Slovakia respectively !! :) |
|
debmohanty
Country : India | kishy72 - 2017-02-21 9:01 PM
So,before the ratings get updated after Round 2 of PR , I wish to state here an observation and a rare coincidence . My current LMI Puzzle ranking(133rd) and LMI sudoku ranking(38th) are exactly the same as my position in WPC(133rd) and WSC(38th) 2016 in Slovakia respectively !! :) Couldn't have been more coincidental. Thanks for sharing.
Deb
|
|
vopani
Posts: 739
Country : India | vopani posted @ 2017-02-22 9:21 AM kishy72 - 2017-02-21 9:01 PM
So,before the ratings get updated after Round 2 of PR , I wish to state here an observation and a rare coincidence . My current LMI Puzzle ranking(133rd) and LMI sudoku ranking(38th) are exactly the same as my position in WPC(133rd) and WSC(38th) 2016 in Slovakia respectively !! :)
Ohhh wow! I often tried to check mine too, but never got close :- )
|
|
aashay
Posts: 46
Country : India | aashay posted @ 2018-01-29 10:10 PM When will the ratings be updated incorporating the SM Round 1 results? |
|
Administrator
Country : India | The ratings shall be updated very soon. |
|
x18349276
Posts: 8
Country : ITALY | What means "very soon"? |
|
Administrator
Country : India | Apologies for a long delay in ratings update.
The Puzzle and Sudoku Ratings are now up to date after Evergreens PR test for puzzles and Odd Even/ Substitution SM test for sudoku.
The top 3 in Puzzle ratings are Endo Ken, Hideaki Jo and Kota Morinishi. For India, the top 3 has Prasanna, Rohan and Amit.
The top 3 in Sudoku ratings are Jan Zverina, Kota Morinishi and Seung Jae Kwak. For India, the top 3 has Rohan, Prasanna and Rishi.
|
|
Administrator
Country : India | The Sudoku Ratings are now up to date after Maths SM test.
Kota is now #1, Nikola is now #12 and Tantan is #17.
|
|
Administrator
Country : India | The LMI Puzzle Ratings are now up to date - after the Casual and Word PR test.
Neil moves up 6 places to #6, Walker moves up 4 places to #5 and Bram moves up 2 places to #8.
|
|
chaotic_iak
Posts: 241
Country : Indonesia | Can we have the link to the ratings placed somewhere visible from the main page, just like the top bar has a link to this thread? |
|
Hemant Kr Malani
Posts: 61
Country : India | chaotic_iak - 2018-05-19 1:41 PM
Can we have the link to the ratings placed somewhere visible from the main page, just like the top bar has a link to this thread?
On the main page, you can find the ratings on the right hand side under the heading 'Monthly Test Ratings' |
|
chaotic_iak
Posts: 241
Country : Indonesia | Whoops, by main page I meant the main page of the forum, because it's almost exclusively the page I visit. But that's better than nothing. |
|
Administrator
Country : India | chaotic_iak - 2018-05-19 9:47 PM
Whoops, by main page I meant the main page of the forum, because it's almost exclusively the page I visit. But that's better than nothing.
Main page bottom right has the links. |
|
Administrator
Country : India | The LMI Sudoku Ratings are now up to date after SM6.
Tiit is now #4, Wendy is now #16 and Kota remains #1 (gains 2 points).
|
|
Administrator
Country : India | The LMI Puzzle Ratings are now up to date after PR6.
- EKBM remains at #1 with 1000 points
- Walker remains at #5 and gains 11 points
- deu remains at #2 and gains 2 points
|
|
mstang
Posts: 74
Country : United States | mstang posted @ 2019-02-04 11:34 AM Will ratings be updated for the most recent PR/SM tests? |
|
tamz29
Posts: 225
Country : Thailand | tamz29 posted @ 2019-03-28 12:17 PM I think we are long overdue for a ratings update :) |
|
Administrator
Country : India | The LMI Sudoku Ratings are updated after Enthralling Sudoku Contest (September 2018)
- Kota is #1 with 995 points
- Tiit is #2 with 980 points
- kwaka is #3 with 977 points
|
|
Administrator
Country : India | The LMI Sudoku Ratings are updated after Deja Vu Contest (October 2018)
- Kota is #1 with 993 points
- kwaka is #2 with 981 points
- Tiit is #3 with 975 points |
|
Administrator
Country : India | The LMI Sudoku Ratings are updated after SM 2019 - Standard (December 2018)
- Kota is #1 with 993 points
- kwaka is #2 with 988 points
- Tiit is #3 with 977 points
|
|
Administrator
Country : India | The LMI Sudoku Ratings are updated after SM 2019 - Outside and Neighbours (January 2019)
- Kota is #1 with 992 points
- Tiit is #2 with 981 points
- kwaka is #3 with 980 points |
|
Administrator
Country : India | The LMI Sudoku Ratings are updated after SM 2019 - Substitution and Odd Even (February 2019)
- Kota is #1 with 991 points
- Tiit is #2 with 983 points
- kwaka is #3 with 972 points |
|
Administrator
Country : India | The LMI Sudoku Ratings are updated after SM 2019 - Math (March 2019)
- Kota is #1 with 985 points
- Tiit is #2 with 982 points
- jaku111 is #3 with 975 points |
|
Administrator
Country : India | The LMI Sudoku Ratings are updated after SM 2019 - Irregular and Hybrids (April 2019)
- jaku111 is #1 with 983 points
- Tiit is #2 with 982 points
- Kota is #3 with 980 points |
|
Administrator
Country : India | The LMI Puzzle Ratings are updated after PR 2019 - Classic (January 2019)
- EKBM is #1 with 1000 points
- deu is #2 with 950 points
- Kota is #3 with 942 points
|
|
Administrator
Country : India | The LMI Puzzle Ratings are updated after PR 2019 - Connected Puzzles (January 2019)
- EKBM is #1 with 1000 points
- deu is #2 with 959 points
- WA1729 is #3 with 946 points |
|
Administrator
Country : India | The LMI Puzzle Ratings are updated after PR 2019 - Shading and Loops (February 2019)
- EKBM is #1 with 1000 points
- deu is #2 with 959 points
- WA1729 is #3 with 950 points |
|
Administrator
Country : India | The LMI Puzzle Ratings are updated after PR 2019 - Casual and Word (March 2019)
- EKBM is #1 with 1000 points
- deu is #2 with 959 points
- Para is #3 with 954 points |
|
Administrator
Country : India | The LMI Puzzle Ratings are updated after PR 2019 - MII and Regions (May 2019)
- EKBM is #1 with 1000 points
- deu is #2 with 954 points
- Para is #3 with 952 points |
|
Administrator
Country : India | |
|
Administrator
Country : India | |
|
rajeshk
Posts: 542
Country : India | rajeshk posted @ 2019-11-26 11:42 AM It will be great if the ratings can be updated with the latest Sudoku and Puzzle tests :). |
|
greenhorn
Posts: 164
Country : Slovakia | Hi, I haven‘t been here for a while and I am surprised that the ratings have not been updated the whole last year. Is there any chance to fix it? The idea of players‘ rankings is great and worth continuing :)
Matus |
|
tamz29
Posts: 225
Country : Thailand | tamz29 posted @ 2020-10-10 12:07 AM I have the same feeling. When this aspect is neglected for more than a year it just seems no one's bothered.
It's a shame LMI can't maintain this. |
|
Gyuszi13
Posts: 17
Country : Hungary | Usually they are refreshing ratings at a start of a year and give a one-time update for the year as I observed it in the past few years.
I share rajeshk's opinion and it would be great to see how are ratings change from test to test both sudoku and puzzle. |