@ 2011-10-18 12:38 AM (#5822 - in reply to #5749) (#5822) Top
Country : United States
motris posted @ 2011-10-18 12:38 AM
There being a penalty, even for a puzzle a solver doesn't complete correctly, is meant to penalize guessing as otherwise a solver can make N "free guesses" and just stop when the puzzle value would no longer be positive if solved. I think there should always be a cost for trying an answer if it is incorrect - the question is how big a cost should it be and should any effort be made to use time or type of error to penalize typos differently from incorrect answers. Even using time is hard. One common error is transposition in a sudoku. I got one entry I remember like XXXXXXX12 and YYYYYYY12 where the correct answer has XXXXXXX21 at the top. This could have been either a puzzle error or a typo error, but it is certainly a small/quick fix error.
I got similar comments on bonus scoring from Melon. My motivations for the system were to have the "hardest" puzzles for a person - the ones they likely do last as they are worst at them - be worth more if actually finished because the flat value of points is not accurate for the difficulty for that solver. Even though Gapped Number Fill was the last, and unfinished by both you and Florian in 12-13 minutes, it was not that much harder a puzzle necessarily than others in the set (and let's agree it is impossible to make a perfectly balanced set even if a perfectly "average" solver existed). My test data had Wei-Hwa finishing it in 7-8 minutes where other puzzles took him longer (but then certainly were solved more quickly by others during the competition).
I think the compromise looking at the final results is a better system would have used 20/60 flat scoring and two 10/30/60/100 step bonuses, still 1000 total points. This would make the final puzzle worth 100 points (compared to 60 for earlier hards), so your score would not be that much higher - 896 instead of 866. But it would be a little less separated. I will say there are other ways you could have gone about solving 19 of 20 puzzles, and you certainly could have sacrificed any easy and completed that hard in my opinion. Only Murat actually attacked the test aggressively. Perhaps a larger point gap in the two types would have encouraged more solvers to go through more hards sooner.
@ 2011-10-18 11:51 AM (#5823 - in reply to #5749) (#5823) Top
Country : India
vopani posted @ 2011-10-18 11:51 AM
I really like Deb's idea of having Instant Grading during the end. Suppose Instant Grading is available during the last 5 minutes.
1. If a player has made a typo, it can be quickly corrected (provided the sheet is found! It might take a few seconds, but I dont believe this is a major issue).
2. If a player has made a solving error, it would be difficult to correct it before the time is up (this solves Para's point to an extent).
3. If a player has made multiple errors, it may not be possible to correct every one of them before the time ends.
4. In many cases, a player completes a puzzle 2-7 minutes before the end time and it is practically impossible to complete another one in the little time left. So, it can be fruitfully used to 'check' answers (in fact, the checking is done automatically).
I would be keen to see how this method works in an LMI test.
@ 2011-10-18 12:31 PM (#5824 - in reply to #5749) (#5824) Top
Country : India
debmohanty posted @ 2011-10-18 12:31 PM
My idea of instant grading only at the end is roughly borrowed from offline events where you are always advised to spend last few minutes checking the already solved puzzles, rather than starting new puzzles. So it is basically a review period, as Thomas put it.
However, I understand Palmer's view - it adds bit of complexity. Instant grading in this test was seamlessly integrated with the overall setup. Doing it at the end adds one more overhead on the players.
We probably can try it once to see how it works.
@ 2011-10-20 3:01 AM (#5825 - in reply to #5749) (#5825) Top
Country : United States
spelvin posted @ 2011-10-20 3:01 AM
My reaction to Instant Grading: It made the test more fun. In the sense that, any time I do an online puzzle competition (especially the USPC), I worry about whether I have typos. Should I double-check this string as I'm typing it in? If I already double-checked it, should I check again at the end? I don't really feel comfortable about anything I submit until it's officially confirmed, which usually happens later. With this competition, once I saw a green number I didn't have to worry about that puzzle ever again, which made the whole experience much less nerve-wracking and more enjoyable.
I also didn't have any incorrect submissions, so I didn't have the experience of making a solving error and being granted the chance to correct it. I can see why some top solvers think that breaks the purity of the experience, but I have to ask, should competitors' scores be more defined by what we solve or what mistakes we make? In the same sense, as a math teacher, when I construct exams, I am often torn about whether to write "trap" questions that deal with exceptional situations where rules work differently, or more straightforward questions. In one sense, the traps are important because I need to assess whether my students can handle those situations, but they also feel like I'm trying to trip up my students rather than educate them. In the same spirit, should puzzle competitions be built around deceptive paths designed to defeat the unlucky saps that fall for them, or around who can most quickly reach the correct answers?
There's a lot of unnecessary philosophy in the above paragraph, but the main thrust is that for me, this system lets solvers worry less about logistics and more about puzzle-solving, and that is a huge plus from my perspective.
@ 2011-10-24 4:00 AM (#5830 - in reply to #5822) (#5830) Top
Country : Canada
figonometry posted @ 2011-10-24 4:00 AM
motris - 2011-10-17 3:38 PMOne common error is transposition in a sudoku. I got one entry I remember like XXXXXXX12 and YYYYYYY12 where the correct answer has XXXXXXX21 at the top. This could have been either a puzzle error or a typo error, but it is certainly a small/quick fix error.That was me. That was a puzzle error. I always do that for some reason, usually with ones and twos.