Exactly Deb. My case is similar to X (9 puzzles, 12 bonus points), and Y solved 10 puzzles but got ranked below me (0 bonus points).

I'm not necessarily sure of the right implementation to fix the bonus problem. Here, I was "done" with the test with 21 minutes left on the clock, started to check my solutions, found my typo in the first mirror puzzle (I typed a 6 instead of an 8 so had two 6s in the string), had my next "done" time at 18 minutes and something on the clock, continued checking. Then checked some more. Didn't change any more solutions, so the 18 minute time lasted. In a live event, with that much time bonus (25% of round value) at stake, I would have done a first pass at checking as above and turned in my paper with 16 minutes on the clock, for less bonus than this test gave me. But that is the difference between online and live. Here there is no "done" button that absolutely ends the test. And I'm not sure there should be. But, playing devil's advocate, if there were such a button, then no one would be earning bonus if they were still trying to solve, and you could separate the cases of someone who made a typo but finished very fast from someone who was still solving but got stuck on one puzzle.

What is dissatisfying here as in rakesh's example, as was true with my Decathlon structure with "friendly" bonuses for completion, is the situation where someone submitting another answer could lead to them losing points and the main motivation should always be to solve more puzzles and not think about these meta-effects of bonus. The simplest system is therefore to not use time bonus at all, but accept raw times of last correct submission (ie when a solver reaches a score) as the tie-breaker. With that system, no one will ever stop solving to collect more partial time bonus. It doesn't deal with the solver finishing 30 minutes early with a mistake problem - which is what started the time bonus discussion here on Broken Pieces I think, if not from earlier Mock Tests in Sudoku - but perhaps the system should be triggered solely by that situation - 100% correct, or just 1 wrong. Because I think most of us are just trying to solve as much as we can, whether we catch our mistakes or not, and more complicated systems simply confuse a ranking in the ways we've run into in the last few tests.

Edited by motris 2010-11-25 8:02 AM

Yes, the partial bonus system triggered after Broken Pieces. Until that point bonuses were given if all puzzles were 100% correct, even during Sudoku Mock tests.
The primary intention was to handle cases where player has submitted all puzzles well ahead of time, but has a submission mistake (like yours in Broken Pieces)
With the recent friendlier bonus systems having undesired side effects, we probably just need to think about how to identify and give benefit to early-completions-with-submission-mistakes.

We should probably give partial bonus points if <= 10% puzzles are incorrectly submitted.
The major flaw in the current system is that it does not differentiate between "incorrect submissions" vs "no submission" vs "random guesses".
Authors and organizers should be allowed to judge what is an "incorrect submission", and what is not. This has to be a manual check. Given that only the top solvers are expected to get bonus, it will be fair to assume that they won't make "random guesses" just to get time bonus. So it really might boil down to differentiating between "incorrect submissions" vs "no submissions"

Partial bonus should be given only if the wrong answer is judged as "incorrect submissions".
No partial bonus in case of "no submissions".

No partial bonus if more than 10% of puzzles are incorrectly submitted, irrespective of how minor the mistake is.

Also, the difference between bonus points per minute in case of all correct and 90%correct should be substantial.

Adding a "Done" or "Claim Bonus" button (the equivalent of "Raising hand" in an offline event) should be fairly easy to implement from technical point of view. The "Done" button should be enabled after all puzzle answers are filled. Once the player clicks on "Done", the submission button should be disabled. The time bonus, if any, will be computed from when the player clicked on "Done".

Just like you, I'm not entirely sure, if we should have this feature. It probably adds one more overhead for players. As such players forget to click on Submit and have suggested to remove the Submit button.

Adding a "Done" button will probably complicate it further, although I believe it is something good to have.

This is a very good point. When watching submissions come in for Decathlon I remember one solver in the top ranks had a typo in a skyscrapers puzzle and finished with a fair bit of time but only 29 correct. He eventually corrected the mistake losing about 2-3 minutes to get to 30 correct (which had, due to the time bonus problem, a net neutral effect on his score unfortunately), but it would be obvious if he had not made the fix that this was an incorrect submission. It wouldn't necessarily get points, but it wouldn't be thrown out in a "this solver did not try the puzzle" sense if time bonus required an incorrect submission. The challenge then is having a good answer submission that can tell you if this is enough progress. A full row or two rows on a sudoku is certainly a good condition to attempt this test. The number of tents on a diagonal in a tents puzzle may not be. But it would be good to try this alternative going forward.

Edited by motris 2010-11-25 10:00 AM

I agree having a good answer key is critical for this aspect.
But even otherwise having a non-guessable and non-error-prone answer key is still required for all puzzles tests, irrespective of whether it is used for 'partial bonus' system or not.

FLIP Special Booklet contains the instruction booklet and puzzle booklet from FLIP along with a bunch of extra content.
Author notes and comments.	Bonus unused puzzles.
Test facts and figures.	Brand new bonus puzzles.
Neat purple design.	Full answer key.
Thank you all once more for giving me the opportunity to host a puzzle test and for participating! Grab the special booklet now!

Click here for SPECIAL BOOKLET!

Edited by davmillar 2010-11-26 11:27 AM

David, Thanks for the Special Booklet.
I'm yet to solve any of the extra puzzles, but I love your comments and notes.
Appreciate your efforts!

Deb