For puzzle tests, we've had more time than actually needed, so that more players get a chance to solve most puzzles. But we realize that it is not necessarily the best thing to do. In this test, apart from you, there is Sebastin who got Scrabble Loop incorrect. It could be a counting mistake or a genuine mistake while solving, but to lose 5+46 or 32+13 (out of 100) points because of one mistake is really too much penalty, in my opinion.

May be we should have 2 different targets (minutes) for top players and beginners. But I'm not sure how to do the points calculation etc.

I've often thought bonus points should be awarded even in the case of imperfect submissions, according to something like the following rule:

Calculate (Points earned - 5 * points missed) / (Total points), and that is the multiplier used to adjust from the bonus points the solver would have been awarded. Maybe the number in the formula should not be 5, I guess that is up to the organizers.

So for this exam, someone who makes a mistake on the answer submission (or is incapable of solving one of the puzzles) could perhaps go from 100% of the bonus points to (95 - (5*5)) = 70% of the bonus points. I believe the multiplier should be large enough to discourage solvers from skipping puzzles (since skipping a 5-point puzzle will cost you much more than just 5 points) entirely but not so large that the bonus points vanish completely due to one mistake or stumper. I also believe those of us who only solve a bare majority of the puzzles should be denied any bonus points, and this ensures that as well.

But enough about bonus points. This was a fun test and shows the true talent of Rohan and Tejal. Not only can they write sudoku puzzles but also these other varieties -- and entertaining puzzles at that! Thank you for yet another enjoyable test.

@Ziti: Very simple, but interesting idea indeed.

I was tempted to do the calculations based on the formula that you had suggested. Click here for the points with adjusted bonus. [As expected, only motris and Sebastian get significant bonus points ]

I think providing bonus point for these imperfect submissions will be correct. Thumps up from my side.

Would like to understand what others feel about this.

We're glad you all enjoyed the test. Thank you once again! :)
Just a little disappointed with the participation level, but as Deb said, it could be because of 5 other competitions during the same time.

This is not a rare occurrence; I missed The Sampler test but I recall in those results that janoslaw if not others had lots of potential bonus lost to one single mistake. I've made a whole blog posting on my stupid errors that cost me time bonus at WSCs and WPCs (http://motris.livejournal.com/93115.html). We tried to accommodate this contingency at our WSC as it is a lingering issue, but all our rounds were so well-timed that it never became relevant. Our concept was 5 points per minute without errors; 3 points per minute for being functionally done but with a mistake somewhere.

On an internet test though I haven't been as worried. This is because the "time stamp" lets you have everything submitted and then check over your work. On Evergreens, for example, I caught my two entry mistakes in the Hitori and Magnets, so while I sacrificed ~10 minutes of time, I got a clean total paper. Today, though, the answer entry itself failed for me since I was not looking for diagonal 90 degree turns and did not catch the problem on my double-check. Since the IB's count actually had such a case, it's my fault for not spending time to very carefully check the submission rules. Regardless, some contingency for time with mistakes should be allowed.

Unlike ziti's system which is still tied to puzzle value (the solver is losing the "extra" points already if the stupid error is in counting scrabble or entering vertical tents or missing a 90 degree turn), I prefer a fixed "cost" on the time bonus per mistake. At the WSC we were going with 60% but I could see on a test like this something like 11/12 correct = .8 points per minute; 10/12 correct = .6 points/minute; ... or something similar. What's important is that the value of the bonus per minute never overwhelms the value of solving the puzzles so that no solver will be motivated to be "incorrect" once or twice to get fractional bonus instead of just solving all the puzzles. Either ziti's system or mine seems to do this.

Edited by motris 2010-08-30 9:52 AM

The idea is indeed interesting and can be implemented easily. I like it too, but, in my view, it is probably against the spirit of bonus points and makes it trivial. Bonus points (so far) is a kind of reward that is given ONLY if someone manages to solve ALL puzzles correctly (in online tests, unfortunately, the most important aspect of this correctness is accurate answer entry and we have to live with it). With this system, there will likely be a few cases where X, who has solved one puzzle less, gets more points than Y, who has solved all puzzles. And that does not sound right.

There are two cases here:

(1) Mistake in answer submission: The participant should not be given bonus points, because he probably had time to re-check his answers for correctness. It is always an accuracy v/s speed balance. The participant took a risk by going for maximum bonus points. It is perfectly possible that those who have actually got bonus points have checked their answer multiple times before submitting.

(2) Incapable of solving one of the puzzles: The participant should try to solve it in the remaining time rather than try to gain partial points through this mechanism.

Another important aspect which was especially relevant to this test is the concept of penalty for wrong answers. Many of the answers were guessable, and we either need to have penalty points or more complex answer keys (not too complex, but not single digits either).

In fact, I had three puzzles left with one minute left. I submitted dummy answers for the three and got one correct too.There may or may not be others who may have done this as well. And, this should not be allowed.

Very valid point Rakesh. [ While I was testing the submission system on Friday, I entered dummy answers. And when I looked at the score page, I had got 18 ]
I would prefer non-guessable (or difficult-to-guess) answer keys to -ve points.

As rakesh mentions, having answer strings that are "guessable" requires either different choices for the submission or penalties for being wrong or both. Guessing a sudoku row is unlikely (oftentimes approaching 9! options), but guessing "0" in a puzzle that could only take 0,1,2,3,4,5,6 as possible entries, and maybe only 0-3 as likely entries is a problem.

Rakesh, your point is valid.
But players make silly mistakes, and to lose 51 or 43 points (out of 100) because of the silly mistakes is too heavy a penalty.
There are multiple stages to make mistakes
a) player solves incorrectly
b) player solves correctly, but computes the answer key as incorrectly
c) player finds the correct answer key, but types incorrectly (if it is a long answer key)
d) player forgets to click submit button :-)

I would think we should give bonus points (based on one of the formulas suggested above or any other) to players doing b/c type of mistake. [because only b and c type of mistake can happen in online tests like ours]
But to determine whether it is a-type mistake or b/c-type mistake becomes very subjective.
So we should give partial bonus points to players. As long as "value of the bonus per minute" does not exceed "value of solving the puzzle".

I think (b) should be treated like (a) - finding the answer key is (practically) part of the puzzle solving. For (c), if the organisers think it is a typing mistake and is a genuine case - go ahead and award the points for the puzzle; the bonus shall take care of itself.

I agree (b) should be like (a)
Differentiating a typing mistake with a genuine mistake is too subjective. Lets just scrap a/b/c/d.

Either we give partial bonus points based on some formula or we don't.

I am happy that world Champs mind and my mind work in the same manner.............He also did the same mistake as I did in the Zig-zag... LOL

Only 9 participants have rated the puzzles (out of 75 non zero scores)
Should we have some bonus points for rating puzzles also ?

it will be too harsh I guess ...
but we can think about not displaying the detailed scores if someone has not rated the puzzles :-)
OR
Not displaying scores which are higher than that player :-)
OR
encrypting all others players' user ids :-)

Anyway, I think its okay, if players don't rate. We should not force them.

Thank You: Nikola, Rakesh, Semax, Dave, Amit, Thomas, Jason and all participants.

Congrats to Hideaki, Ulrich and Takei!

I'm glad most of you'll enjoyed the test. The test could've been a little shorter (maybe 75-80 minutes as suggested by Deb). The low participation was expected due to other contests.

Apologies for the multiple-solutions Zigzag and the incomplete-instructions Tapa.

See you all at Evergreen 2!

Rohan.

Thanks Rohan/Tejal for this great test. I enjoyed solving all these puzzles. Nice to see more and more people creating puzzles. LMI provides right environment for showcasing not only your puzzles solving talent but also your puzzle creation talent. Three cheers to LMI! Looking forward for more such puzzle tests.

Thanks a lot! I truly agree with you and I'm sure there will be better and more exciting contests by Indians (other than you, Amit, Deb, Rakesh, the regulars...) in future :-)

At LMI online we try to design the test timings in such a way that an average solver is able to solve all (or most of the) puzzles in the test and thereby we expect some of the top solvers to get a time bonus.
Even at offline competitions it is difficult to estimate the correct time for a test given the rate at which sudoku solve timings have reduced over the past couple of years.

It was nice to see a discussion on the loss of bonus points due to small oversights.
In offline competitions we can do away with bonus points by introducing bonus puzzles like we have had in the Indian Sudoku Championships in the past and the WSC in Goa.
However since this is not possible for online competitions we have to devise a way in which participants do not loose a large number of bonus points due to silly submission mistakes.
Both Ziti and motris had similar ideas off giving weighted bonus points depending on the number of puzzles solved incorrectly and the number of puzzle points solved incorrectly.

The puzzle points are always related to the time taken to solve the puzzle, so we will compute the percentage of test completed as the percentage of puzzle points solved. The weights will be based on Ziti's approach and hence all participants solving more then 50 % of the test can expect some bonus points for time saved in the test.

Where can I get IB? The IB File is Broken......
(LMIMT_M201008P_IB ? The file is broken..)

Please check now.

Thanks.