I am trying to understand this part of the scoring instructions:

"Total score for a player will be computed by summing individual puzzle scores. For
players who submit all 10 puzzles correctly, their worst puzzle score will be
discarded."

I submitted them all correctly, but this sounds like I would have received the same score if I had stopped at nine puzzles. Or is it that the worst puzzle score is always dropped? Just wondering... either way, I had tons of fun solving!


Edited by jalbert 2012-01-26 8:32 AM

Depends on which puzzle you submitted last. e.g. Suppose you'd got a bonus in the last one you'd solved, and you've exceeded 1 hour in one of the previous 9, that one would be discarded and you'd have got a better score with the bonus. But of course if you'd exceeded the hour on the last puzzle too and got no bonus, then your score would be the same as it would've if you'd stopped at 9. Hope that clears it up. :)

Finally completed it all.. :)
It was fantastic solving all these puzzles.
my favourite puzzle was kakuro. I learned lot while solving that.
Messed up in 4 of the puzzles otherwise where i could have easily fetched up bonus..
Anyways nice ending..
Thanks to all the authors,LMI and Deb.
Will be waiting for Marathon2.

Well done in Tapa, Loop the loops and especially in Different Neighbours. I expect to you to get 10/10/10 in M2.

First I want to thanks to all organizers and authors for this excellent competition. All puzzles were great and I spend a wonderful time solving it. Yes, Kakuro drove me mad but I finally solve it, and I was more proud to myself to do that then if it was easy. I'm also happy for 2 reasons: for solving all puzzles and getting few bonuses (I'm average solver).
Difficulty of this puzzles was perfect. There are many people who already solve all puzzles, many of them get bonuses. I think this should be the point of competition which lasts for a week. Best solvers will spend more time than usually (in other online competitions) to solve puzzles but they will get bonuses and average solvers also could get bonus for some easier puzzles. And if some puzzle is too hard for you to solve it in 1 hour, you still have all week to solve it and get points. If there will be more competitions like this in the future (I hope it will be) there is no need for making easier puzzles. People will solve it too fast and then there's no point to make one week competition.

Thank you,
Will give my best..
and ya most importantly avoid my carelessness.
:)

re large and hard puzzles being too hard - from an author's point of view I think it's nearly impossible to please everyone. When I put together the nikoli selection last July, I retained the idea of larger puzzles as a bonus because I knew the best solvers of nikoli puzzles would be miles ahead of the average solver in terms of time, and would have a fighting chance of getting 1 or maybe 2 (or as I was shown, 3 if you happen to be Hideaki!) done within the time. The test-solving times for each of those 3 puzzles averaged ~50+ minutes! This didn't really matter to me because for mere puzzling mortals the marathons would be a nice extra to complete out of time.

I don't think its appropriate to go into any sort of detail whilst this contest is still open, but for the kakuro I would say the fastest times are exactly in line with my expectations, and there are enough people scoring bonus time to justify the 1 hour bonus window together with the "hard" label. I suppose that's just my opinion though.

The one issue with the current scoring system as is see it is this. Solvers are essentially rewarded for excellent solves well under the par time, without being disproportionally punished for making a fiddly mistake, or simply having an off solve. There was one particular puzzle I was quite grateful for the opportunity to go out into town and get some shopping before finishing it off later in the evening. This is all well and good, so long as for each given puzzle the chance to affect your bonus is about the same, accounting for an individual solvers differing abilities across different puzzle types. This is a massively complex thing to try and encapsulate and measure such inexact science because different people solve different puzzle types in widely varying times.

Perhaps one solution for next time is to have different bonus windows for puzzles of objectively differing difficulties - for example 45/60/75 for easy/medium/hard? Even then, this seems problematic because you don't want one or two puzzles deciding the whole contest because there happen to be only a handful of solvers who are way ahead of everyone else - in this case you would be handing a free 30 points to people who are especially good at sudoku and kakuro (which is clearly a non-trivial overlap!).


Edited by detuned 2012-01-26 8:45 PM

This seems a bit arbitrary and not a complete fix. While the "easy", "medium", and "hard" labels are rather true to my experience, the breadth of styles means for most solvers it will vary and the top times are telling that the labels aren't perfect. I think the best change is to use either rank on a puzzle for bonus scoring purposes or normalized time (like croco-puzzle). Then easy and hard can be compared, and certain puzzles won't end up being dropped just because they are harder than the rest. Thankfully we will have a lot of solving data with this competition as it has been very popular, so we can model what particular scoring systems would look like once it is finished.

Best of luck and skill to all those still competing.

As i see, the objective of this format is see if the solver can do the large puzzles in the first place.The difficulty indicators are certainly misleading for some of these.I really like the idea of using solving time as a primary indicator,as the difficulty only really points at the pleasure aspect.One can construct a very easy huge puzzle.On a negative note,the scoring system cannot be played much with unless
you are willing to make it a lot more informed.

I did all 10 puzzles, and eked out 57 bonus points, but my total score only shows as 957, and not 1057. Am I missing some element? Thanks. as always for being there in so many ways.

TheSubro

I suppose your worst puzzle has been discarded? Its best 9 out of 10.

Missed this post earlier.

It is BONUS/CORRECT/STARTED

Well done Deb, a great concept and I hope we see something like this again.
Many thanks to the authors for 10 quality puzzles!

Really good format- it was nice to be able to spend as long or as little time in a row solving as you wanted, instead of having to find a 3 hour block of free time.
I would say that in my view, it makes sense to try to get the puzzles all at similar difficulty levels- here, someone who messed up the easiest puzzle was penalised a lot more (relative to other solvers) than someone who messed up the hardest puzzle. But I acknowledge that that may not be feasible.
One possible amendment to the scoring, to avoid that issue, could be: fastest solver gets 0 points. Everyone else scores a point for every second longer that they take, up to a max of 3600. Obviously lowest total score wins, so there must be a penalty for not solving a puzzle at all (perhaps 4000). The only problem would be knowing when there is no chance of getting any extra points, and hence being able to take a break (though I think you could assume that Mr. Snyder will finish everything within 30 mins, so you'd never need to continue on for longer than 90 mins).

Finally got back around to trying that Braille Word Search...managed to nail it, would have been quicker had I only noticed one word sooner (and if not for a five-minute interruption phone call).

I started thinking about converting Pennypress puzzles (American publisher) to Braille for fun. Realized it doesn't work exactly that way; in effect, it's almost like three separate small puzzles merging to become a single puzzle, with each horizontal row taking every third line in order.

Cyclone

Edited by Deb to remove the exact word. The test is still running :-)
Further edited by CycloneGU to replace what looks like bad language. Seriously, though, does noting the word here make a difference other than indicating that one word is in there? =)

Edited by Cyclone 2012-01-28 10:33 PM

It probably doesn't.
Since the test is still running, I thought it is best that we don't reveal "a part of the puzzle". After the test is over, I would have un-edited the post, like I've done to several posts in past tests.

Thanks James and Nil for your feedback, and congrats for good scores.
We are certainly going to repeat the format, so we'll discuss about potential changes in the scoring system in the end.


Message to all players who have stopped solving at 7 or 8 puzzles. Please note that the rank that you see now in the partial score page will take a huge dip, you don't get to at least 9 puzzles. You still have 30 hours to participate further.

Was it really mentioned somewhere before the contest? I haven´t seen it until now. I stopped solving seriously after spoiling first puzzle.

Meh, I'll edit it back in myself later.

Or you can.

I suppose it would be wrong to mention that the Samurai Sudoku uses the digits 1 through 9? (Note I haven't solved it yet, or tried...will this evening perhaps.)

Random question. Why are typical smiley codes not used here?

Cyclone


Edited by Cyclone 2012-01-29 12:47 AM

See the Instruction Booklet "Scoring" paragraph, at the end, "Players who solve all 10 will have their worst puzzle discarded" or something like that.

Edited by prasanna16391 2012-01-29 12:59 AM

It is mentioned in "Scoring" section in the IB. See last line. And it was also mentioned in forum, while answering to kiwijam's questions.

The forum didn't have all smileys inbuilt. We added some smileys, but yes, it is not complete. Some other forums (e.g. phpBB) have a more exhaustive list of smileys.

To "solve" the instructions is always the hardes puzzle for me

Just like you should always solve the example of a new puzzle type, you should also "solve the instructions" of a new type test

Phew! Just finished! It was tough but enjoyable. Thanks to Deb and the team for the huge amount of work done and for a very attractive formula. I notice that in spite of the length of the contest the distances between players are minimal (which means that the 10 minutes I trew away stupidly on one puzzle for not reading the instructions costed me several positions....). Also glad I could end in bonus position on 9 puzzle out of 10 (well, I like to leave space for further improvement for next time....). Congratulations to all those which finished at the top, and relized times which are a bit embarrassing for the average player....

The formula is almost perfect. I feel I can support an improvement among those proposed. I do agree with detuned, when I says



perhaps next time the most difficult puzzles will not be of the same type of this time.... however I feel that life has been particularly cruel for a player which solved one of the difficult puzzles in, say, 59'55" and got 0.1 bonus points, i.e. only .1 point more of a player who solved it in 2 hours, particularly if only a handful of solvers could finish that puzzle with the bonus. so the formula 45/60/75 (and perhaps 90), although arbitary, is a good idea and I support it.

while I do not agree with Motris

The formula relies on the possibility, which many players have used, to stop solving a puzzle, putting it in a corner and solve it the next day or later once the possibility for the bonus has been lost. In this case the concept of normalized time is heavily affected and meaningless because the number of player doing this will vary from puzzle to puzzle, depending on the difficulty. Also, a player needs to know in advance the criteria of awarding the bonus.

I renew my thanks to the organizers and authors and look forward for the next one of these.
Stefano

Stefano, I was trying very hard to not give a specific formula, because I think it is more important to speak to the goals of a scoring formula and match them to the test. As a contest constructor I will say that it is very hard before the test, when you've seen just 2 or 3 solvers, to put a fair value on all of the puzzles, particularly with complicated bonus. Also, when having so many constructors (7), it will be very hard to have consistency of difficulty. To say puzzle X should be 75 or 45 might be better, but it might not be correct. And if the puzzle you made into a 75 turned out much easier than a 60 then you are really screwing up the test results by using a guess, and not a more robust formula.

A simple formula might be "top time + 30 minute window" gets some bonus, like what Nilz mentions, which would still have an effective one hour cut-off. I personally prefer something like this:

Solvers in the top 10th percentile earn bonus. Specifically, the top solver earns 100 + X points, solvers in top 10th percentile earn bonus proportion to time gap from 100th to 90th percentile. Solvers at 90th and below get 100 points. With the results I'm seeing on this test, there still is a stable "one hour is too long, so you can start over whenever" effect, and actually for many puzzles the time is now more like 40 minutes. But now you let the competitors scale the times, not 2 or 3 test-solvers.

I'm a bit frustrated that my best relative puzzle performance on this test is the puzzle I have to drop. I'm not frustrated enough because of how I did on other puzzles, but I want a good system that works in general for this kind of test but maintains the goal of removing pressure after one hour. I probably should not have mentioned croco-puzzle specifically, because I did not mean a 3000 to 0 scale. I meant something more like a 3000 to 2700 scale where every solver below 2700 goes up to 2700 whenever they finish. I hope something like this would meet your approval, because my goal is not to ruin the benefit of having comfort to sleep on a mistake and start again the next day.

Edited by motris 2012-01-29 11:49 AM