How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
What was your opinion about the answer key extraction?	Mostly perfect answer keys
How did you feel about the length / time limit for this test?	Just right
What was your opinion of the booklet formatting and printing?	Just right

I agree with that comment, it is good to have all kinds of contests. If we go by the poll results here, there are certainly many takers of Sprint tests. Unfortunately, we just had only one more sprint test in 2012 after the first one.

While Marathons are comparable to Sprint tests in terms of finish-ability, each kind of contests has its own fun and challenges. So I don't think one of them should/can replace the other.

For your information for the last 15 or so months that i have been a part of LMI ,I dont remember any sudoku test in which the time duration was for 60 mins except LMI Screen Test.I had expressed my feedback there too though for a different reason.So it is not that in only puzzle tests that feedbacks of these sort occur.

And please dont equate sudokus and puzzles on an equal platform.If you call say a random person from a group of persons and ask him to choose from a sudoku and a star battle or a sudoku and a fillomino or so on ,it is very probable that he/she will possess the skills required to complete the sudoku rather than the puzzle.This is because sudokus are universally popular and well known.So basically what i m trying to put forth through this is that given an LMI test with a short time duration ,it is more likely that people will solve a lot more sudokus than a lot more puzzles within the time frame which in turn minimizes the short time factor to an extent in case of sudoku tests not to mention the fact that it has not occured so far in a LMI sudoku test.I'l clarify this too.My feedback has got nothing to do with the quality of the puzzles which were on an absolutely high level as usual.It is only the timing of the test that has let me down.

That's what I like to hear. I am surprised that wasn't more of these. In fact, that one happened long before I started doing contests on LMI.

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
What was your opinion about the answer key extraction?	Mostly perfect answer keys
How did you feel about the length / time limit for this test?	Just right
Of the puzzles you solved/attempted, how well did the point values reflect the difficulty?	Most puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

Well, there is none planned at the moment as well. That also means, potential test authors know what they can target to write.

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Fairly Nice
What was your opinion about the answer key extraction?	Answer keys could have been better
How did you feel about the length / time limit for this test?	A bit too many puzzles
Of the puzzles you solved/attempted, how well did the point values reflect the difficulty?	Most puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

To make it clear, my comment was not particularly about this test. I think any test with instant grading can last longer than usual to have more fair ranking of the average solvers. If instant grading were not available the current duration of the test would be perfect, because making it longer would cause early finishers to risk losing a lot of bonus points due to possible mistakes. However, I don't see any disadvantages of making the test longer when instant grading is available.

Just my two cents:

Team USA had a similar discussion to this at the recent WPC -- how rounds can feel much friendlier and fairer by letting more solvers finish. We agreed that an underappreciated feature of Instant Grading is that it really means you can now run any test indefinitely with some form of "Twist" scoring (here, that might be 80% after 60 minutes, 50% after 90 minutes) because solvers are entering solutions throughout the test.

I would not have minded this test running more like the WPC Mini Marathon or LMI Puzzle Marathon tests which seem to be the best solving format for all solvers from beginners to experts. It certainly removes some of the frustrations from puzzle selection, puzzle scoring, and round timing. Of course rules decisions should be left to the authors and LMI organizers, but I wonder if a general trend towards "friendlier" puzzle sets and rules is one way to help grow the audience of competitors.

Edited by motris 2013-10-28 2:09 AM

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
What was your opinion about the answer key extraction?	Mostly perfect answer keys
How did you feel about the length / time limit for this test?	Just right
Of the puzzles you solved/attempted, how well did the point values reflect the difficulty?	Most puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
What was your opinion about the answer key extraction?	Mostly perfect answer keys
How did you feel about the length / time limit for this test?	A bit too many puzzles
Of the puzzles you solved/attempted, how well did the point values reflect the difficulty?	Most puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
What was your opinion about the answer key extraction?	Mostly perfect answer keys
How did you feel about the length / time limit for this test?	Just right
Of the puzzles you solved/attempted, how well did the point values reflect the difficulty?	Most puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

Beautiful theme! Lots of the puzzles were very nice to solve, too, which I know is challenging when the theme constrains the placement of the givens so much.

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
What was your opinion about the answer key extraction?	Perfect answer keys
How did you feel about the length / time limit for this test?	Way too many puzzles (too little time)
Of the puzzles you solved/attempted, how well did the point values reflect the difficulty?	Most puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Average
What was your opinion about the answer key extraction?	Perfect answer keys
How did you feel about the length / time limit for this test?	A bit too many puzzles
Of the puzzles you solved/attempted, how well did the point values reflect the difficulty?	Many puzzles were worth too much or too little
What was your opinion of the booklet formatting and printing?	I have a different complaint

For the booklet complain :
the grid was to dark for slitherlink, rectangles and fillomino, it was really hard to draw visible borders onto the printed grid.

Certainly agree with that. In fact we recognized all 3 images to be a problem, but didn't get the PB corrected before the test started. I should have notified Matej earlier. Sorry about that.

Yay for Top 20 finish :D /end self bragging

It's interesting that this test has both "Way too many puzzles" and "Way too few puzzles" at the same time. (Also "Too many easy puzzles" and "Too many hard puzzles".) Hm...

Edited by chaotic_iak 2013-10-29 3:25 PM

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
What was your opinion about the answer key extraction?	Mostly perfect answer keys
How did you feel about the length / time limit for this test?	Just right
Of the puzzles you solved/attempted, how well did the point values reflect the difficulty?	Most puzzles were worth the right amount

I think you are missing something here Prasanna, and that is how the points are distributed. Puzzle selection for a competition can indeed come down to luck if you have high variance puzzles (often the case on monthly tests as authors want to show off novel designs and tricks) or you haven't taken into account a sufficient number of testers. I think that when puzzle selection has a bigger impact on ranking than overall puzzle solving skills then this can cause a lot of frustration, and it would be a mistake for anyone running a competition not to at least consider this issue.

My own personal preference when running a competition is to have more like 20 finishers, and to make sure there are no puzzles with "outlier" levels of difficulty/points distribution which potentially shake things up if you don't solve them or if you attempt to solve them and waste lots of time scoring 0.

I have to say I didn't really find that too much of an issue with this test, which I enjoyed (despite having to solve in paint); my comments are of a more general nature.

Well, if the criticism was about points distribution being off I wouldn't have argued, because most of the time thats down to personal preference. The criticism I argued with was basically that this test was not timed properly.

Obviously it could come down to luck if the distribution is not done right. Points distribution is always an important aspect of organizing a contest. But that comes into relevance only once the choice is made by the participant. Its unfair to say there's luck involved in the selection process itself. You know you are choosing a puzzle thats "supposed to be" high/low difficulty based on points assigned. If it then proves otherwise, its the points distribution that can be criticized, not the timing of the test, which pretty much followed the usual Monthly Test standards (whether the standards are ideal is an entirely different topic) and any deviation was on the easier side.

It's simple, really. If you scored well on it (or even that if you have the impression that you did well on it, as I have personally found), you will find it easy and say that the test should have more puzzles. If you did poorly, you will say the test have too many puzzles. While more experienced contest takers will try to factor out their personal performance in their review, this process isn't perfect. To the more casual contest takers, they aren't going to consider that at all. If they only managed to do 1/4 of the set, of course they are going to say there are way too many puzzles.

Edited by achan1058 2013-10-29 10:17 PM

Hi everybody,
I am really pleased that Matej fulfilled his promise to prepare a set of puzzles that would make us happy. He often tried to put smiles into his puzzles which sometimes makes me worry because the level of difficulty is closer to tears than to laughter.
In this set, most of the puzzles were human friendly, even though Akari and Masyu where too easy, but in comparism with Kakuro, some easier puzzles were needed. Dispite all, I was not able to finish kakuro within the original time of 60 minutes, and I was a little scared, because only solving the rest of the puzzles took me 58 minutes or so and this does not include printing and submitting answers. That is why we discussed the time limit and finally Deb and Matej have come up with compromise of 75 minutes lenght. From my view, this seemed to be ideal, because 10 puzzlers were able to finnish and 59 puzzlers got the half of points. If we let the initial time limit, Thomas would be the only person to finnish and the rest should have feeling that they participated in badly timed sprint test.
I agree with Deb and some of you, who mentioned the Instant grading, which allows to have a longer competion, but in this case I do not think that adding 30 minutes (instead of 15) will improve the image of this competition - competion full of great and smiling puzzles, do not forget. If some of you were attempted to solve the kakuro and failed, it is not because of bad timing (remember that 10 players have no such problem), but because of revaluation of your skills and bad tactics, which possibly makes the main difference between average players (as me) and the top-solvers.
At the end, I want to express admiration to all contestants, especially to those who finished all the puzzles in time. I am only concerned about the low participation of Slovak puzzlers due to Slovak Puzzle and SudokuTeam Championships held the same weekend. Hope you enjoyed Matej´s puzzles.

Matus

Very nice

compe tition
Thanks Matej
Jirka

This is what I have intended :) J.

OK - lets leave timing of tests out of this discussion, I'm interested in the claim that there is no luck involved in puzzle selection for a test which won't be finished by all but a handful.

I have to disagree with this, to some extent.

For example, what about high variance puzzles? You haven't addressed this despite the fact you will almost certainly have at least one or two of these on every monthly test. It's an interesting topic by itself trying to grade those.

You have to take into account that different people solve puzzles in different ways, and high variance puzzles can amplify the problem. Suppose you had four testers of roughly equal solving ability, who test solved a puzzle in 2 minutes, 5 minutes, 7 minutes and 8 minutes. How many points do you assign the puzzle? The median of 6 minutes seems a good choice, but this is clearly a puzzle which has some intuitive shortcuts, and if you happen to be the right kind of intuitive solver on the test and choose this puzzle, I think it's impossible to argue that you won't be receiving some pretty fortunate points.

The trouble is, if you try and adjust the points total down slightly then you'll probably be undervaluing the puzzle for a lot of other people. Of course things can work out in exactly the same way in the other direction, which can obviously be more frustrating if you you're on the wrong end of things.

I agree with all of this actually. A puzzle, once chosen, can have lucky/unlucky results, depending on solver's strengths vs required logic and a whole lot of other factors.

BUT, again, my main point is that this has to do with points distribution, and something that can be noted only after the puzzles are selected and solved and give out unexpected experiences. It is possible there will be some luck in the puzzle selection. It is also possible that there won't be and all the points are actually pretty accurate, at least for most individuals. The point is, as a solver, no one should go in "expecting" a puzzle to be wrongly valued.

The context here means I wasn't saying luck will never be a factor. I was replying against it being a given that luck will come in during puzzle selection. But, I hope you agree, IF the points are assigned right for an individual, then puzzle selection for that individual becomes more about having their own strengths and strategies. And as a solver, one should go in expecting that to be the case.