How balanced do you think the puzzle types of this test were?	A bit skewed
What was your opinion of the distribution of easy/hard puzzles?	Too many hard puzzles
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Way too many puzzles (too little time)
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	Many puzzles were worth too much or too little
What was your opinion of the booklet formatting and printing?	Just right

An excellent test in continuation with the other Grand prix rounds.All the sudokus which i solved had a very nice hard solving path.
However there are one or 2 things that i wish to bring to the notice of the respected authors.
Points Distribution :
* I dont think the point distribution was fair at all.It was very skewed.It could have been way better.For example,i dont think the quadruple was worth 32 points at all .It was certainly much more than that(atleast 60) compared with how other sudokus are weighed.And the surprise factor in Renban makes it surely deserve much more than just 36 points.By surprise factor i mean how the shaded cells were present extending into other regions.Weighing one sudoku with 157 points and the rest except 2 , not given even half the points is surely not the way to go .
So a person who solves just 2 and sits back and enjoys stands to gain more than a person who has sweated it out and solved 8 or even 9 sudokus.Now Whatever be the difficulty of those 2 sudokus this should NEVER be the case in any test.
Consider this situation.Both A and B represent the same country in the olympics.
A wins 1 Gold medal and 1 silver medal in 2 individual events(does not take part in other events)
B wins 5 silver medals , 3 Bronze medals in 8 individual events.
Now can person A be given more recognition just because he has topped a single event??That seems to be the case here in this test which would reflect on the standings and not give an accurate picture.Mind you i am not trying to say that B should be given more weightage .All i m trying to say is that the balance should not go unfairly down on one side .
Rather it should be more or less in the middle (which could have been done surely by slicing points from the 100 pointer ones and distributing among the rest)
*Time frame :
I know the time frame is only 2 hours which would be convenient for everyone and which has been the case in all the Grand prix rounds so far.But for a test such as this which has a killer worth 157 points,another killer worth 108 points and another one worth more than 100 points,the test should surely be close to 150 mins (atleast).
I was heart broken that i could not enter the solution of the diagonally consecutive inspite of being on the threshold of completing it which surely would not have been the case if i had prevented myself from being a Sherlock Holmes and trying to figure out how the Little killer would proceed (Wasted 10 mins and decided that this was surely not in my reach).
I hope my suggestions are taken in the right perspective.I also hope from the next rounds the point distribution would be better.The above 2 points that i have raised surely dont take away anything from the quality of the sudokus created.The sudokus were simply stunning and stand testimony to the quality of the excellent authors involved.
I really enjoyed the test.
3/3 stars!!!

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	All puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Fairly Nice
How did you feel about the length / time limit for this test?	Way too many puzzles (too little time)
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	Most puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Too few pages / too small grids

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	A bit too many puzzles
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	Most puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

About 15 minutes to the time limit, I started submitting answers, but I couldn`t send my solutions, because the website blew out. I was trying to refresh the website, restart the browser etc., but the situation was still repeating. I lost all answers, that I wrote out. After few attempts of repairing the malfunction, the time finally ended. I don`t know how can I protest, because I cannot prove anything, what I have just written. What will happen with my results, which currently equals 0?
Jan Mrozowski

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Too many hard puzzles
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	A bit too many puzzles
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	All puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	Most puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

This has happened to me before too. I lost a few minutes then as I had to close my browser and reopen as refreshing fixed nothing.
I would have suggested emailing of PM-ing Deb all your answer keys at the moment it happened, so he could confirm your time stamp and adjust accordingly, but I guess that is now too late. I don't know if Deb would be happy with that, but in my mind that would be the best option next time.

How balanced do you think the puzzle types of this test were?	Very skewed
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	A bit too many puzzles
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	Most puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	Most puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Fairly Nice
How did you feel about the length / time limit for this test?	Just right
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	All puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

How balanced do you think the puzzle types of this test were?	A bit skewed
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	A bit too many puzzles

How balanced do you think the puzzle types of this test were?	A bit skewed
What was your opinion of the distribution of easy/hard puzzles?	Too many easy puzzles
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	All puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	All puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	All puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

I think your question should be focused more on the spread of difficulty of the puzzles than on the specific scoring. In other words, the proper constructive spirit of your points are: Should a test have a puzzle that takes five times as long as another? Should it then be scored so that it is worth five times the other?

The first is really the question of test balance. The second is just the organizers determining what is fair. There is a lot of variation in how people view the first question but I don't believe there is much variation with the second. If, in a timed heat, something takes twice as long to do as another task it is most often awarded twice the value without any question. The "points per minute" value is meant to be close to constant.

All test authors make their best attempts to score a test based on the solving data they get from both internal and external test-solvers. While these scores and entry rows are not completely based on my testing, they were definitely influenced by my testing results. So let's just say each puzzle is worth in points about 10x my solving time in minutes. If a puzzle takes me about 3 minutes (quadruple) and another takes me over 15 minutes (Little Killer), then scores of 32 and 157 reflect the spread in the difficulty of those puzzles for a top-level solver. Similarly the 36 point Renban took me close to 3.5 minutes, "surprise" and all. I was not surprised. Perhaps my prior knowledge of tennis and my expectation the Serbian test would use it a lot meant I more quickly grasped that A, T, P made sense as the three regions. Perhaps I'm just good at testing surprise puzzles. The Indian Grand Prix test similarly had complaints about the Even/Odd/Big/Small sudoku. It was not a surprise to me. The score given was pretty reflective of my time and other testers.

Perhaps more testers would help. But having so many solvers want to compete in the Grand Prix means there are few options out there. And recognize that even with a lot of testers, the scoring will reflect how solvers gets through the puzzles on average but cannot possibly represent how you individually will solve the puzzles as you will have your own strengths and weaknesses. If you keep finding that you are spending too much time on under-valued puzzles and never solving the over-valued ones, you are probably taking the wrong approach to these tests and could improve simply by changes in test-taking strategy. Understand, in other words, there is a difference between "over-valued in general" and "over-valued for you".



As a sports fan in addition to a sudoku solver, I can't let this go undiscussed either. Your Olympics analogy seems flawed in that it treats all events as equals. Some competitions involve performing 1 thing for 30 seconds and others involve performing 10 things over 2 days. I do view a gold medal in team basketball quite differently from a gold medal in the 50m butterfly and both of those quite different from a gold medal in the Decathlon. This is why I do not put much credance in the "nation weighting of medals" as there is no real way to equate all medals from quite different competitions. What is essentially different here is we are not comparing the results of an archer with those of a marathon runner.

No, here we are comparing the results of several hundred solvers on a 16 sudoku competition for which they have 2 hours to compete, and the puzzles have varying difficulty indicated in a fair -- albeit impossible to be perfect -- way with variable scoring made known to all solvers in advance.


Edited by motris 2013-04-21 10:48 PM

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	Most puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Fairly Nice
How did you feel about the length / time limit for this test?	Way too many puzzles (too little time)
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	Most puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

Dear Serbian friends, thank you for a such great puzzles! I really liked the tennis theme. During solving I discovered about 10 secrets, and have a great time after competition when discovering the other ones. Converting the ideas to ALL (!!!) puzzles is really unbelievable, specially letters ATP in movable digits. Thank you once again.

In the name of all authors, thank you for kind words.
Great reward for our work.

Branko

Jan,
I sent to you PM.
Please respond.

Branko

Since Thomas' post covered the rest of your feedback, I'll just address this part. The time frame is again (usually) decided on the basis of the testers' timings. Although, here, the time frame's been set to 2 hours for uniformity and I assume the tester's times were used to confirm that the set falls within that time frame. It is fine tuned (usually) so that the top solvers can come close to or just finish the set. Looking at the score page, this set has achieved that goal. I don't see why puzzles being worth more points means the test should go for longer. But I didn't see why the test is imbalanced either. There's all kinds of points from 20, to 30s, to 40s, to 60s, to 70s, and I think the really tough ones should be seen more as a way to differentiate the top solvers, unless they're particularly your strength. The last sentence, well, I'm sorry you couldn't submit on time, but that happens to many solvers in every test, no matter what the time constraints, so that isn't really something time frame will change.

And, thanks to the Serbian authors for a beautifully themed set of puzzles. I had the unfortunate experience of breaking the 2 highest pointers pretty late, and so skipped those two, but overall I really enjoyed the test. I'll post a few more detailed comments after I redo those and others so I know where I went wrong.

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	All puzzles were worth the right amount

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	Most puzzles were worth the right amount

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Fairly Nice
What was your opinion of the booklet formatting and printing?	Just right

Do not really feel qualified to judge contest. I thought I was pretty good at Sudoku until starting the Grand Prix. I enjoy doing it but seem to be completely out-classed. Where can you get experience (try, fail, learn) on these type of 'non-standard' - at least compared to "expert" book/ newspaper puzzles without quitting my day job? This is not a complaint against any contest. I appreciate the challenge being offered - and obviously others do not find it too difficult. Keep up the good work challenging us.