How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	Most puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

Thanks to the puzzlemakers! It was fun to discover thematic secrets - after the contest, of course.
Thanks to all the participants! It is a big pleasure to browse through the result list again and again. I haven't seen my name on the top for quite a long time.
Looking forward to the next sudoku competitons and to the next tennis battles between Serbia and Czech Republic in Davis Cup and Fed Cup...
Krtek

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	Many puzzles were worth too much or too little
What was your opinion of the booklet formatting and printing?	Just right

Thanks to all contestants for the feedback, the praises and criticism alike (it will be silly to expect that one can make all of the people happy all of the time :)).

Nevertheless, I would like to mention two things.
Firstly, there is no such thing as an ideal competition. Secondly, the instruction booklet did read: "Points typically indicate difficulty of the sudokus and time required to solve them. While the organizers have made best efforts to match them, your personal experience and preference may differ."

Whether "To make it worth less than 157 just so the scoring overall looks "balanced" would be to grossly misrepresent the challenge it presents and cause even more frustration from solvers"is the case remains to be seen.In fact i feel that the real frustration creeps when knowing that after attempting 8 or more puzzles with full focus and concentration you get to know that
you are beaten by a person who has completed just 2 puzzles which might be his forte and which might incidentally have carried high points in the test.
Also i dont think the issue of point distribution needs to confused with the spread of difficulty at all.I dont think i have complained anywhere about the difficulty of the puzzles as such so far.The spectrum of our discussion is whether a puzzle deserves so much points because it is that much more difficult.I certainly dont think that just because a puzzle
might be X times more difficult than another puzzle in the same test it should be getting X times more points than that puzzle.This idea
seems absurd to me especially because going that way there is no CEILING LIMIT to the points that a particular puzzle can take.

Let me give a situation where the idea that a puzzle should get points corresponding to the difficulty that it has or corresponding to the time that a test solver takes without a ceiling limit to its total value is true.
Take the case of Puzzle marathon.The above idea might just hold true over here.This is because in such a scenario as this,
i)The Time factor is taken our of question with no fixed time limit under which you are supposed to solve a puzzle
ii)Each puzzle does not have any 'relative' or 'cascading' impact on any other puzzle.
iii)Each puzzle can be started,solved independently on its own and has the right to carry whatever points be it 200 or even 500 ,
depending on the presumed difficulty.

Now does any of these cases arise in a timed test such as the one in discussion?The short answer is a NO.If you get tripped on a puzzle you
are doomed .The time that you lose can never be obtained back.
WHEN the tests have a fixed TIME LIMIT and a DEFINITE TOTAL NUMBER OF POINTS,it just does not seem right to me that the puzzle can carry points in direct proportion to the time that a test solver takes or based on how many times it is more difficult compared to another puzzle without a LIMIT to the total points that it can carry.
Maybe it would not be a bad idea to have a survey asking participants about what they felt regarding the distribution of points in this test,
whether they see it fit to not have any restriction on the total points that a single puzzle or a small group of puzzles might carry.............................

My error was that I had only reviewed the second page of posts. I did not review the first page. As I saw the puzzle addressed on page 2 and I still didn't understand the rule being conveyed (which I see now was on page 1), I took that as the information I used when posting. Clearly I was in error. I'll give both the sample and competition puzzles a shot today knowing the rules better now.

Cyclone


Edited by Cyclone 2013-04-23 8:25 PM

The points distribution was very near to reflect the real difficulty of these puzzles.

Puzzles with 20-30 points was solvable within 2-3 minutes, the harder puzzles needed a few more minutes. On the WSC there are also some harder puzzles and nobody is discussing it, because the goal is to sort out good/better/best solvers from the others.

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Too many hard puzzles
What did you think about the puzzle quality of the test?	Fairly Nice
How did you feel about the length / time limit for this test?	Way too many puzzles (too little time)
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	Many puzzles were worth too much or too little
What was your opinion of the booklet formatting and printing?	Just right

I don't think you're considering everyone here, you are only considering "I completed 8 puzzles with full focus and concentration", which is completely disregarding the population of solvers who will feel "I completed this Little Killer by taking 20-30 minutes on it, within which time I could've easily solved 6-7 puzzles" or even "I took 30-40 minutes and couldn't solve this Little Killer and couldn't get time to attempt others either". All 3 situations have merit, and there will always be people falling in numerous categories of things going wrong in a long test. The solution is not to make things suit one group or the other, but just make sure that practically, the points are distributed fairly. And the testers' timings method is the most practical way. So thats not going to change to suit one side of the argument. So again, I'd say the topic of whether such difficult puzzles should be in a competition format is debatable, but the way the points are distributed is as accurate as can be.

Also, from the last line I can derive that you might have the opinion that people strong at some variants can solve any instance of that variant in a short time. This is not the case. speaking from my perspective, I'd say Little Killer falls roughly in my strong variants. I still took 25 minutes on it before breaking it (I've now found that a typo mid solve cost me and I was solving the right way). I think I solved about 6-7 of the lower pointers in the same time span early on in the test, but this 25 minute expenditure not only cost me directly, I also felt the effects of it while going forward in the test. Now say, I'd solved it right, but the points total was just 120 or so, would I not feel even more frustrated? My point here is, please take a second and consider the enormous risk a person takes attempting these puzzles and the amount of focus and concentration needed to solve these.

So I don't think anyone addressed my post directly, although I think my thoughts are broadly represented by what motris said. The issue is not points allocation, which can only be done sensibly in the one way everyone has already outlined. The issue is the appropriate inclusion of puzzles. And I think it is an issue that hasn't really been discussed seriously before - and I believe this is certainly something that has been got wrong at previous WSC's.

The issue comes down to variation in puzzle difficulty. Shorter, easier puzzles will have much less variation in solving times than longer, more difficult puzzles. Basically this comes down to a couple of factors. First is the fact that you can go wrong in far more exotic and long-winded fashion on a harder puzzle than with an easier one, so often you have no idea that you've gone wrong if you've made a typo or whatever. Second is the fact that a harder puzzle might require one specific deduction that you have to get, and if you miss it for a while then you are going nowhere. I have lots and lots of data to show this is an observable phenomenon with harder puzzles.

Managing the risk of taking on a big pointer is a complex issue. For example, my last correct submission was 35 minutes before the end, at which point I decided to take on the 118 point moveable digits. I broke it after about 15 minutes and then couldn't quite get it finished before the end. In that same time there were 146 points spread across 3 puzzles I'd have got without any problem if I'd attempted them, with perhaps a long shot at the higher variance 108 point killer to boot. Put in context, that decision cost me certainly 33% of what I could have got, and about 30-40 places in the table.

The solution seems obvious right - stick to the safer low variance puzzles for your points right? The problem, especially for a one round format where there is a good chance to finish most if not all of the puzzles on a good day, is that ultimately you'll have to have a go at the harder puzzles, and given they are harder puzzles you need to make sure that you've left sufficient time to have a go at them otherwise again you are left with the problem of having say 10-15 minutes of solving time where you run out of time trying to finish a puzzle. This brings me to the point I was trying to get across, in terms of the difficulty of the test being massively skewed. When you have relatively few puzzles commanding a relatively large proportion of the points, then the lesser pointed puzzles start to lose relevance and the test is exposed more than usual to the elements of chance with the inherently higher associated variance.

Perhaps the proof is in the pudding. Looking at the results there are more solvers (I'm not going to name names) than I'd expect who are way below where they "usually" are. Applying the risk variance thesis gives an explanation: a test with more "risk" than normal is inevitably going to throw up more casualties to variance.

Of course that's not to say managing the harder puzzles is not a skill in itself. It is also no coincidence that there are certain solvers who seem to fare consistently well with more difficult puzzles...

I didn't want to respond directly to your comments detuned because I would have asked if you'd finally come around from my complaints after your last Nikoli Selection test where you forced the large and under-valued "marathon" puzzles to be solved at the very end. I found your criticism a little ironic in that light since in the past you've wielded the severely hard puzzle weapon against all of us.


Managing puzzle difficulty is a very hard thing. I am one of a very small number of LMI test authors that has tried to write a test without any meaningful spread. My Decathlon tests had 10 categories and a consistent "easy" and "hard" puzzle score. So if you wanted a 20-point or 50-point puzzle, you simply found one in every section. This took a lot of puzzle editing to get the difficulties to match a set of testers and not have a large outlier. I'm still not convinced it is easy enough to achieve in practice to encourage it as a goal. It is knowing how difficult this is to do that I have to challenge kishy on the suggestion you can just apply such scoring to any test and not have a revolt on the site from clearly unfair scoring.

So let's say you do allow a spread. How wide should it be? I think it is ok for puzzles to be about 2-3 times above or below the "average" value. A 400 point USPC might have about two dozen puzzles; some are 5 points and a few might be 30 or 40. So a 7x or 8x difference between the easiest and hardest puzzles is ok in my mind. This is matched in this test, but with slightly shorter time and slightly less puzzles, perhaps only a 5x difference would have been better?

Edited by motris 2013-04-24 8:16 AM

How about don't make competition using hard puzzles anymore so no more people will be sad and unsatisfied with the result?

There was definitely more time from me spent on deciding which puzzles to try when, some in advance of the contest and some during, compared with a lower-variance contest.

I've found that doing one or two medium puzzles at the beginning gets me warmed up, but on this test I first looked at the two hard puzzles that I planned to finish (little killer and killer), went over to solve a couple of easier puzzles to get my confidence up, and then after some bad decisions along the way eventually came out with solutions for the two hard ones. Then I tried some more medium-pointers and at the end wasted most of my last 15 minutes working on puzzles that I didn't finish in the given time -- 5+ minutes on the irregular, and then when I felt nowhere near the solution, quickly stomped out the classic and then started but didn't finish the Quad which still seems to me to be worth more points than it was assigned -- which probably means I'm missing an important deduction!

Certainly having the big-point-value puzzles increases the variance (if you spend a lot of time but then break the puzzle, or just get stuck hunting for a difficult logic step, for instance) and it also increases the "metagame" strategy of which puzzles to attempt at what points in the two hours.

I'm sure I'd feel different about this if I were strong enough to hope to solve all or almost all the puzzles in the given time. Still, one thing that surprised me in motris's USPC reports is how often even he will set a puzzle aside for a while, move on to other things, and come back later.

I also wonder about that in the context of things like WPC playoffs -- how different would they be if there was a packet to finish rather than puzzles that had to be done one at a time?

Anyway, I greatly enjoyed this test, and not only because I did well on it! The 15-30-40 theme was beautifully carried out.

I do feel like with the time pressure there were too many places where it was faster to bifurcate instead of using more elegant logic, though. At least, those are the steps that I found led me to reasonably quick solutions on some of the puzzles. Yes, bifurcating carefully is perfectly valid logic, but it's still not as pretty as the logic that I could find later without the time pressure. I wonder if that's a danger of high-point-value puzzles, that bifurcating is too commonly a way to get to the result more quickly.

Will author or anyone please teach how to solve little killer, moveable digits, placement shape, killer and word search for this competition? I need help..

Just one sentence for this Round is suitable: Tennis Round !!
I really enjoyed this round puzzles and they were really wonderful.
I really like tennis and Djokovic and Tipsarevic are both my favorite tensors and I really enjoy while seeing their palyanig.

Thank you to designers and thanks Deb for handling this round on his site.

I just want to say one thing:

---------------

If anybody continue the news about tennis, He should know that last few months which Djokovic was Injured. All Tennis cups and tournaments were less interesting because Djokovic couldn't participate in.
In each game if we don't have all the powerful players, the competitions will have less interesting. Consider that we don't Spain in football!

Here, in the world of Sudoku we have two great champions with Triple Championships (motris ans janoslaw).
But know, one of them (I think) is satisfied in Sudoku Championship and doesn't participate in Sudoku cups and tournaments and other competitions related to Sudoku!

In tennis, by coming back of Djokovic, The Tournaments will be more interesting in the competitions of Nadal and Federer with him.

Now in Sudoku, my dreams is seeing both of (the Lord of Sudoku) in the next WSC and finding more interesting competitions.
This is my dream but my feeling says that it will happen..........!!

-------------

Just one sentence to Jan Mrozowski for this round:

We have a proverb in the world of Sudoku:
Each puzzle has only one solution but there is more than one way to solve it!!
Maybe the competition should have more ways for sending the solutions.
Of course you know that this tournament is 6 of 8, and you still have the chance to be the first.

--------
Mahyar Asadi

http://bia2sudoku.blogfa.com/post/170

I am more like to see myself be in competition rather seen both 'great champions'..

Edited by f4han 2013-04-24 1:49 PM

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	A bit too many puzzles
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	Many puzzles were worth too much or too little
What was your opinion of the booklet formatting and printing?	Just right

i'll respond only briefly re NS, but I'd suggest firstly that is different for a few reasons. firstly, and most importantly in my eyes, the monthly test format gives licence to more creative rules and procedures. secondly, the hard marathon puzzle dilemma was specifically engineered so as not to be a dilemma at all - you were supposed to do the main section first. thirdly even if no marathon puzzles were solved, top solvers were still effectively ranked by how quickly they finished the regular set via bonus points for each minute saved. fourth, i'd argue there is much less "variance" attached to larger, harder nikoli puzzles than to things like the little killer here.

Ultimately, the gimmick with the marathon puzzles was that the points in the bonus section were not equivalent to the points in the main section. Given that this would affect a tiny minority of solvers, together with the fact that this tiny minority of solvers were still effectively rewarded and separated from those who didn't finish the main section, or did finish the main section but in a slower time, there was no qualitative difference in the results. Would I use such a gimmick in a more serious competition? No. Why not attempt to make the points in both sections match up? Because I wanted the 90-95% of solvers to stay on the main section and not be drawn to the marathon section - knowing full well that this group of solvers would at best only just finish the main section if that's all they focused on. In effect, I was removing the option of the big point, high risk, larger and more difficult puzzles. The guiding philosophy were that these were bonus puzzles for "after" the test for the majority of solvers, and something else to really challenge the top solvers who otherwise blasted through the main section in about half the time.

Anyhow, I fully stand by the experiment, and I think there are several reasons why it was suitable for those particular sorts of puzzles rather than sudoku. Bringing the discussion back to the Serbian test, and the spread in points difference. Lets explore your your USPC example. The mean puzzle score there is very roughly 16, and the vast majority of the puzzles are generally worth 10, 15 or 20 points, so the standard deviation isn't hugely high, although I've not taken the time to go into the details. Now take this test. The mean puzzles score is 62.5. You have 7 puzzles from 20-43, significantly lower than the mean, 6 puzzles from 51-72 roughly around the mean and the 3 skewing puzzles from 108-157.

Now I'd agree, trying to edit the difficulties of different puzzle types is going to be a pretty unforgiving task, but I think a think a good test has broadly a consistent level of puzzle, together with a few warm-up puzzles and maybe one or two more challenging puzzles. I think my reaction to my bad result has probably tempered some of my opinion, but there are certainly things worth discussing here, and I'd stand by my observation that difficulty balance was somewhat skewed for this test. Great if you had a good day, but the chances of having a bad day were also increased.

Edited by detuned 2013-04-24 4:20 PM

That's very true!!

So, no one really kind enough to teach me how to solve all these? I'm begging one more time for help please..

I don't know who you are talking about .

I preferred not to react immediately after the tournament, as I was a bit like Tom just after competing: frustrated of my very poor performance.

It was not a good day for me (I could argue that musicians do sometimes excess, like playing in a jam-session until 4 am while drinking too many beers, when they know they have a rehearsal at 8 am, and they'll be very tired the rest of the weekend, including monday... sometimes clichés are not far from reality).

I started quite well and fast with the first easy sudokus: classic, quadruple, tennis, extra-regions. Then the 1rst mistake arrived in the arrow sudoku, almost at the end, which I was not able not fix. I gave up this sudoku, left the word search, as I don't really like this variant. I solved the irregular and the little killer, which took me lot of time, at least I had this impression. Then, like the arrow sudoku, I broke the frameless, almost at the end, without being able to fix the mistake. I solved the renban and submitted my codes. It was quite a good surprise to see that it remained 55 minutes. I made 397 points in 65 minutes, should have been 518 without having broken 2 sudokus. But even with 397 points, I realized after the tournament that if I had gone on this way, I could theoretically have reached the 13th rank...

Then came the second hour. I solved the even sudoku (I realize just now that I forgot to submit it !!! ), the diagonal consecutive, and the thermo (I didn't break it, not like during the 2 last LMI tournament), then I started the killer, but I was stuck (my analysis in the upper right of the grid was not complete), so I switched to the movable. And then it was a black hole: I was repeatedly stuck. I missed lot of (subtle) steps, I realized that I should have worked more on this variant before the tournament (just solved the example). I was so obstinated that I almost forgot to submit the 2 (3 in reality) sudokus I solved. When I decided to submit, there were only 2 minutes left, it could also have been possible that the tournament was finished...
So in the last 55 minutes, I only scored 111 points. I lost my concentration and my motivation, too. I was frustrated, I thought that sudokus were too hard...


But after a while, I finished the set, realized that all sudokus, if hard, can be solved without guessing. Also have seen the beautiful theme and realized that the problem was not in the set of sudokus, but in my performance.

I also would like to say, as contribution to the discussion "hard/easy sudoku, balanced set,...", that some players really don't like when all sudokus are too easy, or when there are lot of classic sudokus, and they know plenty solvers will be faster than them. We had sudokucup 9, and also german tournament, which had this profile, and I know some players who were not very comfortable with this. So I think it's good to have some tournaments with 3-4 really hard sudokus, so these players can also show us that they are strong players, perhaps a bit slower, but really consistent on hard variants.

Fred

Do you mean teach how to solve from the start? Is it possible for you to upload images with the progress you have made? We can then take it one at a time. Authors might be busy processing results, etc., and its a lot of typing to write full solves for 5 Sudokus.

Here are some notes about two of these sudokus (I hope this will be enough for you to solve).

Placement:
There are 4 figures and they overlap in center. You can place 5 in center and note that in other 4 circles in central square are 1,2,6,7.
Look at head of 4 rackets. They are need to be placed in one of corner squares. When you look at givens it is obvious that it is best to start with bottom left racket.
This racket can't be placed in corners with 3 and 4 because he has those numbers in head. This racket also can't be placed in corner with 2 (problem with duplicate number in some row/column - find).
When you place this bottom left racket and transfer numbers in appropriate circles continue with similar logic.

Killer:
First look at top left corner. You have area that sums to 14, one area with ? and single number. ? = 15,30,40 and it gives that R3C3 must be 1.
With similar logic in bottom right corner R7C7 must be 7.
All regions with 3 cells marked with ? must sum to 15. So, with 1 is 5,9 or 6,8. With 7 could be 2,6 or 3,5. Now you look upper to 5 in 6th column.
Two cells in region with 7 must include 2 or 3, so 5=1+4.
Try to identify some more sums and count on that, if you have region with 5 cells that sums to 15 numbers must be small: 1-5, and if you have region with 4 cells that sums to 30 numbers must be big: 6-9

Branko

for placement one..I miss to look properly of each shape..and I thought it is unsolvable when I overlap the last circle for each shape in the center of puzzle..

For killer, I get it..I just don't try that logic that's why I don't get it..

Now I already get moveable digit and word search.. (after try to understand the rules properly and use logic like killer in word search) except little killer.. I just put all possibilities for all cells with clues outside and don't get any number yet..I think it's an embarrassment for me to show my working grid because it just like I never start anywhere..

maybe I should ask where is the first cell or group of cells that can be filled by correct number and it's reason..



Now I know this round is very good except if I still don't know how to solve little killer.. so please tell me asap because I want to know it first before I can rate this competition here..

Edited by f4han 2013-04-26 10:22 AM

I strongly disagree that the situations are different. Is there some contract or set of guidelines for Grand Prix authors that doesn't give them some license to experiment where monthly LMI puzzle test authors can? Are you implying the LMI Puzzle Ratings are less meaningful than the GP ratings? That being "fair" is more important in one context and not the other? I appreciate experiments but I think there are lessons to learn and reconsider.

If anything, what separates the two situations is that at least here solvers could start the hard puzzles before the last 20-30 minutes of the test, and therefore had a better chance to finish them and properly use the test time. There is nothing more frustrating than having 15 minutes left at the end of a test to do a 20 minute puzzle. Good test-takers do not leave themselves this situation, but you as a designer still seem to consider it a "feature" (point two). And the scoring of the marathons did affect the top 10 significantly, particularly on the LMI Puzzle Ratings system. So whether solver 100 and 105 are well rated by only being on the main test, the NS was an example of how not to use long puzzles (whether variance was high or not, and this too is debatable).