This test was timed pretty well. The target is to try to ensure that the top 3-4 finish the test, and this is the standard target in WPC and National Championships too as far as I know. So, on that front, I agree with the author.

About practice = finishing, I disagree. While practice makes you better, I don't think its that easy/common for someone to practice and get to the level of finishing tests. For one thing, you need a lot of things to go right. I mean, deu and Melon both are talented enough to finish the set in 90 odd minutes ideally, but it isn't ideal for them either. Also, everyone has their limitations. There are solvers who I see active on every single place possible online for solving puzzles, and still struggle to finish tests/improve beyond a certain level. I'd suggest, rather than taking that line of argument and trying to guarantee practice = substantial improvement, its important to just note that not everyone needs to compete for the top 2-3 places, and not everyone needs to target finishing the test, the main objective for majority of the solvers should be to have a fun 2 hours + of solving. If there is potential for improvement, it'll come on its own.

Also, I don't agree about Yajilin-Battleships being a problem. I get the dot separation argument, I'm referring more to the twisted ships. Twisted ships/twisted shapes and dimensions have been used in many tests and in WPCs too. I see that as a nice surprise thats a part of puzzling.

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
What was your opinion about the answer key extraction?	Perfect answer keys
How did you feel about the length / time limit for this test?	Just right
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	Most puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Too few pages / too small grids

Excellent puzzles. Thanks for really nice test, Ivan.

Branko

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
What was your opinion about the answer key extraction?	Perfect answer keys
How did you feel about the length / time limit for this test?	Just right
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	All puzzles were worth the right amount

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
What was your opinion about the answer key extraction?	Mostly perfect answer keys
How did you feel about the length / time limit for this test?	A bit too many puzzles
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	Most puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
What was your opinion about the answer key extraction?	Perfect answer keys
How did you feel about the length / time limit for this test?	Just right
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	Most puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

Thanks for everyone participating, and I hope you enjoyed the puzzles. The following post is long. I know.

Participation
There were 137 participants, 132 of them officially participating. Among these 132 official participants, 107 (81.06%) submitted something, and 104 of them (97.20%) got a non-zero score. Among the 5 unofficial participants, one participant submitted something, which also ranked 40th if said person participated officially.

Top Scorers
Congratulations to deu for perfecting the test, with 18 correct answers and no incorrect answers. deu also completed the test in the fastest amount of time, with 13 minutes remaining, topping the scoreboard.

Next, congratulations to MellowMelon (2nd), xevs (3rd), and EKBM (4th) for completing all puzzles. It might worth mentioning that MellowMelon lost 14 points due to penalty but made it up by the 4 minutes margin over xevs (which lost only 7 points by penalty).

It might also worth mentioning that except for motris which is not participating, all Top 10 of the current ratings standing are in Top 12 of the test. Congratulations for rob (7th), daisuke_t (10th), and Valezius (11th) for beating some of the Top 10 rated people.

Puzzles
All nine Top puzzles were attempted more times than the Bottom puzzles, with Semi-Liar Masyu (Top) topping at 117 attempts and Skyscrapers Kropki (Bottom) as the least-attempted puzzle with only 22 attempts. Similarly, all nine Top puzzles had more correct submissions than the Bottom puzzles; the most correct answers was again held by Semi-Liar Masyu (Top) with 90 correct answers and the least correct answers was again held by Skyscrapers Kropki (Bottom) with 19 correct answers. However, the percentage of correct answers didn't have this property: Surveyors Heyawake (Bottom) had 96.66% correct answers (29 of 30), the highest, and Domino Nurikabe (Bottom) had only 60.00% correct answers (27 of 45), the lowest.

The most favorited puzzles were precisely what I expected, although not in the exact order. Domino Nurikabe (Bottom) was rated 8.72 as the highest, Liar Slitherlink (Bottom) with 8.32 followed, Cipher Fillomino (Bottom) with 8.09 was next, and Battleships Yajilin (Top) with 8.07 came fourth. The other puzzles garnered less than 8/10 rating. (My expected order was Liar Slitherlink (Bottom), Battleships Yajilin (Top), Cipher Fillomino (Bottom), and Domino Nurikabe (Bottom).)

The least favorited puzzles were Smullyanic Dynasty (Top) with 6.13, Skyscrapers Kropki (Bottom) with 6.50, Liar Slitherlink (Top) with 6.58, and Smullyanic Dynasty (Bottom) with 6.92. The rest had a larger than 7/10 rating. Apparently the participants in this test didn't like Smullyanic Dynasty :P

Feedback
How balanced do you think the puzzle types of this test were? 52.1% answered "Fairly balanced". Indeed, according to Dr. Sudoku, I didn't include enough puzzles from Number Placement (Skyscrapers Kropki), Object Placement (Battleships Yajilin), and Region Division (Cipher Fillomino), putting too many puzzles on the other two genres: Shading (Smullyanic Dynasty, Elimination Tapa, Surveyors Heyawake, Domino Nurikabe) and Loop (Liar Slitherlink, Semi-Liar Masyu, Battleships Yajilin).

What was your opinion of the distribution of easy/hard puzzles? 65.1% answered "Just right". However, there are still some people (11.5%) that answered "Too many hard puzzles". While I was originally aiming for an easy test, I apparently was incapable of producing easy puzzles on demand; I just began making the puzzles without too much attention to difficulty, and eventually they came out. Apparently they were still too hard. Even the Top puzzles, which were supposed to be easy, had 40-pointers (and the Bottom puzzles also had 40-pointers) and I believe Cipher Fillomino (Top) should have 50 points. A note to make the puzzles easier. And to ensure Bottom puzzles are indeed harder.

What did you think about the puzzle quality of the test? 61.0% answered "Very nice", and 36.1% answered "Fairly nice". I don't have any comment for this.

What was your opinion about the answer key extraction? 73.2% answered "Perfect answer keys", but 20.3% answered "Mostly perfect answer keys" and 6.5% answered "Answer keys could have been better". I do agree with this. As it's Instant Grading-enabled, Deb suggested that the answer keys should be of fixed length (or something checkable), so that added or missed one number is detectable. But Deb told that it's okay, so... I also realize that there were many typos and several claims about this. I do realize that this test is counting-intensive (in particular Cipher Fillomino puzzles and Domino Nurikabe (Bottom), but most of the answer key methods also require counting to some extent).

How did you feel about the length / time limit for this test? 75.9% answered "Just right", but there is 21.0% that answered "A bit too many puzzles" and 3.1% that answered "Way too many puzzles". I do agree with this to some extent. The point value for a puzzle is approximately 8-10 times the number of minutes of the second best test solver (out of 3) time, although many were adjusted downwards. So in total, I think the test requires approximately 120 minutes to finish, if you're at least experienced with each genre. Precisely the time given, so "too many puzzles" is certainly a valid claim.

Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent? 72.4% answered "Most puzzles were worth the right amount". Well, this is the above discussion. Many puzzles have their values adjusted downwards to keep the 90-total for Top and 170-total for Bottom.

What was your opinion of the booklet formatting and printing? 88.4% answered "Just right", but I think the 8.7% that answered "Too few pages / Too small grids" had a valid reason. A 28px cell is too small for writing numbers in, especially when you're doing it quick. I never printed the pages (because I mostly solve on computer, with MS Paint, anyway), so I didn't realize this. Might be my fault.

What was your overall opinion of the test? 67.9% answered "3 stars". Thanks for the positive feedback of my first ever authored test.

Finally
If you really read through them all, congratulations. :P

Anyway. Domino Nurikabe (Bottom) is a rather tricky puzzle, with the 43-island. Even MellowMelon hasn't figured out the uniqueness proof. Here is a detailed solution of it.

Similarly, Liar Slitherlink (Bottom) is the highest-valued puzzle, which also proves to be difficult (second least correct answers). Here is a detailed solution of it.

The remaining 16 solutions will be made on demand, although I do plan on releasing the solutions of Cipher Fillomino (Bottom) and Battleships Yajilin (Top) soon.

And lastly, I do plan to author another test ;)

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
What was your opinion about the answer key extraction?	Perfect answer keys
How did you feel about the length / time limit for this test?	Just right
Of the puzzles you solved/attempted, how well did their point values reflect the difficulty/time spent?	Most puzzles were worth the right amount
What was your opinion of the booklet formatting and printing?	Just right

My experience during the test was quite bad, but that was more down to me making stupid goofs in every single puzzle. But, I liked what I solved, during and after the test. My favorites have to be the 2 Fillominos. However, I didn't like the part where I had to count to 22/27/14 to get the keys right. I guess without Instant grading it'd be even more annoying, to get a mistake for counting errors. From the solving perspective big numbers forced in a Cipher Fillomino are great fun, but I'm just unsure on whether its ideal in an online contest scenario. The Surveyors Heyawake didn't cause me much trouble, I just thought I'd mention that since I seem to be alone on that feeling. I agree it had lengthy rules, but I guess I'm just good at lengthy rules when its all combinations of other puzzle types.

All in all, a fun test. Thanks Ivan!

I liked the contest very much. I am aware of only few types before and was never aware of other types.
Of the puzzles I have attempted I liked all of them. Semi-Liar Masyu (Bottom) is my favourite.
If there is anything to complain, I have to blame myself for not practising the puzzles.

Thanks Ivan, Waiting for another contest of same kind :)

Regards,
Ravi

Edited by macherlakumar 2013-05-22 1:23 PM

Thank you so much for lovely test. You did very well. I agree about being said for S. Heyawake rules. It is really tough to follow them. When I tried to solve both of them in the test, I forgot sometime rules, I broke so many times puzzles. They stole a considerable time from me :) I couldn't solve 2nd Cipher Fillomino, despite the fact that I attempted to solve several times in the test. It was obstacle for me to get bonus. So I easily lost a valuable 110 points. Anyway, that was cool test, I like not only puzzles, but also page formatting. So, thank you for your effort.

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Too many hard puzzles
What did you think about the puzzle quality of the test?	Fairly Nice
What was your opinion about the answer key extraction?	Perfect answer keys
How did you feel about the length / time limit for this test?	A bit too many puzzles
What was your opinion of the booklet formatting and printing?	Just right

You know, when I see "Prasanna only solved 12?", I immediately got to the "okay these puzzles are too hard" stance. Especially after seeing "Kota solved 17, uvo solved 14". I'm relieved after deu proved me wrong with 13 minutes to spare. :P

Heh, ya, I realize that the counting is really bad. I cannot help it though; I feel like almost every time I make Cipher Fillomino, I always have large polyominoes. The one that doesn't has been published before test. I suppose I should have published the Skymin one. (Also, yes, the Skymin theme is absolutely intentional.) However, I do want to toy with the possibility of large polyominoes actually don't come together (ABCD one), and it apparently also gives the chance for an implied number (2 in the middle).

Also, I suppose I should have told that I accept using letters as alternate answers, but then people complain about not knowing which letter should be used for the hidden 2 above. Luckily nobody used letters. Or at least seems so.

Thanks!

---

Really? If you know some of the types before, you like to browse logic puzzles. As I have mentioned before, only two can be claimed to appear a lot. If you state about the original genres (not the variations) though, I tried to pick well-known genres.

SM2? Wow, pretty unexpected.

Heh, probably not the same kind as in Deception 2, but most likely I'll use variations again. (I like variations; they give an equalizing factor to everyone as they are not common. :P )

Thanks!

---

CF2 is indeed difficult, using a rare theme (don't think I ever saw it before) where there are two large numbers and one of them is even split into three. (Raise your hand if you guessed that the A was the largest polyomino before you started solving.)

You should have done the other three Top puzzles (120 points). I'm not sure whether you'll do better (as in you can solve them faster than you can solve CF2), but why not.

Thanks!

Thank you for a nice contest, although I stopped solving after an hour, after I broke too many puzzles, and decided I would feel better if I stop rather than getting more frustrated in the next hour. Not my day. I solved some of the puzzles after the contest, all are nice.

I just wanted to comment on your "variations vs. standards" statement. I think there are different kinds of variations. Some are minor variations with an additional rule that can be used here or there (think of Diagonal Sudoku as an example). And some are variations where most of the solving involves the new rules (with your Masyus beeing a good example). For the later one, I dont see an "equalizing factor", I rather think the performance depends on the time for preparation you do in the week before. And since there are almost no examples, most of this preparation will be constructing.

And for "just do enough puzzles, then you can beat the top". Just forget it. If you can ever do it, you will know it after some months, or you will most likely never do it. I'm solving puzzles for some years now, some weeks with no puzzles, some with 40 hours+. I feel I'm still getting better slowly but constantly every year. But I also know, I will never ever be in Top10. Look where you are standing, and if you want to "compete" somehow, choose your goals for the next year accordingly to that. There are more than enough ratings and measurements to track progress. For example I'm doing sudokus at sudokucup.com. If I can beat the 50%-quantil on a single puzzle, I'm happy. If I can do that for all puzzles of a day, I'm very happy. Maybe it will be 40% in one or two years, assuming the player base stays the same. I never saw something below 25%, and I'm fine with that. I know there are some people that will beat me 98 times out of 100 puzzles, and for the other 2 they are doing a mistake. But does it matter? Or long sentence short, as a member of our german puzzle community said once: "There are worlds without bridges to them."

Edited by Realshaggy 2013-05-22 5:45 PM

Referring to the part about accepting letters, this is obviously in hindsight, but I think worth pointing out that maybe a note can just be added that says "In case of implied numbers, enter a different letter (thats not already been used) for each number that is implied". Could be worded better, but the point is, I don't think that gives away much, I mean I personally won't go away from the solve intended looking for implied polyominos just because the key says how to deal with them, just like I don't go looking for 10 shaded cells in a row if it says "in case of two digit number, enter only the units digit".

We couldn't have done this because of (limitations in) Instant Grading system. The system requires that all valid answers be listed before the test starts. (Ah, well, we could have listed all the 22 valid answers - only involving letters, ignoring case sensitivity - but that is still painful to handle).

So, although, ABAA22BDDA, ABAAEEBDDA and ABAAXXBDDA were listed as valid answers, we never made it public that letters will be accepted.

Ah yes, forgot momentarily about the Instant Grading restrictions. Listing 22 answers is definitely not a solution since there could be a puzzle with multiple implied numbers sometime in the future, increasing the possibilities. I suppose the format for accepting ABAA22BDDA can be made public though? The thing I see against it is its confusing having both letters and numerals, but might be preferable for some over counting, and the implied part is fixed.

Very well documented. Thank you for your effort and for sharing.

Edited by debmohanty 2013-05-23 5:33 AM

Ah, I missed some of those chain reactions on the right of the Domino Nurikabe... followed your Liar Slitherlink path almost exactly though.

On effort: there's not enough of the right kind of data to say anything certain about this, but here's two quick bits from my own opinions:
1. There's little to gain and a lot to lose by believing yourself incapable of reaching a certain level.
2. "If you do lots of puzzles, you get better" is wrong wrong wrong. After solving a puzzle, you must stop and think hard about all the things that you were slow on or that you missed entirely. Figure out how to not screw up the same ways again. I never made any nontrivial improvements without doing this.

Edited by MellowMelon 2013-05-23 9:19 AM

First of all, let me say the contest puzzles are, without exception, cool. Like, seriously cool. There is a surprising, and most welcome, rarity of workmanlike steps. Also, lots of big picture thinking. Congratulations.

Far as format goes: Instant grading is always nice; personally, it helped me eventually reread keying instructions I had radically misunderstood. Scoring was, in a word, unfun. Grouping bonuses are only useful in railroading a solver's path through the test, which can in no way make a contest more enjoyable. Page formating was a mixed bag. I liked the instruction column, but some grids felt cramped. I was particularly annoyed with Skyscrapers. Timing, well, let's just say this was no exception to my usual pattern of losing quite a few points by a few seconds. In this case, 70pts for 10 seconds.

'"If you do lots of puzzles, you get better" is wrong wrong wrong.' is kinda wrong in my experience. When you are first starting puzzles, disentangling things into digestible bits is a tall order. And later on, practice does make perfect. Locating the things I was looking for became a whole lot faster and more efficient with experience. Which is not to say that stop and think after the solve is not excellent advice: I have puzzle books filled with side notes on whatever caught my eye in that puzzle. I find the process of putting stuff to paper commits them to memory. And much longer notes on puzzles where I had to brute-force things the first time around. I'll admit that staring at a completed puzzle may not be everyone's idea of fun though. Another way of marked improvement is simply reading or seeing solutions and chat about particular puzzles, which occasionally happens across the many puzzle blogs around. Some times you'll pick up notation, and sometimes logical attack strategies you hadn't yet formulated. Again, to some this will feel like cheating.

Setting reasonable personal targets in competition settings may help with motivation, provided you keep them tight as you improve. For example, I currently tend to consider things under 33% of top score a failure, and gun for 40%. This wasn't always so, and the goalposts will be moved if keep scoring with ease. But, hey, "There are worlds without bridges to them." wins on style alone.

About group bonuses, I do think that they aren't that fitting in this test. But well, everything has been settled before I realized this, so, yeah. It's mostly to allow solvers to plan how they do the puzzles in order to maximize the bonuses.

I should have made Skyscrapers' grid larger. I'm too much used to binary and loop puzzles that I aim to make smaller grids (so that more can fit on the same page; it's wasteful if the PB has 18 pages IMO).

Thanks!

Thanks for the interesting walk through's for Nurikabe and Slitherlink. Learned some interesting new things!

I am still struggling with the second kropki skyscraper. Can someone write a few lines about the critical steps for this puzzle as well?

Thanks in advance!

It's somewhat trial and error at rows 2 and 6, or at least that's the best path I've found so far. I'll provide a walkthrough ASAP.

Heh, apparently not as terrible as I thought. Skyscrapers Kropki (Bottom)

Edited by chaotic_iak 2013-06-03 9:09 PM

Thanks a lot for this walkthrough. Much less T&E than I expected, so I learned a few interesting elimination steps again!