How balanced do you think the puzzle types of this test were?	A bit skewed
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Way too many puzzles (too little time)
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Very nice

Nice test! There were a few too many hard puzzles among the first 10, I'd say, where I expected them to be relatively easy. There were also a few really beautiful puzzles where you could solve easily if you think about them the right way (which I often only noticed after guessing turned out to give a contradiction, then I'd see what I should have noticed in the first place) -- those were really nice.

I also learned a lot about some types of puzzles where I usually solve on-screen by making tons of notes!

I think the trickiest part is when there's unusual rules and the inverse rule is also in effect; it makes pretty puzzles, but hard to solve on-screen with time pressure.

No problem - and despite the seemingly indifferent colours provided by my feedback, I quite enjoyed the test. Having a quick scan of the results, there seem to be a strikingly large proportion of wrong entries. My best guess would be a large portion of these occurred as a result of guessing. Then again, this will always be a problem with difficult puzzles in smaller grids.

I'd be very interested to see a variation of this format, again with specific time limits per puzzle, but where the solver can choose the order in which puzzles are solved...

When I explained Branko (my perennial test solver) about this format, that is the first question he asked. "Will solvers be able to choose the order in which they want to solve?" So certainly something to keep in mind for the next one.

We can now view each others' exact submissions here

So this finishes with me being the only one in the top 10 to not have a single best time on an individual grid. Never thought consistency would be my positive in a test :P

Sudoku Screen Test is over. This was a different test from regular monthly tests, with every sudoku having time limit and solving in sequence. It was our first attempt, so certainly many points to consider in the next one, if we ever have that. Many thanks to all of you who shared their feedback.

Nevertheless, I sincerely hope everyone had fun to some extent.

Congratulations to uvo, janoslaw and misko for taking the top 3 positions. TiiT and jaku111 are really close to the top 3.

Prasanna has a top 10 rank for the second time now, and it is very significant achievement. Congratulations to Prasanna, Rishi and Rakesh for the top Indian positions.


Edited by debmohanty 2012-12-25 7:35 AM

I was disappointed just now to find that I was too late to try these puzzles. Most of the LMI competitions I've entered have had clearly displayed starting and ending times on the main page, whereas this one just vaguely stated that it would be open over the '96 hour period from Dec 21-24'. As solvers at this site clearly consist of a worldwide community, over many different time zones, it's definitely worthwhile to have the server times displayed for start time/end time/current time, so that we are all able to plan accordingly and be certain that we will have time to compete.

We always follow GMT, but yes, it was not mentioned in the Screen Test main page.
It was there in the demo page / submit page, but I assume you didn't check that. Sorry that you missed it, and we'll be mentioning it more clearly in future.

Thanks for the response. It was clearly my own mistake for leaving it too late. It was a shame for me because I was really looking forward to this competition. I had tried the demo a few days back but don't remember seeing anything about GMT. Of course I should have thought of that and will keep it in mind for the future.

Thanks for taking the time to design these puzzles - they are a lot of fun! I'll anxiously await the next Screen Test.

Just prepared an unofficial link for you - http://logicmastersindia.com/ST/ST3/unofficial.asp

Edited by debmohanty 2012-12-25 8:10 AM

Hi Deb,
I was thrilled with the no of Variant Sudokus given, i tried the demo Sudoku ..and was practising for the screen test..! however before the dead line time of screen test, i was not able to take it up; as i was travelling; can you please tell me when ll this screen test happen next time ?

Thanks Gotroch. You might not have realized that you were the first solver in the official period. It was lot of fun watching your score updates.

About the scoring system : We didn't have much data to come up with a rigid system. I was keen to play safe, so whenever there was a doubt, I thought it is better to award points to players.

I'm not defending the current system, but here is why it was done this way.
Given that there is a 5 minute cutoff on every puzzle, we needed to have partial points. We* expected many players to have half solved sudokus at the end of the cutoff time (and that is precisely what happened). We wanted to give some points if something has been solved - so there was no minimum-cells-to-be-filled-to-get-points condition.

And finally, about no penalty for wrong cells : it was again because of the cutoff. Imagine a situation where at the end of 270 seconds, 2 players have filled in same number of cells. (take the example of Skyscraper). They have 30 seconds solve the remaining part. Player A keeps staring at the grid and 30 seconds gets over. Player B does not find the logic, but decides to take a bifurcation, but unfortunately he takes the wrong route and 30 seconds gets over (and sudoku gets auto submitted) with him having some incorrect cells on the grid. I just thought that Player B shouldn't get less points than Player A. So there was no penalty for wrong cells.

* Thanks to Rohan and Prasanna who very patiently heard about my concept and discussed about it whenever I needed.

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Too many hard puzzles
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Way too many puzzles (too little time)
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Fairly Nice

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Fairly Nice
How did you feel about the length / time limit for this test?	Way too many puzzles (too little time)
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Fairly Nice

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	A bit too many puzzles
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Very nice

Hi Deb, I remember attempting puzzle no. B3 (not sure, if I completed it or not, but I did fill in some cells).
However, on the review submission page, it says that I did not submit it!
Could there be some problem with the auto-submit after 5 minutes were complete?

Hi Jayant, I checked the database. It looks like you have submitted B3 (Touchy) at 88 seconds in your timer without filling any cells. So 5 minutes auto-submit does not have a role to play, since you have submitted much earlier.
You started this puzzle at 2012-12-23 19:48:59 and submitted at 2012-12-23 19:50:28, both in India time. You can check the same in the score page as well.

In review page, we are not showing puzzles which are completely blank.

Is it possible that you are thinking about another puzzle B5 (or some other puzzle) ?

It's so good to see a test with zero `Zero` Scores

Edited by adityasaraf007 2012-12-25 3:33 PM

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	A bit too many puzzles
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Average

Thank you everyone for using the feedback system and letting us know how you felt about the test.

However, I'm bit confused by the feedback about "Number of puzzles". More than 40% of the votes suggest there were too many puzzles / way too many puzzles.
I thought in the IB, it was clear that 12 puzzles are meant for 'All', next 9 puzzles as 'Bonus', and last 9 as 'Extra Bonus'. If you are not a top player, you were not expected to reach section E in any case. Similarly, if you were a beginner, you should have thought that there were 12 puzzles for you.

If we plan for a similar test in future, I am not sure if we can / should reduce the number of puzzles drastically. It will be interesting to understand why players thought there were a lot more puzzles in this test than what it should have been.









Attachments
----------------
TooManyPuzzles..png (16KB - 0 downloads)

A very good explanation by Deb....I think now I should change my vote from a bit too many to just right...

Agree to Neeraj,
Overall i think pencil marks was very much needed, we can do away with UNDO option if one of them is to be exlcuded
Nice variety of puzzles , covered a large no of variants

Edited by utkaarsh 2012-12-26 7:51 PM

First, I would like to thank you for this first sudoku screen test. It was interesting, fun, well organized and with nice sudokus.

I would like to give my opinion on 2 points: The number of puzzles and pencilmarks:

About number of puzzles: I asked myself if it's really useful to ask this question to players. Is it really a subjective topic? If you ask me this question, my first reaction will be to look at the ranking and analize it.
So for this test, what I see is that 5 players submitted all puzzles (without leaving some of them after just a few seconds). Nobody has the perfect score, without mistake; misko didn't make mistake, but he lacked just around 1 minute to finish the test. The other 4 finished the test before the time limit, but with a few mistakes. Janoslaw spared more than 5 minutes, but without the possibility to check the puzzles a perhaps fix some mistakes.
My point is that, with less puzzles, more players would have been able to submit all, but the ranking would not have been fair, because one or 2 little mistakes would have resulted in a lot of places lost, thus perhaps not reflected the performance. For this test, in my opinion, the number of puzzles was just right.
The number of puzzles is always related to the fairness of ranking and scoring. On normal tests (where we can choose the order in which we solve the puzzles): if there are too little puzzles, one mistake can cost a lot, on the other side, with too many puzzles, you need a perfect evaluation of each puzzle's difficulty to be sure that the ranking is fair, what you can reach only by having at least 10-15 testers in my opinion (what is generally not the case).
When I organize a tournament, I have the answer to the "number of puzzles/general difficulty" question when I look at the ranking.

The other point is pencilmarks for this test:

I understand the decision not having the possibility to write pencilmarks for this screen test. However I noticed 2 things:
1. Writing pencilmarks is a matter of habit. Some players like to write lot of them, others prefer to write only a few. In that sense, the ranking would not have been the same if we could write pencilmark. Therefore it's an important decision.
2. Not being allowed to write pencilmarks, we were tempted to use more bifurcation/guesses than logical work at some points (on hard sudokus). I don't like to do some guesses to solve sudokus, normally I try to avoid them when it's possible. I did more guesses than usually on this test.

I would have preferred to have the possibility to write pencilmarks, even if I'm not a big fan of writing lot of them, and even if I'm not sure that my ranking would have been as good.

On a personal note, I'm happy to have made top10 ranking on the last 3 monthly contest, I hope I'll continue the same way in 2013

Fred

How balanced do you think the puzzle types of this test were?	A bit skewed
What was your opinion of the distribution of easy/hard puzzles?	Too many hard puzzles
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Way too many puzzles (too little time)
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Average

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	A bit too many puzzles
What is your opinion about rules, points and scoring for this test?	Exteremely unfair
What is your opinion about the solving interface?	Very nice