How balanced do you think the puzzle types of this test were?	Very skewed
What was your opinion of the distribution of easy/hard puzzles?	Too many hard puzzles
What did you think about the puzzle quality of the test?	Below Average
How did you feel about the length / time limit for this test?	Way too many puzzles (too little time)
What is your opinion about rules, points and scoring for this test?	Exteremely unfair
What is your opinion about the solving interface?	Below Average

'No comments'

Most of your criticisms are valid, and of course we are happy to hear any kind of ideas/feedback as this kind of Screen Test using Sudokus was a concept tried for the first time here. I feel I can answer some of your points though.

The perceived difficulty part is based on test solvers' times which is the same methodology used in mostly any contest as far as I know, and the same I did for my own test too. Obviously, sometimes individual opinions differ from the test solvers' times.

The part about the puzzles not having a flow towards the end, well, the IB states that the attempt was made to place the easier grids in A and B, so thats an indicator that beginners should probably concentrate on solving the stuff at the start as best they can.

The last part too, I feel I can answer. This is, in spite of the small sized grids, a monthly test, and I think we have given warnings in the Beginners' topic that a monthly test is much harder than a Beginners' Contest. I realize of course that you've given other monthlies before, and I'm sure you found them really difficult too compared to the Beginners' Contests. I think its just the fact that you had a 5 minute window per grid to solve here that might make it seem a bit more annoying if you don't finish, but thats probably the reason for the partial points.

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Fairly Nice
How did you feel about the length / time limit for this test?	A bit too many puzzles
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Average

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	A bit too many puzzles
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Average

Not sure how much of this has been said before, couple of comments on the UI.

- During solving the puzzle, it is a convenience feature if, in addition to picking cells with the mouse, you can also navigate between cells using the arrows. I don't like the fact that such arrow key navigation "jumps over" a cell if it has a given number. I am pretty used to navigate in tables with arrows, be it Excel tables or Croco-Puzzle puzzles, I tend to make very fast movements like "3 to the down and 2 to the right". But ignoring given cells throws this up. I believe that ensuring your arrow keys always make a movement of one unit square would much more resemble industry standard, thereby improving de facto usability.

- The submit button was not very well placed at the far bottom. The confirmation request for submitting unfinished grids is a good idea, though, as one can accidentally miss a cell or two. Then again, one comment regarding the "force submit" - the help says Ctrl-click to force submit: this is actually alt/option-click on a Mac (well, at least on mine), may be worth showing that info as well.

- It would be nice to have a way to put pencil marks into cells, e.g. either-or notes or number ranges or so. I understand this being a screentest, that may not have been the intention, but this interface is much more promising to only ever be used for the odd screen test, and once it grows out of that box, such features might come in handy.

- I overran 60 minutes without being told, I notice this was raised before.

Hope that helps - thanks for the work in the UI and the contest
Zoltan

Zoltan -

There's a checkbox in the lower right you can uncheck so that the arrow keys don't jump over givens.

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Way too many puzzles (too little time)
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Average

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Way too many puzzles (too little time)
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Very nice

Hi Zoltan, Thanks for the detailed input.
Issue about force submit not being clear enough in Mac - It was a miss from my part.
Not having pencil marks - Regular Sudoku tests have pencil marks feature. It was disabled purposefully for this test. I realize it is a debatable decision, and has been noted by other players too.
Players not being notified after 60 minutes - probably the biggest flaw here.

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Very nice

Great format - the variety of tests swas very challenging. Increasing the difficulty as you went was a lot of fun too. I would enjoy seeing this format again.

How balanced do you think the puzzle types of this test were?	A bit skewed
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Fairly Nice
How did you feel about the length / time limit for this test?	A bit too many puzzles
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Average

Having a no pencilmark interface is fair for a screentest - after all you are challenging people to solve in their heads - but this is inconsistent with a very powerful undo system which encourages systematic guessing for the harder puzzles.

Given how hard some of the puzzles were, I'd have liked to have seen pencilmarks.

My only other gripe is that puzzles were either quite easy or quite hard, with not too much in between. Other than that, slick as usual.

Thank you so much.

Thanks Tom for very valuable inputs. Certainly will be useful if/when we consider this format again. I'm tempted towards having no pencil mark, no Undo, but also puzzles that don't require any of them. I thought most puzzles in this test fall into that category, but looking at overall feedback, that does not seem to be the case.

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Fairly Nice

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Fairly Nice
How did you feel about the length / time limit for this test?	A bit too many puzzles
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Average

How balanced do you think the puzzle types of this test were?	A bit skewed
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Way too many puzzles (too little time)
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Very nice

Nice test! There were a few too many hard puzzles among the first 10, I'd say, where I expected them to be relatively easy. There were also a few really beautiful puzzles where you could solve easily if you think about them the right way (which I often only noticed after guessing turned out to give a contradiction, then I'd see what I should have noticed in the first place) -- those were really nice.

I also learned a lot about some types of puzzles where I usually solve on-screen by making tons of notes!

I think the trickiest part is when there's unusual rules and the inverse rule is also in effect; it makes pretty puzzles, but hard to solve on-screen with time pressure.

No problem - and despite the seemingly indifferent colours provided by my feedback, I quite enjoyed the test. Having a quick scan of the results, there seem to be a strikingly large proportion of wrong entries. My best guess would be a large portion of these occurred as a result of guessing. Then again, this will always be a problem with difficult puzzles in smaller grids.

I'd be very interested to see a variation of this format, again with specific time limits per puzzle, but where the solver can choose the order in which puzzles are solved...

When I explained Branko (my perennial test solver) about this format, that is the first question he asked. "Will solvers be able to choose the order in which they want to solve?" So certainly something to keep in mind for the next one.

We can now view each others' exact submissions here

So this finishes with me being the only one in the top 10 to not have a single best time on an individual grid. Never thought consistency would be my positive in a test :P

Sudoku Screen Test is over. This was a different test from regular monthly tests, with every sudoku having time limit and solving in sequence. It was our first attempt, so certainly many points to consider in the next one, if we ever have that. Many thanks to all of you who shared their feedback.

Nevertheless, I sincerely hope everyone had fun to some extent.

Congratulations to uvo, janoslaw and misko for taking the top 3 positions. TiiT and jaku111 are really close to the top 3.

Prasanna has a top 10 rank for the second time now, and it is very significant achievement. Congratulations to Prasanna, Rishi and Rakesh for the top Indian positions.


Edited by debmohanty 2012-12-25 7:35 AM

I was disappointed just now to find that I was too late to try these puzzles. Most of the LMI competitions I've entered have had clearly displayed starting and ending times on the main page, whereas this one just vaguely stated that it would be open over the '96 hour period from Dec 21-24'. As solvers at this site clearly consist of a worldwide community, over many different time zones, it's definitely worthwhile to have the server times displayed for start time/end time/current time, so that we are all able to plan accordingly and be certain that we will have time to compete.

We always follow GMT, but yes, it was not mentioned in the Screen Test main page.
It was there in the demo page / submit page, but I assume you didn't check that. Sorry that you missed it, and we'll be mentioning it more clearly in future.

Thanks for the response. It was clearly my own mistake for leaving it too late. It was a shame for me because I was really looking forward to this competition. I had tried the demo a few days back but don't remember seeing anything about GMT. Of course I should have thought of that and will keep it in mind for the future.

Thanks for taking the time to design these puzzles - they are a lot of fun! I'll anxiously await the next Screen Test.

Just prepared an unofficial link for you - http://logicmastersindia.com/ST/ST3/unofficial.asp

Edited by debmohanty 2012-12-25 8:10 AM

Hi Deb,
I was thrilled with the no of Variant Sudokus given, i tried the demo Sudoku ..and was practising for the screen test..! however before the dead line time of screen test, i was not able to take it up; as i was travelling; can you please tell me when ll this screen test happen next time ?