Hi Neil,
I am not sure how exactly the timer behaved for you. We described some of the technical details earlier here. We did not include the instructions reading time in the 60 minutes.

To me, it looks like the score page has a bug, and is not considering your entry which was auto-submitted at the 60 minute mark. The server seems to have received you entries properly.


(Last 3 are submitted after 60 minutes mark)

why did the score page limit my time at 50:00 min ?

Fixed and you move to 11. Refresh your browser.

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Fairly Nice
How did you feel about the length / time limit for this test?	A bit too many puzzles
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Average

hi

How balanced do you think the puzzle types of this test were?	A bit skewed
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Fairly Nice
How did you feel about the length / time limit for this test?	Just right
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Very nice

Thanks Deb!
This will keep my mind active on lazy summer afternoons....

Thanks Deb, I think that explains it. Due to the bug, the clock kept ticking, so I didn't realise that my entry had been auto-submitted (although I'd assumed it would be). It all looks correct, in that it hasn't given me credit for anything I did after that.

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
What is your opinion about rules, points and scoring for this test?	Bit unfair
What is your opinion about the solving interface?	Very nice

Thank you Deb, it was very nice test.
I usually use pencilmarks very often, but this test disables it, so it was very different solving experience for me. Lot of solving in mind and memory was used.

For the scoring - in my opinion penalty system could have been better. In normal tests one wrong digit = zero points. In this test, submission with lots of wrong cells was still awarded with fair amount of points.
Next time, especialy for such small grids, there can be some limit of maximum wrong cells (6-8) for each puzzle to any points being awarded.

Edited by Gotroch 2012-12-23 4:30 AM

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Very nice

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	A bit too many puzzles
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Very nice

Clearly the worst of all the LMI tests i have taken so far....

Do you mean that with respect to experience or performance? I think you can elaborate so future improvements can be made (I do believe that was the main point put forward for needing a comments box in the feedback page).

I meant the test not my performance....My performance is more or less below average in each test....I wish i had unchecked the Ratings checkbox...There was no way to type possible numbers in a box....Besides I dont think the sudokus were arranged according to difficulty levels or 'perceived difficulty levels' as the IB goes....Basically the sentence looks like a joke to me .....overload and a total waste of sudokus....dont like the system which awards points for partially filled cells at all.....Right from the 2nd sudoku none went right...After 20 - 25 mins i was waiting for the test to end....Basically was pressing cntl and submit in each sudoku hoping for atleast 1 sudoku where there would be 'real flow' till the end....sadly none had(i felt)....every sudoku every cell i had to labour to find a number....A TOTAL DAMPENER of spirits after the last 2 months of beautiful sudokus in beginner's

How balanced do you think the puzzle types of this test were?	Very skewed
What was your opinion of the distribution of easy/hard puzzles?	Too many hard puzzles
What did you think about the puzzle quality of the test?	Below Average
How did you feel about the length / time limit for this test?	Way too many puzzles (too little time)
What is your opinion about rules, points and scoring for this test?	Exteremely unfair
What is your opinion about the solving interface?	Below Average

'No comments'

Most of your criticisms are valid, and of course we are happy to hear any kind of ideas/feedback as this kind of Screen Test using Sudokus was a concept tried for the first time here. I feel I can answer some of your points though.

The perceived difficulty part is based on test solvers' times which is the same methodology used in mostly any contest as far as I know, and the same I did for my own test too. Obviously, sometimes individual opinions differ from the test solvers' times.

The part about the puzzles not having a flow towards the end, well, the IB states that the attempt was made to place the easier grids in A and B, so thats an indicator that beginners should probably concentrate on solving the stuff at the start as best they can.

The last part too, I feel I can answer. This is, in spite of the small sized grids, a monthly test, and I think we have given warnings in the Beginners' topic that a monthly test is much harder than a Beginners' Contest. I realize of course that you've given other monthlies before, and I'm sure you found them really difficult too compared to the Beginners' Contests. I think its just the fact that you had a 5 minute window per grid to solve here that might make it seem a bit more annoying if you don't finish, but thats probably the reason for the partial points.

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Fairly Nice
How did you feel about the length / time limit for this test?	A bit too many puzzles
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Average

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	A bit too many puzzles
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Average

Not sure how much of this has been said before, couple of comments on the UI.

- During solving the puzzle, it is a convenience feature if, in addition to picking cells with the mouse, you can also navigate between cells using the arrows. I don't like the fact that such arrow key navigation "jumps over" a cell if it has a given number. I am pretty used to navigate in tables with arrows, be it Excel tables or Croco-Puzzle puzzles, I tend to make very fast movements like "3 to the down and 2 to the right". But ignoring given cells throws this up. I believe that ensuring your arrow keys always make a movement of one unit square would much more resemble industry standard, thereby improving de facto usability.

- The submit button was not very well placed at the far bottom. The confirmation request for submitting unfinished grids is a good idea, though, as one can accidentally miss a cell or two. Then again, one comment regarding the "force submit" - the help says Ctrl-click to force submit: this is actually alt/option-click on a Mac (well, at least on mine), may be worth showing that info as well.

- It would be nice to have a way to put pencil marks into cells, e.g. either-or notes or number ranges or so. I understand this being a screentest, that may not have been the intention, but this interface is much more promising to only ever be used for the odd screen test, and once it grows out of that box, such features might come in handy.

- I overran 60 minutes without being told, I notice this was raised before.

Hope that helps - thanks for the work in the UI and the contest
Zoltan

Zoltan -

There's a checkbox in the lower right you can uncheck so that the arrow keys don't jump over givens.

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Way too many puzzles (too little time)
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Average

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Way too many puzzles (too little time)
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Very nice

Hi Zoltan, Thanks for the detailed input.
Issue about force submit not being clear enough in Mac - It was a miss from my part.
Not having pencil marks - Regular Sudoku tests have pencil marks feature. It was disabled purposefully for this test. I realize it is a debatable decision, and has been noted by other players too.
Players not being notified after 60 minutes - probably the biggest flaw here.

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Very nice

Great format - the variety of tests swas very challenging. Increasing the difficulty as you went was a lot of fun too. I would enjoy seeing this format again.

How balanced do you think the puzzle types of this test were?	A bit skewed
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Fairly Nice
How did you feel about the length / time limit for this test?	A bit too many puzzles
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Average

Having a no pencilmark interface is fair for a screentest - after all you are challenging people to solve in their heads - but this is inconsistent with a very powerful undo system which encourages systematic guessing for the harder puzzles.

Given how hard some of the puzzles were, I'd have liked to have seen pencilmarks.

My only other gripe is that puzzles were either quite easy or quite hard, with not too much in between. Other than that, slick as usual.

Thank you so much.

Thanks Tom for very valuable inputs. Certainly will be useful if/when we consider this format again. I'm tempted towards having no pencil mark, no Undo, but also puzzles that don't require any of them. I thought most puzzles in this test fall into that category, but looking at overall feedback, that does not seem to be the case.