Thanks for all the feedback Fred. I'd like to address those two topics too.

When Deb first explained the concept to me, I personally was pretty excited for it. It seemed like a fun, light way to end the year. I'm glad a lot of others enjoyed it too.

A few of the decisions/features/format of the test are of course debatable, but overall I at least thought it was a fun experience (including the minute-refresh feature which can only be used best in such a test :P ).

To Fred's point about number of puzzles, I look for the same thing when evaluating number of puzzles. Every test is ideally planned so that the top 4-5 scorers can finish it and the rest can come close. As Deb mentioned, Branko was the first one to attempt this test, and then I did it. Seeing my score, both me and Deb were unsure of whether to reduce the time limit to 50 minutes. In the end we went with the belief that I'd had a good day and kept it at 60, and thankfully so, as this turned out to be the ideal limit for the first few to solve, as Fred said. When discussing this test, we thought at length about the beginners and that is why there was the categorization A, B and E in the IB, and I think Deb's already explained about that. So, we tried our best to keep a target for everyone, at the same time keeping enough to give the top players a challenge. At least going statistically, that worked ok.

The point about pencil marks has been debated by everyone, and most seem to feel that its required. When planning the test, I was in favor of disabling pencil marks because of a few reasons -
A. I know most people use notations to proceed, but I've seen it being very much possible to go ahead without them, even on harder grids (There's a site or two that have online solving without pencil marks too, and Fred frequents one of those ;) ). I just felt it was more fun this way.
B. After attempting the test myself, I had solved 29 puzzles logically, guessed on one, and I went back and solved that one too pretty easily after the test. So I told Deb as much, and the system seemed good to go with.

I think Tom's point is good about the Undo button contradicting the decision to disable pencil marks, and here again, it didn't strike me as I didn't use the button during my solves (I used Clear All once I think). We concentrated more on some other features and workings of the page instead, like the warning if an incomplete puzzle is submitted.

I'm glad that some people have commented that they enjoyed it overall, and with a few changes I do think we can have this format again.

With Christmas getting in the way I hadn't gotten around to commenting.

I think much is already said. I, like Fred also opted far more often to use guessing instead of logic on these puzzles than normal. This was mostly caused by using a lot of notation while solving and I just noticed I missed a lot in the beginning and making silly errors as i misremembered things, which led me to place the wrong digits. These would all probably not have happened if I had made notations. The high amount of guessing was mostly the reason I voted a bit lower than normal on the enjoyability of the test. Resorting to guessing always makes a puzzle les enjoyable for me and with the fixed time limit I have to try to get it out somehow. So I would prefer pencilmarks over an undo button, as I never use those anyway. I guess in another colour and find it far easier to just remove the digits of a single colour using the space bar than use the undo button.

I also think there should be a higher cutoff point for how quickly you get points for a puzzle. I would say you need to have at least half of the grid filled to receive points, because I don't think people should be able to get points for looking at a grid for 15 seconds and placing 1 or 2 digits. I also think a wrong digit should count differently than an empty cell. I feel you should score for the amount of points for the total of (correct digits-wrong digits), with at least half the grid filled, with the extra rule that you can't get any negative points for a puzzle, so when the amount of wrong digits is higher the amount of correct digits is should score 0. I get Deb's argument that someone who guesses and then it gets autosubmitted would get less points if it's wrong, even though they did the same parts logically, except he could also have guessed correctly and get the extra points. And as most guesses are a 50-50 chance of being correct, this would in the long run just even out.

I thought the amount of puzzles was good. The best ones managed to get through it completely, without anyone having a lot of time left over. I think the rating might be partly caused by the fact that many people though, "it's only 6x6 sudokus, that's going to be really easy", as most 6x6 puzzles that you can find are just really easy. And then when a 6x6 puzzles stumps you, like some of these puzzles definitely will beginners, it might be demoralising. But that happens to the best of us. I think this test deserves a repeat, and maybe with everyone now better knowing what to expect the ratings will probably go up.

One final comment. I would have liked it if the letter and number of the puzzle were listed somewhere on the puzzle page while solving (in front of the puzzle name for example). I find it handy to know how far along I am in the puzzle set.

Edited by Para 2012-12-28 4:20 AM

Thanks guys for the in-depth analysis. I think it is safe to assume that for a similar test next time, pencil marks should be enabled, if the sudoku difficulties are more or less same. At the same time, I feel adding pencil marks is not going to help much to beginners. (for example : take the Consecutive Sudoku. I see lot of beginners struggling in that. If you miss the 4 (out of 5 possible) dots in row 2, you are never going anywhere. Not sure if pencil mark would have helped here.)

Another theoretical thought I had about pencil marks is that, LMI's pencil marking is very unique. It is not certainly easy to use, especially if you would be using it for the first time. So I was of the opinion that having pencil marks significantly benefits players who are used to LMI's interface. So disabling pencil marks was one way to make it more balanced to paper players.

Anyway, given that almost all comments suggest that pencil marks should be enabled, we should just do it accordingly next time.

About undo : I remember now that during my initial visualization of the test, I had thought of limiting Undo to 2 steps. (Just enough steps to undo mis-types). But I had forgotten about it when the test took the final shape. So, yeah, I agree that not having pencil marks and enabling undo is a perfect recipe for guessing.

Will do next time.

On the contrary, I feel LMI's online interface is the best I have found till now. It's way-way-way better than any other interface out there. And only because of LMI's interface, I have started using Pencil Marks - earlier I was used to solving Sudokus with No Pencil Marks at all. (I still don't use Pencil Marks when solving on paper).

I kept putting off and forgot to officially do this test (would have been my second, I think) despite doing the demo. I think the reason may have something to do with the fact that a puzzle can be skipped as incomplete, and with the appearance as a whole of a 30 puzzle test. You can't please everyone, and if the screen test had been one hour for 12 puzzles - something I don't consider too out of the ordinary, as those not as skilled can then spend five minutes on a puzzle without stressing - then some others would have said "Where's the challenge?" With a test of 30 puzzles, and the ability to skip puzzles, I don't blame someone who skipped over 5 puzzles they couldn't do and otherwise solved 18 (right or wrong, don't matter) before running out of time for saying there was way too many puzzles. Personally, I thought it was too many; you have to literally solve each in, on average, two minutes. However, if the goal had been to solve any 20 and skip any six, that would give a little more time and allow people to customize their test for the best result. Obviously, bonus points could then be awarded for correct solves of 21-30 (not by puzzle number but by number of correct puzzles), and the best performances thus would well exceed the otherwise expected maximum test score based on 20 puzzles (as an example). For 24 puzzles, for instance, that shaves off 12 minutes that can be allocated to other puzzles, giving 2:30 to each puzzle instead of 2:00, and skip any six you don't like the look of.

Just my opinion.

Also, a booklet of the puzzles now that the test is complete might be nice. :)

Cyclone


Edited by Cyclone 2013-01-04 3:48 AM