Good idea. We can easily do something with the timer in the last 5 seconds to catch the eye of the participant. Maybe beep. Maybe changing colour or making bigger, since not all computers may have sound system in place.
Having a more visible timer during the entire session might be distracting to players. We wanted players to see the puzzle and solve it as quickly as possible rather than concentrating on the time left. Thats why we tried to camouflage the timer in the background. But we'll surely look into it if more people suggest a better timer :-)

I'm glad you enjoyed the Screen Test :-)

While we have got bigger green bars for most of the questions, we have got bigger orange / red bar for the question "Were most of the puzzles solvable on screen?"

See here for details.

That obviously means that while players liked the concept and interface, but the main area of improvement is puzzles.
In next few posts, I would cover my thoughts on the Screen Test puzzles.

To start with KnightSafe, Mazes, BeadCount
All three of them visual puzzles. Most players got them right. Mazes being the easiest one, I wanted mazes to be first puzzle in the test. But Rohan thought it might disable players' eyes if it the first puzzle :-)
Either way, I think all 3 puzzles were solvable easily on screen. They all have high percentages of success rate too.

The next 2 are Minimum Mines and Blank Hitori, both needing mild logical steps.
I had planned for a bigger Minimum Mines - an 8X8 one - but I'm glad that we didn't include it.
The 5X5 one needs minor additions, and many players cracked it in less than 30 seconds.
Blank Hitori - It had lot of "black" squares. Many players got it right, but few players answers 2 or 4.
I believe that these 2 puzzles are nice selections for the screen test.

The next two - CellSudoku and SudokuTwins
CellSudoku - One of the few puzzles where we probably didn't get it right.
We wanted a visual sudoku, and not a sudoku with standard solving approach with pencil marks etc.
It is a very simple puzzle, needing a two step approach. See spoiler below.

But given that only few players got it correct - we should have a smaller grid or more time for this puzzle.

SudokuTwins -
Unfortunately, grid A has two solutions because of a misplaced digit. Although the puzzle as a whole had single solution, it definitely pissed off many players.
And also, the labels were different between grids B and C.
I'm to be blamed for the mess in this puzzle.

While we wanted to have both the Sudokus easy and accessible to everyone, it didn't turn out that way.
This is one of my biggest disappointments in ST1.

I was not very confident of Stubborn Snake in the Screen test. But to my pleasant surprise, many players solved it correctly.

MultiESB had 10 circles, so I would think many players simply answered 5.
I believe that this can be solved on screen without much problem (though the original planned puzzle was way too difficult).
The first 3 loops can be drawn almost immediately.


Next puzzle LuckyColumn
If I were to remove exactly one puzzle from ST1, it would probably be LuckyColumn :-)

LMICount and MissingWords are two more visual / observation / counting puzzles.
Original LMICount was 8X8 and we decided to make it 6X6. May be we should have had the 8X8, since the 6x6 was solved correctly in 33 seconds on an average.

The original MissingWords puzzle was too error prone. See image below. We decided at the last moment to change it, and it was a much wise decision.



Operations Twist was the only arithmetic related puzzle. It turned out to be most correct 60 seconds puzzle.

ABCConnect - This was one standard puzzle type put in ST#1 without any modifications. The ideas was that many players solve ABC Connect mentally. We were correct, 75% players who answered it got it correct.

BlackAndWhite If I were to remove 2 puzzles from ST#1, this one will be the 2nd puzzle, the first one being LuckyColumn.
It is not a great B&W puzzle - but it insisted that player solve and count the puzzle on screen, which seemed little tough to me.
Players who had read the Guide must have noticed that we had provided the grid size for this puzzle (and some other puzzles)
The idea of providing grid sizes is that player may want to solve it on paper rather than on screen if the puzzle can be easily drawn on paper.
[ In some WPCs, players are allowed to redraw the images on paper ]

The next two puzzles IrregularSplit and CorrectXV turned out to be as bad as the 45-seconds-sudoku puzzles, very few players solving them.

The IrregularSplit was not spectacular, it doesn't work like a sudoku, but more of a jigsaw fitting puzzle. It all depends upon how quickly one can do all the combinations.
While IrregularSplit had a decent success rate, CorrectXV turned out to be least favorite puzzle fetching -0.16 points per attempt.

I'll explain below how we expected players to solve it, but since most players didn't get it, I accept that there was not enough time for this puzzle.
Just like the example - the four 4 grids were similar looking. Either an X or a V was the only difference between the grids.
We thought players would focus on that, rather than solving the grid.

In grid B, there was an additional V on Column 6, Which made column 6 unsolvable.
In grid C, there was less V on Column 3, which made column 3 unsolvable.
In grid A, there was an additional V in Row1 (which means there was no place for 5 in Row1 and hence grid A is also unsolvable)


The example used a similar technique, and somehow we thought that players will be able to solve this in 60 seconds. It was not to be :-)

It was really good from the regular tests. I think we should go for this kind of test regularly may be once in a month. I am suggesting this because the time for every puzzle is fixed, and many of the times the best comes out when we are under pressure, this is a perfect test for bringing the best from the players and improving their skills.

Thanks a lot for all who made this initiative very successful.

Next one is CarelessArrows another visual puzzle. But unlike many other visual puzzles, it can be quickly checked whether your answer is correct or not.
I liked this puzzle very much.
It is only because of this puzzle that we had a DropDownList rather than buttons for answer keys.

Mastermind is like standard Mastermind. Given that options were given, it again was more like a check-check-check puzzle.

ImagesCount - When I started thinking about ST#1 around 3 months back, ImagesCount was the first puzzle I created. Originally it had 9 images. We reduced it to 6. Given that there were still 120 circles, it seemed too difficult.
Also that, the reflection possibility was missing from the example image (though the instructions had it), it made matters worse.
There could have been a better puzzle!

MiddleStick - Again one of the earliest puzzles. See image below for original version. One of the wisest decision to drop two sticks and make it 2-colored.

That brings us to the last 4 puzzles -

MissingDigitKakuro This type went to several iterations, we made multiple puzzles of this type and we thought we chose the easiest one.
It needed to be solved only in the left top region ( i would think that most players start looking that region first :-)
May be this could be have been easier. A smaller range 1-6 would definitely have helped.



CodedSkyscraper - Our original thought was that CodedSkyscraper and CorrectXV are the only 2 tough puzzles which would challenge the top solvers.
In Coded, it takes a while to realize that {A,D}={3,4} and 3+4 = 7
Looking at either column3 or row3, D can't be 4. so A=4
Given the very low success rate for this puzzle, it was not best fit in 60 seconds.


RareSkyscraper - Irrespective of the success rate, I think that this was an easy puzzle. Especially if one had solved the example carefully and found the pattern of 9-8-7-6....

BrilliantColour
The best puzzle of ST1 in my opinion, and we intentionally had kept it at the end.

Finally, the technical part - It was a good technical challenge to implement the Screen Test since it was very different from what we've done so far at LMI.
But we all love challenges in some way, don't we?

We are glad that ST#1 worked as flawlessly as we designed. Thanks to everyone for sharing improvement ideas, we'll keep them in mind if/when we do ST#2.

I've no idea if making Screen Tests a monthly contest makes sense or whether it is feasible to write good puzzles so frequently. We've to wait and see.

One of the most satisfying things from ST#1 is there were lot of Indian solvers. Also, it really feels good to see many old solvers back (Omkar/Ritesh, are you reading this?)
30 minutes time and no printer requirement seem to be the key things that worked here.

Keshava Murthy continued from his FLIP performance, and got the best Indian rank (he admitted that he made some lucky guesses :-)
Rajesh did reasonably well, though I think he was very conservative.
Rishi stood 3rd, and it is good that he is spending more time on puzzles now.

Rakesh had a disappointing test - he should have got much more.
Amit - I'm not exactly sure what the problem was - but he certainly could have got more points.

Congratulations to everyone and thank you for participating.

Thank you Deb for sharing your views on my suggestion.
I agree that it is not easy to design these kind of puzzles which can fit the time frame (60 or 45 sec).
I think we should increase and decrease time depending on the difficulty level of the puzzle.
We can work on this tests if some of the solvers can become authors and contribute some puzzles, may be not 25 puzzles all the time but decent enough number of puzzles with varying difficulty.
Let see how this goes I also will try to desgin puzzles which suits this test (although I have never designed puzzles )

Wish me luck.

Thank You.

It was a nice test, and a good change. And, most importantly, it introduced the ST concept to solvers like us who have never been to a WPC. The interface worked well overall and the puzzle selection was good. The main challenge was to create puzzles which can be solved without use of paper, and the test was largely successful at that.

Deb: Thanks for your views on the puzzles. They are very comprehensive and almost cover everything that I might have wanted to comment on.

One big mistake which I made in this test was to try to check/verify my answer. Inevitably, the time for the puzzle would run out. I'd try to note the answer for entering during the review period. But, during the review period, I instead concentrated on solving one "almost-finished" puzzle, ultimately getting minus points for that, and even not able to enter the planned ones. A better approach would be to enter answers as soon as you arrive at them. No need to double check. And, to not leave too much to do in the review period.

Many people have commented on the amount of time being inadequate. But, is there really a solution for that? If you make the time 90 seconds for each puzzle, inevitably the complexity would also increase a little bit. I think the timing was alright for 80-90% of the puzzles, as eight solvers were able to cross the par score - 1350.

The only cases which need a re-look are the ones which had the least solvers -
(1) Cell Sudoku (the main problem was the size of the sudoku, which put off most solvers...the in-out concept was actually very well applied)
(2) Correct XV (I was able to eliminate A and B in 60 seconds, then could not look at it in the review period)
(3) Coded Skyscraper (size of the grid was the main obstacle, again)
(4) Missing Digit kakuro (I think, due to the time pressure, most solvers were trying to eliminate digits rather than solve the kakuro. At least this was my approach, so I was able to eliminate 1,2,3,4,5,8,9 easily. But could not arrive at the final answer - between 6 and 7)

Maybe, one suggestion for ST2 could be to increase the review period a little bit (~150 seconds)...I was unable to do what I intended to do in the review period...it went off pretty fast. And, perhaps adequate time for sudoku/kakuro puzzles.

An interesting observation: There were 6-7 solvers who got more than 110 points for a single puzzle, i.e. solved a 60-point puzzle in less than 10 seconds (and two of them twice). But none of them finished in the top 50.

Rakesh - thanks for your views.

After the end of day1 of the test, nobody had solved Cell Sudoku. Thats when we realized that it is trickier that we had expected.
The idea was borrowed from Mock9 Round 1 Page 8
Rohan also got the answer quickly, and we thought it should be solvable. But we were wrong by a huge margin.
I'm not exactly sure if a 6X6 grid with similar trick would have been much better.

As I wrote, I don't have a problem with CorrectXV and Coded Skyscraper being least solvable. We were expecting only the top solvers to do it.

Regarding Review Period :
I have mixed feeling about the review period. Does it really help?
In my original plan, Review Period didn't exist.
But Amit suggested that we must have a review period of at least 120 seconds. Rohan was not very certain too.
I was thinking of an "Add-on" time for each puzzle.
Every puzzle will have an "Add 15 seconds" button. If you are close to solving it but still need few more seconds, you could add 15 seconds to the puzzle (at the cost of losing x% of the points).
To me that is better than Review Period - here is why I think so -
You are working on a puzzle, it is in your brain. It is easy if you have 15 more seconds to solve the puzzle.
If you switch to another puzzle, and again look at it in Review Period, you again have to bring the puzzle back in your brain.

We discarded Addon time for ST#1 because it looked complex to us.

Some players made heavy guesses... Solving MinimumMines in 2 seconds, KnightSafe in 2 seconds.
I'm not sure how to prevent that. Suggestions welcome.

I also have mixed feelings about the -ve scoring that we had.
Ideally -ve scoring should be applicable to random guesses.

Let me take an example -
BeadCount - It is a visual puzzle. J has 11 beads, A has 10.
J is the correct answer. Some players made minor counting mistakes and answered as A. [ A is most incorrect submission ]
But there are players who would either randomly guess or made major counting mistakes, and answered as C / D / E or whatever else.

158 | J
17 | A
3 | C
2 | G
2 | D
1 | F
1 | E

Take MultiESB - The correct answer is 4. If someone answers 5, it could be a counting mistake. But how on earth can the answer be 7 (there are only 10 circles - so the answer could be max 5 without even drawing any loop)
54 | 4
43 | 5
14 | 6
7 | 3
6 | 7

The point I'm trying to make is -
the negative scoring didn't consider this difference - a minor counting/solving mistake OR a random guess.
May be there should not have been any negative scoring for the most incorrect submission.

The negative scoring was introduced to dissuade players from randomly guessing. But given the nature of the -ve scoring, lot of players played conservatively.

I think the drop down should be active after 4-5 sec this will force few of the solvers who try to guess to solve the puzzle since they dont't know what are the options

For cell sudoku:
The options will be clear because of grid size, to prevent from guessing, the answer must be manipulated to some other form may be some mathematical operations, in this case both the mathematical expression to be used and the drop down will be visible at the same time.

For Lucky Column:
The options will be clear because of alphabets used for the naming of columns, to prevent from guessing, numbers can used to name the columns an then the answer must be manipulated to some other form may be some mathematical operations, in this case both the mathematical expression to be used and the drop down will be visible at the same time.

For ABC Connect:
Same as lucky column replace the alphabets with numbers.

For Black and White:
Manipulate the answer using a mathematical expression which will be shown at same time of dropdown.

For IrregularSplit:
Interchange numbers and alphabets, again same process manipulating the answer.

For Middle stick:
Replace alphabets with numbers, again same process manipulating the answer.

For Missing digit Kakuro, Coded Skyscraper, Rare Skyscraper, Brilliant Color:
Same process of manipulating the answer, replace the colors with numbers in brilliant color puzzle.

Are there any puzzles in ST1 that can be solved in less than 4 sec?

The number of options in the dropdown should increase also not all the answers will fit the mathematical expression (to avoid reverse calculations).

I hope I explained it clearly, any modifications or suggestions to this are welcome.

Thank You.

Are you suggesting something analogous to "answer key being displayed after some time in sudoku tests"? How does this change the situation? The heavy guessers will punch their answers in 5-6 seconds.

I did not understand this totally, perhaps. But it seems you are suggesting that we use a drop down (as in ST1) PLUS an answer key. Is that correct? And that the answer key should not be known beforehand. The idea is good and will certainly bring down guesswork but whether it goes with the spirit of a Screen Test or not needs to be determined...

Maybe not 4, but definitely within 10 seconds.

Can you explain your point with some example here? I am unable to understand it fully.

I am not able to understand it at all

Do you really want to prevent that?

Guessing is inevitable if you have multiple options as answers. But then, if you do not have options, the test will look more like a normal test. And, heavy guessers got more wrong than right. So its not as if someone is going to finish in top 20 by heavy guessing.

Still, if you want to reduce the probability of success of guesses, simply increase the number of options.

Hi guys I am extremely sorry for not making my points clear enough .

As rakesh said to increase the number of options to prevent from guessing in not always possible, like in the case of missing digit kakuro, rare skyscraper, irregular split etc. the number of options are fixed or rather can be decreased but not increased.

The idea is
1) There will be a mathematical expression in which there will be one variable, i.e is the answer obtained after solving the puzzle. Say 6(A) + A^2 is the expression where A is the answer.
Player after solving the puzzle needs to calculate the result of the expression and then choose it from dropdown box, so now we can increase the number of options.

2)Regarding the reverse calculation I guess I made a mistake it should have been like this, all the possible options that will result from the mathematical expression and some other numbers (to increase the options) should be there in the dropdown.

3)The dropdown and the mathematical expression will be active only after 4-5 sec (which I think should be fine) so that no one gets to know the answers or the expression before hand.

Even after all this if a player is able to guess the answer then it can't be helped, but I think this will decrease the number of playes from guessing.

Thanks for explaining, and I now understand what you are saying.
But it would simply increase the solving time unnecessarily, and adds an error-prone element to the puzzle.

To answer Rakesh's question "Do you really want to prevent that? " - I don't think we really need to do anything to prevent it. Definitely not by adding more complexities for genuine solvers.

Hi,

I feel, we should not do anything to prevent Guess work, True Champions will always emerge in a long run.
Guess work will not produce results always, Just to prevent Guess work, we should not complicate Puzzle Interface.
Easier/simple Interface will attract more puzzlers, I think which is also one of the main aim of LMI.

This is my personal feeling.

Thanks again for your precious time in organizing wonderful event.

Best Reagrds,
Keshav

Edited by keshava.hs 2010-12-07 9:45 PM