How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	A bit too many puzzles
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Fairly Nice

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Way too many puzzles (too little time)
What is your opinion about the solving interface?	Very nice

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Fairly Nice

...I have an urge to rate the puzzle quality as below average due to the cooked puzzles. D:

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Fairly Nice
How did you feel about the length / time limit for this test?	Just right
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Fairly Nice

Its given in the guide that the categories were done according to familiarity/accessibility more than difficulty.

About the evil JOCs, firstly, I'm really sorry We're looking at a way to fix the problem. We're thinking of going with canceling B2 and E5, and awarding [points scored over remainder / seconds spent over remainder * seconds spent on B2,E5]. We would like to know general thoughts on whether this is fair.

This was mainly because of the rules being too long. The rules have now been shortened to be accommodated on the applet. Thanks for the notification.

Heh, I think that may have bitten me. When I was solving E4, I don't think the "No arrow means no sequence" part had been added. I looked over my solution, glanced at the rules summary, saw no contradiction, and submitted and moved on. Not sure I would have had time to find the correct solution even if I had the right rules, but oh well. Much happier with my performance regardless of the issues. I still wasn't going to finish, but it was great to get so much closer. Thanks for the cool puzzles!

I'm curious whether people actually continue solving after the time is up or not. If they do, you can try removing the seconds spent in B2 and E5 and count the unofficial time for the length of time spent on B2 and E5? (For example, in my case, I spent 300 seconds in B2 and like 50 seconds in E5, so I gain an extension of 350 seconds, counting the first 350 unofficial seconds as official.)

(Ok this suggestion is a bit biased since with an additional of 300 seconds or so I would have finished the test. :P )

Edited by chaotic_iak 2013-11-16 9:23 AM

This would be ideal if everyone continued solving after time. But thats not the case, so its unfair to those who stopped.

I can't see a good way out then. My opinion would be to drop B2 and E5, then let the total score be multiplied by 3600 / (seconds spent in all other puzzles)...basically the same thing as you suggested. I feel something is unfair about this, but I cannot really point it out...yeah it's weird. xP

If two players solved correctly the 28 remainder sudokus in the same time, and one spent much time on B2 and E5 as the other, would they get finally the same number of points ?

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Very nice

Yes. I think in the event of the puzzles being canceled, that is fair.

Oh wait, the displayed rules for E4 were incorrect? (incomplete, not mentioning that no arrow means not in order). That's REALLY EVIL, and I'm pretty sure it caused me to submit a solution that was valid for the displayed rule but counted as wrong.
Hmmm... Prasanna, can we hope that future screen tests get implemented more carefully? Thanks!

-- Denis

Wheeee… going to http://logicmastersindia.com/ST/ST4/ after having finished the test (was wondering whether that allowed me to review the puzzles), my browser goes into something like a redirect loop. (Safari 6.1 (7537.71) / Mac)

The test was fun, but kind of stressful. Good practice.

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Too many hard puzzles
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
What is your opinion about rules, points and scoring for this test?	Bit unfair
What is your opinion about the solving interface?	Very nice

I thought about what could the best decision to have a fairer ranking, given the 2 wrong puzzles. It's a bit complicated and I don't know if it's technically doable, but that's what I thought:

-cancel B2 and E5 and remove the times of both puzzles for every player.
-calculate a new time limit as 3600-(maximum time spent by a player on puzzles B2+E5)
-For players who didn't finish the 28 puzzles in the new time limit, you've to estimate points for the last puzzle (the puzzle player was solving when the new limit was reached). For example, you can take a percentage of points proportional to points gained and solving time of the player in the old ranking (if a player solved his last puzzle in 200 seconds and the new time limit was reached after 100 seconds, assign 50% of the points the player gained).

In my opinion, it's the best way to do as if both puzzles never existed.

Fred

I'm sorry Denis, and others, for the problems. We will indeed work to make sure these don't happen in future Screen Tests. The problem here was the rules were so long they slipped out of the interface even though we entered them right. I confess I hadn't considered that possibility, and definitely will in future. About this time I agree that Rossini would've been a big problem for folks who hadn't seen/remembered the rules on the guide and relied on the interface. This seems a harder issue to fix than the 2 JOCs which can be dropped as suggested previously.

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Fairly Nice
How did you feel about the length / time limit for this test?	A bit too few puzzles
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Fairly Nice

First of all, I really enjoyed the set, so thanks for putting the puzzles out there!

I don't necessarily care about my ranking, but wanted to throw out there that, because I didn't think I was getting to the end, I did get bitten by the lack of converse (aka I was low on time, took a guess, and my solution was correct without the converse). I don't think there is a way to really fix that one (aside from either the JOC solution of throwing it out, but now we're possibly talking about taking 1/4 of the time spent on the test out, or alternatively grading that problem correct if they hit a correct solution without the converse rule). I do have the feeling that I would've come close to finishing Arrow had all of the rules appeared on the screen, but I don't know if I would've actually done it, so I'm certainly not going to push for any "advantage" from that puzzle.

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Very nice

A nice test with nice puzzles. However, given that on the last sudoku screen test I ran out of time, this year I skipped a couple of puzzles more quickly. Obviously, I then found I had several minutes wasted at the end. Clearly this is my fault, and there was a note somewhere that these puzzles were generally easier than last years, but I didn't realise just how much easier they would be. Given that a lot of people won't expect to complete every puzzle, 'tactics' and puzzle choice becomes very important (given that the strategy 'never skip' clearly cannot be optimal, except for the best), so I wonder if it would be better to have a more accurate gauge of the level of difficulty of the puzzles, if possible?
Anyway, thanks for the test; I really enjoy screentests, and look forward to the next puzzle one. It's also nice to only have to set aside an hour or so, rather than two or more!

Prasanna: don't obsess over how to fix things -- all in all this test was still a lot of fun and I'm sure (at least I hope) that any vaguely reasonable solution to the problem of the two faulty JOCs will be fine for everyone. Also, no worry about the Rossini -- it's too bad, but it's not something I'll hold against you. Stuff happens.

Denis

Few notes :

Firstly, we're making some changes to the score page now (computing bonus, canceling B2/E5 using my previous suggestion, and computing the changes). So if the score page does weird things for a while today, please bear with us

Secondly, we're not considering E4 errors for the bonus, and the bonus will be computed for the rest as announced. Something that isn't announced is, even players who have just one Sudoku wrong are eligible for bonus. So its either players who have just one grid wrong, or players who have 10 cells wrong across multiple grids.

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Average