I thought about what could the best decision to have a fairer ranking, given the 2 wrong puzzles. It's a bit complicated and I don't know if it's technically doable, but that's what I thought:

-cancel B2 and E5 and remove the times of both puzzles for every player.
-calculate a new time limit as 3600-(maximum time spent by a player on puzzles B2+E5)
-For players who didn't finish the 28 puzzles in the new time limit, you've to estimate points for the last puzzle (the puzzle player was solving when the new limit was reached). For example, you can take a percentage of points proportional to points gained and solving time of the player in the old ranking (if a player solved his last puzzle in 200 seconds and the new time limit was reached after 100 seconds, assign 50% of the points the player gained).

In my opinion, it's the best way to do as if both puzzles never existed.

Fred

I'm sorry Denis, and others, for the problems. We will indeed work to make sure these don't happen in future Screen Tests. The problem here was the rules were so long they slipped out of the interface even though we entered them right. I confess I hadn't considered that possibility, and definitely will in future. About this time I agree that Rossini would've been a big problem for folks who hadn't seen/remembered the rules on the guide and relied on the interface. This seems a harder issue to fix than the 2 JOCs which can be dropped as suggested previously.

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Fairly Nice
How did you feel about the length / time limit for this test?	A bit too few puzzles
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Fairly Nice

First of all, I really enjoyed the set, so thanks for putting the puzzles out there!

I don't necessarily care about my ranking, but wanted to throw out there that, because I didn't think I was getting to the end, I did get bitten by the lack of converse (aka I was low on time, took a guess, and my solution was correct without the converse). I don't think there is a way to really fix that one (aside from either the JOC solution of throwing it out, but now we're possibly talking about taking 1/4 of the time spent on the test out, or alternatively grading that problem correct if they hit a correct solution without the converse rule). I do have the feeling that I would've come close to finishing Arrow had all of the rules appeared on the screen, but I don't know if I would've actually done it, so I'm certainly not going to push for any "advantage" from that puzzle.

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Very nice

A nice test with nice puzzles. However, given that on the last sudoku screen test I ran out of time, this year I skipped a couple of puzzles more quickly. Obviously, I then found I had several minutes wasted at the end. Clearly this is my fault, and there was a note somewhere that these puzzles were generally easier than last years, but I didn't realise just how much easier they would be. Given that a lot of people won't expect to complete every puzzle, 'tactics' and puzzle choice becomes very important (given that the strategy 'never skip' clearly cannot be optimal, except for the best), so I wonder if it would be better to have a more accurate gauge of the level of difficulty of the puzzles, if possible?
Anyway, thanks for the test; I really enjoy screentests, and look forward to the next puzzle one. It's also nice to only have to set aside an hour or so, rather than two or more!

Prasanna: don't obsess over how to fix things -- all in all this test was still a lot of fun and I'm sure (at least I hope) that any vaguely reasonable solution to the problem of the two faulty JOCs will be fine for everyone. Also, no worry about the Rossini -- it's too bad, but it's not something I'll hold against you. Stuff happens.

Denis

Few notes :

Firstly, we're making some changes to the score page now (computing bonus, canceling B2/E5 using my previous suggestion, and computing the changes). So if the score page does weird things for a while today, please bear with us

Secondly, we're not considering E4 errors for the bonus, and the bonus will be computed for the rest as announced. Something that isn't announced is, even players who have just one Sudoku wrong are eligible for bonus. So its either players who have just one grid wrong, or players who have 10 cells wrong across multiple grids.

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Average

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
What is your opinion about rules, points and scoring for this test?	Bit unfair
What is your opinion about the solving interface?	Fairly Nice

Very pleasant time. I usually don't play Sudoku contests, but this one was very attractive. It would have been better for me to unterstand what "Converse rule applies" meant...

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Very nice

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit easy
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	A bit too few puzzles
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Fairly Nice

Overall it's a fairly nice test, I enjoyed it.

It's a bit unfortunate that there was no JOC example in the test case. For submitting, after ten puzzles without popup I was pretty sure I had done something very very bad when there was a popup after the eleventh puzzle. I was confused and read the popup twice very carefully, just to more than double my solving time.

And another little thing that I don't understand: Why is there an automatic submission after 5 minutes? There are so many situations where this is bad. Can you tell me one situation where this is good?

How balanced do you think the puzzle types of this test were?	A bit skewed
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Way too many puzzles (too little time)
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Very nice

What did you think about the puzzle quality of the test?	Average
How did you feel about the length / time limit for this test?	Just right
What is your opinion about the solving interface?	Fairly Nice

I finally got to participate in an LMI screen test! (after co-authoring the first, authoring the second and testing the third).

Even though the B2: Repeated Neighbours has issues, I wanted to share what I did.
'3' cannot occur in R5C1, R5C2, R5C3, R6C2 due to the puzzle rules. If '3' is in R6C3, it also has to be in R5C4 but since it is a 'Just One Cell' sudoku (so, non-unique), I submitted R6C1 as '3'.

Pretty cheeky huh? Did anyone else get this weird funda? ;-)
Also, an interesting thought about using 'uniqueness' in Just One Cell, it sounds funny though.

Edited by Rohan Rao 2013-11-18 2:30 AM

Hey my friend-despite-the-fact-you took-part-in-WPC,

Interesting reasoning, although it is wrong.

"...If '3' is in R6C3, it also has to be in R5C4..." you can conclude that 3 in R6C3 is not the answer of the JOC sudoku, but you can't conclude that 3 cannot be placed in R6C3 (it can be one of the multiple solution).

I was in such a hurry about submitting my solution after having found the 3 in R5C4 that I didn't thought that it could not be possible, as it has to be in R6C3 as well (what I've seen however).

Fred

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Way too many puzzles (too little time)
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Very nice

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Fairly Nice

Lots of fun! Some of them were pretty darn hard, but most of them were perfect for the quick online solve. It was definitely in reach even for someone like me to finish within the time limit.

A few of them were a lot harder to do without a pencil! A good memory exercise for me to see how much of that I could keep in my head.

The colors interface would have been nice for bifurcating at times if I could have figured out how to make it work for me instead of changing the color of everything when I clicked on a color.

Oh, forgot to mention: the popup warning about just one cell was very confusing because I had filled in just one cell! So I worried that somehow the interface thought that I had submitted a puzzle with a non-1 number of filled-in cells. It would be nice to be warned about the warning.

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	A bit hard
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
What is your opinion about the solving interface?	Very nice

(Semax's point too) I agree, we should've said that this will happen. That'll be noted for next time (if at all we do variant JOCs again, that requires a think through in itself), and having a JOC in the example series on interface is also a good idea.

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	A bit too many puzzles
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Fairly Nice

In the solving interface, it would have to better if we could have been given different colors for backtracking the puzzles.

How balanced do you think the puzzle types of this test were?	Fairly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Fairly Nice
How did you feel about the length / time limit for this test?	Just right
What is your opinion about rules, points and scoring for this test?	Fair
What is your opinion about the solving interface?	Very nice

How balanced do you think the puzzle types of this test were?	Perfectly balanced
What was your opinion of the distribution of easy/hard puzzles?	Just right
What did you think about the puzzle quality of the test?	Very nice
How did you feel about the length / time limit for this test?	Just right
What is your opinion about rules, points and scoring for this test?	Very fair
What is your opinion about the solving interface?	Very nice

Yes, I am also an 'exception' B-) B-)



I know, I feel so stupid. I had a hearty laugh after I realized what I did.