7

First impressions from the review audit system: I am slightly amused but more annoyed.

  • The first two audits I got were ones that I had voted to close myself - so it was a mild form of deja vu
  • yesterday I got 5 audits - seems a little heavy for one day - and the "congratulations, this was just a test" message disrupts my concentration.

I noticed that I was looking for tell tale signs on the posts before really dealing with the text. So it might be a good idea to reduce the frequency a bit.

Apart from that I want to +1 the appeals that evaluation should not happen before the reviewer clicks on "I am done".

7
  • I'm speculating, but when one runs for mod, I would suspect the system SHOULD expose them to higher review audit prejudice. Before I got bothered about the review audits, I would wait for the mod election to end and see if it just goes away.
    – RobotHumans
    Commented Feb 23, 2014 at 12:07
  • Doesn't apply here :-))
    – guntbert
    Commented Feb 23, 2014 at 12:15
  • Sorry. I had you confused with @Mitch. And upvote.
    – RobotHumans
    Commented Feb 23, 2014 at 12:27
  • 2
    I was about to post the same question. I totally agree with everything you said: deja vu (we should not be tested with questions we interacted with), frequency (users that fail the test should be exposed to more tests, and the ones that pass should get less tests), "I am done" (I was tested by this askubuntu.com/q/416380/65926 and I rememberer something similar was asked before, so I pressed close just to search for a duplicate and BAM, I failed the test).
    – Eric Carvalho
    Commented Feb 23, 2014 at 15:59
  • Failing prior to "I am done" seems extreme. Failing for attempting to leave a comment on a good answer seems ludicrous at best.
    – Elder Geek
    Commented Jun 20, 2014 at 14:49
  • 1
    I agree that the frequency can be a bit extreme (to be kind) and in some cases borders on annoying. My response to the annoyance factor has been to just go away for awhile. I doubt this is the desired result so perhaps revisiting the audit frequency code would be helpful.
    – Elder Geek
    Commented Apr 7, 2015 at 16:05
  • 1
    Is anything actually done about this? I just reviewed 21 questions and late answers and had 5 tests! waisting 15min of my time, because I made the effort of leaving a long custom comment on those, only to discover that they had already been deleted.
    – Minos
    Commented Mar 25, 2016 at 10:03

2 Answers 2

Reset to default
5

With 6 audits under your belt in 3 days, you're near the top of the list for "most audited". Mostly, that's just because you do a lot of reviews, but some of it is due to plain old luck...

Review audit frequency is semi-random: you shouldn't get in a situation where every review is an audit, but if you're unlucky you can end up encountering them fairly often: so far, you're averaging about one audit per 20 reviews; expect this to vary over time.

2
  • 1
    <sarcastic> ah, I get the message, "there are far to many reviewers, you should review less than 20 posts per day" </sarcastic> - please understand that I am not complaining (and I had definitely more than 6 in the first two days) - it was intended as feedback, no immediate "solution" was expected.
    – guntbert
    Commented Feb 24, 2014 at 20:00
  • Oops, I was just looking at one queue - you're up to 14 across all of them as of right now.
    – Shog9
    Commented Feb 24, 2014 at 20:04
3

The review audit system really isn't that smart, which is why you get questions that you have already dealt with.

Remember the audits aren't here to test you, they're here to catch robo reviewers who fly through the review queue applying the same action to every post. They should be easy for you to catch.

I do agree however, that the frequency could be lowered a bit, they are starting to get disruptive. I also disagree with how close vote review audits work, especially with duplicates as explained in this question on MSO: Close vote review audit flawed - fails without casting vote.

I started this as a comment, but it kinda evolved beyond that, so here it is

2
  • According to @shog9 ("Remember: review audits are meant to be helpful, a tool for teaching new reviewers what is expected and maybe staving off fatigue in experienced reviewers.") the purpose is not only to catch robo reviewers. If it were then just one audit/week would be sufficient. I don't really want to catch them (in the sense "I see the trap").
    – guntbert
    Commented Feb 23, 2014 at 18:24
  • @guntbert Yes, they have that purpose too. You should want to catch the robo reviewers though, they hurt the site and make review useless. By "catch" I don't mean eternally ban, but, like Shog said, teach them what review is really all about. They should get eternally banned however if they refuse to review properly after many lessons. Review is an important aspect of site moderation, not a game.
    – Seth
    Commented Feb 23, 2014 at 18:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .