Skip to main content
2 of 2
added 88 characters in body
NotTheDr01ds
  • 20.6k
  • 4
  • 21

2021: a year in moderation - Handled Question Flags

A question (but first a comment) regarding 2021: a year in moderation:

First, that's 32,646 actions taken by our hard working Mod team in 2021. I know that there's even more that you do that isn't even represented in these stats -- Thank you! You are definitely appreciated!

On the "Community" side, I do want to question one stat.

This is repeating a point I made when I sanity-checked the Stack Overflow results, but since it's related to a chat in AUGR that a few of us (@Zanna, @RandomPerson, and others) had back in September, I figured I'd check the AU numbers as well.

From AUGR chat:

Zanna: I mean, I agree with @RandomPerson that reviews automatically getting invalidated due to nobody ever doing them is real real bad

At that time, the concern was primarily over the First Posts Review queue (now, of course, split into First Questions and First Answers). But the "Close Vote Review" queue is, in my understanding, fed partly (mostly?) from Flagged questions.

In these AU moderation stats, we see that:

  • A total of 14,011 questions were flagged either by Mods or by Community Members (or Community Bot).
  • A total of 13,646 question flags were "handled" (97.39%)
  • Leaving, presumably, only 365 question flags (2.61%) that weren't handled in 2021.

But are things really that good? According to a comment from @Makyen (mod) on my meta post on SO:

Close-flags, like close-votes, do age out but those flags are not presented to moderators. They go to the CV review queue.

So are we (the community) really "handling" nearly every question flag?

Zanna pointed out a SEDE query that checked whether a review had "aged out" due to expired votes, and I've attempted to adapt that to Close Votes Reviews. I've extended the range of days it was in the queue to:

  • Between 4 and 7
  • Between 14 and 17

... based on this information that Close votes expire at a rate of 1 per day after day 4 (for highly viewed posts) and day 14 (for low-eyeball posts).

If the query assumptions are accurate, we actually had 4,037 questions here on AU that (likely) aged out of the Review queue in 2021. That's almost 30% of the ones that the stats above say that we (the community) "handled."

IMHO, that's hiding a rather significant problem that, as Zanna said, "is real real bad".

Related side note: It seems very difficult to get a Close Vote Review to muster enough votes (either "Close" or "Leave Open", really), even in the most obvious of cases. Even when the OP says in the comments that the problem was a typo. Even when the OP agrees in the comments that the problem isn't Ubuntu related and reposts on Super User.

So:

  • Are "invalidated"/"aged out" reviews being counted as "Handled" in the Moderation Stats just because they moved from "Flagged" to "Review Queue"?
  • Should they be? Or should the stats instead show that there's a potential issue with Review Queues not being "handled"?
NotTheDr01ds
  • 20.6k
  • 4
  • 21