9

More than half a year ago, Stack Overflow lowered the close/reopen vote threshold from 5 to 3.

Ask Ubuntu has an order of magnitude smaller community than Stack Overflow1 but we still require 5 close/reopen votes here. Since there are fewer users with the close/reopen privilege, it can currently take days to close/reopen a question on Ask Ubuntu and I guess it is less probable here that a question gets closed/reopened at all after the initial vote or flag happens.

The experiment done on Stack Overflow a year ago showed that closing, editing, and reopening become more effective with 3 votes threshold. I find myself doing more reviews on Stack Overflow than on Ask Ubuntu now while my ratio was reversed before the change.

Shouldn’t we lower the close/reopen vote threshold to 3 votes, too?


1 I compared the number of pages of the user lists: https://stackoverflow.com/users/ and https://askubuntu.com/users/

3 Answers 3

5

The post you linked to on Stack Overflow has strong community support. It seems the change was received very positively. However, my feeling is that the change was made to cope with the fact that the Close Votes queue was, according to a Stack Overflow mod, "like a black hole" due to having 8000 posts in it. So, the community response may reflect the fact that something just had to give, and finally reviewing Close Votes became actually worthwhile and possible. The feature request had been made many years earlier when the number of items in the queue was an order of magnitude higher.

At the other end of the scale, very small sites like Vegetarianism & Veganism, where I have been active, also can't manage to close questions with five votes, because they have a very small number of active reviewers.

Ask Ubuntu ought to be in the middle, with few enough posts in the queue and enough active reviewers to keep on top of it, as Kulfy argues.

I have not come to a conclusion on this myself, but I have some material for the discussion.

Catija kindly provided me with some statistics.

90 days prior to the most recent 30 days (120-30 days ago):

  • Questions getting one vote/flag: 6073
  • Percent handled "close" - 60%
  • Percent handled "leave open" - 5%
  • Total percent handled - 64%

So... not terrible but that's ~35% going unhandled.

This, to be honest, surprised me. 35% unhandled attempts to close seems very high. This really challenges my assumption that we are dealing sufficiently well with attempts to close.

Also, 5% leave open seems very low. In recent years, I have generally been reviewing at least 50% of items in the Close Votes queue as Leave Open. Assuming that a decent proportion of my votes were correct, this suggests that very many posts get wrongly closed.

I have had many of my own close votes age away over the years, and at times, due to low numbers of active reviewers and a full queue, it has been difficult to get things closed and even more so reopened. One of the main functions of the Downboat is to coordinate closing, leaving open and reopening (and to discuss them, of course).

Of course, mods can always step in and help out. But, Catija wrote, a high proportion of questions being closed by mods is an indicator that lowering the close vote threshold will be useful to the site. But Ask Ubuntu has a very low proportion of questions closed by mods compared to other sites. [For the same period]:

329 questions closed by mods, 3857 by users, so about 8% by mods

I know there is a post somewhere on MSE that has detailed stats on that for all sites, but I couldn't find it and only have this for comparison (from a post by Catija on the Stack Moderators Team)

there's dozens of sites where the mods are closing as many posts as the users on the site... sometimes in the hundreds - for example, the SO in Portuguese mods have closed 4301 questions in the same 90 days that the users have closed 4763 - six mods have closed 4301 posts in 90 days [...] Super User is 2076 for mods vs 2492 for users; Server Fault is 1260 vs 1439. This is true across the spectrum of sites.

So, mods have relatively little involvement in close voting on Ask Ubuntu. But it's unclear to me whether this in itself is an argument either way for lowering or not lowering the threshold. Maybe mods could pick up some of the unhandled slack here, rather than the threshold being lowered - would that carry a smaller risk of incorrect closures?

I also received some reopen stats:

For the same period as above:

  • Edited after close: 609
  • Percentage reopened: 5%
  • Percentage marked "leave closed": 23%
  • Total handled: 27%

Assuming every post in the reopen queue (which never has more than a handful of posts) was handled, it seems about 165 of the 609 went to the reopen queue, and so about 18% of those were reopened.

But, it would be more interesting to know which posts were wrongly closed, and information on posts that were reopened without being edited would be more useful for that, assuming that posts were edited into an answerable state and thus reopened, having been rightly closed earlier.

Perhaps the most important question for this discussion is how many posts receive at least 3 incorrect close votes. Even if posts could be more easily reopened, it may be that, if there are many more wrongly closed posts, more of them will go unseen and the wrong closures will not be addressed. Assuming my belief that many posts on Ask Ubuntu receive wrong close votes has some basis, lowering the threshold should, if it is to happen, perhaps be accompanied by some initiative to train new close vote reviewers. At present, such an effort would be hampered by closed-as-off-topic posts now being displayed uninformatively to users who lack the close voting privilege, but maybe we could put together a crash course or something.

I have not figured out any way to determine how many posts receive 3 wrong close votes, or whether the difference between 3 and 5 close votes required is a difference that allows such posts to be prevented from being closed, or whether the greater ease of reopening could compensate for that difference. If anyone has any data, or can even suggest what data should be sought to inform this, it would be of great help. Perhaps we could even devise and perform a data-gathering exercise. We probably have time to do that, because, again according to Catija, while lowering the threshold can be done, for technical reasons, it can't be done right now, although I do not know when that will change.

4

I don't think we need those changes to be implemented right now on Ask Ubuntu. Stack Overflow is big, very big. Daily ~6000 are posted there. On the other hand, Ask Ubuntu gets ~200 questions per day and in my honest opinion, that's a manageable number.

As of now there are ~4500 items in close vote review queue and ~300 in reopen queue on Stack Overflow while there are ~200 items in close vote review queue and exactly 1 in reopen queue on Ask Ubuntu. Looking at those numbers we have less than 5% pending items as compared to Stack Overflow. These numbers may also reflect the chances of close/reopen votes getting aged.

These are the closure statistics for past week retrieved from 10k tools:

enter image description here

So, I think reducing the threshold isn't needed for now.


Note: Similar experiment was carried away on Software Engineering 5 years ago but as far as I know they never implemented that on that site.

2
  • wouldn't the 90 day stats be more informative?
    – Zanna Mod
    Aug 8, 2020 at 7:05
  • @Zanna I think that's a matter of opinion.
    – Kulfy
    Aug 8, 2020 at 7:11
4

I don't advocate lowering the vote threshold for closing questions from 5 to 3 like Stack Overflow does for two reasons.

  1. Stack Overflow has many more users than Ask Ubuntu in general and many more gold badge tag users than Ask Ubuntu in particular. Most duplicate questions at Stack Overflow are closed by gold badge tag users.

  2. Stack Overflow also has many more 100k+ reputation moderators than Ask Ubuntu. Stack Overflow reviewers are moderated hard by moderators, to reduce the number of incorrect question closures.

1
  • 2
    +1 it totally makes sense that review needs to be very well moderated to allow this to work
    – Zanna Mod
    Aug 9, 2020 at 7:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .