Should we consider unanswered questions a problem yet?

Question

_{In the fashion of the discussion on meta.SO about unanswered questions on Stack Overflow}

Currently, out of 120,000 odd questions, only about 70% of them are considered answered. But it wasn't this bad some time ago. Last year, after our cleanup event, our answered rate went as high as 86%. And that stat has been on the downfall ever since.

Here's some number crunching to get the idea:

Percentage of open questions with answers, by quarter

The number of questions we get have significantly increased since 2012 but the answers haven't caught up with them yet. Actually, they are not even trying to catch up.

Question quality by quarter

Likewise, the number of questions that have received at least one vote have been on the decline as well. But there haven't been much increase on downvoted questions. I think votes should be used to quickly sort good content to the top and bad content to the bottom. If we are to club decent quality questions with poor quality content by not voting on either of them, we are doing no justice to the system. Anyway, that probably should be raised in a separate discussion.

Question Votes by Quarter

People predominantly prefer to upvote questions or not vote at all. Just 6% of the votes are downvotes. Maybe we shouldn't be so averse to the downvote button.

Now that I have established all of that, what we can do about the unanswered questions? Should we be concerned yet?

---

Me being a little concerned about the state of the site, I have a few ideas that I would like to throw here for discussion. You are strongly encouraged to brainstorm a few more ideas and discuss the validity of the ideas raised here. The answer section is open for all.

1. Enforcing a stricter policy on content quality

   I believe we should consider actively putting questions that don't have adequate details and information that are needed to provide a satisfactory answer "on hold". Historically, we were quite lenient on questions that didn't have enough details and didn't close them right away on the hopes that OP would edit the details in.

   But this workflow is horribly broken and more often than not, that's exactly what is not happening. Questions lie around on the site for months together without enough information to answer them. And OP will not be active either. Maybe we should start to put them "on hold" first and then reopen them when they have enough details. This workflow should work better since the system now automatically adds recently closed and edited questions in the reopen queue.

2. Restricting the asking privileges to registered users

   I think this little tweak to our system would do a world of good. For a start,

   • it would keep away a decent chunk of spam posts
   • prevent drive-by users from posting their questions and then fleeing the site
   • tremendously decrease the chance of a user losing access to their cookie-based accounts thereby avoiding anonymous edits from OP or non-answers posted by OP from another account to provide further information.

3. Downvoting incomplete questions so they fall to the bottom

   This probably would help on multiple folds.

   • Bad questions will be eventually auto-deleted from database
   • Good questions would be better sorted from junks making it easier for answerers to focus on them
   • Improves the overall quality of content on the site

   Now, if you are one of those good people not comfortable with downvoting questions because "it is not the right/best/kind/nice thing to do", I think we should start putting the best interests of the community first instead of sacrificing the community at all.

4. Inviting and encouraging 'experts' to answer questions

   It is a known secret that most of the questions we receive right now are either very mundane or highly specific. Maybe all we need is more people in each of the specific field answering those questions.

   Let me end of with another data query showing how askers compare with answers..

   Number of users answering or questioning

   

---

Answering Oli's concerns.

jokerdino's first three suggestions have problems:

• Our existing policies stipulate quality... People ignore policy.
• Forced registration won't float with SE and doesn't solve the problem of people asking bad questions.
• Downvoting is just another way of cleaning. It's still time taken from members to review something.

We should always be trying to attract people in.

• People may ignore policy. But that's not an argument for never enforcing one. Skeptics.SE expects a remarkably high standard (in comparison with the rest of Stack Exchange network sites) from their users and the general quality of the posts over there would tell you the fruits of enforcing such a quality policy.

• Forced registration is not a new concept, not to Stack Exchange. Stack Overflow has it for a long time and Programmers.SE has required registration for quite some time now. And no single solution will overnight solve the problem of people asking bad questions and requiring registration will help prevent people easily losing access to their accounts and questions. This will be beneficial especially with mods no longer being able to merge user accounts at our discretion.

• Now I am not sure why we are so averse to any form of cleaning. Downvoting is a form of cleaning (actually, I would call it sorting) and that's pretty much the reason why it's even there. And downvoting will help highlight good content. It may take some time from the reviewers (realistically speaking, voting costs you may be 10 seconds but the benefits are far reaching) but it will definitely help signal to the general audience that a post is not great.

When we are getting 180+ questions on average everyday, I am sure we can afford to keep some of the questions away. It is as much about keeping our community happy and healthy as it is about getting more people in.

Answer 1

When people mention cleanup and our "answer rate", they're talking about a historical number. A number that I frankly feel isn't that important. It's a stat that we can manipulate if we close a big bunch of questions.

The important number is the rate we're dealing with new questions. We need to deal with the questions that are coming in faster, or lower the number. jokerdino's first three suggestions have problems:

• Our existing policies stipulate quality... People ignore policy.
• Forced registration won't float with SE and doesn't solve the problem of people asking bad questions.
• Downvoting is just another way of cleaning. It's still time taken from members to review something.

We should always be trying to attract people in.

I have a theory...

We're spending 13 hours every day fixing questions missing vital information

On any given day, ~50% of the questions will be about hardware in some capacity. 80% of these need additional detail that we have to tease out of the user and a good 10-20% of those don't respond. Either way, we're sinking 5-10 minutes into each of these questions just to make them answerable. Collectively that's 13 hours.

4-8% of our new questions are unanswerable, abandoned crap. That's over an hour every day completely wasted.

And because these questions are so stop-start, they completely bypass the useful stages of the review process. By the time we can work out that it's a dupe, everybody has forgotten all about it.

What do you think the opportunity cost of 13 hours is? Let me put it another way: How many questions do you think you could answer in 13 hours? If we could ensure that every question were answerable, I reckon I could get to all of them in 13 hours. And that's just the time we regain.

Cleanup is not our saviour

So going back to cleanup. The few sprints we've done have been great at improving our historical "answer rate" but they do nothing to improve our current answer rate. If anything, they're diverting resources (people) from answering new things.

Additionally there are some questionable behaviours and side effects cleanup brings like closing good questions because they're too hard for us at the moment and voting for crappy answers so the question is technically "dealt with". When we do this... Well I think I said it best in the moderator chat room:

Closing something because we (as a site) can't answer it just compounds one failure on top of another.

Cleanup is a very blunt tool that lets us hide how bad we've been at keeping on top of things in realtime. It's a last resort.

So here's one way to ensure that new hardware questions have all the detail we require... A big-bearded, lightning-bolts-and-all WIZARD!

When somebody goes to ask a new question, before they see a text box we ask them a few questions to determine where their problem is.

• Is you question about the following problems?
  • Blank screen on boot
  • No wireless connection or other networking issues
  • The resolution on my screen isn't correct
  • I don't know how to install the right graphics driver
  • I can only see the desktop
  • Nope, just take me to the form

This list needs to represent four or five of the most common and hardest to diagnose issues. The above are just examples.

Let's say the user picks wireless. They're then asked:

• Please open a terminal, type in lspci -nn | grep -i net and copy the output into the following text box. (Help: How to do this - a link to a Q/A on how exactly to follow this procedure)

They paste in the following:

00:0a.0 Ethernet controller [0200]: NVIDIA Corporation MCP79 Ethernet [10de:0ab0] (rev b1)
05:00.0 Ethernet controller [0200]: Atheros Communications Inc. AR242x / AR542x Wireless Network Adapter (PCI-Express) [168c:001c] (rev 01)

We then ask:

• Which best represents your issue:
  • A problem with your "NVIDIA Corporation MCP79 Ethernet"
  • A problem with your "Atheros Communications Inc. AR242x / AR542x Wireless Network Adapter (PCI-Express)"

They click the Atheros wireless and here's where the magic happens. We do a quick API search for answered questions that contain 168c:001c and offer them up to the user in a "We have questions about this hardware already, please check them to see if they help".

This is where I envisage most people getting off. We have a huge resource of old answered hardware questions and if people knew what to search for, they'd find their answer.

At the bottom of that list we have a "No, none of these helped" link. If the user clicks that, we prepopulate the question with a code-formatted version of the lspci dump (so future lookups can find this version) and any other details we've collected along the way (other examples may ask more questions).

To recap, this has three ridiculously good benefits:

1. Most people will find their answer before posting and that means less reviewing, less duping.
2. Those questions that survive don't need follow-up questions because they've already been asked (remember I was suggesting that 40% of questions previously needed human interaction before they could be answerable!)
3. The unanswerable crap is limited to questions that aren't part of this wizard.

The best thing is this can be built, tested and implemented purely in javascript, using a userscript immediately. It's not a small project but most of the work is in designing thoughtful questions to problems we see a lot of.

Answer 2

We're 2 slots away from being the worst in the entire network:

We need to figure something out, we're two slots away from being the worst. I'm not saying we have to be awesome, but surely we won't be throwing parties over this:

~6%-7% variation from way higher traffic sites, we need to step up!

Answer 3

Here is my recipe for dealing with the whole "unanswered" thing:

• I adopted a tag (or more like a set of them) that are more or like what I clean up going from most upvoted and newer questions to far back.
• I go sometimes to a random page of the Unanswered -> no answers list (ie https://askubuntu.com/unanswered/tagged/?page=20&tab=noanswers) and check up what's there, or just select a tag, and start answering.
• The normal flagging when I'm doing this.
• I struggle that most of questions have at least a comment whenever they arrive to the newest page, so new unanswered don't pile up (but I won't do this if I'm busy on other tasks).

You could take some of my behavior, propose some, or just continue what your are doing ;). Although some pre-question quality insurances is always welcomed, since it's tiring having to comment asking for more information about the same things: lspci, lsusb, GRUB-UEFI configuration (the comments vary somewhat, but still is almost the same information that we need), etc.

Answer 4

From the question:

The number of questions we get have significantly increased since 2012 but the answers haven't caught up with them yet. Actually, they are not even trying to catch up.

The increase in the number of questions is partly a natural consequence of the publicity given to Ask Ubuntu over other "help" options. While publicizing Ask Ubuntu, the "ethos" of Stack Exchange is not given prominence and that perhaps accounts for the change in quality. With Ubuntu going mobile, I suspect things will only get worse and the number of people posting in SMSese will rise.

One aspect that hasn't been covered in the question is the number of views per question. I find the low number of views particularly depressing.

Despite what numbers may say, I feel that the community's resources are being spread too thin. Support is being offered

• at Launchpad
• via several chat rooms and mailing lists
• at ubuntuforums.org (despite attempts to depict it as some sort of place for unwanted questions) and
• at Ask Ubuntu

In addition, someone presumably has to "moderate" the various "social" pages that are mushrooming. The people at Discourse did have a help section but terminated it otherwise there'd be yet another support option. Is the community big enough?