Usability issues for first-time Stack Exchange users - a micro-study

Question

I come from lands afar, also known as Hardware Recommendations Stack Exchange. Over there, we have a serious problem with first-time Stack Exchange users coming along and asking off-topic questions - to the extent that 50% of incoming questions are closed. We've had a few attempts at solving this, but none have really worked.

So, this afternoon, I got together a group of people who had never seen Stack Exchange before. I gave them each a question (on various topics, each intended for a different site in the network), and asked them to find the most appropriate site to ask it, noting down the troubles they had along the way.

(They were all terrible questions, though the topic was clear. I did ask them not to actually post, but, by happy chance, the one who actually did demonstrated my issue perfectly.)

This post is a list of their most common problems with the sites as first-time users.

Help! How do I start?

People didn't start reading from the top bar. When they landed on a Stack Exchange site, the most common behaviour was:

• read the site title
• read the banner, be confused by it (more on that later)
• read question titles

After that, actions start diversifying. The point is, they don't see the topbar. Which, in turn, means they don't see the help link. Of 20 participants, only one ever saw a help center - and that wasn't by getting there from the topbar link. The topbar help link fails its purpose.

It's Free! Now sign up.

When you're a first-time user, your view of a site has a dismissible banner across the top giving you a 30-second introduction to the site:

That seems helpful enough, until you read the wording for the first time and notice this:

It's 100% free, no registration required.

Sign up

It's not technically incorrect or in any way wrong, but it reads very strangely for a first-time user. "You don't need to register - click me to register!"

Why can't I answer?

Okay, my test group weren't meant to be posting answers. Still, they identified this as another weird thing. Having just read that banner, and seeing "anyone can post an answer", the next thing many of them did was click on a question to see if its topic matched their question. When they scrolled down to read the answers, there weren't any, and they discovered that they couldn't post an answer either.

"What? But it said anyone could post an answer - how am I meant to do that?"

Of course, the question they're looking at has been closed. This raises several points:

• they didn't notice [on hold] in the title (or didn't place any significance on it)
• they scrolled past the close message
• there was no way to tell why they couldn't answer

When they did notice the close reason, it was generally fairly good at explaining what had happened, so that much is working. Perhaps we just need to emphasise the reason, and add a note to the bottom of the page in place of the answer controls explaining why the controls aren't present.

Hot Network Questions - what?

HNQ was one of the most confusing features mentioned. On a page full of questions about gardening, looking at the sidebar and seeing questions about Star Wars and Windows 10 next to each other confused my test group. Some sample reactions:

• Aren't those questions off topic here?

• What's the network?

• What's Hot?

• Why are they here with the gardening questions?

Ah, so I ask here. Or do I?

Many sites' scopes are not immediately obvious. That's not a bad thing, it's just a product of the system that a site can't take absolutely all questions about one topic. However, it seems that we do need to make sure the guidance is available.

On the Ask Question page (/questions/ask), there's an info box on the right-hand side of the page. Some sites have customised this with a custom message; others use the default.

A customised message (Hardware Recommendations):

The standard message (Open Source):

In both cases, my test group didn't notice, or didn't fully read this information until after they'd typed their question into the box.

At that point, if the guidance makes the question they've just typed off-topic, the majority reaction was "but I've already typed it out... they won't mind, right?". Clearly, that's not the reaction we want. If this guidance could be moved to the left hand side of the page, in consistency with most people's reading direction of left-to-right, I think more people would read the guidance before posting an off-topic question.

---

I've got a small table of results, which shows the site I intended the question for, the site the user decided to post on, the time it took them to reach a decision (hh:mm) and whether the question was on-topic at their decided site.

Intended	Decision	TTD	On Topic?
SU	HR	00:12	n
HR	HR	00:08	y
SO	SO	00:03	y
Progs	SQA	00:20	n?
Workplace	Workplace	00:19	y?
SF	SO	00:05	n
Gardening	Gardening	00:15	y
ELL	EL&U	00:10	y?
CS	TCS	00:12	shrug
Apple	Apple	00:07	y
SF&F	Movies	00:25	y?
Bicycles	Bicycles	00:09	y
Aviation	Aviation	00:11	y
Security	Progs	00:07	n?
Physics	Physics	00:06	y
Homebrew	Beer	00:17	n
Writers	WB	00:13	n
DBA	SO	00:15	y?
Crypt	Security	00:08	y?
Biology	Biology	00:06	y

So that's the results of my mini-research study on Stack Exchange. No doubt there are some obvious and some less obvious solutions to each of these points; if we could consider doing some of these things we'd make the sites a lot friendlier to first-time users. As a side benefit, we'd also give them more guidance, potentially reducing off-topic questions on some sites. So - can we make SE more friendly to first-timers?

Answer 1

Good job! This is awesome. There's a lot of useful observations here, but here are a few I'd highlight specifically:

The default copy in the hero is kinda goofy. It has that, "No charge - EVER. Enter your credit card below!" vibe. This actually got fixed on SO:

Normally, weird copy getting updated would get pushed to all sites, assuming it makes sense, but in this case, it was discovered in the context of some SO-specific A/B tests, so that didn't happen. We'll work on fixing that ASAP.

The absence of an "answer" button on closed questions is bad UX. It's happening more on HR due to the higher close rate, but it's sub-optimal anywhere: Sometimes a button isn't where the user expects it to be, and the only signal for why (that the question is closed) is off-screen, plus it requires you to know something most people won't intuit (that closed questions won't accept new answers). I personally hadn't seen that raised before. It's slightly more work - requires some minor design thought on whether to grey out a button, what copy to use, etc. But I think it's low cost to benefit - we'll look at adding something there.

The "ask question" page (and sidebar guidance specifically) aren't really great at helping new users be successful. I think your points here are dead-on, and this one needs more work than what you've described. At some point not-too-far-off, I'd like to see us put some real work into more of a "guided" version of how to ask, with more structured fields ("What did you want to happen?", "What was the actual result", etc.), and some guidance for each that's not easy to miss. I also like enderland's Pre-fill question-box with text for new users (A/B study proposal), or something similar. This needs a bit more time available to kickoff, but I really want to see if we can't devote some focus to this page before this year's up.

Answer 2

I love your idea to move the infobox and while at it, I would extend the idea. After the user entered the question's name, the system already do a lookup with the question name on the site in question, but if the lookup would find more result on another site, could the system display a hotlink direct there in the infobox, like "Is this question could fit on sitename instead ?"

To prevent too much load, it could do a simple search on site that tend to overlaps (like for me on SF, it could be on SO & SU)