Why doesn't Stack Exchange pass the Joel Test with flying colors?

Question

I was randomly browsing through the job offerings page on Stack Overflow Careers and noticed THIS monstrosity:

How can Stack Exchange, the company created by Joel Spolsky, not pass his own test?

Is it just me or is there something off about this?

Answer 1

Do you have testers?

Shadow Wizard is right; our testers are you lot! If you read the Joel On Software post about the Joel Test, you'll notice his recommendation (emphasis mine):

If your team doesn't have dedicated testers, at least one for every two or three programmers, you are either shipping buggy products, or you're wasting money...

We could hire a handful of testers, but because Stack Overflow has been all about the community from the beginning, we prefer to work more publicly and roll things out incrementally to real users. There are far more people here on meta and on Stack Overflow who will find and report bugs, or suggest improvements, or give us general feedback, than we could ever afford to hire. And we love the dialogue with the community: you suggest a feature, we build a version of it, roll it out to Meta, ask for feedback, make some changes, and roll it out to the rest of the network. The process is public, we don't have to hypothesize too much about what users might want or think (because we hear from you directly), and we can iterate much faster.

The development of the Stack Exchange engine is very different from most other software development. We're very lucky that our primary audience is programmers, that our community is global, and that the community feels so invested in the development of the product. All of that combined allows us to have "testers" who give us knowledgeable feedback (and often suggestions for solutions) very, very quickly.

I wouldn't want it any other way! :)

Alright, second question...

Do you have an up-to-date schedule?

The answer to this one is "sort of". We have a roadmap for the year, but it is high-level and generally outlines goals / areas of focus rather than being made up of detailed project plans and timelines. We do plan more detailed schedules at the project level (we try to break large projects up into roughly one- to two-week chunks), but we aren't as rigid about it as Joel recommends – I don't think we've ever kept a single timesheet, for example.

The type of scheduling that Joel goes into a lot of detail about is focused mostly on big releases. ("If you wanted to ship in six months, but you have twelve months on the schedule, you are either going to have to delay shipping, or find some features to delete.") For Stack Overflow and Stack Exchange, we continuously release (we have multiple production builds per day), and there's less coordination needed than for, say, a new Careers feature. The Careers team operates a little differently – due to the need to coordinate with marketing, train the sales team on the new feature(s), etc. – but even that is pretty flexible, and we rarely have large enough releases to need to detailed scheduling 6 months in advance. We mostly ship new features, not new versions.

Our preference is to ship things quickly and iterate based on real feedback and data. We love the Joel Test and its recommendations, but I think it's best for product development teams to be aware of best practices but make choices about what processes work best for their specific team, product, and circumstances. We constantly look to improve our processes here, but we aren't going to adopt new strategies just for the sake of checking off an item in a list.

Answer 2

Well, Stack Exchange fails on two requirements of the Test:

• Do you have an up-to-date schedule?
• Do you have testers?

Dunno about up-to-date schedule, but as for testers I'm pretty sure I know the answer: they don't hire testers because they have... us! :)

That's right, this very site (MSE aka Meta Stack Exchange) is used to test new major features before they are being launched network wide. With dozens and hundreds of eager users, they can save those dollars for better use.

Some recent examples:

• Help us test the shiny new "User Activity" page! (Plus a bunch of new features.)

• New navigation for Stack Overflow is in alpha testing