XP Doesn’t Give Us Enough Information

Developers don't like to create a lot of documentation (or any, we suppose), but
they do like to have enough information to act on. They don't want to fly blind.
Nothing hurts quite as badly as working like a dog to deliver something nobody
likes. We hear often that XP sets you up for failure by not having enough design
documentation, and by depending on stories instead of comprehensive specs.
Ken and Roy have worked on lots of projects. In most cases, your first day
follows the same pattern. You walk in and somebody hands you a functional spec, a
technical spec, and/or a detailed design document. Nobody ever reads them and
they're hopelessly out of date, you're told, but here they are. This is no better than
having no documentation at all. Often, it's worse.
Several years ago (pre-XP), Ken went to help a group who had lost a developer
due to a family emergency. He was told that, when he got there, they were a day or
two away from delivering "Phase 1", a functional prototype of a subset of the
system. Since the chances were that he couldn't help finish that, (by the time he got
up to speed, they'd be done), he should begin thinking about the next version of the
system. Being an agreeable sort, Ken said, "fine, how would you suggest I get the
background I need without getting in your way?". "Read these documents. They are
fairly complete, but mostly out of date. Section 6 gives a pretty decent account of
the motivation for what we want to do. The prototype doesn't follow much of the
implied design, but it works and is probably a bit more thorough."
much design to speak of (at least nothing that could be reasonably implemented),
and Section 6 was full of acronyms and phrases that were not in Ken's vocabulary.
There was a lot more to read, but at a cursory glance Ken didn't think he would get
any more benefit from reading it (other than the bliss that comes at the end of the
day when you get to stop reading), so he asked if he might review the prototype to
help him understand how they had approached the problem for the prototype and
begin to make recommendations of what approaches they might keep and what
approaches should change. Permission was granted since it turned out that Phase 1
still wasn't over due to a lot of bugs in the prototype that the rest of the team was
working hard to get out.
As he studied the prototype, he discovered that it was built on top of a
framework that another group had built for a different application. Although there
was little documentation on that framework, an occasional class comment explained
some of the trickier parts and the code was clear enough that he could figure out the
rest.
On day three of reviewing the prototype and doing some experiments to explore
how they might approach things differently (day 5 of being there… 4 days over the
day Phase 1 was supposed to be complete), Ken overheard the morning panic
meeting that happened to be held in the middle of his cube set. (Each developer had
their own corner of the square, and a small round conference table was in the
middle. The other 3 developers were working on finishing Phase 1). One of the
developers was sharing with the manager that the bug they were discussing was a
bear and that it would be nice if he could do something like X-Y-Z, but that was
going to take some time. Ken interrupted, "Excuse me, Bob, but I couldn't help
overhearing. There is a feature of the framework that already does X-Y-Z."
"Really?". "Well, at least I think so. After you're done the meeting, I'll look at it with
you and we can make sure it helps you do what I think I heard you say." "Cool.
(turning back to the manager) Well, if Ken's right, then we can knock that one out
today and I can look at some of the other bugs that should be easier."
Ken was right, and the next day at the "we just gotta get the last bugs out of this
thing and we'll be done with it" meeting a few more hairy bugs came up. This time,
Ken was called on. "Hey, Ken, you wouldn't happen to know if there is anything in
the framework that will let me U-V-W, do you?". "Actually, I think there is
something that will do U-V for you. And the W part is a piece of cake.".
What's the point of the story?
All the developer's had read the documentation and realized it wasn't that helpful
so they threw something together anyway. They were fortunate enough to have
some decent code from the other group to build upon, but they never "read the
code". The documentation was what was supposed to give them understanding.
Since the framework they were sitting on didn't have much documentation, they
never tried to understand it. Somebody showed them how to do something with it,
and they ran forward. If they would have taken two days to read the code instead of
reading the documentation, they probably would have been done Phase 1. The code
was up to date and it provided something useful to study. The documentation
provided almost no value. It certainly provided less payback than the months of
time and money that was put into producing it.
When Ken asked the "Phase 1" developers how they built the prototype based on
the information in the document, he was told that they didn't. They had read the
document and still didn't know what to do. So, they started meeting with the users
who explained to them what they really wanted, and built the prototype in between
meetings based on what they learned during the previous meetings.
XP does NOT recommend that you don't produce documentation. It just says
you should not spend time creating any unnecessary documentation. You need to
figure out what that is. Our recommendation is to throw out pre-conceived notions
of what is necessary and assume you don't need it. Then, when somebody asks for it,
ask them why they need it. If they "need to understand something", offer to explain
it to them. If they still need it, consider it. See the section on Communication, Not
Documentation.
Unless there is some external requirement to do so (e.g. in a regulated
environment), we do the simplest thing that could possibly work for documentation:

 Stories, tasks, the developer-customer discussions around them, and the
acceptance tests are the only spec you'll ever need.

 The "system documentation" is mostly in the code and the unit tests
which always tell you how the code works today.
Recently, we were in a meeting with a prospective client. Part of the
conversation went like this:
Client: We have this software that was written several years ago by someone
who is no longer with the company. We brought in a new person to try to add some
new functionality and he's having a rough time.
Ken: Do you have out of date design documentation?
Client: Yes.
Ken: Would you rather have a set of working tests?
Client: Absolutely.
Specs are never comprehensive. In fact, until you have clear consistent
acceptance tests, they are far from exact. A recent study indicated that the average
requirements document is 15% complete and 7% accurate, and that the length of
lie" of BDUF. You simply can't think of everything to the last detail and if the
requirements aren't complete, the chance of the design being adequate is slim to
none.
It is better to admit that the requirements are vague from the beginning, and
work on the details in the form of acceptance tests as they are needed. That's really
the hard part. It is seemingly impossibly in some cases until you have something
half-working.. This is how it happens anyway, but usually the customer and
developer expectations are all out of whack. Customers think they'll get the
implementation of the whole spec at once, but developers prioritize it and work on it
sequentially. The developers will most likely have different priorities than the
customer, so the wrong people are prioritizing. If you make customer prioritization
the norm, both groups have the same expectation. We'll identify all the big pieces
(stories), we'll prioritize them so that everybody knows what we're working on and
why, and we'll flesh out the details when it comes time to do each one (tasks and
estimates).
Our initial estimates are no more unreliable than the requirements. (Our
estimates are at least 7% accurate). Once requirements are specified as
unambiguous acceptance tests, we will get much better estimates. Until then (and
beyond), we should work within reality.
Tests and Code are a better system documentation than anything else could be. It
is also the most efficient way of getting accurate documentation. If you're trying to
learn a new part of the system, you eventually have to go from words to code
anyway in order to really understand what's going on. Why waste time on the
overhead of keeping paper updated? We have never heard a good reason.
Earlier this year, Ken was in London and was a guest at the eXtreme Tuesday
Club4. The first question he got was something like, "How does XP map
requirements to code?". Ken's answer was "Why do you need to map it to code?".
"So you can see how a the requirements are met." "The requirements are met when
the acceptance test passes." "Yes, but what if the requirements changed". "Then the
acceptance test changed". "Yes, but how do you know what change in requirements
caused what change in the code." "Why do you need to know that?". "To make sure
that the requirements are met." "The acceptance tests do that… Look, you are
assuming that there is a requirement to track requirements to code. I'm trying to
build a product that meets the requirements. Why would I waste time doing all of
this mapping. The customer doesn't want a map of requirements. They want the
requirements met. Of course they are going to change. So, we need a new test or set
of tests and they have to pass." End of discussion.
XP gives you enough information to do your job as well as you can, without
distracting you with too much.