Current development trends are much the same as they were 10 or more years ago. Perhaps 10 years ago, not everyone had a PC, but most of the concepts are the same. A single person is responsible for a static set of documentation handed out to various individuals, all of whom will be ultimately responsible for the final product.
In the last 10 years, the static document has become more dynamic, with almost everyone having access to the program used to generate the document, and the document is typically emailed, rather than existing as a "dead tree edition" hard copy passed out in a meeting, but there is still one central person responsible for the "golden" version of this documentation.
This collection of documents is emailed to all individuals involved in the design, as the design evolves. The designers will have meetings to discuss the design, and make changes to the design. Some features will seem obvious, and not be written down, and others will be written in painful detail.
Once development is started, a copy might be emailed (or a scratched-up hard copy given) to the developers as they are involved, and usually one or, worse, multiple versions will be put in a public place, making the current one difficult to determine. The developers may not understand the text as it is written, or other knowledge the designers had won't adequately be conveyed to the developers in the document, and key features will not be coded correctly. Usually, inadequacies will be corrected in prototypes, but the documentation will not cover this.
After the project goes to QA (all projects have some form of QA, even if it is the marketing department who plays with a prototype), some of the documents deemed important will be given to QA so they have the ability to judge the quality of the work the developers created. Some of the test plan will be devised by the designers (if they happened to consider that testing would be part of their responsibility), and this knowledge may be covered in various documents given to the QA department. Subtle details that were given to the developers by the designers will not be covered in the documentation. Features and design updates will be designated as bugs by the QA department, since these changes might not be covered in the design documentation.
Users are given instructions based on what the technical writer understands the product to do. Generally, the product will be what the designers originally intended, if enough time is given to the project. If the project was time-limited, there will be substantial changes, and features just left out. Subtle changes made during development will not be reflected in the original design documents for various reasons (including resource constraints), and these changes may (though this is unlikely) be reflected in the documentation.
Support is typically left out of all documentation. Maybe they will have access to the original documentation, but probably not. Various tester notes will have been completely ignored, limiting support's ability to do even basic troubleshooting. Normally, because the support people are thought to be bothers to developers, they will only be able to call the developers under dire circumstances (which will be the norm, since they have no idea how things are supposed to work). The developers end up late on the next project since they spend so much time supporting the current project. (If they spend no time, then the company goes broke, and the issues all go away).
It seems support organizations are used to not having resources to support them, so they don't bother asking. When some is available, it is completely ignored, and trouble keeps festering. Development accuses the support organization of being dumb, and the support organization accuses the developers of being uncooperative. Self-fulfilling prophecies. It has to change.
If a new person is added to a project, she must get all the same knowledge everyone else has. That can mean email from multiple individuals, archives of related email, and a host of meeting notes. Organization of all this information will be quite difficult for her, or it will take an excessive amount of time for someone familiar with the project to organize this information and present it to new people after the fact.
When the project is to be revised, many of the "why" decisions will have to be addressed. Why did the project not get this feature? Why does it do things this way instead of that? This knowledge is gone unless it is recorded somewhere. There may be no way to capture this information after the fact, since people move on or just plain forget. Rarely will one person have all the answers.
Sometimes, the goals are not clear. Sometimes, the whole reason for doing the documentation, the project, or whatever is some other agenda.
The best way for documentation to be done is in a more dynamic sense. All of the people involved in the project usually have a computer to use to manage the documentation. Turn responsibility of the documentation over to the whole group. Put the files in a central repository where everyone can manage them.
HTML doesn't yet offer the same formatting abilities as a word processor, but it provides easy access during development. It allows people to read any document anytime (or can be password protected, if necessary), and editing can be done offline or online, in sections or as a whole. (Early in the design, word processor documents are usually a hindrance, since people concentrate on format over content, and content is very important until the details are finalized.) HTML also allows easier crosslinking of public documents, perhaps enhancing sharing of ideas (and code) between projects.
Change meetings to electronic discussions where the "wild ideas" are captured, and someone can say "we did that because...", rather than having multiple people suggest the same change and have it rejected because of some subtle reason, multiple times. Having electronic discussions will also allow people to participate after the fact by reading the notes and being able to question the participants.
Documentation needs to be a continuous process. If the project leader is aware of details such as a User Manual and begins thinking about the users of the product, the quality may be better. Internal products may not need marketing literature, but it might not be a bad idea to at least think about how to sell the change. If the developer is aware of testing documentation, some "scary" part of the code can be incorporated into the testing plan, and the product can be more thoroughly tested and made more reliable.
Once the project goes from design to development, the developers and designers can add clarifications and corrections to the design. As the design goes from development to QA, the changes (and the reasons for them) can be made clear, and non-problems will be handled as non-problems. Once the product is finally documented by a technical writer, all the details will be nearly correct as designed. The support people will have the opportunity to ask for "features" to help them do their job.
Once the project is "finished", the designers, developers, and QA should get together and add to the document, outlining the pros and cons of the current design, the details that were desired, and the pitfalls from the user's perspective, as well as from the developer's perspective. This document would be the start of the next revision's design document, even if that project is months (or years) away.
Properly tracked, changes can be easily managed. I'll argue that the changes won't need to be tracked if everyone shares the success of the project. If the people involved with the project are truly participating at the same level, there won't be any need for finger pointing or tracking who did the bad thing.
The only way this will happen is if everyone buys in. Designers have to give up their home directory. Their work will have to be put in a central public repository. There will have to be a sysadmin (or tools) available to deal with daily changes to permissions for this repository (designers are the only ones who can change until development has started, etc.).
Developers have to realize they are part of the documentation process. Many developers feel that documentation is up to the tech writer and that designers don't know coding. This is mostly correct, especially since tech writers don't know coding, either. The tech writers need something to start with. Everyone needs to help everyone. Developers change jobs, and everyone knows you cannot always look at the code to get the full details of the project. Tech writers usually write user documentation, and coders write coder documentation. The coders can start with the design documentation and the tech writer can start with the coder documentation, each creating a public document for a specific target.
In extreme cases, everyone's C: drive should be made read only, and Word should be removed from everyone's hard drive. At a minimum, Word, Visio, Excel, and PowerPoint email attachments should be outlawed. The key to making all this happen is that all data is in a central place and can be updated by appropriate people at any time. Once a document is emailed or printed, it is obsolete, and changes are not reflected in old email.
People will argue that the above tools are ubiquitous, and if the files are put on a shared drive, everyone can access them and update them. I would agree that the tools are nearly ubiquitous and nearly shareable. For odd reasons, people sometimes share things with various read/write bits set so that only the work group can share items, or only they can read them. Visio isn't on everyone's desktop, especially if they aren't using a Microsoft operating system. Until there are tools that are more ubiquitous and sharing can be more automatic, minimizing the non-ubiquitous tools will be the best way to interact with the documentation.
Once the culture has adopted these suggestions as a good idea, there need to be tools in place to make this happen. The documentation repository should exist, with the tools to allow all users to use it. There need to be electronic discussion tools available. There need to be documentation templates. Old project documentation should be made available to allow current products to be updated.
Many tools are available to make a system like this happen, including Lotus Notes and various chat programs. Each has benefits and drawbacks. Programs like Lotus Notes and Exchange, being proprietary, require everyone to have the program and don't leave room for extensions or flexibility in their use (e.g., the use of other clients). Chat programs have no longevity; if the discussion has moved beyond a subject, typically, it will have to be brought up again.
Since most of the Internet was built on email, newsgroups, and the Web, and these tools seem to have longevity, I suggest that they would be the tools to use. Email mailing lists can be used to call attention to new things, including updates to documents, questions that a newsgroup is discussing, or other issues. The newsgroups would be the electronic discussion tools. The Web would provide the golden documents, incorporating issues discussed on the newsgroups and in email. Another tool that may be useful is WikiWikiWeb, which allows full collaboration on Web pages by anyone with a browser (see PhpWiki).
When a new project is started, templates for all the documentation must exist. There should be templates for all types of documentation, and the project leader should begin by putting the title of the project in the document repository and at least blank templates for all documents. All the documentation should include:
Obviously, not all products will need all these documents, while some will need more. Having access to templates covering a minimal set of knowledge for most projects will make getting documentation easier. Final documentation for a small project may include some parts of many of the templates in a single document, and a large project may have multiple instances of various parts of these documents broken into subprojects.
Conversion of current documentation will be difficult. This is a one time cost, however, so once done, it may be considered an investment. The difficulty will be to collect all the information that isn't available in the current form to be collected, just as a new user would have to collect and organize the information.
Much of this is brought about by frustrations with current systems, and many of the ideas are quite utopian, inasmuch as radical changes to current culture are almost impossible to accomplish, but doing nothing will produce the least risk and the least rewards. Adopting any of the changes here with a complete buy-in will provide a better world. Without complete buy-in, these ideas will certainly create chaos, and have the potential to make the world a worse place.
These radical ideas are not wholly mine; other people have suggested change, include the EXtreme Programming (XP) group at Chrysler. This group actually advocates less documentation, but in a way different than this document suggests. They do advocate developers coding tests that can be used by other groups.
A similar way of thinking is advocated by Chuck Musciano's Don't Cramp My Style comments. Tools are tools; they don't build products. They can be used to build products, but only people build products. Communication is the cornerstone to good products.
Certainly, the Open Source movement has tainted my thoughts. Knowledge and information should be free. There shouldn't be an owner to parts of a project, and no single individual should be allowed to hoard any of this information. That feeling must include respect for everyone involved in the project. There are responsibilities that everyone in the project assumes, and everyone incurs the same risk. Not everyone has the same abilities, and people's strengths must be reckoned with their weaknesses (nerds don't have interchangeable brains).
Of course, there are ESR's Cathedral/Bazaar writings, in which he quotes Brooks, Chapter 9: "Show me your [code] and conceal your [data structures], and I shall continue to be mystified. Show me your [data structures], and I won't usually need your [code]; it'll be obvious." I want to change this to "Show me your code and conceal your design docs, and I shall continue to be mystified. Show me your design, and I won't usually need your code; it'll be obvious." :-D
One improvement might be the addition of a transition. Once a project's development is complete, the Wiki has stopped. The "final" documentation (user manual, support notes, etc.) needs to be quantified and written. This documentation can (perhaps should) stay in the Wiki, but it needs to be done. Perhaps the templates described above could be incorporated into the Wiki so when a new project is started, is has links to all this documentation. This is where a tech writer could come into play, a professional to manage the documentation. This person could insure that all the needs for these "final" documents can be gathered, then could be the follow up documenter.
I tried to make it available to support, but they aren't interested. This is a whole 'nuther article -- how to get a support organization to cooperate. When I can quantify the issues there, I'll write it.