Throughout my professional career as a software developer, one challenge has remained a constant regardless of my employer…
There, I said it. One word. No description necessary. Every software developer knows exactly what I’m talking about. We all hate doing it and sometimes we just do not do it to the extent at which we should. The hypocrites in all of us then get utterly frustrated when we are asked to jump right into a project for the first time and are expected to know all the things necessary to hit the ground running when there is a complete lack of documentation anywhere to be found.
This brings me to my next point. This blog post is not our end all, be all solution for all things documentation related to Andculture Code. No, this is, instead, written to focus in on the point of documentation that matter most to us as coders… project documentation with a large emphasis for project on boarding.
OUR PRIMARY GOAL
Over the last couple months, a few of the engineers here at Andculture Code were asked to tackle this issue of project documentation. Our leadership team saw a need for a more efficient process that removes the documentation issues we all continue to face. So naturally, our end goal was to further define the need and devise a solution. In the end, we were asked to present our solution to the entire engineering department and then to encourage / foster a culture that adheres to our proposed solution.
OUR PLAN OF ATTACK
Our team needed to come up with a plan of attack. We took the approach some engineers take when it comes to problem solving.
- Define the Problem
- Define the Goals
- Define the Solution
- Iterate and Improve
We began by asking ourselves what questions we typically ask when we start working on a new project. Some of the questions we asked ourselves were...
- What language was this written in?
- What is this random terminology I keep hearing?
- What is the development workflow?
- Who has previously worked on this?
- Where do I go to fix my computer configuration issues?
- How do I set up my development environment?
- Where are all the requirements?
- What are all the things?
- How do all the things connect?
- Oh my… where did all these GIT branches come from and what are they for?
Using questions like these, we then asked ourselves how we would currently find the answers. Going through this process made the problems pretty self-evident. We found that…
- ...onboarding of new project resources is a timesuck both for the new resource and the existing project resources.
- ...we often have multiple, fractured sources of information.
- ...we often are missing documentation.
- ...we too often do not prioritize documentation as part of our development workflow.
We established our goals as such. We wanted to…
- ...make onboarding of new engineers as easy and as efficient as possible.
- ...identify a single source of information.
- ...prioritize the documentation in an order that makes the most sense.
- ...standardize on a documentation format.
- ...make it easy to search and find things by way of better organization.
- ...connect all the things so you don’t need to go anywhere else.
Our solution to our problems ended up actually being pretty simple. If you would prefer to jump right into our documentation template solution and skip all the blog talk, feel free to jump over to the documentation template out on our Andculture Code Github account. As we iterate and improve on this template, we plan on updating this repo so stay tuned for updates.
We ultimately did not want to break with the industry standard use of markdown files so we decided to continue down that path. Instead, we found that better organization of our readme was crucial. All too often, our readme files in the past have become large, cluttered, unmanageable, documentation messes.
In its place, we now define several markdown files. The first is a single root level README.md file which provides high level information and also serves as a linking mechanism out to all the things related to this project. In some cases, these links will be links to externally hosted resources. In many cases, though, these links are merely redirects to more detailed markdown files containing their own relevant subject material within a subdirectory named `documentation`. Each of these markdown files will also contain their own table of contents. Typically, these links will just be jump links down to their respective information contained within their own file.
The various sections listed in this file were intentionally ordered with the priorities of a new project engineer in mind. The idea is that any new project engineer could jump on this project, clone the repo, open the readme, and then have access to all the information they need in the order in which they need it.
As its name suggests, this is a place to summarize the project and its functionality at the highest of levels. This should be short and to the point. Any software given a name or an acronym should be called out here and clarified.
This is an opportunity to list the most important technologies and frameworks that are being used. Where it makes sense and is important to the engineer, note the major versions of the tech you’re using but limit it to major versions only. Do we really care what each and every node dependencies you’re using in this project and what specific version it is? Of course not… be smart about what you fill in here. Don’t create more technical debt for yourself.
This should be a rather simple listing of all the people involved in the project. Clearly state their name and the role they played. No more, no less. The idea is that any new resource now has the ability to see who’s worked on this in case they need to question anything.
Table of Contents
As stated earlier, this is merely a listing of various links to more detailed read me files for internally hosted documentation in addition to various links to externally hosted documentation where it makes sense.
Developer Machine Setup
/documentation/developer-machine-setup.md This markdown defines every step necessary to get your development environment set up. Using Docker? Define those configuration steps here. What about Vagrant? Bingo… here. Got a Mac and you’re using .NET Core right on it? Yep, it should all be listed here. In addition, there should be a troubleshooting section contained within this file to identify any problems that were encountered during configuration and how to resolve each.
/documentation/production-troubleshooting.md This markdown is an area where ongoing production issues and their resolutions should be listed. Let’s use the example that something happened to a software package that is running in production and now your client’s business comes to a screeching halt. The hours count down while your team scrambles to find the problem. Nearly a day wasted and you and your team finally discover that there is a manual process that someone needs to run that was never automated. Yep, that should have been written down somewhere. Put it here.
/documentation/branching-strategy.md What do you use for your branching strategy? How many different environments do you have? Do your branches relate directly one to one to your environments? What are they and what is the use of each? How do you name your feature / bug branches? This is all relevant information that should be defined here.
/documentation/maintenance.md Use this markdown as an opportunity to list anything that you should be doing on a periodic basis to maintain appropriate performance of your software. Maybe you need to clear your transaction log in your database? Maybe there’s a file system that fills up and needs to be archived periodically? Jot it down. It’s important.
/documentation/release-management.md Define your rules for releases here. Does your client require production releases only during certain periods of time? Maybe there’s a required frequency if you’re working on a product? What are the steps for releasing to production? What does your version number scheme look like and define the rules when the number needs to change. This type of information belongs here.
This is a section that links out to an external document. At Andculture, our requirements have historically been stored in Google Documents but have recently moved to Confluence to further integrate with the Atlassian suite of products. For us, then, this link would simply redirect to the one of those sources. Detailed requirements will NEVER live in this file.
System Architecture Design
This is another section that links out to an external document that is optional. The sourced document should be a visual representation of all the things involved in the project at a high level. This is an extremely helpful source of information when conceptualizing more complex systems. Consider a project that uses several different data sources of different platforms such as MS SQL Server, Redis, MySQL, and MongoDB. There are 5 different web applications built using a combination of .NET Core, Node, and PHP. Now consider that each of those has different integration points to things like Google Maps, Mailgun, Twilio, Sales Force, and Microsoft Dynamics? Oh the list can go on an on. Do yourself (and everyone else) a favor and diagram this system.
/documentation/project-architecture.md Have you ever opened the source of a project and been completely confused by the folder structure and organization of all the projects contained within it? Me too. This markdown’s purpose is to address this issue. This should clarify the purposes of the higher level folders and give a brief description of each of the projects (if there’s more than one).
/documentation/design-patterns.md I cannot emphasize the importance of this section enough. As the name suggests, this markdown is to provide referential material to the design patterns being used in code. My suggestion is to provide at least one code example per design pattern with a quick description of what it’s doing and why. Clearly, this is something that will evolve throughout the life of the project. It is our duty as engineers to do our best to keep this up to date.
Andculture uses Slack for its communication platform. It’s become a very integral part of our daily work life. That said, we have a Slack channel for anything and everything. If you’re like us, you can get lost in the black hole that is Slack channel overload. That is why this section exists. If you’re also like us, you have multiple slack channels for a project. We usually have at least 2 or 3 channels per project but we have some projects that have upwards of 6 channels. Use this section to define the name of the Slack channel and give a brief description of its purpose and appropriate use. It helps new resources get connected to the right people fast and know the rules for Slacking.
/documentation/environment-production.md For each of the environments in your project, there should be a link to another markdown file. The example link provided here is for the production environment. Each of these markdowns should contain every step necessary to configure the environment from start to finish. This could include things like environment variables and installation steps.
3rd Party Integrations
This section serves to quickly outline the 3rd party integration points that this project utilizes and in what way. This should not be anything more than the name and a quick description. We encourage that each integration point link out to the 3rd party API documentation (if one exists).
Andculture uses Google’s Drive and the G Suite of products for everything. Every project gets its own folder in Google Drive. This section simply links out to that folder.
This section serves as a catch all for all external sources that were not linked out to in previous sections. Maybe your client uses DropBox for transfering files? Perhaps there’s a portal the clients wants you to use for client communication or project workflow? These type of things should be listed here as links with a description of each.
Changes to Development Workflow
Defining our new documentation template is only one important step in the process of migrating our current documentation strategy over to this one. The truth is that unless there’s a shift in culture where this becomes a more important task as part of our workflow, nothing will ever change. Therefore, we are instituting the following guidelines by which our engineers are now incorporating into their daily development workflow.
- Clone the Documentation Workflow project from our Github Andculture account and start watching it for updates.
- When setting up a new repository for a new project, copy the documentation template files including the README.md and the `documentation` folder as well as its contents into your new repository. Fill out as many of the sections as possible. This must be done before any code is committed to the repository.
- When working on an existing project with no template, copy the documentation template files including the README.md and the `documentation` folder as well as its contents into your new repository. Fill out the sections that pertain to the part of the project that you’re working on. Be smart here. Don’t try to fill out all the things unless time allows.
- When working on an existing project with a template, fill in the sections that pertain to the part of the project that you’re working on. Again, be smart.
- Are you ever not busy? Probably not. But if for some reason you are or you just want to make a difference in another engineer’s life, spend some time revisiting an older repository you’ve worked on and give its documentation template some T.L.C.
The idea here is that as we work on projects, their documentation will little by little begin to fill in. Eventually, someday, all of our projects will adhere to the same format.
A FEW MORE IMPORTANT POINTS
This is currently being used strictly by our engineers. There are several items that were mentioned above some may argue do not belong in the repository. The reality is that this is documentation meant strictly for our engineers… maybe the design team as well. This is only something we will be managing (unless your product owner, sales people, project managers, accounts reps or whoever else knows how to use GIT). If it makes us more more organized and more efficient at our jobs in the end, we’ll deal with whatever headaches are associated with it.
Making this part of our development workflow will consume time albeit a very small fraction of time hopefully. Make sure you plan for it accordingly in your estimations. The good news is that we now have a documentation template which probably covers all the situations you may run into. Just plug in your information where appropriate and move on with your work.
Last but not least, you should NEVER store password or any other sensitive information directly in your repo anywhere. Instead, we suggest you use a password manager and reference it as such when and where it makes sense. Setting up environment variables in the Environments markdown files is one example of this.
ITERATE AND IMPROVE
As this template continues to be used in the future, we realize, though, that things change and ideas and concepts will be added to this template. That is why we will continue to keep the template updated with our latest documentation practices as it evolves.
So what do you think about our documentation template? How do you approach defining and standardizing your project documentation? Feel free to drop us a line using. Looking forward to hearing what you have to say! We value your opinion as well!