|Be a lazy developer! You know you deserve it.
I’ve been developing websites professionally for almost nine years now and although I still party like it’s 1999, my productivity has increased greatly thanks to better tools and technologies. Due to recent dealings with the IT department of another firm, I remembered the fact that although we think that all developers are using source control (CVS, Subversion, etc.), this is not the case. There are lots of corporate developers out there who don’t follow the industry’s best practices. This post is not about using version control tools for source code… it’s about re-using the same tool for deploying websites, instead of FTP. We assume you know what source control is and are interested in using it in novel ways.
A few days ago we were visiting the facilities of a company which provides services of interests to one of our franchisor customers. As our specialty is the integration of external systems with FranchiseBlast, our franchise management tool, we wanted to know how the data would be able to move back and forth. One of the sophisticated options available to us was the use of FTP to transfer flat files in a specific file format, not that there’s anything wrong with that! Indeed, when your core business doesn’t require lots of integration with your customers, no need to re-invent your solution every three years with newer technologies. You can afford to stick with the same working solution for a long period of time! (We will obviously continue to push web services, as it is much easier to write code against a web service than picking up flat files from an FTP server!).
Integration and automation reduce support costs for the franchisor
We’re always looking at pushing the envelope and we know that software integration and especially automation is the key to cutting down labor costs. If your business processes include lots of manual labor, we feel it is worthwhile to take the time to investigate replacing a few steps software-based solutions (focus on the 20% of steps that make you lose 80% of your time). Wouldn’t you rather play with your new puppy than copy-paste data from one place to another? A typical example for the integration we built into FranchiseBlast and the point of sale is the automatic creation of products in stores, once they are created on FranchiseBlast. Our franchisees save lots of time not having to create their own products and we avoid the situation where an incorrect UPC is entered only to be discovered months later.
Furthermore, although your mental picture of a franchisor might boil down to someone lounging in their satin pyjamas in front of the fireplace, sipping some piping hot cocoa, while waiting for the royalty check to come in, this is very far from the truth. Supporting the franchise is a major time consumer but if you can manage to reduce and/or simplify all the work done inside the store, you can greatly reduce time spent supporting the stores.
Enough about the franchise tangent; talk about web development!
Integration and automation does not only apply to your customers: any serious web development firm still using FTP to deploy websites should consider the following questions. By a serious firm, I mean you’ve got more than your mother’s recipe website to deploy and you build dynamic web applications that change regularly, not a static business card for your trucker uncle.
Are you re-uploading the whole site every time you make an upgrade?
Are you selecting the changed files manually or telling your FTP client to only overwrite new files?
Is someone else also uploading files and you’re never sure what changed?
Do you care about being able to deploy frequently without wasting your time moving files around?
Do you have to re-upload large files (DLL, images) even if you know you only changed a few bytes?
Did you ever have to revert your website back to a previous version when a bug was discovered?
Do you upload to a staging area for client approval, then deploy to the production server?
If you answered yes to one of these questions, you’re probably wasting your precious time and (now cheap) bandwidth. Yes, I know it is fun to read comics while you’re uploading a site but you’re not using technology to its full potential.
Source control technology has been around for decades and hopefully you’ve been using it to collaborate with other developers or designers when creating websites. Even if you work alone, there are several advantages to using CVS or Subversion, for example. You may be wondering why I am talking about source control in the context of web deployment but I hope the astute reader will know exactly where I’m headed.
Why not deploy your websites using source control tools such as Subversion?
There are probably lots of you out there that already do this but there may be some people that never thought outside the box. By sharing this with you today, I hope to help at least one person cut down an hour per week spent in deployment. We’ve experienced the benefits and wanted to share them with you.
We prefer Subversion over CVS for multiple reasons, but one that is of particular interest here is the fact that it can do binary diffs. If you recompile your ASP.NET website, you’ll generate new DLL files that are very similar to the previous ones. The same thing happens when you’re changing images. Thanks to this Subversion feature, you only have to upload the bytes that have changed inside your files… as opposed to the whole files! Furthermore, as with most source control management tools, you can copy a file from one project inside your repository to dozens of others, without taking up additional space on the disk.
You can create a separate repository for your deployment files (separating them from your source code) and checkout the project on the production server. Later on, when you commit your changes, you can simply perform an update on the production server. You could even automate the update process using post-commit actions.
There are numerous advantages to deploying using a tool such as Subversion:
You only commit what has changed and the tool tracks these changes for you.
Only changes are transferred, not the whole site.
If someone other than you fools around with the deployed site, you can immediately see what was changed.
You can easily revert back to an older version of the site
You should automate the deployment to the staging area (using continuous integration or post-commit scripts) but keep deployment to the production server manual. Automatic deployment to your staging area means you can commit at 4:50PM on a Friday afternoon, wait for the successful build confirmation, and head off to play hacky sack with your high school buddies without having to manually redeploy the development machine. At 5AM on Saturday morning, when your early-riser client gets up and has a few minutes to spare before reading the paper, he can load up the development server at play with the latest build.
What about my database?
The concept of database migrations is very useful in this context; if you use tools that have database migrations built-in (such as Ruby on Rails), then you are in luck. Otherwise, it gets more complicated. We’re waiting for database migrations to be supported by SubSonic before investing much effort in this area (although I don’t recall ever having to revert a production server back to a previous version). For our business, this is a must have feature because it allows us to revert a store’s point of sale to a stable build should the software exploded in the middle of a busy Saturday. Even better, should a fire destroy the store’s computers, we can reload this store’s customized version within minutes. (We also do nightly off-site data backups of the sales information).
In any case, we recommend you take a peek at the series of posts made by K. Scott Allen, referenced in this article by Jeff Atwood.
How can I save more time?
The answer is simple: by scripting the copying of files (locally) from your development copy to the checked out folder of the production repository. This can be as simple as using the build-in deployment tools available in your IDE (such as VS.NET’s deployment functionality) or writing a script that copies all files of a particular extension from one place to the other. Eventually, you’ll need to adapt your script to your particular needs, if you’re wasting too much time copying large files that never change, for example. This step depends on your project and its environment. I will describe in a future post how we use NAnt to manage our software product line. Kudos to Jean-Philippe Daigle for helping us out in the early days.
LavaBlast deploys its point of sale application via Subversion; every computer we manage has its own path in the deployment repository. This allows for per-store customizability and is not as redundant as one may sound because of Subversion copies. Furthermore, when the POS communicates with FranchiseBlast, we track exactly which version of the software is running in that particular store (via the Subversion revision number). We also track a revision number for the database schema. Having this bird’s eye view of the software deployed in the franchise lets us easily schedule updates to subsets of the stores. At the point where we are now, we could easily write code that would signal to the store it should upgrade itself to a certain revision at a certain time & date. Ultimately, instead of spending my time copying files, I am available to write blog posts like this one!
By moving away from FTP, we’ve considerably cut down the time it takes to deploy a website. We invested time in this infrastructure early on, allowing us to concentrate on design, coding, and testing as opposed to deployment. Of course, FTP still has many uses outside of the context we describe here! FTP has been around for a long time and will not disappear any time soon!