I’m a planning whore. It’s true. I’m one of those weirdos that really enjoys creating data flows, use cases, wireframes, and functional requirements documents. My bizarre predalictions aside, wireframes are a critical part of planning any website or web based application.

Spending the time to plan your project out well at the very beginning may seem like a luxury your timeline and budget can’t allow, but most people realize pretty quickly that failing to do so often ends up costing exponentially more time and money towards the end of the project, when you have exactly none of either to spare.

Scope starts to creep, or features and interface challenges that hadn’t been thought through before are suddenly at the center of a series of critical and rushed decisions that have to be made. This means that you (and/or your crew) are stressed, and because you’ve run out of time and money, you end up having to cobble something together that works as a compromise, but doesn’t stand up to the caliber of work you pride yourself on creating.

For those of you who have been reading this blog for many years, you’ll recognize the tune I’m singing. I constantly harp about planning and documenting. My love affair with planning and documentation isn’t just because of that head injury I suffered as a kid – its because I’ve been fucked over so many times and on such a grand scale in the past when I (or my client/manager) have decided to cut corners and just “crank it out”. While the “crank it out” method may work sometimes, the times that it fails, it fails hard, at the expense of time, money, quality and developer sanity.

Okay, I’ll get off my soapbox. For now. </rant>

So, wireframes. I’ve done them in just about every application you can imagine. Visio, Smartdraw, Omnigraffle, even hand-drawn – you name it, I’ve probably tried it. I’ve been on the eternal quest for the perfect wireframing solution, but I’ve always been left wanting.

But What ARE Wireframes?

Wireframes are a graphical representation of a website or application that help illustrate the user interface without the clutter of stylized design elements. If you skip the wireframe process and go right to Photoshop comps, the client can often get so hung up on the fact that they want that button to be blue with rounded corners instead of green with a square corners that critical elements in the UI can be forgotten altogether. Wireframes let you plan through the user interface elements that are required for each screen in a barebones, no-frills manner.

I’m a big fan of sketchy wireframes, especially in the beginning of a project. I find that if you go to a client with something too polished, they perceive it as a structure that has been decided upon, even when that’s the furthest thing from the truth. The sketchy styled wireframes have a hand-drawn, loose feel to them that seems to better communicate that it’s a discussion, not a decision. By avoiding a polished, rigid wireframe, I have found that clients are more able to understand that what they’re looking at isn’t meant to represent the finished design.

I used to hand-draw my wireframes – but the obvious challenges there is that if a change is made to an element that appears on many pages of the site, you now have to re-do all of those drawings. Another disadvantage to pen and paper wireframes is that you have to write a crapload of code if you want to extend the drawing into an interactive format where the client can actually click on elements and see how they behave.

Evolution: Easy Prototyping

More recently, I had found a few apps that were a lot closer to what I was looking for. Balsamiq Mockups is a great application that has a gorgeous sketchy style and a great library of user-contributed downloadable interface elements, so just about every imaginable interface element is available for free download once you’ve purchased the product. More interestingly, Napkee is a commercial addon for Balsamiq that allows you to export your wireframes into HTML (with jQuery) or a Flex file, turning static wireframes into a prototyping tool with very little additional work. Neat!

Unfortunately, Napkee ended up being a little buggy sometimes, and perhaps more importantly, Balsamiq doesn’t natively support Master templates yet. So once again, when a change was made to the main site navigation, I’d have to go in and edit 30 files instead of being able to update a Master template and have those changes reflected across the boards.

Another drawback is that Balsamiq does not support the ability to switch back and forth from sketchy style to straight lines. In my workflow process, what I wanted was to be able to do the rough sketchy style wireframes in the beginning, update them as needed, and then be able to click a button and convert them into a more polished, final-looking set of wireframes that I could hand over to the graphic designers with a client sign-off.

A later contender, Flairbuilder, looked like it might be the answer. It does support the ability to toggle back and forth between sketchy and regular line styles, however the sketchy-style lines somehow didn’t look as nice as Balsamiq’s, and (this is the big one) in order to show the client a working prototype that was created with the desktop version, you had to export the file and then ask them to download a proprietary viewer. Fuck that noise. Half my clients are still on IE6 for chrissakes. I’m not asking them to download anything. So back to Balsamiq I went. Until now.

The Next Generation

Enter iPlotz. I like it already because the name makes me think of a pixelated rabbi pitching a fit. What are their next product lines going to be, the iKibitz and the iSchmaltz? HAH! I’m hilarious. But probably more importantly, it fills in the gaps that Balsamiq+Napkee and Flairbuilder couldn’t – and even adds some great project organization, management and collaboration tools in to boot.

The interface is pretty intuitive, and they have a nice library of elements from which to choose – plus, like Balsamiq, they have a public repository of snippets and projects available for download to fill in the gaps. Like Balsamiq, iPlotz has both a web-based version and a desktop version. The desktop version is a must-have for me, since I spend a hideous amount of time (4.5 hours a day) commuting through mountains where I have no connectivity.

What’s Awesome

• Works on PC, Mac and Linux. (I normally despise AIR applications – but it works here. Nicely executed.)
• Support for Master template files. If our main app nav or layout changes, change it once and it automagically updates on all the pages using that Master. HUGE time saver.
• Send work you created in desktop client to your web account, so you can do a lot of the work wherever you want and then upload it when you’re ready to use the collaboration tools.
• Toggle between sketchy-style, Mac skin and PC skin, great for rough mockups and then converting to polished final versions with just a click.
• Export PDF list with thumbnail of annotated elements an annotations, perfect for an organized breakdown of more specific details on each element. LOVE this – didn’t know I wanted this feature until I found it here.
• Online collaboration with others, allowing clients and co-workers to add notes and comments right into the file.
• Detailed version history.
• Built-in task management system with milestone, great for organizing work on large projects where tasks can get lost.
• Export wireframe pages as PNG.
• Share the HTML prototype on their server, or export to ZIP file and upload it to your own.
• One-click grid overlay.
• Easily embed the prototype in your own websites.
• Easily create or import additional UI elements from user contributed snippets.
• Properties menu lets you specify exact pixel dimensions for every element if needed.
• Separate menu of iPhone UI elements (Android coming soon).
• Automatic site map generation.

What Needs Work

Of course nothing is going to be perfect, and iPlotz is no exception. Here are a few of the things that bug me, none of which are even close to being deal-breakers in my world.

• Hotkey mapping is a little wonky in some browsers, but that is more an artifact of AIR/Flash than it is iPlotz, I think.
• Some elements do not maintain the same sketch style. For example, the module-style widget has rounded corners, which is inconsistent with the rest of the sketchy style’s squared corners. I also feel it is introduces a level of graphic design that I specifically wish to avoid by using sketchy frames. I have only noticed this on a handful of elements, however, and I spoke to iPlotz founder Mark Vernon today and he assures me that properties like that will be customizable in the near future.
• Element library isn’t as extensive as Balsamiq, but user-contributed snippets seem to be on the rise
• I didn’t see any way to save a frequently used imported snippet to your element library so you can get to the easily for future projects. UPDATE: Mark informed me that they’re going to be pushing out this functionality in one of their next releases. I’ll be checking it out on their staging server.

My Demo

In order to give it a fair review, I decided to whip up a small set of mockups, both to give me an idea of what the UI is like, and to be able to show you some “finished” products.

Below is the embeddable widget that I could easily stick into a branded HTML page to show my client. Click around in it, and you’ll see it’s fully functional. The size was dictated by my blog size, not a iPlotz limitation. You can display it at whatever size you’d like in your own sire.

Check out my sitemap demo version here. You can set your wireframes to public or private – I’ve got mine set to public so that you can see them, but I probably would keep them private and just assign roles to team members on a real project.

Take a few minutes and click around in the protoype. The nav menus, homepage accordian box, contact form, etc all work, and it required no extra work on my part other than to specify a link in the element’s properties. (If you’ve used Balsamiq+Napkee before, the process is basically identical.)

For client deliverables, in addition to the fancy shmancy working protoype, I can easily export single wireframes (using the sketchy style, the Mac OS style or the PC style):

Or I can export them all into one PDF [PDF link]. And this is my favorite part – the bit I didn’t realize I needed until I had it – I can generate a PDF that contains only the elements in the pages that have annotations [PDF link]. This is fantastic because it allows me to explain any special considerations around specific elements in great detail without cluttering up the wireframe itself. I normally have to manually create this documentation, so this feature alone has just saved me hours of repetitive work.

Another Potential Option. Sorta.

Another interesting contender is ProtoShare. It looks to have a lot of the same functionality as iPlotz, with online collaboration, version history, project management elements and easy prototyping, however their product is web-only, which I’m not crazy about, and their free demo required a credit card, which I can’t stand on principle.

Protip: The free trial is supposed to woo new clients and get them on-board and buying as quickly as possible – asking for a credit card number just so someone can try out your software is a little sleazy, as if you’re hoping I’m going to forget to cancel and you can suck me for at least a few bucks before I get around to it. If your product is good and brings value, I will buy it. Don’t force me. Dick.

For me, web collaboration might work fine for my personal web clients, but some of the clients we have at the agency I work for have strict rules about online collaboration in apps that do not live on our server or theirs, so my particular circumstance might limit the times I can use the collaboration in a practical situation.

The price points on ProtoShare is also a little higher. iPlotz starts at exactly free, albeit with a very limited plan, only 1 project, max 5 pages. The next plan is $15 a month (although the best value seems to be the yearly plan of $99 which comes with a license for the desktop version), whereas ProtoShare starts at $29/month with restrictions on how many people can review/collaborate. Who knows – it might be worth the extra scratch, but I hate using web-only tools, and I’m not going to give them my credit card information just to try out a demo. You can compare their prices in more detail here: ProtoShare vs iPlotz. When they come out with a desktop version, I might be willing to check it out. I don’t see any option for the sketchy-style mockups in ProtoShare either though, which is another negative in their column.

But hey, don’t take my word for it – check out the video demos on both sites and download iPlotz. I’m curious to know what you think.

Note: The title of this article refers to “cheap”. While to many people, “cheap” might mean “free”, I consider ~$99 a year a reasonable price for such a flexible tool that is such an important part of my job, especially since Balsamiq is $79 and Napkee is $49, so you’re spending that much even if you go the “cheaper” route. Seriously, you spend more than $100 on Starbucks (or whatever corporate coffee swill you’re into) a month, and I spend more than that on hookers and blow in a single week. You can also buy the iPlotz software outright without the web collaboration, which ends up being about $80 for a one-time purchase.

So that’s iPlotz. I’m a now paid member, and I’m very happy with what they provide. Plus, in talking to Mark today, it sounds like they’ve got more very cool features coming down the line soon.

What prototyping/wireframing tool do you use? Let me know in the comments!

Possibly related posts:

1. Cheap or Free Website Status Monitoring Its a call you never, ever want to get. “My...
2. Generate lists of banned words for forums and other applications If you develop software for a living, or if you...
3. Extending Facebook Static FBML Tabs with Dynamic Content This tutorial walks you through how to use DynamicFBML to...

In part one of this series, you learned what Subversion is and how it can help you, in addition to the terms you’ll need to become familiar with in order to use Subversion in your development process. Part two of this series goes into more specific detail about potential development environments that may work for you.

Subversion creates a bunch of hidden files and folders inside of every directory in your Working Copy that’s checked into SVN. If your Working Copy is on your local computer, you’ll see these as .svn directories if your system preferences allow you to see hidden files and folders. You don’t need to worry about these, and you never really need to look at them, but as a result, it is not recommended that your Working Copy be on your live production box. If you decide not to listen to me, you should at least consider adding a few lines to your .htaccess to deny prying eyes access to your .svn directories.

Working Copy & Repository on your local machine

Create a Working Copy on your local machine, create the Repository on your local machine, and FTP/SFTP the files once they’re committed. This will really only work if you are the only developer, or your local machine is the only machine changing files. (Okay, that’s not entirely true – you can set up the Repository on a network drive and have multiple developers Check Out their own Working Copies to their own computers, but I have run into issues running Subversion over a network.)

If you’re running OSX on your local machine, see this tutorial on WikiHow on how to set up Subversion locally. If you’re running Windows, there is an exceptional walkthrough on The WebSqueeze written by Christopher Hyne that is a must-read. Setting up Subversion on Mac or PC is really quite painless, and this is a perfectly valid way to go.

Pros: Easy, local setup. No external services to set up, your workflow doesn’t change that much, and you don’t need an internet connection to commit things to your repository.

Cons: Possible challenges with running Subversion over a network, you still have to upload your files manually, and you don’t get the benefit of a slick gui with which to view revision history, as you would using Beanstalk or Unfuddle for hosted solutions, or by setting up Trac (a gui reporting interface/wiki/ticket system) on your server. You can set up Trac on your local machine with Python and some extra work, but that’s always been too much trouble for me. Just MHO though – it works great for some people.

Working Copy on your local machine, remote Repository

For my personal stuff, since I use Mosso as my web host, which does not currently provide Subversion (though I hear they’re working on it), I use a third party Subversion host, like Beanstalk or Unfuddle. I create my Repository on Beanstalk or Unfuddle, Import whatever files I may have created for the project already, Check Out a Working Copy to my laptop, and I’m all set. I work on files on my laptop, test them on my local installation of Apache/PHP using XAMPP, Commit them to Beanstalk or Unfuddle (depending which service I’m using to host my Repository), and upload them via FTP/SFTP to the development server area.

The result is a little extra work – I have to deal with SVN on top of FTP/SFTP, but the security of knowing my ass is covered and I have a detailed revision history makes it entirely worth it to me.

One benefit to using a third-party SVN host, such as Beanstalk or Unfuddle (besides the fact that it gives you SVN capabilities if your server doesn’t support it) is that you get a shmancy web interface with which to view your revisions. Below is a screenshot of the Unfuddle web interface. You can see that it gives me a nice, organized view of all of the history, including revisions messages, numbers and person committing. (This is a project for JPM Morgan that I worked on with my friend Vidyut Luther.)

In getting Vidyut’s blog url, I see his latest blog posts states that he can, in fact, use Subversion on Mosso, by using Expandrive. I haven’t tried it yet, and he has, so if you’re interested in exploring that as an option, check out his blog post about it. I may give it a shot at some point, but I do like the simplicity of working with a third-party SVN host. I don’t have to administer anything, just create the Repository and go.

Remote Branches on dev, remote Repository, rsync to live

One way of handling your development environment is to set up a Repository on the remote server, Check Out a Working Copy on a dev area on the remote server, such as dev.example.com, and set up individual Branches for each developer. An SVN Branch is simply a copy of the filesystem separate from the main version, called the Trunk. Using Branches allows you to save your half-broken work frequently without interfering with others, yet you can still selectively share information with your collaborators.

In this scenario, no one ever touches the live files directly, ever. Ever ever. Everything gets pushed to dev for review, and once it’s approved, you run the rsync script and rsync is the only utility that ever moves stuff over to live.  It may sound confusing, but check out this simple blog post that explains how it can be done.

This can be daunting to set up, since there will be directories on live you may want to skip during the rsync (uploaded images, cache files, etc), but once it’s set up, it generally works like a charm, and it insures that no one ever screws up the live server, since no one is ever touching the live server.

Creating Branches is incredibly easy – check out the section on Branches in the free Version Control with Subversion book. They cover this concept and process very well. To give you an idea of what Branches look like, we’ll use their example structure, with a project named calc. This is what the original directory structure looks like:

You can see in this example, the ‘branches’ directory is currently empty, since we haven’t created any branches yet. The three file icons represent the total files in the project, which live in the Trunk.

Once we create a Branch, named ‘my calc branch’ (since it’s a Branch within the ‘calc’ project), you can see that a new directory is shown in the ‘branches’ directory, and a copy of the files from the Trunk live there:

Parts of each Branch can be selectively merged into the Trunk, so each developer can work on their tasks within their own Branch, break (and hopefully fix) as many things as they need to, Committing their work as they go, without affecting the main Trunk until the revisions are merged.

If you didn’t want to use Branches, you could add a staging server in this process. At a place I used to work, we had a development version that was actively worked on, a staging server, and a live server. We would Commit changes to the development server, rsync over to the staging server, and the staging server would automagically rsync every 10 minutes to live. Nothing was synced to staging unless we had kicked the tires and done QA on it on the development box, and if we couldn’t wait the 10 minutes, we could do a manual push to live by running the rsync script directly from the command line.

These are just a few of the possibilities. How are you using Subversion? Leave your configuration in the comments! And keep your eyes out for the next section that deals with the basic Subversion commands you’ll need to get started and optional gui applications you can use to make your Subversion experience easier.

Possibly related posts:

1. Getting Started with Subversion As a web developer, hopefully you’re already using subversion, and...
2. Dropbox versus FolderShare for Syncing Files Between Computers In my eternal quest to sync up all of the...
3. Questions to Ask Before Starting a Project We’re all human, and we all make mistakes and overlook...

As a web developer, hopefully you’re already using subversion, and this tutorial is utterly useless to you. If so, good job, and carry on! If you’re a web developer (or web designer, or web programmer, or are in any way creating files that display or do something on the internet) and you’re not using subversion (or any other version control), this tutorial is for you.

Since this is a question that comes up a lot, I’ve scoured the web for a really basic subversion primer, and although many come close (including the outstanding free online book, Version Control with Subversion), some people may feel overwhelmed at such an immense amount of information. Truly, I believe the only flaw in the Version Control with Subversion book is that they don’t include any real life set-up examples when discussing the Subversion Architecture.

There are many different ways you can set up your development environment, so I can understand why they would be hesitant to give specific examples, however some people learn best by seeing examples that relate to their lives, so that’s what I’m going to attempt to do.

The example set-up we’ll be looking at is one in which the developer is currently working either locally or remotely, and using FTP/SFTP to move files to the remote development environment webserver. For example, you either:

• Have a copy of Apache running on your desktop/laptop computer. You store and work on the files on your desktop/laptop, test them locally, and then upload them to your server  - or you -
• Make changes to the files locally on your desktop/laptop and test directly in the development space of the webserver (no local webserver) - or you -
• Use a text editor that allows you to pull files from FTP directly. You open them from FTP, make your changes, and save to FTP (no local copy stored at all)

I should mention that the third option here is absolutely the least recommended, especially since you’re not using version control. If your FTP process konks out mid-way through and the file isn’t saved completely, closing your application could mean losing that file forever. (Yeah, I’m looking at you, UltraEdit.) Regardless, if you’re using any of the above scenarios, or something fairly similar, this is aimed at you.

Because the Version Control with Subversion book is so good at explaining so much of this, I’ll be linking to it liberally, and just filing in the blanks where some people may get a little lost.

Disclaimer: This article is a very basic overview of subversion. There are many ways you might have your system and repository set up, and I won’t try to list them all, or even most. I’m going to use what many web developers would consider a “real world” example. You will not walk away from this a subversion guru, or able to deftly administer a subversion server. You will hopefully walk away knowing how to set up a simple repository, connect to it, and begin using svn in your day to day development. If you fuck something up, it is not my fault. Start on a test environment until you get your svn legs and feel pretty comfortable with basic commands.

This article is not a replacement for Version Control with Subversion – more of a supplement for newbies. As you begin to delve further into using subversion, that book will become your bible.

What is Subversion?

According to the Version Control with Subversion book:

Subversion (SVN) is a free/open-source version control system. That is, Subversion manages files and directories, and the changes made to them, over time. This allows you to recover older versions of your data, or examine the history of how your data changed.

Subversion can operate across networks, which allows it to be used by people on different computers. At some level, the ability for various people to modify and manage the same set of data from their respective locations fosters collaboration. Progress can occur more quickly without a single conduit through which all modifications must occur. And because the work is versioned, you need not fear that quality is the trade-off for losing that conduit—if some incorrect change is made to the data, just undo that change. (more)

This is a pretty good explanation of what it is, but it can be hard to imagine what that actually looks like or how you’d use it in practical terms.

So what can Subversion actually do for you?

It means you can have multiple developers working on a project without fear of one overwriting the other one’s work. Have you ever worked on a project with multiple developers, where everyone is working on the same codebase?

“Hey Bob – I’m working on index.php – don’t touch it for now. I’ll let you know when I’m done, okay?”

or how about

“CRAP! Did you make changes to view.php? You blew out my work! I spent half a day working on that frakking function!”

All of that goes away. Each developer can have their own version of the application or website (called branches), where the changes they make will not impact the other developers, and the changes can then be selectively merged into the main development area, or trunk.

It also means that you have a method by which to track all of the changes you’ve made to any given file. Each time you commit changes to the file, subversion assigns that change a unique revision number. This means that if you ever need to roll a file back to a specific revision, you can easily do so with a simple command.

A good example of where you might need to do this is where you’ve “fixed” a bug that broke something else on the site (and really, who hasn’t done that?). If you know the now-broken functionality was working before, you can roll back to a previous version of the changed file (or any version before that, if needed) to the point where the functionality wasn’t broken.

You can also compare revisions side by side, so if you don’t want to do a full rollback, you can see what changed from one revision to the next to figure out what you (or they) did to break it so you can undo just that part or find a new solution. Plus, since each developer has their own subversion username, you can see who did what (so you know who to fire/berate/pummel/defenestrate).

In short – whether you’re flying solo or working with a team, Subversion can help keep you sane, and bail your ass out when you need it most.

Sounds awesome, right? Let’s get started!

Before we go any further, there are some terms that you’ll need to understand in order to grok Subversion. Don’t worry too much if they are hard to remember – we’ll be using them a lot, and they’ll become second nature to you.

Repository - this is the central location where Subversion keeps all of the information about your files, folders, and revisions. The repository can live on the same server as your development environment if your hosting setup has Subversion installed, or it can live somewhere completely separate.

If your hosting company does not have Subversion installed, you can even use a third-party Subversion host like Beanstalk or Unfuddle, whose sole purpose is to store your repository, and only your repository. Your local files are still on your hard drive, and your development files are still on your webserver.

Working Copy - this is the set of files you work with to make changes to your code, build your application, etc. The working copy usually lives on your hard drive, and would replace the directory you upload from and download to if you were using FTP. Simply put, this is the copy of the files you are meant to be working with. You can have multiple working copies of a site for any single repository, and in fact, you would have multiple versions if you have multiple developers. Each developer would have their own working copy.

Import – Importing in Subversion allows you to pull your existing files into a new repository. So, for example, if you have a site that’s already halfway built before you have the epiphany to use version control, you’d import your existing stuff into the repository.

Check Out – If you already have files in a repository and need to create a new Working Copy, you’d use Check Out.

Update - Using the Update command pulls the most recent version from the repository. If you had multiple developers working on the same files, or if you’re working from multiple computers, you’d want to make sure you update before you start making any edits to the files. If you fail to update before you make changes, you risk running into a Conflict when you Commit. (see below)

Commit – When you’ve made changes to a file or files in your Working Copy and you’re ready to push those changes into the repository, you Commit them.

Conflict – A Conflict occurs when there are changes on both your local version (Working Copy) of a file and the Repository of that file. For example, Sally makes changes to the file sandwich.txt in the repository. Harry has just changed the file in his working copy and commits it. Sally doesn’t update her working copy before committing it and she gets a conflict (since Harry committed his version after Sally’s last Update, and Subversion therefore sees Sally’s version as out of date). More on predicting and resolving conflicts later.

Trunk - the location within your Working Copy where your main project files live.

Branches – individual copies of the project that allow each developer to have their own working version. Changes can be Committed without breaking the Trunk, and are then merged into the Trunk when they are deemed worthy. (More on this in the bonus section of part two.)

The image below gives a very high level view of how Subversion works.

The Repository is your gatekeeper. It is the part of Subversion that makes sure files don’t get over-written with outdated versions, and allows you to work in tandem with other developers without risking stepping on each others toes. Other than through the command line (to execute import, update, commit, and status commands) or through whatever gui application you prefer using, you never access the Repository directly.

How often to Commit?

This is largely a matter of preference. Some developers wait until entire classes or functions are added/edited/etc before committing a file. I personally tend to commit more often, since it gives me a more granular way of seeing what broke something else and rolling it back, etc – but the only wrong way to Commit is to not Commit at all. You do not need to commit every time you change every tiny thing, though. For example, if I had to do some copy changes, I wouldn’t Commit after changing each word. The revisions would be so broken down, it would be hard to find important changes in the sea of revisions.

You (and your team, if you’re working in a group) will find a Commit frequency that works for you, as you become more comfortable using Subversion, so don’t sweat it too much in the beginning.

Now that you know the lingo and what it does, you need to decide how you’re going to set up your subversion workflow. Check out the second part in this series!

Possibly related posts:

1. SVN Working Copies and Repository Locations In part one of this series, you learned what Subversion...
2. Introducing FBMHell.Com Good news, everyone! My compulsive need to make websites and...
3. Dropbox versus FolderShare for Syncing Files Between Computers In my eternal quest to sync up all of the...

On *nix-based servers, mod_rewrite can be a powerful tool in any web monkey’s arsenal, however it still voodoo to many, while others may not even be aware that it can help them at all.

What is Mod_Rewrite?

Simply put, mod_rewrite is an Apache module that let’s you rewrite urls based on rules you define. That’s it. Seriously.

Regardless of how confusing some of the rules you may have come across appear to be, all they are doing is taking one url and rewriting as a different url. This rewriting happens at the server level, before the user’s browser sees anything, so the end result is seamless to them.

When you hear about “search engine friendly” urls, you’re most often seeing mod_rewrite in action. Mod_rewrite is the Apache module that let’s you turn a url like:

http://www.example.com/shop.php?category=Books&Title=Foo

into:

http://www.example.com/shop/books/foo.html

Some other common uses for mod_rewrite:

• Directing all traffic from multiple domain names to one domain
• Directing all traffic from www and non-www to one location
• Blocking traffic from specific websites
• Blocking spammy searchbots and offline browsers from spidering your site and eating your bandwidth
• Mask file extensions
• Preventing image hotlinking (other web pages linking to images on your server)

Apache’s mod_rewrite can be intimidating if you start where you’re supposed to start – the Apache documentation, however there are some very useful, common – and simple rewrite rules that you may wish to consider implementing into your site development plan, if you’re not doing so already.

Note: If you’re using Microsoft IIS, you have a few options, but I don’t use IIS, so I’m afraid I won’t be of much help to you beyond telling you where to look. ISAPI ReWrite seems to be very popular, and there is a free “lite” version available.

Getting Started

Your mod_rewrite rules typically live in an .htaccess file in your web root. You can only have one .htaccess per directory, but you can have individual .htaccess files in sub-directories under the web root. I generally do not recommend doing this. If mod_rewrite rules from one .htaccess conflict with the rules from the .htaccess in a sub-directory, it can be a real pain in the ass to troubleshoot. Try to avoid it.

When you’re adding mod_rewrite rules to your .htaccess file, you’ll want to start by using a conditional that checks to see if mod_rewrite is installed on your server. This can prevent getting a 500 Internal Server Error if you don’t.

<ifmodule mod_rewrite.c>
# Start your (rewrite) engines...
RewriteEngine On

# rules and conditions go here...
</ifmodule>

Directing Multiple Domain Names to a Single Domain Url

If you have multiple domain names pointing to the same site, mod_rewrite can also help you direct all traffic to a single domain url, to improve your search engine rankings. Search engines don’t take too kindly to the same content living at multiple urls – they usually think its an attempt to spam the search engine – and you can actually be penalized for it. To redirect all traffic to one specific domain name,

RewriteCond %{HTTP_HOST} !^www\.snipe\.net$ [NC]
RewriteRule ^(.*)$ http://www.snipe.net/$1 [R=301]

This basically says “if the domain requested (the HTTP_HOST) does not match www.snipe.net then rewrite the url as www.snipe.net“. (Note the escaping slashes after the www and before the .net in the condition.)  The R=301 specifies that the redirect should be a 301 redirect, meaning that the address has moved permanently and search engines should use the new url instead of the old one.

To www or not to www

Even if you have only one domain name, if you’re not redirecting traffic from the “www” version to the “non-www” version (or vice versa), you may encounter this problem. Whether or not you choose to use the www in your url is largely a branding decision more than anything else (i.e. it doesn’t really matter in most cases) – but you should pick one and stick with it.

Require (force) the www

RewriteCond %{HTTP_HOST} !^www\.snipe\.net$ [NC]
RewriteRule ^(.*)$ http://www.snipe.net/$1 [R=301,L]

Remove the www

RewriteCond %{HTTP_HOST} !^snipe\.net$ [NC]
RewriteRule ^(.*)$ http://snipe.net/$1 [R=301,L]

Deny traffic by referrer

There may be a few reasons why you want to block traffic by referrer. Maybe you’re getting a lot of bandwidth-sucking hits from a spammy website – or maybe someone is linking to you that you feel does not represent you very well, and you want to pull the plug on traffic from coming their site.

RewriteCond %{HTTP_REFERER} onebadsite\.com [NC,OR]
RewriteCond %{HTTP_REFERER} anotherbadsite\.com [NC]
RewriteRule .* - [F,L]

In this snippet, the rule is saying “If the referring url is onebadsite.com OR anotherbadsite.com, redirect the user to an HTTP Forbidden error.” The NC specifies that the condition is not case-sensitive, and the OR flag is… well… an “or”. OR is used with multiple RewriteCond directives to combine them with OR instead of the implicit AND.

Keep in mind – this method of blocking traffic is hardly foolproof, at least in the latter of the two scenarios above. If the webmaster of onebadsite.com is linking to you in a way or context you do not want (and you’ve asked them to remove the link), the above method will cause a user on onebadsite.com’s website who has clicked on the link to you from onebadsite.com to hit a Forbidden error. If that user has half a brain, they may very well just google your site name or try to access it later from a bookmark – but it’s a simple measure you can take to keep the idjits out.

Blocking Bad Bots and Spiders

While there is some potential debate as to what is a “bad” bot or spider, the consensus seems to that a bot is bad if they do more harm than good, such as e-mail harvesters, site rippers that download entire websites for offline browsing, etc. Even if bandwidth isn’t so much an issue, I like to block these just on principle.

Please note – this list is not mine – it was directly nicked from a list on JavascriptKit.

RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]
RewriteCond %{HTTP_USER_AGENT} ^Bot\ mailto:craftbot@yahoo.com [OR]
RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR]
RewriteCond %{HTTP_USER_AGENT} ^Custo [OR]
RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR]
RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR]
RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR]
RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR]
RewriteCond %{HTTP_USER_AGENT} ^Express\ WebPictures [OR]
RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR]
RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE [OR]
RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetWeb! [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It [OR]
RewriteCond %{HTTP_USER_AGENT} ^GrabNet [OR]
RewriteCond %{HTTP_USER_AGENT} ^Grafula [OR]
RewriteCond %{HTTP_USER_AGENT} ^HMView [OR]
RewriteCond %{HTTP_USER_AGENT} HTTrack [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Stripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} Indy\ Library [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^InterGET [OR]
RewriteCond %{HTTP_USER_AGENT} ^Internet\ Ninja [OR]
RewriteCond %{HTTP_USER_AGENT} ^JetCar [OR]
RewriteCond %{HTTP_USER_AGENT} ^JOC\ Web\ Spider [OR]
RewriteCond %{HTTP_USER_AGENT} ^larbin [OR]
RewriteCond %{HTTP_USER_AGENT} ^LeechFTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader [OR]
RewriteCond %{HTTP_USER_AGENT} ^MIDown\ tool [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX [OR]
RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR]
RewriteCond %{HTTP_USER_AGENT} ^NearSite [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetZIP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Octopus [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Navigator [OR]
RewriteCond %{HTTP_USER_AGENT} ^PageGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^Papa\ Foto [OR]
RewriteCond %{HTTP_USER_AGENT} ^pavuk [OR]
RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR]
RewriteCond %{HTTP_USER_AGENT} ^RealDownload [OR]
RewriteCond %{HTTP_USER_AGENT} ^ReGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger [OR]
RewriteCond %{HTTP_USER_AGENT} ^SmartDownload [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR]
RewriteCond %{HTTP_USER_AGENT} ^tAkeOut [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro [OR]
RewriteCond %{HTTP_USER_AGENT} ^VoidEYE [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Image\ Collector [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebAuto [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebFetch [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebLeacher [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebSauger [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ eXtractor [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ Quester [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebWhacker [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget [OR]
RewriteCond %{HTTP_USER_AGENT} ^Widow [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus
RewriteRule ^.* - [F,L]

Once again, this method isn’t foolproof. The HTTP_USER_AGENT is quite easily spoofed, and some site ripping applications even allow you to specify what user agent you want to appear as. But if your site is large, implementing this list may make a significant impact on your monthly bandwidth bill.

Mask File Extensions

If for some reason you want to hide the fact that you’re using PHP (or Perl, or whatever), all it takes is a simple line in your .htaccess to have your .php files look like .html files:

RewriteRule ^(.*)\.html$ $1.php [R=301,L] 

You could even completely obfuscate it if you wanted to, for example serving files that end in .snipe that are really .php files:

RewriteRule ^(.*)\.snipe$ $1.php [R=301,L] 

In these examples, redirects all files that end in .html (or .snipe) to be served from filename.php so it looks like all your pages are .html (or .snipe) but really they are .php. Notice again that we’re using a 301 redirect.

Prevent Image/File Hotlinking

This snippets prevents people from hotlinking to your files – that is, linking directly to files hosted on your server from their website, thus sucking your bandwidth. It should be noted that in my experience, this rewrite rule seems somewhat spotty, and doesn’t always work, so be sure to test thoroughly.

RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?snipe.net/.*$ [NC]
RewriteRule \.(gif|jpg|swf|flv|png)$ /images/dont_steal_bandwidth_jackass.png [R=302,L] 

This rule basically says “If the request’s referrer is not blank (meaning the file was accessed directly in a browser) AND is not snipe.net (case insensitive), rewrite any files that end in .gif, .jpg, .swf, .flv or .png to display the file /images/dont_steal_bandwidth_jackass.png.

I have a friend who was trolling through their server logs (as he tended to do) and he realized that someone was using an image from his server, a Debian logo, if I recall correctly – as their MySpace background image. My friend spicyjack, being of the snarky persuasion, set up a leech=prevention mod_rewrite that directed all requests for that image that were not coming FROM his server to an image that was… shall we say…. not something you’d want as the background image of your MySpace page. (Google “hotcurry.jpg” if you’re really curious. It’s NSFW. Or anything, or that matter.)

Search Engine Friendly URLS – Make Dynamic Pages appear Static

To turn site.com/index.php?category=foo&subcat=bar into site.com/category-foo/bar.html, just use:

RewriteRule ^category-([0-9]+)/([0-9A-Za-z]+).html index.php?category=$1&subcat=$2 

The category number and the subcategory name are variables, these are now represented by $1 and $2. The round brackets and the stuff inside them will be replaced by the variables. The things inside the round brackets are the regex rules for what the variable can contain. Example [0-9] means that it can contain any number from 0 to 9 and the + sign means that the number can be repeated 1 or more times.

The [L]ast Word

The [L] flag tells mod-rewrite that no more rules should be processed after that. Also remember that the order these rules are in DOES matter, so you’ll want to consider their intended behavior carefully when you’re planning your .htaccess. If at all possible, have your Apache error logs accessible while experimenting. when mod_rewrite goes wrong, it very often gives you a generic (and infuriating) “500 Internal Server Error” instead of anything actually useful to you – the Apache error logs may help shed some light on things.

Also be sure to test thoroughly whenever using mod_rewrite, since you can seriously break stuff if you’re not careful.

More Resources

Can’t get enough mod_rewrite? Check out these links for additional information and tips.

• Mod_Rewrite flags
• Mod_rewrite Tips & Tricks
• Mod_Rewrite Cheat Sheet

And if you’ve got a handy mod_rewrite rule you can’t live without, let us know in the comments.

Possibly related posts:

1. Upgrading to WordPress 3.0 and Adding Multi-Site WordPress 3.0, code name “Thelonious”, has been released, and it...
2. And Still More Notes on Mosso Continuing in the Moving to Mosso series, I’ve come across...
3. More About register_globals If you’ve been directed to this page, that means that...