We’re finally at a point where someone who spends a reasonable amount of time at a server command line can actually get real work done, and I gotta say, it’s pretty cool. Just last night I was discussing an obscure Apache config issue with a friend at a bar, and rather than working from memory, I busted out the iPad and my Bluetooth keyboard, and 5 minutes later, the configuration issue was solved.

Having the freedom to go to the park to read for a bit but knowing I have the ability to handle an emergency should it come up is very freeing. Yes, I have become that douchebag at Starbucks – and you know what? I fucking love it.

Anyway. Point is, the iPad (or iPhone) can be used for more than just porn now (which is good, because the folks at Starbucks get surprisingly upset when you try adding your own “cream” to your latte), and I’ve spent some time and money to try out some of the most promising apps in the app store that allow you to do actual work, and edge the iPad closer to being a viable option for a netbook replacement.

I didn’t address any design/mockup/mindmapping apps in this list, but that may be a topic for another post sometime. This list isn’t meant to be all-inclusive, and doesn’t reflect the totality of what is available in the app store – it’s a short list of personal recommendations of products I actually use and like.

Disclosure: Some of the links below are hooked into the iTunes affiliate program so that I might get a penny or two if you decide to buy, however the recommendations are legit, and I wouldn’t recommend something unless I had used it. Click through on the affiliate links or don’t – but do leave me a comment if you’ve fallen in love with something I haven’t mentioned here.

Code Editors/FTP

There are quite a few nice code editors for iPad in the app store, but I won’t consider any that only offer FTP instead of SFTP and neither should you. I am just as likely to use vi in an SSH app on my iPad as I am to use a code editor, but for handling multiple open files at one time, sometimes an editor is kinda nice. Unfortunately, 90% of the code editors in the app store are complete and utterly shit-tastic garbage. Seriously. Even if you don’t pick one of my recommendations, make sure you read the comments on the code editor apps before you buy so you don’t get burned.

Textastic

I think Textastic might be my new favorite code editor for iPad. The interface is very clean, it supports FTP and SFTP, integrates with Dropbox and WebDav (if you’re into that sort of thing) and comes with syntax highlighting for around 80 different languages. It’s a little pricier than some of the other options, but I think it’s well worth the investment. I want to make sweet ASCII love to it all the time.
In iTunes: Buy Now ($9.99)
Developer: Alexander Blach

Gusto

Gusto is pretty sexy and has come pretty far in a short time. (When it first appeared in the app store, there was no SFTP support.) It supports projects, one-touch uploading, background processing so your state doesn’t get lost when you have to switch apps, pretty Coda-like site thumbnails, tabbed editing, and remove and local preview support. Three obvious features that are missing are syntax highlighting, line-wrapping and public-key authentication, but it’s a great start and a solid option for busting out quick changes on the road.
In iTunes: Buy Now ($6.99)
Developer: Horse and the Rook

An alternative to Gusto that’s an app to watch would be Markup for iPad, but I’ve heard such crap things (crashy, no SFTP) about it that I haven’t tried it. Sounds like it’s worth keeping an eye on, but not ready for prime time yet and not worth the $10 pricetag until it’s a bit more stable and can handle SFTP.

FTP on the Go (Pro)

Feature-packed FTPS app. Honestly, too many spiffy features to list – the best FTP app I’ve come across so far. Comes with a built in FTP Server and Web Server allow viewing and adding files to the iPhone or iPod touch. Browse files on your iPhone from your computer with a web browser. Madness. Madness, I say!
In iTunes: Buy Now ($9.99)
Developer: Headlight Software

MySQL

MySQL Database Client

Small, simple MySQL client for iPad and iPhone. Supports stored profiles and custom queries, but don’t go too nuts. It can handle basic queries, but more complicated stuff like JOINS will return unpredictable results. Still, it’s $0.99, and is worth at least that much, contrary to the cheesedick who “wants a refund” in the reviews. Seriously. It’s a buck. Get over it, kid.
In iTunes: Buy it Now ($0.99)
Developer: Kyle Hankinson

MySQL Editor Pro

A much more full-featured app with a price tag that reflects it, MySQL Editor Pro is the real deal. If the cost doesn’t scare you off, this is well worth the month for such a strong db admin app.
In iTunes: Buy it Now ($14.99)
Developer: Pasha Topchiyev

SSH/VNC

Prompt

That Apache configuration issue I was having? Solved in 5 minutes using Prompt. It’s made by the same folks that make the super-sexy Coda code editing app for Mac. The UI is pretty nice, and it supports special characters and keystrokes like CTRL which one ends up using frequently in a shell. Prompt supports DSA/RSA keys, automagically remembers your frequently used commands, runs in the background so screen-switching won’t disconnect you, and you can map commonly used keystrokes easily for speedy access. An added bonus – it’s a universal app, so you buy it once and it works on your iPhone and your iPad. (Given my horrible typing on the iPhone and the iPhones even more horrible auto-correction, I don’t know that I’d want to use it on my phone much, but it’s nice to know it’s an option.
In iTunes: Buy it Now ($4.99)
Developer: Panic, Inc.

iSSH

Less sexy than Prompt but still one helluvan app is iSSH. iSSH boasts a pretty impressive feature set, including a tunneled VNC client, tunneled X server, the fact that SSH, telnet and VNC all work via EDGE, WiFi and 3G, transparent keyboard, Bluetooth keyboard mapping, RSA and DSA key generation and exchange, tons of keyboard customizations and holy shit a lot more. It’s a solid client, and a universal app, so you can buy it once and use it on your iPhone, iPad, iPod touch, etc. Even works with older iPhones running iOS 3.0.
In iTunes: Buy it Now ($9.99)
Developer: Zingersoft

Network Tools & Miscellaneous Hackery

IT Tools

Puts a whole handful of diagnostics just a tap or two away, with DNS, Ping, Route, ARP, active sockets and Interface tools. 45 supported DNS record types, including A, AAAA, CNAME, LOC, MX, NS, SRV, TXT – and it come with a database of MAC addresses so you can look up manufacturers of devices on your network. All of these things can be done through SSH if you’ve already got a terminal running, but this app makes it so much easier.
In iTunes: Buy it Now ($4.99)
Developer: Kevin Koltzau

Server Admin Remote (Mac OSX Server)

Called a Swiss army-knife for the mobile Mac OS X admin, with Server Admin Remote IT administrators can monitor the alive status of Mac OS X Server services, start/stop services and observe the services’ logs (Mac OS X Snow Leopard, Mac OS X Leopard Server and Mac OS X Tiger Server). Works on EDGE, WiFi and 3G connections. No further installation on your Mac OS X Server needed, since Server Admin Remote uses the same interface as Mac OS X Server Admin.
In iTunes: Buy it Now ($11.99)
Developer: Harlekins

Rackspace Cloud

If you’ve got a Rackspace Cloud Servers account, this app is the shit. Reboot, rename, resize, and rebuild your Cloud Servers, spin up a new server or delete an existing one, change your root password, bootstrap Cloud Servers with Chef from your Chef server or the Opscode Platform, open and manage Cloud Files assets and control your CDN settings for Cloud Files containers, play Cloud Files audio and video over Airplay to your Apple TV (iOS 4.3 and up) – a ton more. It’s not a complete replacement for their control panel, but you can do a heck of a lot with it.
In iTunes: Download Now (FREE)
Developer: Rackspace

Vtrace

Simple visual traceroute (or TracerT, if you’re this kid) that uses your current location to take you down the bunny trail to whatever IP or hostname you’re looking up.
In iTunes: Download Now (FREE)
Developer: Vlad Alexa

iAccess for Nagios

Mobile Nagios client that gives you direct access to the /nagios dashboard. (Obviously, you need a Nagios server configured for this to work.)
In iTunes: Buy Now ($3.99)
Developer: ASION IT Services

Flame for Bonjour

Flame is a browser for Bonjour network services. It lists the services advertised on your wireless network and you can browse them by server or by service type. When selecting a service, its advertised details are displayed. If an application on your iPhone or iPod touch can handle any of the advertised services, a command to open it right away is provided.
In iTunes: Download Now (FREE)
Developer: Tom Insam

Ping A Majig

Handy app that lets you check the ping status of multiple hostnames at one time. It’s a bit handier as a monitoring tool than the other apps that include ping as an available tool, since the at-a-glance view lets you see if any of your hosts are in trouble on one screen.
In iTunes: Buy Now ($0.99)
Developer: Pingysoft

RBL Status

Simple but effective Real Time Blacklist looker-upper.
In iTunes: Buy Now ($1.99)
Developer: Pavel Ahafonau

iPortscan Pro

iPortScan PRO is a port scanner for your IPhone or IPodTouch. It does not feature any network discovery; however, this tool is useful for sysadmins checking what services are listening on a known system. This is very handy for the system admin who can use this tool to quickly portscan all of their systems to make sure nothing is open that shouldn’t be.
In iTunes: Buy Now ($1.99)
Developer: Whiteside Solutions LLC

Default Logins

This app contains a database of over 300 common and uncommon manufactures and the usernames and passwords they pre-configure their devices with (which there are 1,000 + in the database). Can come in handy for more nefarious reasons (if you’re that kinda person), but also super useful for fixing a relative’s biffed router when they ask you to come over and fix their internets.
In iTunes: Buy Now ($1.99)
Developer: anthony lamantia

So that’s my list – for now. Did I miss any that you love? Leave me a note in the comments.

PS – yes, that’s a photo of my actual license plate at the top of the post. And yes, that makes me more awesome than you.

Also check out:

Background

The MSWDS is one of only four significant annual events within the PHP community (others include tek, DPC and ZendCon), and this summit is a bit harder to get invited to. Unlike most other conferences, where all you need is the cash to pony up for a conference pass and a hotel room to crash in, invites are very limited and attendees are selected because they have had some interaction with the folks at Microsoft, and are believed to be leaders and influencers within the open source community. To be blunt, these summits cost Microsoft a lot of money, so they need to make sure they’re getting the best bang for their buck.

Keeping that in mind, it would be easy to assume that we were being brought out there so that Microsoft could pitch us on the latest and greatest Microsoft products, trying to get the movers and shakers of open source to drink the corporate kool-aid and switch to Microsoft products. While more acceptance of Microsoft products within the open source community is obviously a goal, they are making a concerted effort to learn from us – what we need, where they are falling short, and how we can move forward together.

Discussions and Format

The summit itself was a total of three days, with the last day being optional for those open source developers who were willing to sign an NDA to discuss some of Microsoft’s emerging technology. During the three days, different representatives from Microsoft’s product teams sat down with us and asked for our comments, thoughts and ideas about where they’re at, and where we think they should be going. We met with folks from the IIS Web Platform team, the SQL server team, as well as some representatives from Codeplex, Silverlight, Powershell, ASP.NET Ajax (which is not exclusive to ASP.NET, despite the name), and Bing maps.

We had a chance to air grievances, which was cathartic in some ways, but I think it was more important to us to be able to sit down with the actual teams who are working on this technology at Microsoft, and really get into the specific challenges we face. The approach was not generally pitchy, and with very few exceptions, a great deal of effort was put into making all of us from the open source community feel like respected authorities in our field whose opinions really matter.

Something they did this year which was apparently not done last year was to include representatives from well-known PHP-based projects who are not normally parts of the PHP community. I honestly hadn’t realized that the many of the folks over Joomla, WordPress and Drupal often don’t consider themselves as part of the greater PHP community, and getting a chance to discuss that with them brought up some interesting perspectives. I don’t think the guys representing these projects were there in an official capacity, but their point of view was one that had honestly not occurred to me before, so that was a really interesting and unexpected benefit. There was some debate on whether or not these types of projects should be a more involved part of the PHP community, with good points on both sides, but I think most walked away with some ideas on how to move forward in making those lines of communication more accessible and open.

My Perspective

Of course the ultimate question from Microsoft was “What would it take for you to switch to Microsoft products for your clients?” My smartass remark was, of course “A fucking miracle.” But everyone in the room knew I was joking. I hope. If we weren’t willing to work with Microsoft on improving their products to work with open source better, we wouldn’t have been there.

As often as Microsoft has been an easy target in the past, and as much bad blood as there may have been in the past, there are people at Microsoft that care about working with the open source community, and who are making progress to get there. It is our job as technology professionals to fairly evaluate technology and make recommendations based on what makes the most sense technologically and financially. It is NOT our job to make religious decisions based on zealotry.

That means that if and when Microsoft can meet my needs and/or the needs of my clients, it can and should be part of that evaluation or I’m not doing my job. Does that mean I’m ready to switch back? No. Not yet, anyway. But I believe they are listening, and I saw some things during this summit that make me far more likely to start including some parts of Microsoft’s products into the technology I suggest as being potentially viable for client projects, which is a far cry closer than I was last week. Specifically, some of the stuff I learned about Silverlight, Bing’s geolocation products and Windows Azure (Microsoft’s cloud hosting platform) was pretty impressive. As I get to play with these products a little more, I’ll be blogging about them with my fair evaluation of pros and cons, so stay tuned.

I’m also excited to see where the Microsoft Web Platform Installer product heads. Right now, the WebPI product is a very easy to use, slick solution for the less techy individual who wants to, for example, deploy a WordPress blog in 5 minutes or less and may not have the savvy to do the install themselves – basically a MS version of Cpanel/Fantastico, which we have had available to us as web administrators for over a decade. That product is less interesting to me right now, but some of the directions they could go in for more advanced users like us hold real potential. We had some suggestions that were well-received, and if they are actually implemented in the way I envision them, it could honestly turn the table and make some of the Microsoft web server products something that I could consider recommending, or even using myself. (I should also mention that Cpanel is the most horrific, insecure, hack-prone web control panel I’ve ever used, and I am NOT endorsing it as a solution.)

The reality is that competition inspires innovation, and Microsoft getting better means progress for everyone. I saw a post on Twitter that basically implied that open source representatives attending this conference were traitors or sellouts. I don’t see it that way at all. We have amazing open source products like Firefox because the open source community worked together to create a better product, and Microsoft responded by making vast improvements to Internet Explorer, building in more security and standards compliance. When we work together to innovate, everybody wins.

Another transition I’ve been seeing in Microsoft which was really made more obvious by this summit is that there is a less omnipresent feeling of “all or nothing” within many Microsoft departments. As open source advocates, we enjoy having choices. Previously with Microsoft, you’d get the most benefit from their products by committing to an entirely Microsoft development process (“drinking all of the kool-aid, since the best stuff is the sugary goop at the bottom”), with benefits sharply falling off if you opted to pick and choose. This philosophy has always been distinctly in opposition with the open source philosophy, and I believe was likely the cause for some of the distrust coming from the open source community. Seeing this transition into a paradigm of being able to cherry-pick what we like for some things and sticking with open source solutions we like better for others is a step in the right direction, in my opinion.

An additional unexpected benefit to sitting down with all these MS product people was that I got a chance to better understand some of the legal/licensing challenges Microsoft faces. I’m not making excuses for them, but I hadn’t considered some of the obstacles in the way of people at MS who care about working with us. Microsoft is a big target with deep pockets, and they have to cover their own asses. I was quicker to dismiss some of the corporate decisions as being “evil” prior to sitting down with some of them and understanding why they do what they do. Don’t get me wrong – some of their decisions (*cough*sudo*cough*) still don’t make sense to me and I believe they are wrong, but I think I have a better understanding of where they sit than I did before

Wrapping Up

Overall, I would consider this summit a great success, and I hope I get to participate again in the future. There are several people who really deserve a shout-out for all of the hard work that went into this and are directly responsible for it’s success. From the PHP community, Cal Evans was a co-host and an absolute rock star, always quick to make sure things ran smoothly and kick-start conversations and redirect us back when we went off on tangents. From Microsoft, Karri Dunn, Tonya Young, Josh Holmes, Peter Laudati, Lauren Cooney, and others were amazing. I may be forgetting a few – I am still a little wiped from the week and the traveling.

Was this an instant fix? Certainly not. Do we all have a lot more work to do before we’re “there”? Absolutely. But as Cal Evans put it on his own blog roundup, “The more people I get to know at Microsoft, the less I’m able to despise the company.” They took the time to find out what we think, even when it may not have been what they wanted to hear. Time will tell whether or not they actually act on it.

Other PHP Representatives Blog Post Roundups

I’ll be updating this list as more people finish their blog post roundups, so you can get take their on the summit. Many of them are far smarter than I am, so it’s worth reading what they have to say.

• Cal Evans (@CalEvans)
• Chris Cornutt (@enygma)
• Maarten Balliauw (@maartenballiauw)
• Rafael Dohms (@rdohms)
• Keith Casey (@CaseySoftware)
• Romain Bourdon (in French) (@le_vrai_roms)
• Marco Tabini (@mtabini)
• Sam Moffatt (@Pasamio)
• Helgi Þormar Þorbjörnsson (@h)

MS Representative Blogs

If you’d like to see more about the fabulous people at MS who are working hard to move the company forward in a way that works with open source, check out their blogs. I’m proud to call these guys friends, and as long as we continue to have people like this working for Microsoft, I think the lines of communication and cooperation between both sides of the aisle will keep moving forward.

• Dave Bost (@DaveBost) – Developer Evangelist
• Josh Holmes (@JoshHolmes) – UX Architect Evangelist
• Tobin Titus (@tobint) – MSDN Site Manager
• Peter Laudati (@jrzyshr) – Developer Evangelist
• Mark Brown (@MarkJBrown) – Product Manager for Microsoft Web Platform
• William Coleman (@will_coleman) – Developer Evangelist
• Lauren Cooney (@lcooney) – GPM for Web Platforms at Microsoft
• Jas Sandhu (@jassand) – Interop Strategy Evangelist
• Ruslan Yakushev (@ruslany) – Program Manager on IIS team in charge of FastCGI and PHP support
• Scott Hanselman (@shanselman) – Principal Program Manager Lead

Funnily, as I’m writing this, Futurama: Into the Wild Green Yonder has been on television, and the scene that just played is the one where Calculon says “I’d like to thank the academy, my agent, and most of all my operating system – Windows 7, for everything it –” at which point his OS locks up. Windows 7 is actually a great product, and I run it on my Mac using Bootcamp and VM Fusion, but I thought the timing was amusing.

Kool-aid photo taken in Belize by me – just thought it was funny. Lead post image by Eli White.

Also check out:

Subversion creates a bunch of hidden files and folders inside of every directory in your Working Copy that’s checked into SVN. If your Working Copy is on your local computer, you’ll see these as .svn directories if your system preferences allow you to see hidden files and folders. You don’t need to worry about these, and you never really need to look at them, but as a result, it is not recommended that your Working Copy be on your live production box. If you decide not to listen to me, you should at least consider adding a few lines to your .htaccess to deny prying eyes access to your .svn directories.

Working Copy & Repository on your local machine

Create a Working Copy on your local machine, create the Repository on your local machine, and FTP/SFTP the files once they’re committed. This will really only work if you are the only developer, or your local machine is the only machine changing files. (Okay, that’s not entirely true – you can set up the Repository on a network drive and have multiple developers Check Out their own Working Copies to their own computers, but I have run into issues running Subversion over a network.)

If you’re running OSX on your local machine, see this tutorial on WikiHow on how to set up Subversion locally. If you’re running Windows, there is an exceptional walkthrough on The WebSqueeze written by Christopher Hyne that is a must-read. Setting up Subversion on Mac or PC is really quite painless, and this is a perfectly valid way to go.

Pros: Easy, local setup. No external services to set up, your workflow doesn’t change that much, and you don’t need an internet connection to commit things to your repository.

Cons: Possible challenges with running Subversion over a network, you still have to upload your files manually, and you don’t get the benefit of a slick gui with which to view revision history, as you would using Beanstalk or Unfuddle for hosted solutions, or by setting up Trac (a gui reporting interface/wiki/ticket system) on your server. You can set up Trac on your local machine with Python and some extra work, but that’s always been too much trouble for me. Just MHO though – it works great for some people.

Working Copy on your local machine, remote Repository

For my personal stuff, since I use Mosso as my web host, which does not currently provide Subversion (though I hear they’re working on it), I use a third party Subversion host, like Beanstalk or Unfuddle. I create my Repository on Beanstalk or Unfuddle, Import whatever files I may have created for the project already, Check Out a Working Copy to my laptop, and I’m all set. I work on files on my laptop, test them on my local installation of Apache/PHP using XAMPP, Commit them to Beanstalk or Unfuddle (depending which service I’m using to host my Repository), and upload them via FTP/SFTP to the development server area.

The result is a little extra work – I have to deal with SVN on top of FTP/SFTP, but the security of knowing my ass is covered and I have a detailed revision history makes it entirely worth it to me.

One benefit to using a third-party SVN host, such as Beanstalk or Unfuddle (besides the fact that it gives you SVN capabilities if your server doesn’t support it) is that you get a shmancy web interface with which to view your revisions. Below is a screenshot of the Unfuddle web interface. You can see that it gives me a nice, organized view of all of the history, including revisions messages, numbers and person committing. (This is a project for JPM Morgan that I worked on with my friend Vidyut Luther.)

In getting Vidyut’s blog url, I see his latest blog posts states that he can, in fact, use Subversion on Mosso, by using Expandrive. I haven’t tried it yet, and he has, so if you’re interested in exploring that as an option, check out his blog post about it. I may give it a shot at some point, but I do like the simplicity of working with a third-party SVN host. I don’t have to administer anything, just create the Repository and go.

Remote Branches on dev, remote Repository, rsync to live

One way of handling your development environment is to set up a Repository on the remote server, Check Out a Working Copy on a dev area on the remote server, such as dev.example.com, and set up individual Branches for each developer. An SVN Branch is simply a copy of the filesystem separate from the main version, called the Trunk. Using Branches allows you to save your half-broken work frequently without interfering with others, yet you can still selectively share information with your collaborators.

In this scenario, no one ever touches the live files directly, ever. Ever ever. Everything gets pushed to dev for review, and once it’s approved, you run the rsync script and rsync is the only utility that ever moves stuff over to live.  It may sound confusing, but check out this simple blog post that explains how it can be done.

This can be daunting to set up, since there will be directories on live you may want to skip during the rsync (uploaded images, cache files, etc), but once it’s set up, it generally works like a charm, and it insures that no one ever screws up the live server, since no one is ever touching the live server.

Creating Branches is incredibly easy – check out the section on Branches in the free Version Control with Subversion book. They cover this concept and process very well. To give you an idea of what Branches look like, we’ll use their example structure, with a project named calc. This is what the original directory structure looks like:

You can see in this example, the ‘branches’ directory is currently empty, since we haven’t created any branches yet. The three file icons represent the total files in the project, which live in the Trunk.

Once we create a Branch, named ‘my calc branch’ (since it’s a Branch within the ‘calc’ project), you can see that a new directory is shown in the ‘branches’ directory, and a copy of the files from the Trunk live there:

Parts of each Branch can be selectively merged into the Trunk, so each developer can work on their tasks within their own Branch, break (and hopefully fix) as many things as they need to, Committing their work as they go, without affecting the main Trunk until the revisions are merged.

If you didn’t want to use Branches, you could add a staging server in this process. At a place I used to work, we had a development version that was actively worked on, a staging server, and a live server. We would Commit changes to the development server, rsync over to the staging server, and the staging server would automagically rsync every 10 minutes to live. Nothing was synced to staging unless we had kicked the tires and done QA on it on the development box, and if we couldn’t wait the 10 minutes, we could do a manual push to live by running the rsync script directly from the command line.

These are just a few of the possibilities. How are you using Subversion? Leave your configuration in the comments! And keep your eyes out for the next section that deals with the basic Subversion commands you’ll need to get started and optional gui applications you can use to make your Subversion experience easier.

Also check out:

Since this is a question that comes up a lot, I’ve scoured the web for a really basic subversion primer, and although many come close (including the outstanding free online book, Version Control with Subversion), some people may feel overwhelmed at such an immense amount of information. Truly, I believe the only flaw in the Version Control with Subversion book is that they don’t include any real life set-up examples when discussing the Subversion Architecture.

There are many different ways you can set up your development environment, so I can understand why they would be hesitant to give specific examples, however some people learn best by seeing examples that relate to their lives, so that’s what I’m going to attempt to do.

The example set-up we’ll be looking at is one in which the developer is currently working either locally or remotely, and using FTP/SFTP to move files to the remote development environment webserver. For example, you either:

• Have a copy of Apache running on your desktop/laptop computer. You store and work on the files on your desktop/laptop, test them locally, and then upload them to your server  - or you -
• Make changes to the files locally on your desktop/laptop and test directly in the development space of the webserver (no local webserver) - or you -
• Use a text editor that allows you to pull files from FTP directly. You open them from FTP, make your changes, and save to FTP (no local copy stored at all)

I should mention that the third option here is absolutely the least recommended, especially since you’re not using version control. If your FTP process konks out mid-way through and the file isn’t saved completely, closing your application could mean losing that file forever. (Yeah, I’m looking at you, UltraEdit.) Regardless, if you’re using any of the above scenarios, or something fairly similar, this is aimed at you.

Because the Version Control with Subversion book is so good at explaining so much of this, I’ll be linking to it liberally, and just filing in the blanks where some people may get a little lost.

Disclaimer: This article is a very basic overview of subversion. There are many ways you might have your system and repository set up, and I won’t try to list them all, or even most. I’m going to use what many web developers would consider a “real world” example. You will not walk away from this a subversion guru, or able to deftly administer a subversion server. You will hopefully walk away knowing how to set up a simple repository, connect to it, and begin using svn in your day to day development. If you fuck something up, it is not my fault. Start on a test environment until you get your svn legs and feel pretty comfortable with basic commands.

This article is not a replacement for Version Control with Subversion – more of a supplement for newbies. As you begin to delve further into using subversion, that book will become your bible.

What is Subversion?

According to the Version Control with Subversion book:

Subversion (SVN) is a free/open-source version control system. That is, Subversion manages files and directories, and the changes made to them, over time. This allows you to recover older versions of your data, or examine the history of how your data changed.

Subversion can operate across networks, which allows it to be used by people on different computers. At some level, the ability for various people to modify and manage the same set of data from their respective locations fosters collaboration. Progress can occur more quickly without a single conduit through which all modifications must occur. And because the work is versioned, you need not fear that quality is the trade-off for losing that conduit—if some incorrect change is made to the data, just undo that change. (more)

This is a pretty good explanation of what it is, but it can be hard to imagine what that actually looks like or how you’d use it in practical terms.

So what can Subversion actually do for you?

It means you can have multiple developers working on a project without fear of one overwriting the other one’s work. Have you ever worked on a project with multiple developers, where everyone is working on the same codebase?

“Hey Bob – I’m working on index.php – don’t touch it for now. I’ll let you know when I’m done, okay?”

or how about

“CRAP! Did you make changes to view.php? You blew out my work! I spent half a day working on that frakking function!”

All of that goes away. Each developer can have their own version of the application or website (called branches), where the changes they make will not impact the other developers, and the changes can then be selectively merged into the main development area, or trunk.

It also means that you have a method by which to track all of the changes you’ve made to any given file. Each time you commit changes to the file, subversion assigns that change a unique revision number. This means that if you ever need to roll a file back to a specific revision, you can easily do so with a simple command.

A good example of where you might need to do this is where you’ve “fixed” a bug that broke something else on the site (and really, who hasn’t done that?). If you know the now-broken functionality was working before, you can roll back to a previous version of the changed file (or any version before that, if needed) to the point where the functionality wasn’t broken.

You can also compare revisions side by side, so if you don’t want to do a full rollback, you can see what changed from one revision to the next to figure out what you (or they) did to break it so you can undo just that part or find a new solution. Plus, since each developer has their own subversion username, you can see who did what (so you know who to fire/berate/pummel/defenestrate).

In short – whether you’re flying solo or working with a team, Subversion can help keep you sane, and bail your ass out when you need it most.

Sounds awesome, right? Let’s get started!

Before we go any further, there are some terms that you’ll need to understand in order to grok Subversion. Don’t worry too much if they are hard to remember – we’ll be using them a lot, and they’ll become second nature to you.

Repository - this is the central location where Subversion keeps all of the information about your files, folders, and revisions. The repository can live on the same server as your development environment if your hosting setup has Subversion installed, or it can live somewhere completely separate.

If your hosting company does not have Subversion installed, you can even use a third-party Subversion host like Beanstalk or Unfuddle, whose sole purpose is to store your repository, and only your repository. Your local files are still on your hard drive, and your development files are still on your webserver.

Working Copy - this is the set of files you work with to make changes to your code, build your application, etc. The working copy usually lives on your hard drive, and would replace the directory you upload from and download to if you were using FTP. Simply put, this is the copy of the files you are meant to be working with. You can have multiple working copies of a site for any single repository, and in fact, you would have multiple versions if you have multiple developers. Each developer would have their own working copy.

Import – Importing in Subversion allows you to pull your existing files into a new repository. So, for example, if you have a site that’s already halfway built before you have the epiphany to use version control, you’d import your existing stuff into the repository.

Check Out – If you already have files in a repository and need to create a new Working Copy, you’d use Check Out.

Update - Using the Update command pulls the most recent version from the repository. If you had multiple developers working on the same files, or if you’re working from multiple computers, you’d want to make sure you update before you start making any edits to the files. If you fail to update before you make changes, you risk running into a Conflict when you Commit. (see below)

Commit – When you’ve made changes to a file or files in your Working Copy and you’re ready to push those changes into the repository, you Commit them.

Conflict – A Conflict occurs when there are changes on both your local version (Working Copy) of a file and the Repository of that file. For example, Sally makes changes to the file sandwich.txt in the repository. Harry has just changed the file in his working copy and commits it. Sally doesn’t update her working copy before committing it and she gets a conflict (since Harry committed his version after Sally’s last Update, and Subversion therefore sees Sally’s version as out of date). More on predicting and resolving conflicts later.

Trunk - the location within your Working Copy where your main project files live.

Branches – individual copies of the project that allow each developer to have their own working version. Changes can be Committed without breaking the Trunk, and are then merged into the Trunk when they are deemed worthy. (More on this in the bonus section of part two.)

The image below gives a very high level view of how Subversion works.

The Repository is your gatekeeper. It is the part of Subversion that makes sure files don’t get over-written with outdated versions, and allows you to work in tandem with other developers without risking stepping on each others toes. Other than through the command line (to execute import, update, commit, and status commands) or through whatever gui application you prefer using, you never access the Repository directly.

How often to Commit?

This is largely a matter of preference. Some developers wait until entire classes or functions are added/edited/etc before committing a file. I personally tend to commit more often, since it gives me a more granular way of seeing what broke something else and rolling it back, etc – but the only wrong way to Commit is to not Commit at all. You do not need to commit every time you change every tiny thing, though. For example, if I had to do some copy changes, I wouldn’t Commit after changing each word. The revisions would be so broken down, it would be hard to find important changes in the sea of revisions.

You (and your team, if you’re working in a group) will find a Commit frequency that works for you, as you become more comfortable using Subversion, so don’t sweat it too much in the beginning.

Now that you know the lingo and what it does, you need to decide how you’re going to set up your subversion workflow. Check out the second part in this series!

Also check out:

In the hopes that this saves you some of the headaches I’ve had to suffer through, I’m making my list available to you. Some of these questions don’t have yes-or-no answers, and require additional discussion – but that’s okay. The point is that they were asked in the first place, and you’re including them in your plan.

Every time I start a new project, I run through this list of questions with the project managers or client, one by one, until I have all the answers I need – and after every project, I revisit this list of questions to see if there is anything I should have asked up-front that would have made the project go more smoothly.

Yes, this is long. But better to spend the time asking questions now and save yourself days or weeks worth of work later. Also, there may be whole sections you can skip entirely if they don’t pertain to your project – just be sure they really and truly don’t before you give them the brush-off.

I will be out of the country for a week in Belize starting oh-my-god-o’clock tomorrow morning, so this post is my gift to all of you who are stuck working on projects while I’m on a beach in Belize sipping Mai Tais. Enjoy!

Basics:

What is the main purpose of this website? General overview, specifically addressing goals the website or application hopes to accomplish. Online sales, brand reinforcement, etc. This answer doesn’t have to be long, but I find it helpful to include them here so that the tech process always compliments and enhances achieving those goals.

How “big” is this project? This is a generalized question and doesn’t require specifics most of the time – is it a 4 page site, a 40 page site, or a 400 page site. This helps to understand the basic scope. The answers to this question should include:

1. Approximate number of screens/pages in the website/application
2. What are the basic functions of the site? i.e.Registration, Game, Ecommerce, Search, catalog, photo upload, discussion boards, voting, etc

What data will be stored/accessed locally? Will the website/application require a database? You don’t need to make a concrete decision about what kind of database at this stage – just knowing there needs to be one is enough.

How will the site be updated?

1. Will it require a CMS?
2. What content is dynamic vs static?
3. How frequently is the data updated (hourly, daily, weekly, monthly, periodically)?
4. Where will the data come from? (client or administration back-end of any kind)
5. Who is responsible for updating the data?

What are the reporting needs?

1. Basic site traffic?
2. Will it require content‐specific reporting? (i.e. how many new users signed up between specific dates, etc.)

Will there be a mobile/WAP component? The answer to this question is usually “no” for the projects I work on, but it’s important to know in advance. Phone browsers these days are so similar to desktop web browsers that creating a WAP version of a site is far less trouble than it used to be, when you had to take black-and-white screens into consideration, etc.

What operating systems, browsers and browser versions are we targeting? Do we care about IE6 or other older browsers (pleasesaynopleasesaynopleasesayno)?

Are there additional considerations such as printer‐friendly versions, BOBBY/ADA‐compliance, age verification or other that we need to be aware of? Make sure you’re familiar with the COPPA-compliance guidelines before you create a site or application that asks for user information and doesn’t screen for the user’s age.

Domain Name & Hosting:

Where will this application/website be developed? If it will be developed on a different machine than its final hosting situation, we must provide time for file transfers, additional QA once the site is moved, etc.

Where will this application/website be hosted? If we are hosting, will it require a dedicated host, or virtual hosting environment?

If the client is hosting, have we confirmed that their server supports the technology we intend to use? This is a key point, but one that is often overlooked. If you’re expecting PHP on a Linux server and the client has an IIS box without PHP or MySQL, you need to know that up-front, so you can decide how to handle it. Maybe the client installs PHP and MySQL, or maybe you do the application in ASP – either way, the answer here has a huge impact on how you proceed, before even a line of code has been created, so you need to know that right away.

Also, if possible, try to obtain access to the hosting server as soon as possible, so you can run some basic configuration tests. One follower on Twitter remarked that the client confirmed that they had PHP and MySQL (on IIS), but the environment was not set up to allow PHP to talk to MySQL. Not something that’s impossible to fix, but if you’re pushing it close on your deadline, you want to know that right away so you can have the hosting company make whatever changes you need with plenty of time to spare. A simple phpinfo() would have spit out everything you needed to know (not to rub anything in, @derekobrien!). To be fair, I might have just assumed the same way he did, so it’s important to remind ourselves to check everything up front.

I would imagine that hosting blunder (and that was a blunder – who sets up PHP and MySQL but forgets to allow them to talk to each other?) is hopefully uncommon, but if you’re using libraries or modules like PEAR, cURL, imagemagik, the GD library, and so on, you’d best confirm that everything is all set up before you move forward.

In @derekobrien’s case, they weren’t granted server access until last minute, so he was caught off-guard by this. One way to protect yourself against issues like this is to set clear milestones in the tech brief or feature scope that you create based on these questions (which the client signs), with the caveat that if the client doesn’t provide server access by xyz date, the launch date can and will be pushed back, and additional budget may be required to get things up to snuff.

@derekobrien also included a clause in the fine print that says “If your server admin is a fool we reserve right to point and laugh at you.” I generally consider that implied, but it’s just as good in writing

Will it require its own domain name?

1. If yes, will the client provide, or will we?
2. If yes, we must allow time for propagation
3. If no, at what url will this live? (sub‐domain of noise, sub‐domain of primary client domain, etc)
4. If sub‐domain of client, we must provide time to work with the client’s IT dept to configure DNS settings

Will we need to manage any email accounts for the project? If so, what are the account names, and are they pop3, IMAP or forwarding accounts? Who will be checking them?

Will this project be migrated to the final hosting location from an existing location? If so, a migration plan (including e‐mail address, DNS, content, database, etc migration should be created.)

Is this project ongoing, or for a limited time? If for a limited time:

1. What ongoing service or support will be required? Webmaster support email? Unsubscribe administration?
2. What is the plan for the domain name and/or website once the project is over? Do we keep hosting the project? Do we let the domain name expire, or simply redirect it to the main client site?

If the project is hosted on a server we are managing, server setup should include:

1. Realistic bandwidth estimate. We should confirm with the hosting company we select that if we surpass anticipated bandwidth, there will be no interruption in services. (Extra charges will apply, but service should NOT be suspended for bandwidth issues.)
2. Maximum throughput configuration available. There is generally an insignificant additional monthly charge for this.
3. If the project relies heavily on mySQL, we should consider making adjustments to the my.conf file to tune the database configuration for optimal settings. (I.e. if the database queries are largely SELECTs, optimize for SELECTS, etc.)
4. Fine‐tune the apache httpd.conf file to allow for maximum concurrent connections

Backup and Restore – if we are hosting the project on a server we manage, dedicated or virtual, some level of backup must be implemented for every account.Backup should include files, user data, and applicable databases – and a restoration plan should be created, should catastrophic system failure occur. The backup process MUST be tested during non-emergency circumstances. An untested backup is no better than no backup at all.

Flash/Video:

Will there be substantial flash or video aspects to this project? If yes, we should consider hosting arrangements and ensure we have the bandwidth available and ensure the throughput is set up to handle maximum traffic.

If the primary site or application content is Flash‐based, we must budget time to create an SEO‐optimized version of the page in plain text. Copy will need to be obtained from the client, and MUST accurately reflect the true content of the Flash. Attempts to keyword bomb or deceive search engines will result in being blocked from the search engine altogether.

Email:

Will the application or website be communicating with users via email or text messages? If so, we may wish to consider outsourcing email/text messaging for ease of management and to circumvent potential SPAM blacklisting issues. This decision needs to be made on a case‐by‐case basis.

Integration:

Will the application or website be required to interface (push or pull) with a third party system or service? If yes, documentation on the integration should be obtained at the beginning of the development planning so any potential obstacles can be addressed.

User and Content Filtering:

In projects where users will be contributing or creating content using the tools we develop, is there a need to build in support for:

1. Banning or blocking specific users by username or IP address?
2. Moderating content before it is displayed publicly or at least the ability for and administrator to delete to unapproved content to remove it.

If yes, by what method will offensive material be flagged as offensive? Does the application require a “report this content” function, and if so, what needs to happen when content is flagged? Who is notified, and what actions do they take? Should the system automatically unapprove content that is flagged many times?

Scrubbing or filtering bad words. If yes, what action should be taken if an offensive word is found? (i.e. is the offensive word replaced with ****, is it rejected by the system, etc.)

Pre‐Existing & New Content:

Does this project require us to import existing client data or content into the application? If so, in what format will the client be providing this data? (Excel, csv, Word, etc.) Adequate time must be allowed to scrub the data and create custom import scripts, and then to QA the imported data.

If content being provided runs on a timeline (such a daily quiz, where a new question is presented every day), do we need to set up an alert system that notifies an administrator if content is about to or has already run out?

Will there be ongoing new content that would be appropriate for use in an RSS feed? (Events, articles, blog content, etc.)

User Support:

What options does the user have if they have experiencing a problem with the application or website? Is there a Help/Tech FAQ page, a contact page, or some other way that they can reach out to us for assistance? What email address should be used for replies?

If we are going to be sending the user emails or text messages, we should outline the nature and frequency of this communication very clearly on a page available to the user.

A privacy policy should be built into the site or application whenever we are collecting any information from the user.

How does the user opt out? We should make it very easy for a user to globally opt out of our communications. Where applicable, on the opt‐out form, we should ask the user why they chose to opt out so we can better tailor future communications. Any email or text communication should require a double‐opt‐in confirmation.

Statistics & Reporting:

What aggregate statistical reporting needs to be done on this website/application? (Omniture, Google Analytics, Refresh Analytics for FB apps)

Do we plan on implementing click heatmap technology and/or multiple page versions for conversion rate evaluations?

Marketing and Promotion:

1. What media is driving traffic to this site? Online/Offline?
2. What is the schedule for that media?
3. What is the total exposure?
4. Are there any specific activities that would drive spikes in traffic? Superbowl Ad, etc.

PDF

So, that’s my list. After all of the questions are answered (which usually doesn’t happen all at once – it very often requires a bit of back and forth with the client and project managers), I create a tech brief or feature scope document based on the answers I from this questionnaire.

If the client doesn’t know the answers to some of the questions (“are you running linux or Microsoft?”), I do my best to find out who does. If the client can’t decide on something “do you need a content management system?”), I advise my project managers that we have to be given authority to make that decision for them. NO development will be done until the tech brief is done. Period.

I then get the client to SIGN the scope/brief so that they agree, in writing, to everything we’ve decided – and more importantly, what I expect from them, when I expect it, and what they can expect if I don’t get it on time – that way when they don’t deliver server access or copy/creative assets on time, there are no surprises when I tell them that we are not going to make their deadline.

The tech brief is written to protect both parties, the client and the service provider. When done properly, it helps manage client expectations, and protects you from being expected to deliver something that was not in the scope. Everyone knows what happens next, how much it will cost, and what the final product is supposed to do.

I’ll be updating this as new screw-ups happen and we (hopefully) continue to learn from them.

Leave your own in the comments, and feel free to download the (slightly longer) PDF version if it will help you do battle with middle management that doesn’t understand why you have to take so much time to plan your projects.

Also check out:

What is Mod_Rewrite?

Simply put, mod_rewrite is an Apache module that let’s you rewrite urls based on rules you define. That’s it. Seriously.

Regardless of how confusing some of the rules you may have come across appear to be, all they are doing is taking one url and rewriting as a different url. This rewriting happens at the server level, before the user’s browser sees anything, so the end result is seamless to them.

When you hear about “search engine friendly” urls, you’re most often seeing mod_rewrite in action. Mod_rewrite is the Apache module that let’s you turn a url like:

http://www.example.com/shop.php?category=Books&Title=Foo

into:

http://www.example.com/shop/books/foo.html

Some other common uses for mod_rewrite:

• Directing all traffic from multiple domain names to one domain
• Directing all traffic from www and non-www to one location
• Blocking traffic from specific websites
• Blocking spammy searchbots and offline browsers from spidering your site and eating your bandwidth
• Mask file extensions
• Preventing image hotlinking (other web pages linking to images on your server)

Apache’s mod_rewrite can be intimidating if you start where you’re supposed to start – the Apache documentation, however there are some very useful, common – and simple rewrite rules that you may wish to consider implementing into your site development plan, if you’re not doing so already.

Note: If you’re using Microsoft IIS, you have a few options, but I don’t use IIS, so I’m afraid I won’t be of much help to you beyond telling you where to look. ISAPI ReWrite seems to be very popular, and there is a free “lite” version available.

Getting Started

Your mod_rewrite rules typically live in an .htaccess file in your web root. You can only have one .htaccess per directory, but you can have individual .htaccess files in sub-directories under the web root. I generally do not recommend doing this. If mod_rewrite rules from one .htaccess conflict with the rules from the .htaccess in a sub-directory, it can be a real pain in the ass to troubleshoot. Try to avoid it.

When you’re adding mod_rewrite rules to your .htaccess file, you’ll want to start by using a conditional that checks to see if mod_rewrite is installed on your server. This can prevent getting a 500 Internal Server Error if you don’t.

[source='c#']
# Start your (rewrite) engines…
RewriteEngine On

# rules and conditions go here…
[/source]

Directing Multiple Domain Names to a Single Domain Url

If you have multiple domain names pointing to the same site, mod_rewrite can also help you direct all traffic to a single domain url, to improve your search engine rankings. Search engines don’t take too kindly to the same content living at multiple urls – they usually think its an attempt to spam the search engine – and you can actually be penalized for it. To redirect all traffic to one specific domain name,

[source='css']RewriteCond %{HTTP_HOST} !^www\.snipe\.net$ [NC]
RewriteRule ^(.*)$ http://www.snipe.net/$1 [R=301][/source]

This basically says “if the domain requested (the HTTP_HOST) does not match www.snipe.net then rewrite the url as www.snipe.net“. (Note the escaping slashes after the www and before the .net in the condition.)  The R=301 specifies that the redirect should be a 301 redirect, meaning that the address has moved permanently and search engines should use the new url instead of the old one.

To www or not to www

Even if you have only one domain name, if you’re not redirecting traffic from the “www” version to the “non-www” version (or vice versa), you may encounter this problem. Whether or not you choose to use the www in your url is largely a branding decision more than anything else (i.e. it doesn’t really matter in most cases) – but you should pick one and stick with it.

Require (force) the www

[source='css']RewriteCond %{HTTP_HOST} !^www\.snipe\.net$ [NC]
RewriteRule ^(.*)$ http://www.snipe.net/$1 [R=301,L][/source]

Remove the www

[source='css']RewriteCond %{HTTP_HOST} !^snipe\.net$ [NC]
RewriteRule ^(.*)$ http://snipe.net/$1 [R=301,L][/source]

Deny traffic by referrer

There may be a few reasons why you want to block traffic by referrer. Maybe you’re getting a lot of bandwidth-sucking hits from a spammy website – or maybe someone is linking to you that you feel does not represent you very well, and you want to pull the plug on traffic from coming their site.

[source='css']RewriteCond %{HTTP_REFERER} onebadsite\.com [NC,OR]
RewriteCond %{HTTP_REFERER} anotherbadsite\.com [NC]
RewriteRule .* – [F,L][/source]

In this snippet, the rule is saying “If the referring url is onebadsite.com OR anotherbadsite.com, redirect the user to an HTTP Forbidden error.” The NC specifies that the condition is not case-sensitive, and the OR flag is… well… an “or”. OR is used with multiple RewriteCond directives to combine them with OR instead of the implicit AND.

Keep in mind – this method of blocking traffic is hardly foolproof, at least in the latter of the two scenarios above. If the webmaster of onebadsite.com is linking to you in a way or context you do not want (and you’ve asked them to remove the link), the above method will cause a user on onebadsite.com’s website who has clicked on the link to you from onebadsite.com to hit a Forbidden error. If that user has half a brain, they may very well just google your site name or try to access it later from a bookmark – but it’s a simple measure you can take to keep the idjits out.

Blocking Bad Bots and Spiders

While there is some potential debate as to what is a “bad” bot or spider, the consensus seems to that a bot is bad if they do more harm than good, such as e-mail harvesters, site rippers that download entire websites for offline browsing, etc. Even if bandwidth isn’t so much an issue, I like to block these just on principle.

Please note – this list is not mine – it was directly nicked from a list on JavascriptKit.

[source='css']RewriteCond %{HTTP_USER_AGENT} ^BlackWidow [OR]
RewriteCond %{HTTP_USER_AGENT} ^Bot\ mailto:craftbot@yahoo.com [OR]
RewriteCond %{HTTP_USER_AGENT} ^ChinaClaw [OR]
RewriteCond %{HTTP_USER_AGENT} ^Custo [OR]
RewriteCond %{HTTP_USER_AGENT} ^DISCo [OR]
RewriteCond %{HTTP_USER_AGENT} ^Download\ Demon [OR]
RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR]
RewriteCond %{HTTP_USER_AGENT} ^EirGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR]
RewriteCond %{HTTP_USER_AGENT} ^Express\ WebPictures [OR]
RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR]
RewriteCond %{HTTP_USER_AGENT} ^EyeNetIE [OR]
RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetRight [OR]
RewriteCond %{HTTP_USER_AGENT} ^GetWeb! [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla [OR]
RewriteCond %{HTTP_USER_AGENT} ^Go-Ahead-Got-It [OR]
RewriteCond %{HTTP_USER_AGENT} ^GrabNet [OR]
RewriteCond %{HTTP_USER_AGENT} ^Grafula [OR]
RewriteCond %{HTTP_USER_AGENT} ^HMView [OR]
RewriteCond %{HTTP_USER_AGENT} HTTrack [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Stripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^Image\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} Indy\ Library [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^InterGET [OR]
RewriteCond %{HTTP_USER_AGENT} ^Internet\ Ninja [OR]
RewriteCond %{HTTP_USER_AGENT} ^JetCar [OR]
RewriteCond %{HTTP_USER_AGENT} ^JOC\ Web\ Spider [OR]
RewriteCond %{HTTP_USER_AGENT} ^larbin [OR]
RewriteCond %{HTTP_USER_AGENT} ^LeechFTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader [OR]
RewriteCond %{HTTP_USER_AGENT} ^MIDown\ tool [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mister\ PiX [OR]
RewriteCond %{HTTP_USER_AGENT} ^Navroad [OR]
RewriteCond %{HTTP_USER_AGENT} ^NearSite [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetAnts [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Net\ Vampire [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetZIP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Octopus [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Explorer [OR]
RewriteCond %{HTTP_USER_AGENT} ^Offline\ Navigator [OR]
RewriteCond %{HTTP_USER_AGENT} ^PageGrabber [OR]
RewriteCond %{HTTP_USER_AGENT} ^Papa\ Foto [OR]
RewriteCond %{HTTP_USER_AGENT} ^pavuk [OR]
RewriteCond %{HTTP_USER_AGENT} ^pcBrowser [OR]
RewriteCond %{HTTP_USER_AGENT} ^RealDownload [OR]
RewriteCond %{HTTP_USER_AGENT} ^ReGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^SiteSnagger [OR]
RewriteCond %{HTTP_USER_AGENT} ^SmartDownload [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^SuperHTTP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Surfbot [OR]
RewriteCond %{HTTP_USER_AGENT} ^tAkeOut [OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport\ Pro [OR]
RewriteCond %{HTTP_USER_AGENT} ^VoidEYE [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Image\ Collector [OR]
RewriteCond %{HTTP_USER_AGENT} ^Web\ Sucker [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebAuto [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebCopier [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebFetch [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebGo\ IS [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebLeacher [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebReaper [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebSauger [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ eXtractor [OR]
RewriteCond %{HTTP_USER_AGENT} ^Website\ Quester [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebStripper [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebWhacker [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebZIP [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget [OR]
RewriteCond %{HTTP_USER_AGENT} ^Widow [OR]
RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE [OR]
RewriteCond %{HTTP_USER_AGENT} ^Xaldon\ WebSpider [OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus
RewriteRule ^.* – [F,L][/source]

Once again, this method isn’t foolproof. The HTTP_USER_AGENT is quite easily spoofed, and some site ripping applications even allow you to specify what user agent you want to appear as. But if your site is large, implementing this list may make a significant impact on your monthly bandwidth bill.

Mask File Extensions

If for some reason you want to hide the fact that you’re using PHP (or Perl, or whatever), all it takes is a simple line in your .htaccess to have your .php files look like .html files:

[source='css']RewriteRule ^(.*)\.html$ $1.php [R=301,L] [/source]

You could even completely obfuscate it if you wanted to, for example serving files that end in .snipe that are really .php files:

[source='css']RewriteRule ^(.*)\.snipe$ $1.php [R=301,L] [/source]

In these examples, redirects all files that end in .html (or .snipe) to be served from filename.php so it looks like all your pages are .html (or .snipe) but really they are .php. Notice again that we’re using a 301 redirect.

Prevent Image/File Hotlinking

This snippets prevents people from hotlinking to your files – that is, linking directly to files hosted on your server from their website, thus sucking your bandwidth. It should be noted that in my experience, this rewrite rule seems somewhat spotty, and doesn’t always work, so be sure to test thoroughly.

[source='css']RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^http://(www\.)?snipe.net/.*$ [NC]
RewriteRule \.(gif|jpg|swf|flv|png)$ /images/dont_steal_bandwidth_jackass.png [R=302,L] [/source]

This rule basically says “If the request’s referrer is not blank (meaning the file was accessed directly in a browser) AND is not snipe.net (case insensitive), rewrite any files that end in .gif, .jpg, .swf, .flv or .png to display the file /images/dont_steal_bandwidth_jackass.png.

I have a friend who was trolling through their server logs (as he tended to do) and he realized that someone was using an image from his server, a Debian logo, if I recall correctly – as their MySpace background image. My friend spicyjack, being of the snarky persuasion, set up a leech=prevention mod_rewrite that directed all requests for that image that were not coming FROM his server to an image that was… shall we say…. not something you’d want as the background image of your MySpace page. (Google “hotcurry.jpg” if you’re really curious. It’s NSFW. Or anything, or that matter.)

Search Engine Friendly URLS – Make Dynamic Pages appear Static

To turn site.com/index.php?category=foo&subcat=bar into site.com/category-foo/bar.html, just use:

[source='css']RewriteRule ^category-([0-9]+)/([0-9A-Za-z]+).html index.php?category=$1&subcat=$2 [/source]

The category number and the subcategory name are variables, these are now represented by $1 and $2. The round brackets and the stuff inside them will be replaced by the variables. The things inside the round brackets are the regex rules for what the variable can contain. Example [0-9] means that it can contain any number from 0 to 9 and the + sign means that the number can be repeated 1 or more times.

The [L]ast Word

The [L] flag tells mod-rewrite that no more rules should be processed after that. Also remember that the order these rules are in DOES matter, so you’ll want to consider their intended behavior carefully when you’re planning your .htaccess. If at all possible, have your Apache error logs accessible while experimenting. when mod_rewrite goes wrong, it very often gives you a generic (and infuriating) “500 Internal Server Error” instead of anything actually useful to you – the Apache error logs may help shed some light on things.

Also be sure to test thoroughly whenever using mod_rewrite, since you can seriously break stuff if you’re not careful.

More Resources

Can’t get enough mod_rewrite? Check out these links for additional information and tips.

• Mod_Rewrite flags
• Mod_rewrite Tips & Tricks
• Mod_Rewrite Cheat Sheet

And if you’ve got a handy mod_rewrite rule you can’t live without, let us know in the comments.

Also check out:

Back in the day, it wasn’t uncommon for a website to look and work fine in one browser, and be utterly, horrifically broken in another. These days, if you’re coding to standards, your woes are more likely to be weird (but subtle) padding issues, some javascript issues, etc – not the complete and utter train wreck you’d have seen 10 years ago.

Planning and frequent testing will save your hide

Just because things are better now than they’ve ever been doesn’t mean we can get lazy. Cross-browser testing should be done often, and with gusto. One common pitfall with cross-browser testing is the temptation to leave testing until the very end, once the project has been completed. Problem is, if something is broken, you won’t necessarily have time to fix it, and depending on what’s wrong, the fix might take longer than you’ve allowed for.

You can avoid finding yourself in that kind of situation by testing more frequently throughout your development process. Whenever a large chunk of code or markup gets written, take the 5 minutes to check it on multiple browsers. By addressing these issues during these mini-milestones, you can take care of problems as they arise and avoid finding yourself in a bad situation the day before launch.

If you’re a small design shop, or an independent developer, you may not have access to the complete array of potential browser and OS combinations. I personally have a Mac laptop running Leopard, a Mac desktop running Tiger, four Windows XP machines, and a RedHat box set up, but even that doesn’t cover every possible combination – and you may not have as many machines at your disposal.

Fortunately, you’ve got some options, some better than others. The three primary ways you can test for cross-browser compatibility, short of actually running all of the browser and OS combinations on physical machines, are screenshot generators, browser emulators and virtual terminals. Each serves a slightly different purpose, and you may end up deciding that a combination of the three is most appropriate for your needs.

Screenshot Generators

Services like BrowserShots.Org and Litmus allow you to enter a url, click submit, and use their resources to generate multiple screenshots based on your selection of browser, setting and OS choices. Both have free and paid versions. (For the record, Snipe.Net looks like dog balls in IE4 for Windows… Hah!)

The free version of BrowserShots allows you to run a test on the full spectrum of browser/resolution/OS choices available, but your request will be lower priority than paying customers, so it can take a little while to get your results. Upgrading to a premium processing membership of $30 for one month bumps your job up higher in the queue for faster screenshots. Personally, I don’t think the improvement in speed is worth $30 a month, but you may disagree.

Litmus allows you to sign up for a free account which allows you 50 tests per month, but on only two browsers, both only on Windows, rendering the free version basically useless. If you don’t have access to IE7 and Firefox 2 somewhere, you probably shouldn’t be in this business. So the only useful way to use Litmus is through their paid version. They offer a few different packages, each with increasing features, starting from a day pass at $24, moving up to a monthly subscription of $49 a month for individuals and $199 a month for “team” membership. Once you get into the paid accounts, tests are done on 23 browsers, including Mac and Linux browsers. (A side note, I love how they say you get access to “all 23 browsers”, as if 23 is the final number.)

While their price structure is a bit steep and their browser/OS selection isn’t complete (no Firefox for Mac? Really?), it’s worth noting that Litmus offers HTML email testing across muiltiple email clients in their paid packages – something Browsershots doesn’t do. If you do a lot of email marketing for clients, this service may be worth it for that aspect alone.

Litmus also has a few value added features that may make it worth shelling out the money, depending on your needs.

• Nicer design – more suitable to showing to clients directly if you don’t feel like downloading dozens of screenshots for each iteration of a site.
• Ability to test password-protected areas of a site.
• Automatically checks for validation errors in both HTML and CSS, with a link to the W3C validator pages – so if something doesn’t look right in the screenshots you get, you’ve got a shortcut to start figuring out why.
• Version support, so each time you do a test, it creates a separate instance of that test as a new version for the site. If you test a page 6 times, you’ll see your current test and 5 historical versions in your control panel.

Me personally? I use the free BrowserShots service. They have more browser/OS combinations available and the wait time doesn’t bother me. I might use a paid Litmus account to test emails at some point though, as testing that manually can be a real pain in the ass.

Browser Emulators

The good news is, browser emulators are usually free. The bad news is that they are not available for all scenarios, and Macs seem to be at a real disadvantage here. Of course, Mac users can always use Parallels, WINE, or some other virtual OS that will allow you run Windows on your Mac, but that does require extra work.

If you usually develop on Firefox, the Firefox addon IE View can be very helpful. It simple provides a shortcut to an IE-engine rendered version of the page right in Firefox, that opens in a FF tab. Back when I worked on Windows, I loved this extension, but alas, it is for Windows only.

Also for Windows users, downloadable application DebugBar puts multiple versions of IE at your fingertips, including IE8 beta 2, IE7 IE 6 and IE5.5 on Vista and XP, as well as the installed IE. This one is handy because you can compare displays side by side, within one program.  I have gotten this to successfully install on my Parallels version of Windows on my Mac.

And the third one that comes to mind is Multiple IEs, a small downloadable application (again, Windows only, sorry) that lets you create separate instances of IE in many versions, going back as far as IE3. I managed to get this installed on Parallels, but it crashed every time I tried to use it. I had been using it for years on my native Windows computers though. I cannot speak to whether it works on Vista or Windows 7.

Virtual Terminals

Virtual terminals for cross-browser testing is a relatively new option for web developers, and brings something to the table that the other options cannot. Rather than simply seeing a picture of your site’s layout with screenshots, and rather than just being limited to IE as with the emulators, virtual terminals actually let you physically login to a machine running the OS and browser you’re looking to test. They are created using disk images, so when you logout, the image is destroyed and recreated for the next user.

What makes virtual terminals so unique and advantageous is that you are using the actual OS and browser you’re testing on. You can click around, test out your javascripts, login to protected areas and test functionality such as cookie behavior, and so on. What you end up with is a much more sophisticated test environment that results in a more genuine reflection of what the user will see and experience.

The funny thing is, with the pricey packages of the screenshot services, you’d probably expect to pay dearly for this much more advanced option, but in reality, they’re about the same as the screenshot services, with far more bang for your buck.

CrossBrowserTesting.Com even offers free access, and although the sessions are limited to 5 minutes and based on priority (paying customers come first if there’s a line), the free sessions are not limited in any other way. Their top-level paid package is still quite reasonable, at $29.95 for the first month and $19.95 each month after. For that top-level package, you get unlimited sessions (capped at a total of 40 hours of connected time), with a maximum of 30 minutes per session. They have a huge list of available OS and browser combinations, and for my money, this is the way to go. Check out their pricing structure here.

RealityBoxLabs offers a very similar service with a slightly different pricing structure. Unlike CrossBrowserTesting.Com, they offer a day pass for $5, and then and unlimited membership for $49 a month. They don’t specify which OS/browsers they offer, but they do mention that Linux will be available soon.

Which browsers to test?

The question of which browsers to test for is hotly debated in the web development community, but the realty is, there is no one single right answer. I can’t give you an answer, but here are some tips on coming up with the answer for your own situations.

• Consider existing browser/OS stats for the site, but don’t limit yourself to that if you’re pushing for a large increase in traffic or reaching new demographics, since your browser/OS stats may change once you succeed.
• Consider the browser/OS stats for the media outlets you may be pushing media from (such as Facebook and Facebook ads). If 90% of your traffic is going to come from a media outlet, make sure your site works on their top browsers.

My general rule of thumb, unless the stats indicate otherwise, is to develop and test on all of the major browsers, while still maintaining some level of normalcy for IE6 users. Depending on the project (and the client), I may have to work particularly hard to make sure the display on IE6 looks exactly the same as it does on current browsers, but if that’s not a requirement, I make sure it looks good on IE6. There may be some padding that isn’t quite right, but only the designer and the client would even notice.

For personal projects, it really depends on the audience. Snipe.Net has a user base of over 50% FireFox users, with the remaining large chunk being IE7 users. I made the decision to sacrifice some minor display points on IE in order to optimize for newer browsers. I am never so cavalier with paying clients.

Developing cross-browser compatible sites is easier than ever, both because of improved adherance to sandards, and because of advancing testing technology. Whether you choose to do it the old fasioned way, hopping from one computer to another hitting “refresh”, or whether you use some of these new-fangled, high-tech methods, it is your responsibility to your clients – and to their users – to make sure it works.

Also check out:

Get it free at tredoft.com.

Also check out:

General Browsing:

URL Fixer
Seamlessly fixes the fat-fingered “.con” typos in a domain name. (Other similar extensions only suggest the alternates after the page has failed to load)

Copy Plain Text
Allows you to copy without formatting

AdBlock Plus
Blocks out inline ads, such as banner ads, Google Adwords, etc. (Note, if you’re a regular at the IWG site, please consider disabling this plugin while there. The ad revenue actually does help keep it running. Its not a lot, but its enough.)

Link Alert
Pops up a teenie icon when you mouseover a link, indicating the type of file a hyperlink goes to (.doc, .pdf, etc).  This is particularly handy if you (like me) hate clicking on a link and then waiting 10 for Adobe Acrobat to open.  Obviously, yes, you can look at the status bar and find out where it’s going – but this extension makes it much more intuitive.

URL Link
Reconnects broken links that are only broken because web-based email wrapped them, etc.  Also has some other neat features.

PDF Download
Makes FF ask you what you want to do with a PDF file, rather than automatically opening it in a browser window.  I’m on a lot of law and court websites often, and many of them offer PDFs, so this is a nice one for me.

Resizable textarea
Just what it sounds like – it lets you make a textarea box bigger for more room to type.

Aging Tabs
Changes the color of the tabs in FF based on how long they have been idle.  This isn’t a critical one, but its helpful to me, since I switch back and forth from tabs so often. At a glance, I can see which are the most active, so I know which ones to click on.

Foxmarks Synchronizer
Syncs all of your Firefox bookmarks across multiple computers.  Requires you to sign up for an account at Foxmarks, but the account sign-up is built in to the initial setup, and it literally took less than 30 seconds. As of October 2008, Foxmarks now supports browser password syncing.

Download Helper
Grab images and video from YouTube, MySpace, etc.

FavLoc
Allows you to set specific favorite folders to save your downloads.

Background Image Saver
More and more sites are utilizing a transparent gif overlay to prevent their images from being saved via the right+click “save image as”. Since Firefox doesn’t have a “Save background image as…”. this extension gives it back to you.

Google Adsense Notifier
Shows you how much you’ve earned in the status bar.  I have found a bit of a glitch with this since the last update, where I can no longer view the amount from last payment (it shows the amount for all time instead), but I am hopeful that this will get fixed soon, since I really like this extension.

Gutil!
Just a quick shortcut to Google services (mail, news, etc.)  Simple, but nice.

Read it Later
This is a handy little addon that lets you flag a page to read later. If you’re like me, you constantly find interesting pages that you want to delve into, but don’t always have time, the very moment you find them. While you could certainly just set up a Bookmarks folder for stuff you want to get to later, Read it Later adds a spiffy icon in your address bar. One click and its saved for later.

Google Preview
Provides a thumbnail screenshot next to the site link and summary in Google search results. I wouldn’t have thought I needed this when I first heard about it, but its become one of my favorites.

Web Development:

IE Tab
This is perhaps more important as a web developer, but there are times when a page won’t render correctly in FF, so it’s nice to have an alternate way of viewing it. There is also a trimmed down version of this extension available as IE Lite. Alas, for PC only.

Aardvark
This one is a must-have for all web developers.  It allows you to quickly and easily view and select distinct elements of a webpage, displaying their characteristics, including id. Great for troubleshooting HTML and CSS.

Web Developer Toolbar
This is THE first extension I go for when setting up a new computer.  Tons of web dev tools, including viewing submitted headers, debugging tools, and so on.

Firebug
This is the SECOND extension I go for when setting up a new computer. If I ever get to meet the people who came up with this, I’m going to march right up to them and plant a wet one on their face. This extension has saved me so much time and aggravation – I can’t imagine being a web developer without it. How did I do it before Firebug? Firebug gives you easy access to CSS styles, HTML attributes, even network calls for external files.

User Agent Switcher
Helps debug any weird encoding issues you might run across.

ShowIP
Shows the IP address of the website host.  Useful if you’re working with a site that does load-balancing, dev environments, and so on.

Server Spy
This addon lets you see what type of server the website you’re visiting is running, similar to Netcraft’s “What that site running” lookup.

Character Identifier
This extension adds a context menu item for selected text that provides more information (from the Unicode database) about the characters selected.

Server Switcher
If you work with a dev and live server environment, such as dev.yoursite.com and www.yoursite.com, this extension allows you to easily switch between the two servers from any page on either server.  It would have been nice if you could add one more version, such as staging, but overall its really quite handy.

Fireshot
A very slick, lightweight screenshot addon that allows simple annotations and highlight boxes. Very handy for explaining elements to clients, helping non-techy people understand what and where to click, and so on. There is a free version and a paid version, but I’ve found that the free version is more than adequate for most use. Only available for Windows… *sniffle*

Colorzilla
Great little addon that gives you the power of a color eyedropper in your browser. Need to grab a color from a website to make additional graphics? No problem. Two clicks later, you’ve got the RGB or hex color code copied to your clipboard. This addon has saved me a ton of time.

MeasureIt
Handy little addon that lets you whip out a ruler and measure the size of anything in your browser, in pixels.

HttpFox
Lets you see the headers being sent through the browser – can be helpful for debugging scripts.

The Pencil Project
Great addon that lets you plan and tweak GUI interfaces right in your browser. Click here to read more about the Pencil Project in a previous Snipe.Net article.

Social

Twitterfox
Simply THE easiest way to Twitter. There isn’t a time when I’m on the computer without a browser window open, so this slim Twitter client makes it unbelievably easy to post to and read Twitter updates. Don’t use Twitter? Click here to learn why you should!

Not at All Useful, But Still Fun

Bork Bork Bork!
Translates webpage text into Swedish Chef

Also check out: