The $10,000 Space

If you are a programmer, have you ever had one of those bizarre situations where you know something is not working, but you aren’t getting an error or anything else that would make debugging easy?

A couple days ago I migrated a cluster of web servers to PHP 5.1.x (they were running PHP 4.4.x before). One of the things I run on them is a geo targeting mechanism, and in the case where my screw-up was, a function to block a subnet of IP addresses.

[code=php]echo sprintf(“%u”, ip2long(‘ 80.1.2.3’));[/code]

That little bit of code was my fuck up… PHP 4.4.x returns “1342243331” (as expected), but PHP 5.1.x would return “0“. PHP4 was more forgiving of the stray leading space before the IP address, while PHP5 was not. The end result was that every IP address before 81.1.2.3 was blocked, instead of blocking FROM that IP address to an end IP address (that was not very many IPs away).

Awesome… so what happened here? I ended up blocking about 40% of the potential traffic for awhile, which equated out to probably about $10,000 in lost revenue.

The lesson learned? Don’t fuck around with spaces, because they will cost you many hours of debugging/tracing as well as ~$10,000.

11 thoughts on “The $10,000 Space”

  1. That’s why you need to have a development server that has the same software (os, php, libraries….) as the production server….

    It costs less than 10.000$ and it can take you out of troubles 🙂

  2. I do have development servers (actually 2 of the 10 blades in the chassis are test/development machines currently). This issue would only surface under high traffic though… I only knew something was wrong because the overall conversions were down (but not gone).

Leave a Reply

Your email address will not be published. Required fields are marked *