For anyone using Firefox, this is a great plug-in (made by Google). It will sync your cookies, passwords, bookmarks, history and window tabs (you can pick and choose just certain things if you want) to your Google account.
What that means is you can use Firefox on a different computer, and all your stuff will show up. I mostly wanted bookmarks, but it’s handy to have the other stuff too.
//www.google.com/tools/firefox/browsersync/
Okay, is anyone tired of my server admin tips yet? Yes? Too bad.
Monitor everything… Put as much info at your finger tips as easily as possible. Put that info in a place where you will always be looking at it for some reason. For example, I made a vBulletin plug-in that monitors 4 memcached servers (including a latency test it runs) as well as 10 blade servers. This shows every time I’m in the admin of the forum (which is a lot), so I can’t help but to not see it.
I wrote a little daemon that runs on my servers that can quickly report back whatever info I want (time, disk RAID status, server load, MySQL replication status, etc.)
The more info you have in one place (especially when you run a bunch of servers), the easier it is to see anything wrong. For example, I had an issue with a web server serving requests slow one day… it ultimately ended up not being a problem with the web server, but the memcached server it was using. The latency test was showing ~2,000 ms latency (2 seconds) vs. the normal 0.5 ms (1/2000 of a second).
And be proactive about monitoring stuff… don’t wait until something bad happens to start doing it! Then it’s too late.
Can I go to bed now?
Okay, normally Apple products are extremely polished and well thought-out. But this airline power adapter they came out with today is just plain stupid.
I bought one today when I ordered one of Apple’s new MacBook Pro laptops at the same time. Well I just noticed some fine-print on it…
Well isn’t that lovely… there is no technical reason why they couldn’t have made both of those things happen, that’s just plain stupid IMO.
//store.apple.com/1-800-MY-APPLE/WebObjects/AppleStore?productLearnMore=MA598Z/A
After all the beta and release candidate versions, Firefox 2.0 is finally finished and released.
Go get it.
If you don’t know what Firefox is, then you especially need to go get it (really).
Yay, more server admin fun! 🙂 Here’s a useful *nix command that will let you determine what system calls a program uses… For example, I wanted to double check that libevent calls within memcached were using epoll() and select() or poll() calls (epoll scales better) on my SuSE Linux machines…
...clipped...
epoll_ctl(5, EPOLL_CTL_ADD, 3, {EPOLLIN, {u32=5313904, u64=5313904}}) = 0
epoll_ctl(5, EPOLL_CTL_ADD, 4, {EPOLLIN, {u32=5313920, u64=5313920}}) = 0
epoll_ctl(5, EPOLL_CTL_ADD, 7, {EPOLLIN, {u32=5313968, u64=5313968}}) = 0
epoll_wait(5, {}, 1024, 1000) = 0
Sure enough! 🙂
Purely out of necessity, I’ve become a system administrator/architect for digitalpoint.com servers… and a few people have been asking me for general admin tips to make things stable and scalable, so here’s a good one for you…
Use memcached… no really, use it.
memcached is a distributed memory caching system that allows multiple servers to access the same shared memory (you can use it just for single local server too of course). You can cache pretty much anything… SQL query results (especially ones that can’t hit indexes), dynamically generated web pages, dynamic RSS feeds, etc.
If you compile the memcache() functions/class directly into PHP, you have a an easy (yet powerful) way to incorporate it.
For example, let’s say I have some existing code that makes a call to some sort of API (I use memcached for the keyword suggestion tool in this way)… you can add a couple lines of code to cache the results.
[code=php]$data = @file_get_contents (“//some_api_call_url?keywords=$keywords”);[/code]
could become this:
[code=php]$memcache = new Memcache;
$memcache->addServer(‘192.168.1.1’);
$memcache->addServer(‘192.168.1.2’);
if (!$data = $memcache->get($keywords)) {
$data = @file_get_contents (“//some_api_call_url?keywords=$keywords”);
$memcache->set($keywords, $data, false, 86400); // 24 hour cache
}
$memcache->close();[/code]
Now we are only hitting the API for the same keywords once every 24 hours… that setup utilizes 2 memcached servers working as a single entity for pooling and failover.
I use memcache for all sorts of things (this blog’s pages are cached for 60 seconds as another example because WordPress sucks, and protects against a massive influx of traffic [front-page digg for example]).
Here’s some things I personally use it for…
Any high-traffic site that is dynamically generated (especially if it includes SQL queries) could benefit from it… you could manage which memcache items need to be updated when you insert/update to the database, or an easy way to do it is just check the cache for the pages that potentially have high traffic… if they aren’t in the cache, put them in the cache with a certain expiration date. Then the page is dynamically generated no more often than the cache expiration time (in the case of this blog, the pages are generated no more than once every 60 seconds).
P.S. – Make sure you block outside access to your memcached port since there’s no authentication used!
Here’s something cool that Google has added to Google Sitemaps for site owners…
Let’s you see how much Google’s spider crawls your site in the last 90 days, and even gives you the option to instruct Googlebot to increase or decrease their crawl activity on your site.
Here’s the stats for the Digital Point forum… Google crawling an average of 52,000+ pages per day (without putting it on “increase activity”)… Getting close to a page per second 24/7. 🙂
Did I miss this somehow, or is this new?
Google’s Code Search is *really* handy for programmers. For example, say I wanted to see sample code that uses the preg_match() function in PHP, I could do this query:
Neat!
I found some bugs in the web interface for a Linksys WRT54G router I bought to fix some 2Wire “issues”. One of the bugs just would give me a false error when trying to do something and keep the form from being submitted.
Anyway, the short version of this entry is that I found out that you can flash your Linksys with an entirely new firmware. Ultimately, I just wanted to fix the bug so I could do what the Linksys is supposed to do to begin with. But after installing it, it looks like you can do all sorts of other interesting things… like run crontab processes (since it’s Linux).
Anyway, if you have a Linksys WRT54G (they are pretty popular), and you are a dork, you may want to give it a go… but don’t blame me if you screw yours up (mine was fine).
If you are a programmer, have you ever had one of those bizarre situations where you know something is not working, but you aren’t getting an error or anything else that would make debugging easy?
A couple days ago I migrated a cluster of web servers to PHP 5.1.x (they were running PHP 4.4.x before). One of the things I run on them is a geo targeting mechanism, and in the case where my screw-up was, a function to block a subnet of IP addresses.
[code=php]echo sprintf(“%u”, ip2long(‘ 80.1.2.3’));[/code]
That little bit of code was my fuck up… PHP 4.4.x returns “1342243331” (as expected), but PHP 5.1.x would return “0“. PHP4 was more forgiving of the stray leading space before the IP address, while PHP5 was not. The end result was that every IP address before 81.1.2.3 was blocked, instead of blocking FROM that IP address to an end IP address (that was not very many IPs away).
Awesome… so what happened here? I ended up blocking about 40% of the potential traffic for awhile, which equated out to probably about $10,000 in lost revenue.
The lesson learned? Don’t fuck around with spaces, because they will cost you many hours of debugging/tracing as well as ~$10,000.
This really is stupid IMO… if you are worried about your weight so much that you need to go out of your way to buy a camera that actually makes you look slimmer than you really are, maybe you should just go to the gym or something. Unless you are a model and that’s your livelihood, I don’t see the point of this…
//www.hp.com/united-states/consumer/digital_photography/tours/slimming/index_f.html
I was just thinking… Google needs to build an API for Gmail. Then we could do things like POP our email into an email client, and then run a script from within the email client that is able to go back to Gmail and tag stuff as spam, apply a label, etc.
That would be handy… can someone please make it so? 🙂
I’m not sure if this is supposed to be a joke, or what… but this guy might as well been speaking Chinese and doing backflips. I think I may be able to put one in my new car (maybe)…
Did I tell you how much I hate the 2Wire router/gateway you have to use for fiber Internet connections? You can’t get into the admin (which is web based) remotely, you can’t use DDNS so when your IP address changes weekly, you have no idea what it changed to, the firewall assumes you are an idiot and can’t do basic stuff with it, etc…
So the IP address changed over the weekend, and the only way to get the new IP is to drive out there and look to see what it is. On the way out there, I had an idea… why not buy a Linksys WRT54G broadband router and just route all in-bound traffic to it. Then you can use it’s firewall and DDNS functions. So that’s what I did… and it works *so* much better… remote admin access and everything else. I even tried to get tricky and route traffic to the Linksys and then right back to the 2Wire to see if I could get to it’s web admin remotely, but no such luck… It ends up redirecting it to a different URL (I’m assuming based on the client not being on the local network)… whatever that part isn’t that important anyway now.
I guess I missed it, but Motorola released a new phone (MOTOKRZR) a few months ago that looks pretty tight. Like a new and improved RAZR. The one thing it has that I’d love to have in my RAZR is support for the high-speed EDGE stuff for fast Internet. But it does seem to have some other handy stuff like a GPS, memory slot, etc.
Now if they could somehow fit a QWERTY keyboard on it, we would be all set…
//www.motorola.com/motoinfo/product/details.jsp?globalObjectId=160
Dude, I want this *really* badly now… I’ve been using fast user switching to create separate environments I can switch between quickly, but Spaces seems to be a much better solution.
More than anything, this is just a bookmark for myself. That way when/if I need it at a later date, I remember what it was! 🙂
I tried to find a PHP class to control the new camera, and I couldn’t find one… so a little packet sniffing later, I was able to figure out how it’s little Java applet communicates with the camera. Anyway, I spent a few minutes to make a PHP class to control the camera, so if you need one, here you go (it hasn’t been extensively tested, so it probably has problems… if it does, leave a comment)…
So here’s a basic PHP script that uses the class to take control of the camera, set the pan/tilt/zoom to a certain position, capture the image and save it to a file… handy for uhm… I dunno… a cron job to take time lapse photography. 🙂
[code=php]connect(‘your.camera.ip.address’);
$cam->get_control();
$cam->goto( -7463, -548, 4126);
sleep(5); // Wait for movement to finish and lens to focus
$handle = fopen(‘images/’ . time() . ‘.jpg’, ‘w’);
fwrite ($handle, $cam->image());
fclose ($handle);
$cam->disconnect();
?>[/code]
class_camera.php:[code=php]host = $host;
return $this->connection_id = substr(file_get_contents (‘//’ . $host . ‘/-wvhttp-01-/OpenCameraServer?client_version=LiveApplet_4125&image_size=640×480’), 14, 9);
}
/**
* Disconnects from the camera
*
* @return none
*/
function disconnect()
{
file_get_contents (‘//’ . $this->host . ‘/-wvhttp-01-/CloseCameraServer?connection_id=’ . $this->connection_id);
}
/**
* Attempts to get control of camera
*
* @return none
*/
function get_control()
{
file_get_contents (‘//’ . $this->host . ‘/-wvhttp-01-/GetCameraControl?connection_id=’ . $this->connection_id);
}
/**
* Get camera info
*
* @return array of camera info variables
*/
function get_info()
{
preg_match_all (‘#(.*?)=(.*)#’, file_get_contents (‘//’ . $this->host . ‘/-wvhttp-01-/GetCameraInfo?connection_id=’ . $this->connection_id), $matches);
foreach ($matches[1] as $key => $value) {
$this->camera_info[$value] = $matches[2][$key];
}
return($this->camera_info);
}
/**
* Move camera
*
* @param string Movement type (pan | tilt | zoom)
* @param number Position to move to
*
* @return none
*/
function move($operation, $position = 0)
{
file_get_contents (‘//’ . $this->host . ‘/-wvhttp-01-/OperateCamera?’ . $operation . ‘=’ . $position . ‘&connection_id=’ . $this->connection_id);
}
/**
* Set pan/tilt/zoom on camera with one call
*
* @param number pan position
* @param number tilt position
* @param number zoom position
*
* @return none
*/
function goto($pan, $tilt, $zoom)
{
$this->move(‘pan’, $pan);
$this->move(’tilt’, $tilt);
$this->move(‘zoom’, $zoom);
}
/**
* Set parameter
*
* @param string paramter to set
* @param string value
*
* @return none
*/
function set_value($parameter, $value)
{
$mapping = array (
‘back_light’ => ‘B’,
);
$parameter = $mapping[$parameter];
file_get_contents (‘//’ . $this->host . ‘/-wvhttp-01-/OperateCamera?’ . $parameter . ‘=’ . $value . ‘&connection_id=’ . $this->connection_id);
}
/**
* Get current image
*
* @return none
*/
function image()
{
return file_get_contents (‘//’ . $this->host . ‘/-wvhttp-01-/GetOneShot?image_size=640×480’);
}
}
?>[/code]
In theory, Gmail’s spam filtering is supposed to get smarter as you “train” it by tagging spam emails that slipped through their spam filters. I’ve been anal about going through every single email and tagging every spam email as such in the hopes that it would get smarter. In reality, I think it might actually be getting dumber though, considering how much spam is making it to my inbox these days (it’s not a new thing BTW, I’ve been getting tons of spam in it for the last year, and it’s only been getting worse).
I doubt there’s an “easy” way to fix it (otherwise it would have been done), but I really wish Gmail would let us define our own custom spam rules. For example, if an email isn’t in English, it’s spam (for me anyway) considering I don’t read Russian, Chinese, Japanese, Korean, Greek, Hebrew, etc.
Check out the most recent emails in my inbox this morning (only 1 of them [the white line] is not spam).
Anyone know of some super awesome secret way to make a 2Wire Internet gateway also act as a dynamic DNS client? Since it’s a FTTP (fiber to the premises) gateway and not a “normal” DSL modem, I’m pretty sure I can’t just swap it out with something else that supports DynDNS.
The little I poked around the web interface, it seems pretty decent, other than no DDNS support. 🙁
In case anyone can help, the one I have is a HomePortal 300 series residential gateway.
I was talking to the SBC/AT&T guys that installed the Internet connectivity at the job site today and having fiber to the door is going to have some really cool advantages early next year (right now they just run phone and 6Mbps Internet across it). At that point the “standard” Internet connectivity on it should be 20Mbps down/3Mbps up. But they were also talking about the U-Verse television service which more or less streams television on demand. They were saying you could be watching a football game for example and you could control which camera angle you want to watch the game from. 🙂
That’s going to be pretty dope!
Someone asked me why I hide the sidebar on this blog by default, then show it in a different part of the HTML, so here’s the explanation…
Most blog themes float the main body to the left, and make the sidebar on the right “static”. I did the opposite when creating my custom WordPress theme… My sidebar is a DIV that is floated to the right. The reason for this then the content below the bottom of the sidebar wrap around it, which I just think looks better. So here’s the problem… Google weighs content at the beginning of a page higher than content at the bottom, so Google was seeing 99% of my pages as “similar content” because my sidebar needed to be at the top of the HTML source.
So to solve that problem, I just put the sidebar at the bottom of the page in a hidden (with CSS) DIV, then call a simple script to set the contents of an empty DIV (<div id=”sidebar”></div>) at the top of the page, where the sidebar needs to be, to the contents of whatever is in the hidden DIV at the bottom…
For web browsers, the sidebar renders at the beginning of the HTML source, but Google sees the sidebar at the end of the HTML and pages no longer are seen as 99% similar.
Just for fun (since I’m a dork), I was looking for a wireless stumbler for Macintosh that supported a GPS unit because I thought it would be interesting to map how many wireless networks there are in my neighborhood (I usually can see 15-30 unique wireless networks from any given point). In my search, I ran across one called kismac that does exactly what I wanted (it even generates the maps for you, so I didn’t need to code something to plot the GPS coordinates on a map):
I download it and start playing around with it. It turns out it also has security testing functions within it (although I would guess that most of the people using the cracking functions are just trying to gain access to “secured” networks… which is beside the point I suppose).
Anyway, so I start monkeying around with those functions to see if I could learn something about WEP encryption on my own 2 wireless networks (I have a Linksys WRT54G and an Apple Airport Express which I use for beaming iTunes music to the living room stereo), both are currently secured with 128-bit wireless security and I did not change anything in them for the purpose of this video. My “word list” is just the standard dictionary word list that comes with most any UNIX distribution (like Mac OS X) and resides in /usr/share/dict/.
So here’s the scary part, from the time it started scanning for wireless networks to the time I was able to crack both wireless network keys (which is all you need to gain access to the wireless network), it took right around 60 seconds. Check out this video…
Okay, so what just happened here? I just cracked my two 128-bit wireless networks in roughly 60 seconds from start to finish.
Even as a relatively knowledgeable tech guy, this seems like utter insanity to me. Okay, obviously I didn’t have some crazy, ultra-secure password for my networks, but I would guess 90% of all the wireless network passwords out there are based on simple (easy to remember) word(s). After doing some reading, an “ultra-secure” password/MD5 seed would be relatively useless anyway… all it would do is force the attacker to spend 10 minutes on it instead of 10 seconds (see this FAQ and this FAQ), all of which is easily done from the kismac Network menu. It doesn’t even matter if you setup your wireless network to be public or not, because kismac can see it even if the base station isn’t showing the SSID publicly.
I’m going to poke around and see how secure RADIUS authentication is for a wireless network, but even if RADIUS is more secure, what normal person is going to have the technical knowledge and an extra few thousand dollars to setup and run a RADIUS server for their wireless network? I’m not even sure if I want to run a wireless network anymore to be honest… or maybe shut them down except for the times I’m actually using them (talk about annoying though).
MySQL is working on version 5.0.25 already. 5.0.23 was never released, and 5.0.24 was released on July 27, 2006.
So if 5.0.24 was released, where the hell is it? (it’s not on the download page) 🙂
//dev.mysql.com/doc/refman/5.0/en/news-5-0-x.html
5.0.23 has a couple key bug fixes in it that I’ve been itching to get my hands on.
…so please join with me, and let’s pray to the MySQL God’s for a swift delivery of *some* version higher than 5.0.22…
It’s been out for a couple days now in Digg labs, but I have to finally say that Digg’s Swarm is pretty pimp… It will be fun when they release their API and we can start building our own stuff.
Check it out for yourself:
A buddy of mine has a new blog where he has a cartoon of himself reading his blog entries to you. That’s pretty cool if you ask me… I think we need a cartoon Shawn! 🙂
(I think it’s done with SitePal’s technology)
I bought an exterior dome for the camera I got (to be installed at the construction site), and the damn thing was missing the two electrical transformers it was supposed to come with. No biggie though, Videolarm shipping them out today, so should have them soon.
I guess it’s not terribly important right now since there’s no electricity at the site anyway…
Okay… I was able to recover everything from my corrupt FileVault volume and convert my home directory back to not using FileVault. The only issue right now is I’m running on 1 hard drive, not 2… but at least my computer is alive (with all my junk).
It sounds like a good idea in theory, but Apple’s FileVault sucks, and here’s why…
I installed the Mac OS X 10.4.7 update today (from 10.4.6) and it didn’t go so well (it’s not like I have some old/bunk Mac either, I have the newest quad processor PowerMac)… A reboot would give the system a kernel panic. Gee, cool… so I put the machine into firewire target disk mode and mount it as a volume from my laptop so I can pull some suspect extensions out of there. Okay, so far so good… ended up being some dependency issue with the USB driver, so I just yanked the kernel extension for the USB driver out of there for now.
Well guess what I found out? If you use your computer in firewire target disk mode and you go into your home directory, your FileVault image becomes corrupted. Hmmm… interesting… that means I just lost everything important (everything in my home directory). Would have been nice if someone tells you this (or better yet, just not let you do it). So now I have no home directory. 🙂 Rad.
Thankfully, I just realized that the RAID mirror on my computer broke about a month ago and never rebuilt itself… so worst case scenario is I have to go back a month. Right now I’m disabling FileVault on the month-old volume (only another 1 1/2 hours to go). When that’s done, my home directory is going to be backed up to something (iPods are handy for more than just music). Then I can see about figuring out a way to “uncorrupt” my up to date FileVault image.
Just for the record, if anyone is using FileVault, disable it. If your single FileVault disk image gets corrupt (by whatever means), you just lost all your files.
I’m going to be one pissed off bitch if I end up having to go back a month…
If you are *really* dorky you can now get a USB puppet that stands up when your friend comes online via IM and “dies” when they go offline.
I kind of want one of these actually. 🙂
This is something interesting that someone from my forum ran across while using Google (an error page that Google spewed back at them)…
I knew Google was using some sort of Borg technology, I just didn’t have any solid proof until now. “–using_borg”
Some dorky/interesting stuff in there… it looks like they may be using 2.8Ghz Xeon processors (same as my blade servers). But the more interesting stuff is the parameters they are using (pagerank and supplemental cutoffs for example).
I have no clue if this is true, but it sure would be a fun toy if it turns out to be true. 🙂
Either way, it’s a little late, I could have already used it for something useful.
//www.t3.co.uk/…next_apple_os_to_track_stolen_ipods
(via digg)
My 10 year old laser printer finally died on me earlier this week (it’s needed to be replaced/updated for a LONG time, so it’s kind of a good thing).
The replacement I ended up ordering was an Epson CX11N. I remember when color laser printers first came out, they were close to $10,000. Not only that, but this one also scans and copies (and does it well, unlike most other multi-function machines). You can also get it to send/receive faxes for an extra $100 (which I didn’t need, so I passed on that option).
But check this out… it’s normally $699.99 (which seems pretty cheap for everything it does and compared to stuff from other manufacturers), but Epson is offering free shipping (which normally is $150 or so because of how heavy it is), AND a $150 instant rebate. So that brings the cost down to $549.99 plus tax (that includes shipping).
$550 is a pretty f’ing good deal for this sucker… great printer (color laser, 25ppm B/W and 5ppm color) and an excellent copier/scanner. The fax version also has a 25 page auto-feeder for scanning/faxing/copying, but whatever… I don’t do that much, so. Either way, if anyone is thinking of getting a new printer any time soon, I would check this one out (especially before the instant rebate and free shipping expires).
Oh yeah, and it’s network capable, so you just need to plug it into your network switch/hub. 🙂
I decided I didn’t have enough to do (that’s a joke BTW), so this morning I made a “spy” system for my forum (the general idea of course was borrowed from Digg). It’s “extra” slick in Firefox because of the fades and opacity ability Firefox has, but it still works in IE, Safari, etc…
It more or less lets you see what’s going on in the forum in realtime.
Check it out: //forums.digitalpoint.com/spy.php
Unfortunately it also means that I’m still up right now and haven’t gone to bed yet. Goodnight! 🙂
The new servers and equipment were installed into the data center yesterday (I also had to move the existing servers/equipment to a new rack), so everything is physically at the data center now (it’s not actually in USE yet, but at least it’s at a place where I can start moving stuff over to them).
Kind of funny to see a load of servers that’s worth 100x as much as the car they are in. 🙂
A long time ago (when I was 17) I ruptured my spleen and they had to cut through my stomach muscles for the surgery. Well, apparently I had a weak spot just under my belly button from when my stomach muscles were sewn back together where I got a little hernia from lifting the servers earlier (I didn’t even notice it until about 10 hours later). I noticed a little bulge and knew I probably had a hernia of some sort from lifting that crap. Anyway… I just got back from Urgent Care (it’s a 24/7 place you can go in case you don’t know). 5 hours, 1 CAT scan and 3,827 games of Bejeweled on my cell phone later, I found out that I have a little piece of fat that popped through that weak (remember the spleen thing?) point.
Not that big of a deal… they are going to call me when they can take care of it with a quick little surgery.
More importantly, the new servers are in (oh yeah, I said that). 🙂
 A bit of a blurry picture, before the cables were tied up. Image stolen from Julien, who helped me move and install servers today. |