Bigbro's foray into the scary world of blogging
27 06 2006
From the annals of Steve-Teh-Wonder-Cook(tm)...
[19:18] diamond huh. burning smell. that's not good. [19:19] atlas Are you "cooking" again? [19:19] diamond atlas: you mean pouring milk into a bowl of cereal? no, not yet. [19:20] atlas Hmm. I've no idea what it could be then [19:21] diamond found the issue. luisa was on fire in the kitchen. [19:22] atlas How inconsiderate of her. [19:22] diamond yeah. i'm ringing the immigration fire brigade to get her put out.
Theo Schlossnagle (OmniTI) spoke about a solution for serving a massive amount of largely static images.
Why pay Akamine(sp?) or NetApp lots of money for bandwidth, network infrastructure or webcache devices, when you can roll your own.
Peer-based HA caching with something like Apache + mod_proxy (Reverse proxy + caching) might work well. There are some caveats - such as other people's caches, which might not respect cache directives.
The example given was a three-site HA solution. Finding the 'closest' image server was achieved through using local DNS servers, colocated with the image servers. Anycast (all the DNS servers have the same IP address and BGP takes care of finding out which is the 'closest' DNS server.) DNS uses UDP which means that it's safe to use Anycast for serving - however, the image servers would use TCP connections, which will not work if the internet topology/routing changes in the middle of a session.
So if you have 3 DNS servers, each with the same IP address in 3 geographically diverse locations, 3 servers ready to serve static image content, your own AS for BGP routing and a large amount of static image content - you can use F/OSS software and commodity hardware to make a HA/LB solution that will handle an enormous load.
What about distributed reliable logging? Something like a spread patch to syslog-ng allows logs to be written in 'real' time to multiple servers, reliably.
Blogs can benefit hugely from caching. Something like memcached might work for a read-heavy, write-light dataset. User preferences can be stored in user cookies which will provide all the nodes you ever need, along with all the resilience you'll ever want. If someone loses their cookie (or deletes it) you can just look up their preferences in your database and regenerate their cookie - if their cookie gets corrupted or their browser breaks, they only remove service from themselves.
Apologies for the terseness of this article, but information was coming hard and fast - and I can only typo at a certain rate ;-)
posted at: 16:39 | path: /technical | permanent link to this entry
Afer a largely uneventful, though delayed, flight, I arrived in Dublin. Immigration was less crowded than usual, the walk from the RyanAir terminal to the main airport building was longer than usual and the arrival of my bags was more prompt than usual. I guess it all averages out in a woefully uninteresting manner though. In a similarly uninteresting vein, Dublin's weather had returned to the bleak grey dampness to which I have become accustomed, leaving my sunglasses to dangle forlornly from my collar.
After topping up my phone and calling one of the ApacheCon organisers to see where best to meet up with them, I did my usual inept dance of trying to exchange all Sterling money on my person for Euro - failing utterly to complete this by the time I had to pay for a bus ticket. I now have an unnerving weight of Euro shrapnel (coins) in my pocket to attempt to get rid of over the next few days.
HEANet are generously providing internet connectivity for the event, but I've been warned that this means there's a collection of wireless APs to be configured and installed. I suspect this will keep me occupied for at least a chunk of the remaining afternoon. It also means there's a good chance that I (and probably many like-minded, but superior writers) will be able to keep a mostly up to date blog of the event as it happens. I guess you, dear reader, will find out whether this happens or not. ;-)
posted at: 16:05 | path: /technical | permanent link to this entry
Having only just returned from Ireland, I'm flying back to Dublin again to attend ApacheCon 2006. It should be a most interesting opportunity to hear some differing views from various industry sources on the future of Apache, web applications and Open Source Software in general. I'm certainly looking forward to it.
I'll be blogging parts of the ApacheCon 2006 event here (and on skynet) as the event unfolds.
posted at: 08:37 | path: /technical | permanent link to this entry
It is the way of things... (from poisonedminds.com)
posted at: 01:05 | path: /lotd | permanent link to this entry
