Hungry Machine

UPDATE: See Part 2 of this for a better solution
Note: This monkey-patch only works on Rails 2.2
We recently noticed our mongrels, upon startup, were 244M. Eek.

PID USER PR NI VIRT RES SHR S %CPU %MEM [...]

As we have been internally discussing how to scale our databases from 10’s of millions of rows to 100’s of millions, database sharding came up.
Depending on your data model and your application, sharding data into tables by some natural key is great if any given request uses only one shard. FiveRun’s DataFabric seems [...]

Instead of relaying on running cleanup of old releases via capistrano, we have a cron job to only keep releases for last two days (but at least three latest).

#!/usr/bin/env ruby

require ‘fileutils’

KEEP_RELEASES = 3
KEEP_DAYS = 2
EXCLUDE_APPS = %W(uploadr)

cut_time = (Time.now.utc – KEEP_DAYS*24*60*60).strftime("%Y%m%d%H%M%S").to_i

Dir['/u/apps/*'].each do |app|
next if EXCLUDE_APPS.include?(File.basename(app))
dirs = Dir["#{ app }/releases/*"]
fresh [...]

UPDATE: Animoto just raised a round of investment from Amazon! Congrats Guys!
Animoto is a great idea. They take your photos and create a production quality video to the music of your choice. Its the end of those boring slide shows, for good.

(From a recent Techcrunch article here)

We had [...]

Curb is a ruby binding for libcurl. We’ve had sporadic issues with Net::HTTP, which this might aleviate via native dns, native timeouts, performance improvements, etc. It wouldnt be hard to re-implement ActiveResource, rfacebook, myspace-ruby, etc to use it instead. Anyone using this already?

sudo gem install curb

require ‘rubygems’
require ‘curb’
require "net/http"
require ‘benchmark’

iterations = 40
Benchmark.bm [...]

One of the challenges with writing a Facebook or Bebo application is staying within a limit it gives you to respond with data before it shows the Application Did Not Respond page to a user. Having a content reach application calling external APIs, like Amazon or YouTube, with response times beyond your control, forces you [...]