Archive for the ‘Ops’ Category

Cleaning up old releases

Instead of relaying on running cleanup of old releases via capistrano, we have a cron job to only keep releases for last two days (but at least three latest).

#!/usr/bin/env ruby

require ‘fileutils’

KEEP_RELEASES = 3
KEEP_DAYS = 2
EXCLUDE_APPS = %W(uploadr)

cut_time = (Time.now.utc – KEEP_DAYS*24*60*60).strftime("%Y%m%d%H%M%S").to_i

Dir['/u/apps/*'].each do |app|
next if EXCLUDE_APPS.include?(File.basename(app))
dirs = Dir["#{ app }/releases/*"]
fresh [...]

Reviewing Application Health with HAProxy Stats

One of the methods we use for checking the health of our applications is stats collected from HAProxy. We utilize it to see how many requests are scheduled for execution on mongrel instances. The graph is one indication of how our applications perform. When we launched the new version of the site three weeks ago, [...]

Reconfiguring the whole rails stack via a central YAML file

The challenge with hosting of multiple Rails-based Facebook applications is that the amount of users grow quickly. To address this problem we are using EC2 nodes that we can expand/shrink as the demand grows. The price/performance ratio isn’t quite what we first expected, so we are moving toward having a few dedicated boxes instead. Another [...]

Killing sneaky mongrels

We found that sometimes monit fails to restart all mongrel instances after deployment and some of them end up running with the pid file gone. Since there is no pid, monit believes the instance is not running so it tries to start a new one on the same port and, of course, fails. Which leads [...]

Using a ruby-based AIM notifier in nagios

If you use nagios for monitoring of your rails instances, you might want to get notification not only via email or SMS-messages but to your AIM when you are online. The script (libexec/aim_notifier.rb) utilizes the Net::TOC gem for sending out notifications:

#!/usr/bin/env ruby

require ‘rubygems’
require ‘net/toc’

user = ‘your_bot_name’
password = ‘bot_password’

msg = ARGV[0].to_s.gsub(‘\n’, [...]