Reviewing Application Health with HAProxy Stats 0

Posted by val
on Thursday, March 27
One of the methods we use for checking the health of our applications is stats collected from HAProxy. We utilize it to see how many requests are scheduled for execution on mongrel instances. The graph is one indication of how our applications perform. When we launched the new version of the site three weeks ago, the graph for a single vertical (ReadingSocial) on a typical Tuesday looked like this:
So, between porting all verticals to Myspace, Orkut, Bebo, and enhancing the functionality, we spent some time on optimization. In addition to analyzing slow-query logs with mysqlsla, Aaron wrapped all external API calls (and we do a lot of them - to Amazon, Facebook, Myspace, etc) in slow monitoring so we could see where the latest external bottleneck was so we could fix it one by one. Three weeks later the graph became much more peaceful:

Monitoring Rails Apps: Pulse + More 1

Posted by aaron
on Friday, October 26
Paul Gross from Thoughtworks recently created a pulse gem. The gem adds a simple action to your rails app, "/pulse", which acts as a heartbeat.

The pulse gem currently defines the method as: 
def pulse
   render :text => "OK"
end
Then you can configure haproxy to monitor your application by hitting http://server/pulse and verifying the response is "OK".

We've had a similar action in our applications for quite some time, although I really like the idea of externalizing it to a gem. Our implementation is slightly different though. Since so many rails applications depend on a database, I instead added: 
def pulse
    rows = ActiveRecord::Base.connection.execute("select 1 from dual").num_rows rescue 0
    render :text => rows == 1 ? "OK" : "Error!"
end
IIRC, "select 1 from dual" is the fastest query you can run against a database, in MySQL, Oracle, and Postgres. And if you have mutliple databases, you can add union in the one pulse request, or have multiple actions.

Now you are not only testing your application is live, but that it can connect to the database.