Saturday, January 11, 2014

Daemonized rvm ruby tasks using start-stop-daemon

I am very new to Ruby so the solution described in this post might be very obvious to some but I could not find all the parts of this solution in one place. I cobbled together bits and pieces of other peoples startup and capistrano scripts to get this working. My research has also shown there are ruby gems that might handle some of this better but I wasn't trying to reinvent the wheel. Anyhow, on with the show. At work, during our capistrano deployment we have a ruby process that has to be launched in the background. We are using start-stop-daemon to daemonize the process but our use of rvm complicates running rake because the rake binary is stored in the .rvm/gems directory. Normally the path to the rake binary is set from .bashrc and .bash_profile when you log in through a shell but when you execute things from cron, start-up scripts or other non-shell environments the paths don't exist. After some googling and tinkering I finally got something that worked:

task :start_mytask, :roles => :mytask, :except => { :no_release => true } do
  run "RAILS_ENV=#{rails_env} start-stop-daemon --start -b -m -o -d ~/current -p ~/pids/mytask.pid -a /home/ubuntu/.rvm/gems/ruby-1.9.3-p194@global/bin/rake mytask"
end

The downside with this script is I was calling rake for a specific version of ruby. At the time this was good enough to get us by. Fast forward several months and now we are preparing to upgrade to ruby 2.0. Our existing start script needed to be updated to use ruby 2.0 but I wanted to find a better way that wouldn't need to be tweaked each time we upgrade ruby. My first shot at updating the script I switched to executing rvm and calling 'bundle exec rake':

task :start_mytask, :roles => :mytask, :except => { :no_release => true } do
   run "RAILS_ENV=#{rails_env} start-stop-daemon --start -b -m -o --chdir ~/current --pidfile ~/pids/mytask.pid --exec ~/.rvm/bin/rvm -- current do bundle exec rake mytask"
end

This eliminates directly calling rake for a specific version of ruby but introduces a different issue. Executing it this way causes rvm to launch in a bash shell which then executes the rake task. start-stop-daemon creates a pid file based off of the first process started which is the bash process not the rake task. So when you try to stop it start-stop-daemon tries to kill the bash process which won't end because it has a child process running the rake task. I was getting closer but it wasn't perfect. More googling ensued and I discovered I could use rvm-exec instead of rvm. The rvm-exec command fixes the bash shell problem. It was created specifically for calling rvm in scripts. Here is what I finally came up with:

task :start_mytask, :roles => :mytask, :except => { :no_release => true } do
  run "RAILS_ENV=#{rails_env} start-stop-daemon --start -b -m -o --chdir ~/current --pidfile ~/pids/mytask.pid --exec ~/.rvm/bin/rvm-exec -- current bundle exec rake mytask"
end

It will now run independent of a ruby version number and since only one process is launched the pid file is created correctly. One other benefit I realized is now it is much easier to monitor this process. Previously the process was listed as just "rake". If you have more than one running that would get tricky to monitor. Launching it with this new script the process is listed as "rake mytask".

Finally to stop the daemonized rake task run this command:

task :stop_mytask, :roles => :mytask, :except => { :no_release => true } do
   run "start-stop-daemon -o -p ~/pids/mytask.pid --stop"
end


[UPDATE 01/13/2014] I realized today after doing more testing that I should be calling 'current bundle exec rake' instead of 'default bundle exec rake'. When you are upgrading ruby versions it doesn't necessarily mean you want to change the default ruby version in rvm. Using 'current' will cause the script to use whatever ruby is specified in the .rvmrc. I have edited the above scripts to reflect my new findings.

4 comments:

  1. How do you handle starting the background daemon at server bootup time? Do you somehow call the start_mytask rake task from an init script?

    ReplyDelete
    Replies
    1. I have a simple bash script. Here is the meat of it:

      exec /sbin/start-stop-daemon --chdir $railsdir -b -o -m --pidfile $pidfile --start --exec ~/.rvm/bin/rvm-exec -- current bundle exec rake $mytask

      I also use this during log rotation.

      Delete
  2. Google led me to your solution, embedded RVM process start inside a start scripts. This replaced some clunky bash shell encapsulated stuff that was broken quite badly after upgrading RVM to a newer version. Thank!

    ReplyDelete

Please note all comments are moderated by me before they appear on the site. It may take a day or so for me to get to them. Thanks for your feedback.