Ruby on Rails
HowToRunBackgroundJobsInRails

Still partially an OpenQuestion.

How do I run background jobs in rails?

All the doc I’ve found so far is for response based code e.g. user pushed button, Ruby bootstraps itself → runs some code → sends back a page.

How can I just plain, run some code and, for example, change the escalation level on a “bug” record from “normal” to “too old” if a record ages too much?

Alternately can I write a background job that’ll, for example, spit out a nicely formatted text file every night at 2:00 AM?

There are several things that might work, depending on your needs.

Ap4r

The new kid on the block is Asynchronous Processing for Ruby (Ap4r), an implementation of reliable asynchronous message processing. It provides message queuing, and also message dispatching.

Using asynchronous processing, we can cut down turn-around-time of web applications by queuing, or can utilize more machine power by load-balancing.

Documentation is still quite limited, however it appears to be the most actively developed project in this area.

More information:

*NIX Cron and Runner

Using the RunnerScript in a cronjob like:


.script/runner -e production "Model.do_something"

is by far the most stable way of running background processes in Rails.
mits.

Cron, when used with RoR, has the following shortcomings:

CronEdit – Editing Cron from code

Crontab entries can be managed from within your code very conveniently using CronEdit gem:


require 'cronedit'
include CronEdit
Crontab.Add  'task1', "5,35 0-23/2 * * * #{YourRunnerTask}" 
Crontab.Add  'task2', {:minute=>5, :command=>'script/runner lib/task/task1.rb'}
Crontab.Remove 'task1'

Your entries does not interfere with the actual crontab. You can also do bulk updates/removals or use files.

Controller Actions

How to Execute a Rails Controller Action via. Cron

*NIX ‘at’ suite and Runner

Similar to cron in that you use runner, but via the ‘at’, ‘batch’, ‘atq’ and ‘atrm’ command suite. ‘batch’ is particularly useful as it only runs jobs when the load level on the computer drops below a particular level.

The abilities of the ‘at’ suite vary amongst Unix flavours. Some have multiple queues ‘a’ to ‘z’ which run jobs on the computer with increased ‘niceness’ (lower priority) as the letter increases.

Jobs can be monitored and controlled by interfacing with the atq and removed with atrm, thus providing greater flexibility than cron (which is really designed for jobs that repeat regularly, rather than jobs that are instigated by user action).

daemon_generator

RailsCron’s author, Kyle Maxwell, has deprecated RailsCron in favor of daemon_generator. RailsCronDeprecationMessage

Installation:


./script/plugin install -x svn.kylemaxwell.com/rails_plugins/daemon_generator/trunk/

sudo gem install daemons

./script/generate daemon <your_daemon>

RailsCron

RailsCron is a way to execute background tasks using your Ruby on Rails environment. The RailsCron object is an ActiveRecord, so you can manipulate it in familiar ways. See the README for details

I had a problem with RailsCron tasks not releasing database connections (using Postgres and postgres-pr driver). I had to add:

ensure
  connection.disconnect!

at the end of my processing method.

—Scott Persinger

BackgrounDRb

Stable Version 1.0 of BackgrounDRb has been released. Does not work on Windows.

BackgrounDRb is a distributed ruby server daemon that runs outside of rails but allows for you to kick off and manage long running tasks from your rails app. It has hooks for creating ajax progress bars and status updates in the browser while your long tasks runs in the background and thereby does not hold up the rails request/response cycle.

Spawn

I made a plugin that allows you to either fork or thread a background process. You can get it from rubyforge by running this command:

The way you use the plugin is simply:


  spawn do
    do_stuff  # stuff takes a long time
  end

Where you replace do_stuff with your stuff. If you want to do threading instead of the default forking then you can add a configuration setting or override it like this:

  spawn(:method=>:thread) do
    do_stuff  # stuff takes a long time
  end

Check the README for complete details on usage.

—Tom Anderson

Forking with Ruby

Ruby contains a fork method in the Kernel class that starts a background process. Using it in rails has been discussed:

I just tried this, and it seems to stall the engine (which is good in that it prevents race conditions, but it means forking isn’t the right tool for long running tasks). Here’s the code I used:


class HalController < ApplicationController
  def tick
    Process.detach fork do
      now = DateTime.now
        sleep 30
        Race.new do |r|
          r.system     = System.find_all.first
          r.start_date = now.to_s
          r.end_date   = DateTime.now.to_s
        end.save
    end
    render_text "tock #{DateTime.now}" 
  end
end

When several users go to hal/tick at once the race-condition test blocks are all consecutive (no overlap) but the GUIs return in sequence (e.g. several minutes later for the last ones).

—MarkusQ

Well, I’ve had pretty good luck using fork, at least under Mongrel. Here’s my post on using fork to run background tasks:

http://geekblog.vodpod.com

—Scott Persinger

Don’t forget to set 

config.active_record.allow_concurrency = true

in config/environment.rb, otherwise your models will behave strangely. However, at the present moment (Rails 1.2.3) there’s a bug that causes the dispatcher to hang.

ActiveMessaging

Another option is to send off a message, and have it get queued up and run asynchronously.

ActiveMessaging is one way to incorporate a messaging system into a rails app to handle work queues, or other mechanisms to throw work over the fence and out of the synchronous thread of responding to user input.

Threading with Ruby

http://www.rubycentral.com/book/tut_threads.html

It should be noted that Spinning off Thread’s via Thread.new { code_block } will not work with apache. It will however work with lighttpd (reasons why?)

—To answer this question, your apache musty be compiled with Thread support to do this. My company uses apache and spins of emails with Thread.new to keep requests that generate emails working as quickly as possible.

—See the comment about allow_concurrency above.

Store the request in a database table with the asynchronous plugin

http://code.google.com/p/asynchronous/

Don’t even have to change your methods, just add an “asynchronous” right after it to get it queued. Only seems to work right now from ActiveRecord callbacks, but still works pretty well.

BackgroundFu

p If you need just a simple and straightforward solution, try BackgroundFu

Workling: starling and asynchrous tasks in ruby on rails

how do you offload asynchrous tasks out of your rails request cycle? we needed this ability in boomloop.com, where we write data into a statistics database.

frankly, i find backgroundrb a bit scary. rumours about instability persist, and it seems like a lot of weight for a little problem. what else is out there? topfunky has a few ideas here. after checking out a few of those options, i asked evan weaver of chow.com what he thought. he said:

…If you really need a queue, use Starling, which Blaine Cook of Twitter released, like, yesterday. Or SQS if you need really huge storage. If you just want to fire and forget a local process as you say, I think Spawn is pretty good ( http://rubyforge.org/projects/spawn ). I haven’t actually used it but seems like the best of the forking bunch. That should eliminate the startup overhead. On the other hand, you don’t get any message reliability or cross-machine scheduling. ..I agree that BDrb is shady (actually all of Drb is shady). ap4r is too bloated. Thruqueue is promising if you make it past the crazy dependencies list. BackgroundFu is like a worse Spawn.”

result: i wrote a rails plugin called workling that integrates starling into your rails app. it also lets you swap starling for any other system you might want to offload work to, without needing to change your client code. along with starling, i’ve implemented a spawn runner and a local runner.

installation instructions and use after the link:

http://playtype.net/past/2008/2/6/starling_and_asynchrous_tasks_in_ruby_on_rails/

Updated on April 18, 2008 10:00 by Anonymous Coward (130.149.154.39)
Edit | Back in time (75 older versions)