Comments on: Ruby Background Tasks with Starling - Part 2

By: Dave

Dave — Thu, 24 Jul 2008 00:35:14 +0000

I’ve had it running for months without a single thread-related issue. The trick is to make sure you have:

ActiveRecord::Base.allow_concurrency = true

when you startup your thread. To be complete, you should also have:

ActiveRecord::Base.verify_active_connections!

at the end of your thread.

Don’t ever try to have threads in your web application, but I haven’t seen problems with daemons.

By: graham

graham — Wed, 23 Jul 2008 23:52:59 +0000

I’m wondering if threading is the best direction to go for this. I’ve been looking around, and it seems like rails/activerecord threading generally meant a world of hurt (is this view outdated?).

By: Dave

Dave — Sat, 28 Jun 2008 11:45:00 +0000

Balancing sleep_time with CPU is pretty much your only option. Since Starling is a really simple queuing system, it only supports polling. We could probably come up with a signaling scheme to prevent the polling and make jobs start immediately, but that is not the case that starling/workling were meant to solve.

By: Andy Watts

Andy Watts — Sat, 28 Jun 2008 04:01:01 +0000

default poll is 1 second. Pretty fast.

Adding sleep_time to config/starling.yml can make it faster.
Making it faster drives the cpu load up.

development:
listens_on: localhost:22122
sleep_time: 0.001

By: Andy Watts

Andy Watts — Sat, 28 Jun 2008 03:38:44 +0000

Workling seems to poll memcache for starling messages every two seconds. This means a delay before a worker starts on a job.

Is this sleep between polls necessary?
Is there anything else I can do to minimize the delay in jobs starting?

Thanks Andy

By: Dave

Dave — Wed, 18 Jun 2008 15:48:44 +0000

As an FYI, the original Working author has integrated my changes as well as several other improvements. I will be switching back to his very soon. Because everything is on git now, it’s easier to suggest new additions so everyone can get the benefit.

The author’s site: http://playtype.net/past/2008/2/6/starling_and_asynchrous_tasks_in_ruby_on_rails/

The github for the sources: git://github.com/purzelrakete/workling.git

By: Dave

Dave — Wed, 18 Jun 2008 13:06:49 +0000

I’m curious how the wrong port caused a loop.

By: Ram

Ram — Wed, 18 Jun 2008 06:16:08 +0000

aaaahhhh… Starling was running on the wrong port !!!

By: Ram

Ram — Wed, 18 Jun 2008 05:17:08 +0000

Any idea why my worker functions would be called in a loop?

I have exactly one call:

BgWorker.asynch_set_enable(:ref => 1)

in my bg_worker.rb file I have

class BgWorker <1, :uid=>”bg_workers:set_enable:61bb317b03aaf804bdbc21ce635a779e”})

called in a loop. I don’t understand why. This worked before, and now runs in a loop.

Thanks

By: Dave

Dave — Wed, 11 Jun 2008 02:28:19 +0000

Updated the patch again to be a ZIP instead of an svn patch. Some people were having problems with that.