Comments on: Ruby Background Tasks with Starling – Part 3

By: Replacing Mongrel with Passenger (mod_rails) » Big Dave’s Blog

Replacing Mongrel with Passenger (mod_rails) » Big Dave’s Blog — Mon, 23 Mar 2009 13:35:33 +0000

[...] bother with it since I have a bunch of recipes for Apache and Mongrel (see a previous posts about background tasks with Workling). The other day, I figured I would give it a shot. Now, I think I found my new way to deploy Rails [...]

By: Dave

Dave — Tue, 17 Mar 2009 02:26:52 +0000

No problem. Glad I could help.

By: Jason

Jason — Tue, 17 Mar 2009 01:58:36 +0000

wow, my unix sucks. i didn’t realise you could assign sudo privileges for a specific command only – awesome!

I added:
myappuser ALL= NOPASSWD: /usr/local/bin/god

into /etc/sudoers and bingo – god in a capfile – thanks dave!

By: Dave

Dave — Mon, 16 Mar 2009 23:47:43 +0000

Not sure how to do that with god.rb. What I normally do is give sudo access to the production account but ONLY for the god application. Combine that with no ssh passwords (public/private key auth), and I feel pretty good about things.

There are other monitoring applications that may do what you’re thinking, however, but I haven’t spent too much time looking.

By: Jason

Jason — Mon, 16 Mar 2009 22:54:12 +0000

Great post,
One question – I don’t really like the idea of giving sudo access to my application account in production ie:
* it would be nice if your deploy script didn’t use sudo for god commands.

I wonder if theres a way to configure god.config to monitor the applications ‘current’ (deployed) directory in order to trigger it’s workling restart? Monitoring ports might work for mongrel but not phusion passenger. Any ideas on how to do that?

By: Dave

Dave — Wed, 04 Feb 2009 19:13:12 +0000

I have the same thing in my Inquisix app. User can upload 1000′s of contacts, and that process can take a while. Once the file is received, the user is informed that they will get a message/email sent to them when the process is complete.

By: Nanda

Nanda — Wed, 04 Feb 2009 18:42:13 +0000

Ok thanks, got the info I needed. In our app, there are couple tasks that take 5-10 mins and there could be situation where users invoke the task(from different pages) in short span of time. Anyway, we might just have to educate users that wait time may vary.

By: Dave

Dave — Wed, 04 Feb 2009 18:17:36 +0000

The threading model I implemented created a thread per worker class, not per task. I wanted to create a balance between workers not stepping on each other and creating too many threads. In most cases, when you have a job that takes 5-10 minutes to run, you don’t want to have more than one executing at a time anyway. It is often more efficient to run things serially in that case. What I wanted to avoid, however, is the single 10 minute task preventing the hundreds of 100 millisecond tasks from running.

By: Nanda

Nanda — Wed, 04 Feb 2009 18:04:52 +0000

Dave,
One more question, so you had added threaded functionality to the workling base code right? Is there anything particular we need to specify in workling.yml or sth to ensure its threaded. Basically, even for the same profile(app), say i have campaign_worker.rb, and the task there takes say 5/10 mins to run. What’s happening now is when i invoke a campaign task thru app, it seems to wait if there is another campaign task already running in the workling. Ideally, we would want to fork another thread forr each of the task, any thoughts on what we might be missing here. Thanks!

By: Dave

Dave — Thu, 18 Dec 2008 17:27:38 +0000

Very interesting. I have not used the namespace option with memcache, but it sounds like it would do the trick as well.