Comments on: Ruby Background Tasks with Starling – Part 2

By: Dave

Dave — Mon, 23 Mar 2009 13:27:33 +0000

Workers are still single-threaded. The concurrency level is only at the worker level. Essentially, each worker class gets its own thread.

By: Taurus

Taurus — Mon, 23 Mar 2009 12:27:48 +0000

@Dave:

If I’ve understood your posts correctly, you indicated the latest version of workling now includes a facility to create concurrent threads in the case of long running jobs. Is there something special I need to do to get that working? My workling runs fine but still only serially processes jobs from the same worker. For example, I have a report_queue_worker which routinely spins off long-running jobs. All other jobs seem to still wait until the last one finishes.

By: Dave

Dave — Thu, 12 Feb 2009 23:07:18 +0000

Very interesting. The behavior I see in that case is for workling to write something in the log and exit immediately.

By: Nanda

Nanda — Thu, 12 Feb 2009 21:43:52 +0000

We just had the latest github version about 2 months ago, since then haven’t updated. To me it seems like, there was no starling, no spawn, bjrunner running, so looking at the plugin code, it invoked notremoterunner section of the code .. still doesn’t explain why it went in a loop. But now we started the starling and it’s working fine.

By: Dave

Dave — Thu, 12 Feb 2009 02:54:20 +0000

I know there is code in there to quit on startup if Starling isn’t running. I just tried it locally, and Workling quit immediately when Starling was not running. Are you running the latest version from Github?

By: Nanda

Nanda — Thu, 12 Feb 2009 02:51:34 +0000

Ya, we never had starling running at any point, thing is we dont have much control over the server and not sure how it was set up. But I dont think starling was ever started there, no folder where the task files are stored like /var/spool/starling. Anyway, Ram had mentioned in the comments above he had similar issues, but he had starling running on wrong port, in our case we dont have it running at all…..tried to reproduce it locally and appropriately workling complains it can’t find starling in specified port. Anyway, we will try to debug more, thanks.

By: Dave

Dave — Thu, 12 Feb 2009 02:35:50 +0000

Very interesting. I would have expected that without Starling running, workling throws and error and stops. Was Starling running when you started Workling, or did Starling stop while Workling was running? In the later case, Workling will keep trying to talk to Starling, but it will pause for a while between tries.

By: Nanda

Nanda — Thu, 12 Feb 2009 00:12:32 +0000

Dave,
I am running into a really weird situation. Starling process is not running at all, but workling task seems to be running in endless loop..any thoughts? In workling.yml, we specified starling server as localhost.
Thanks!

By: Dave

Dave — Tue, 10 Feb 2009 19:45:48 +0000

I always use the command line rather than a YML file because I didn’t see any doc either. However, I looked through the code and it looks like the format would be:

starling:
  host:
  port:
  queue_path:
  log_level:
  daemonize:
  timeout:
  pid_file:
  log_file:
  user:
  group:
  syslog_channel:
  timeout:

Then, run with starling –config starling.yml

I haven’t tried this, so YMMV. Check out load_config_file and parse_options in starling-0.9.8/lib/starling/server_runner.rb. It looks like load_config_file simply maps yml names to symbols, but it maps queue_path –> path and log_file –> logger.

Let me know if this works.

By: Tom

Tom — Tue, 10 Feb 2009 18:37:42 +0000

I can’t find any documentation on the starling.yml file? I only have a workling.yml. What options are available? How do I start starling and reference the .yml?

Thanks!