More Asynchronous Processing

In a previous series on using workling and starling for asychronous processing, I described how to setup background tasks. Here is a quick way to use this for emails without a lot of changes to your application.
First, create lib/asynch_mail.rb:


# Makes an actionmailer class queue its emails into a workling queue
# instead of sending them sycnhronously
#
# From now on all MyMailer.deliver_whatever_email calls create an entry in
# the MailerWorker.deliver_mail corresponding queue and should be processed
# by a worker. If you still want to deliver mail sycnhronously add a bang to the method call:
# MyMailer.deliver_whatever_email!
#
module AsynchMail
  def self.included(base)
    base.class_eval do
      class < < self
        alias_method :orig_method_missing, :method_missing

        # Catch deliver method calls to turn them into asynch calls
        def method_missing(method_symbol, *parameters)
          case method_symbol.id2name
          when /^deliver_([_a-z]\w*)\!/
            orig_method_missing(method_symbol, *parameters)
          when /^deliver_([_a-z]\w*)/
            mail = self.send("create_#{$1}", *parameters)
            MailerWorker.asynch_deliver_mail(:class => self.name, :mail => mail)
          else
            orig_method_missing(method_symbol, *parameters)
          end
        end
      end
    end
  end
end

Then, create app/workers/mailer_worker.rb:


class MailerWorker < Workling::Base
  def deliver_mail(options)
    Object.const_get(options[:class]).deliver(options[:mail])
  end
end

Now, all you need to do is include AsynchMail in your mailer class.


class MyMailer < ActionMailer::Base
   include AsynchMail
end

That’s it! No other changes to all your mail calls are required. All will automatically become asynchronous. If your application sends a lot of emails, I recommend you do this. There is no reason to make your users wait for an email to be sent.

The only downside with this is that now emailing happens outside of Mongrel. This means that link_to does not have access to the HOST. As a result, you can can’t easily use the route helpers to generate urls. You will need to build your urls manually. At one point, I had some code that got around this, but it stopped working when I upgraded to Rails 2.x. In my app, it was easier to fix the urls than make the other work. If someone can provide this, it would great.


Posted

in

by

Tags:

Comments

6 responses to “More Asynchronous Processing”

  1. Neil Avatar

    Great post. That’s handy a snippet – if it wasn’t for the loss of helpers (although I’m pretty sure I don’t have any link_tos in mailer templates anymore), I’d give this a go right now.
    I was looking for a similar solution a little while back, but the engine yard engineers (the app being hosted on engine yard) said that a single outgoing email per action isn’t such a big deal because of their shared architecture – it seems they take the outgoing emails from your app and send it themselves. So, I’ve implemented async processing for the four or so mailer methods which can potentially create more than one recipient (activity notifications within ‘groups’) and have to cycle through recipients to do so (plus I couldn’t work out how to do your auto-async trick at the time, but now I know!).

  2. Dave Avatar

    To be specific, link_to and other helpers still work. It’s only the routing helpers that require a HOST that do not work. That means all the *_url helpers will have a problem. You can play games with _path methods and prefix your own HOST to it. That’s what I do.

  3. Neil Avatar

    Looks like the Workling guys have done it again – their latest post shows a really, really easy way to create async outgoing email by default;
    http://playtype.net/past/2008/11/11/sending_mail_asynchronously_in_rails/

    I haven’t tried it but I could end up implementing it soon. How does that look to you, Dave?

  4. Dave Avatar

    This is pretty much exactly the same thing as I did, but it is wrapped up in a plugin. The other difference is that they are not using Starling. If you are already using Starling, then there is no need to add another queue server to the mix.
    However, either way will work perfectly.

  5. Aaron Gibralter Avatar

    Doesn’t the create_xxxx call happen synchronously? So shouldn’t the link_tos function “correctly” (with host and all) since they get called in the mongrel process before it gets handed off to workling?
    Also, have you noticed any problems with large messages? I thought the idea of Workling/Starlign was to send short messages… The entire TMail seems like a potentially large thing to send via a starling queue. Is there a way to instead send the Mailer’s arguments rather than the TMail itself?

  6. Dave Avatar

    Yes, the create_xxxx call is synchronous, and you’re right. I was confusing the original issue I had. My original issue with link_to was that I was sending emails from other workling tasks, and it’s those that do not have access to the HOST value. Therefore, emails sent from the web app should work fine. Thanks for catching that!
    Regarding large messages, I have not noticed any issues. Generally, the emails that I send are pretty small (a few k), so it’s not a big deal. Looking at the Starling source, I suspect the issue with large messages has more to do with the queue being in memory than anything technical.

Leave a Reply to Dave Cancel reply

Your email address will not be published. Required fields are marked *