Big Dave’s Blog

UK Snubs Support for Home of WWII Enigma

Here we go again. Bletchley Park continues to get little love. Here we have what is basically one on of the birthplaces of modern computing. On top of that, the group of people that worked here, along with their US partners in Building 26, did more to shorten WWII and save countless lives than just about any other group. Makes your life as a military leader a whole lot easier when you know what your counterpart is up to.

It seems now that the UK will not give Bletchley Park the same status as Imperial War Museum.

“We have no plans at present to associate it with the Imperial War Museum,” Lord Davies said. “The House is all too well aware of the significance of designating any area in association with a museum of that rank, but I want to give an assurance that Bletchley Park will continue to develop under the resources made available to it.”

OK, I know I’m a little biased because I’m a history buff, but I’m also aware of the history of my profession. Without these two groups of scientists, we might not have the same level of computing we have today. This is where Alan Turing (of Turing Machine fame) cut his chops.

Let’s not forget about these people and what they did. I know it was super-secret, but it was almost 70 years ago now. 

As you probably noticed already, I use Workling a lot, and I wrote about it a few times (Part 1, Part 2, Part 3). One minor gotcha to be aware of is that you need to make sure you handle it when Workling is too fast. A common use of Workling is to make a call in an after_save method. For example:

class User < ActiveRecord::Base
  def after_save
    MyWorker.asynch_geocode_address(:user_id => self.id)
  end
end

class MyWorker < Workling::Base
  def geocode_address(options = {})
    user = User.find(options[:user_id])
    user.geocode_address
  end
end

Pretty straight forward, right? Unfortunately, what you will find is that often your worker will get called so fast on create that ActiveRecord hasn’t committed the transaction yet, and User.find will fail. To get around this, you need to write your workers that could be called from after_save/after_create to handle this condition. Instead of the above, you need to do the following:

class MyWorker < Workling::Base
  def geocode_address(options = {})
    retries = 3
    begin
      user = User.find(options[:user_id])
      user.geocode_address
    rescue => err
      if (retries -= 1) > 0
        sleep(0.2)  # Give ActiveRecord a chance to save
        retry
      end
      raise err
    end
  end
end

Now we catch the exception that ActiveRecord throws when it can’t find the new user yet. Waiting a short time and trying a few times gives ActiveRecord a chance to commit the record.

Enjoy asynchronous processing…

Thousand Yard Stares: Ruins and Ghosts of the Battle of Peleliu, 1944, 2008 « The Wired Jester

In reference to Memorial Day, I came across the above link. More than most battles, Peleliu is one of those that defines sacrifice. The amount of suffering and carnage endured on that small island in the Pacific is difficult to imagine. This is where the term “thousand yard stare” came from.

"Thousand Yard Stare" by Tom Lea

"With the Old Breed" by Eugene Sledge

Prior to going to Peleliu, Lea was known more for ‘Go America!’ paintings, but something changed after that. No longer was it a matter of glory. It was survival.

Peleliu is also the subject of what I believe to be one of the finest wartime biographies ever written. If want a real description of combat and sacrifice, you must read “With the Old Breed” by Eugene Sledge. Sledge was on the island for entire 4 months of fighting. 10,000 Japanese soldiers and about 2000 Americans died on this island 3 Miles Long and 1 mile wide, but that wasn’t the worst of it. You have to read the book to fully understand his descriptions of living among corpses for days/weeks in 100 degree heat.

Again, thank you to all those that serve and have served. Freedom is not free, and I try to remember every day what it took to create and keep what we have in this wonderful country of ours.

Next post will be back to technical stuff.

Rails 2.3 added a couple of nice new methods – find_each and find_in_batches. Both methods accomplish the same thing in a slightly different way. Unlike a normal finder these methods grab objects in batches instead of all at once. For instance, if you have 500,000 users, you don’t want to do the following:

User.find(:all).each { |user| user.some_method }

The reason is that you just loaded all 500,000 records into memory, and your server is not happy. Instead, you could do:

User.find_each { |user| user.some_method }

By default, the above will only load 1,000 User objects into memory, and your server will thank you. If 1,000 is too big/small for you, use the :batch_size option to change it. The find_in_batches method is similar except that it provides the array to the block instead of one object at a time. For example:

User.find_in_batches do |users|
  users.each { |user| user.some_method }
end

If you ever used the wonderful will_paginate gem, you are probably familiar with the concept from the paginated_each method that will_paginate provided.

So, what is the problem? The problem is that you have to aware that unlike paginate_each, find_each and find_in_batches work by setting up a with_scope block. Therefore, if you need to do any other finds on that same model, the scope will apply. Usually this only affects relationships, but it isn’t hard to forget. Here is an example:

# This is a purely made up example
class User < ActiveRecord::Base
  # There is a last_login_at attribute
  named_scope :recent_login,
              lambda { |*args|
              { :conditions => ["people.last_login_at >= ?", (args.first || 1.week.ago)] } }
  belongs_to :parent, :class_name => "User", :foreign_key => "parent_id"
end 

User.recent_login.find_each do |user|
  parent = user.parent # This will include the recent_login scope.
end 

It’s no different than other with_scope issues, but it isn’t as obvious. You can get around it by doing:

User.recent_login.find_each do |user|
  # Got use send because with_excusive_scope is protected.
  User.send(:with_exclusive_scope)
    parent = user.parent # This will include the recent_login scope
  end
end

Now, go and be kind to your server with find_each and find_in_batches – just remember the scope.