1. You could flood your inbox if there is a bad problem on a popular page.
2. It’s difficult to collect all the emails and track errors.

Enter Hoptoad. Hoptoad solves both problems, and it is super simple to use and setup. It’s free for one project and two users. If that isn’t enough for you, the premium services are very reasonable.

How does it solve both problems? First, Hoptoad will still send you an email when there is an error. However, it won’t flood your inbox if there are hundreds of the same email. Second, Hoptoad provides a simple web-based system that groups your errors by exception class. That makes it easier to track down problems. Finally, you can mark errors resolved so you don’t have to wade through noise to solve the real problems.

Hoptoad installation:

REMOVE EXCEPTION_NOTIFIER

1. In your ApplicationController, REMOVE this line:include ExceptionNotifiable
2. In your config/environment* files, remove all references to ExceptionNotifier.
3. Remove the vendor/plugins/exception_notifier directory.

INSTALL HOPTOAD_NOTIFIER

From your project’s RAILS_ROOT, run:

script/plugin install git://github.com/thoughtbot/hoptoad_notifier.git

CONFIGURATION

You should have something like this in config/initializers/hoptoad.rb.

HoptoadNotifier.configure do |config|
  config.api_key = '1234567890abcdef'  # You get your key when you sign up
end

(Please note that this configuration should be in a global configuration, and is not enrivonment-specific. Hoptoad is smart enough to know what errors are caused by what environments, so your staging errors don’t get mixed in with your production errors.)

Once you do the above, any Exception not caught by your controllers will be sent to Hoptoad where where they can be aggregated, filtered, sorted, analyzed, massaged, and searched.

Now, if you have read anything I’ve done before, you know I do a lot of asynchronous processing with Workling. By default, Hoptoad will not log errors from anything outside of your controllers. Fear not. Hoptoad provides a webservice API to send errors. Here is an example.

In workling/lib/workling/base.rb:

    # takes care of suppressing remote errors but raising Workling::WorklingNotFoundError
    # where appropriate. swallow workling exceptions so that everything behaves like remote code.
    # otherwise StarlingRunner and SpawnRunner would behave too differently to NotRemoteRunner.
    def dispatch_to_worker_method(method, options)
      begin
        self.send(method, options)
      rescue Exception => e
        raise e if e.kind_of? Workling::WorklingError
        logger.error "Workling Error: runner could not invoke #{ self.class }:#{ method } with #{ options.inspect }. error was: #{ e.inspect }\n #{ e.backtrace.join("\n") }"
        # DND: Let HopToad know of the issue
        params = options || {}
        HoptoadNotifier.notify(
          :error_class => "Workling Error - #{e.class.name}",
          :error_message => "Workling Error(#{e.class.name}): #{e.message}",
          :request => { :params => params.merge(:worker_class => self.class.name, :worker_method => method) })
        # DND: end of change
      end
    end

Now, any error not caught by your workers will be sent to Hoptoad for processing. You can use a similar method from Rake tasks or any scripts that run outside of controllers. Remember, Hoptoad will collect errors by :error_class, so you can use different classes to separate errors into bins based on where they came from.

Give Hoptoad a try. You will not be disappointed.

Check it out. You may find that you have a different strategy when you negotiate your next deal. Remember, deals are made all the time, and it’s not always a big business deal.

Good stuff.

In the early morning hours on a Tuesday morning 65 years ago today, D-Day paratroopers began jumping into the French countryside. At first light, six divisions of soldiers from the United States, Britain, and Canada began landing on the beaches of Normandy.  The beaches all had code names Omaha, Utah, Gold, Juno, and Sword. The United States landed on Omaha and Utah. The British took care of Gold and Sword, and the Canadiens landed on Juno.

Just prior to the invasion, General Eisenhower read what is now an historic passage:

“You are about to embark upon the great crusade, toward which we have striven these many months.”

Obviously, you can read about the Normandy invasions everywhere, so I’m not going to describe all the events again here. I will, however, talk a little bit about my experiences touring the area as well as introduce you to Charles Durning.

Who is Charles Durning? You probably know him by his movies. His film career began in 1965. Some credits include “The Sting”, “Dog Day Afternnon”, “North Dallas Forty”, “The Best Little Whorehouse in Texas”, “O Brother, Where Art Thou?”, and dozens of others. What you may not know about Charles Durning is that he survived two most horrific periods in WWII. You wouldn’t know because like many veterans of that era, he rarely spoke about it until years later when asked.

Durning was awarded the Silver Star and three Purple Hearts. He was among the first wave of troops that landed on Omaha Beach. Suffice to say, the nickname of “Bloody Omaha” is descriptive because of the more than 2,200 casualties suffered on 6 JUN 1944.

By 17 JUN 1944, Durning was back in England recovering from  shrapnel wounds in the left and right thighs, the right hand, the frontal region of the head, and the anterior left chest wall. He was pronounced fit again on 6 DEC 1944, just in time for the Battle of the Bulge and his second historical experience.

In 2008, Durning received France’s National Order of the Legion of Honor (each year France honors 100 veterans that served with distinction in France). At the ceremony, he described his experiences during the Battle of the Bulge. Early in the battle, Durning was stabbed 8(!) times during a hand-to-hand fight with a young German soldier. The fight did not end until Durning was able to reach for a rock and bludgeon the German soldier to death. He said when it was over he wept with the dead German soldier in his arms.

Soon after, Durning was taken prisoner and would have been shot on the spot were if not for an english speaking German officer that accepted his surrender and had his wounds tended to (the 8 stab wounds – one to the chest). As a prisoner now, Durning was led to a small town called Malmedy. It is here that the infamous Malmedy massacre took place. At Malmedy, some 150 prisoners were rounded up and dozens were executed. Durning and two others of his group managed to escape the carnage. There were many incidents in an around Malmedy, resulting in 72 bodies being discovered.

Durning was on the TV show “Rescue Me” this year. It was here that I was reminded of his service and felt the urge to talk about it. I’m thankful for men like Charles Durning and thousands of other’s like him. Like most veterans, I agree that the true heros are the ones that didn’t make it back.

I was lucky enough to pay a short visit to Normandy many years ago. The visit was far too short, but I was able to stand on Omaha and Utah beaches and see what men like Durning faced. Think about 2-3 football fields of open beach to cross with an amphitheater of cliffs all around. Unlike what you see in the movies, it’s not the machine guns right in front of you that are the most dangerous. It’s those to the side. This is one of the reasons some units saw casualty rates of over 50% in the first few minutes of the landings. Powerful stuff.

In this time of robots and smart bombs, we will thankfully never again see mass invasions and infantry action. I only hope it isn’t replaced with something far worse. So today, while you are enjoying a wonderful Saturday, send a “Thank You” to those men and women that started the Great Crusade to clear Europe of Nazi tyranny 65 years ago today.

I’ve lost track of how many times this simple fact has been proven to me. The qualities of a great engineer carry over to any platform, and a great engineer will pick up a new platform quickly — mostly because they love learning new things. If you’re starting out in software development, concentrate on being a great engineer. That’s far more valuable than an engineer that knows a platform.

What makes a great engineer? To me, it’s pretty simple. You have a passion and ability to craft outstanding, maintainable, and testable code. You know your algorithms, design patterns, and data structures like the back of your hand. Finally, you posses the other skills necessary to round things out — communication, time management, risk assessment, strategic and detailed design, and quick decision making (and sticking to those decisions). You can write a loop in Java? I don’t really care. You understand dependency injection or how an outer join works. Now we’re talking.

What do you think?

UK Snubs Support for Home of WWII Enigma

Here we go again. Bletchley Park continues to get little love. Here we have what is basically one on of the birthplaces of modern computing. On top of that, the group of people that worked here, along with their US partners in Building 26, did more to shorten WWII and save countless lives than just about any other group. Makes your life as a military leader a whole lot easier when you know what your counterpart is up to.

It seems now that the UK will not give Bletchley Park the same status as Imperial War Museum.

“We have no plans at present to associate it with the Imperial War Museum,” Lord Davies said. “The House is all too well aware of the significance of designating any area in association with a museum of that rank, but I want to give an assurance that Bletchley Park will continue to develop under the resources made available to it.”

OK, I know I’m a little biased because I’m a history buff, but I’m also aware of the history of my profession. Without these two groups of scientists, we might not have the same level of computing we have today. This is where Alan Turing (of Turing Machine fame) cut his chops.

Let’s not forget about these people and what they did. I know it was super-secret, but it was almost 70 years ago now. 

class User < ActiveRecord::Base
  def after_save
    MyWorker.asynch_geocode_address(:user_id => self.id)
  end
end

class MyWorker < Workling::Base
  def geocode_address(options = {})
    user = User.find(options[:user_id])
    user.geocode_address
  end
end

Pretty straight forward, right? Unfortunately, what you will find is that often your worker will get called so fast on create that ActiveRecord hasn’t committed the transaction yet, and User.find will fail. To get around this, you need to write your workers that could be called from after_save/after_create to handle this condition. Instead of the above, you need to do the following:

class MyWorker < Workling::Base
  def geocode_address(options = {})
    retries = 3
    begin
      user = User.find(options[:user_id])
      user.geocode_address
    rescue => err
      if (retries -= 1) > 0
        sleep(0.2)  # Give ActiveRecord a chance to save
        retry
      end
      raise err
    end
  end
end

Now we catch the exception that ActiveRecord throws when it can’t find the new user yet. Waiting a short time and trying a few times gives ActiveRecord a chance to commit the record.

Enjoy asynchronous processing…

In reference to Memorial Day, I came across the above link. More than most battles, Peleliu is one of those that defines sacrifice. The amount of suffering and carnage endured on that small island in the Pacific is difficult to imagine. This is where the term “thousand yard stare” came from.

"Thousand Yard Stare" by Tom Lea

"With the Old Breed" by Eugene Sledge

Prior to going to Peleliu, Lea was known more for ‘Go America!’ paintings, but something changed after that. No longer was it a matter of glory. It was survival.

Peleliu is also the subject of what I believe to be one of the finest wartime biographies ever written. If want a real description of combat and sacrifice, you must read “With the Old Breed” by Eugene Sledge. Sledge was on the island for entire 4 months of fighting. 10,000 Japanese soldiers and about 2000 Americans died on this island 3 Miles Long and 1 mile wide, but that wasn’t the worst of it. You have to read the book to fully understand his descriptions of living among corpses for days/weeks in 100 degree heat.

Again, thank you to all those that serve and have served. Freedom is not free, and I try to remember every day what it took to create and keep what we have in this wonderful country of ours.

Next post will be back to technical stuff.

User.find(:all).each { |user| user.some_method }

The reason is that you just loaded all 500,000 records into memory, and your server is not happy. Instead, you could do:

User.find_each { |user| user.some_method }

By default, the above will only load 1,000 User objects into memory, and your server will thank you. If 1,000 is too big/small for you, use the :batch_size option to change it. The find_in_batches method is similar except that it provides the array to the block instead of one object at a time. For example:

User.find_in_batches do |users|
  users.each { |user| user.some_method }
end

If you ever used the wonderful will_paginate gem, you are probably familiar with the concept from the paginated_each method that will_paginate provided.

So, what is the problem? The problem is that you have to aware that unlike paginate_each, find_each and find_in_batches work by setting up a with_scope block. Therefore, if you need to do any other finds on that same model, the scope will apply. Usually this only affects relationships, but it isn’t hard to forget. Here is an example:

# This is a purely made up example
class User < ActiveRecord::Base
  # There is a last_login_at attribute
  named_scope :recent_login,
              lambda { |*args|
              { :conditions => ["people.last_login_at >= ?", (args.first || 1.week.ago)] } }
  belongs_to :parent, :class_name => "User", :foreign_key => "parent_id"
end 

User.recent_login.find_each do |user|
  parent = user.parent # This will include the recent_login scope.
end 

It’s no different than other with_scope issues, but it isn’t as obvious. You can get around it by doing:

User.recent_login.find_each do |user|
  # Got use send because with_excusive_scope is protected.
  User.send(:with_exclusive_scope)
    parent = user.parent # This will include the recent_login scope
  end
end

Now, go and be kind to your server with find_each and find_in_batches – just remember the scope.

This is a great read for anyone that is interested in great sound. I’ve said it a few times, I’m not into expensive audio gear because I like gadgets (although that part is fun too). I’m into expensive gear because I flat-out love music, and I want to hear it like the artist meant it to be heard. I will tolerate MP3’s on my phone, but not when I want to hear some really choice music played like it was meant to. Sadly, between my kids and my schedule, I do not get near enough time for pure listening.

This is Michael Fremer. He’s listening to “Avalon” by Roxy Music on his $350,000 stereo system. It sounds excellent. He’s a bit crazy, but if you love music, you need him.

BTW, “Avalon” is one of those pieces of music that screams for high quality. My system isn’t quite in the league of $350k, but it does pretty well.

I think the following quote sums up my thoughts exactly:

Because the thing is, Fremer loves music first and foremost. The audiophile I had feared was one who cares far more about the overpriced gadgetry than the actual music. This is not who I ended up meeting. This man listens to music and makes sure it was recorded with the best fidelity, that the intents of the artist have been preserved. And thank God he does, because we certainly don’t.

If you ever get a chance to listen to a favorite, well-recorded piece of music on a choice system, take it. You may be surprised at the subtleties and feeling you are missing. Be careful though. Many recent recordings are mixed for MP3 players and will sound pretty bad on a great system. The really scary part is that some recent “Remastered” versions of classic albums were mastered for MP3 players as well. Many times, it is better to stick with the original.

In a future post, I will describe what I built/coded to control my entire media system from one remote. Note, since I have small children, all my equipment is in a locked closet.

What an amazing day. Yesterday, my friend Brian sends me a tweet offering a ticket for Opening Day at Fenway Park. Now, I’ve been a fan of baseball and the Red Sox since I could walk, but I’ve never had the privilege of seeing Opening Day. On top of that, after I can’t count how many Red Sox games I’ve been to, I’ve never seen the Red Sox Ace pitch in a regular season game (only Pedro during the All Star game, but that’s another story.) Of course, I jumped at the chance.

The tickets were were actually from a friend of Brian’s. They are season tickets, and we couldn’t figure out why they wanted us in our seats before 3:20PM for a 4:06PM start (bonus points if you know the significance of the 4:06PM start time.)  So, we get there pretty early, and no one seems to care about whether we go to our seats or not. Of course, that signals we have time to get a beer. A couple of minutes later we find out why the Red Sox wanted our sections in their seats so early. This is where the players were going to come down to the field from! It was hard to tell, but I think they had players coming out from more than one section because it wasn’t the whole team. Our section had Big Papi, Mike Lowell, J.D. Drew, and a few others.

Our seats were great. Check out Ted Kennedy throwing out the first pitch to Jim Rice. I only had my phone, so I didn’t have any chance of zooming closer.

Oh, yeah. There actually was a game to be played against the Tampa Bay Rays. It turned out to be a great game. Josh Beckett started for the Red Sox and struck out 10 over 7 strong innings. He looked outstanding! Dustin Pedroia started off 2009 well by hitting the first pitch he saw over the Green Monster. Papelbon struck out 2 in a 1-2-3 ninth to get the save and a 5-3 Sox victory.

Thanks again Brian for the invite.

One more thing, if you ever go to Fenway Park and expect to get any kind of internet access for Twitter or looking up a stat during the game, forget about it. Even the Fenway WiFi couldn’t handle all those phones trying to do the same thing. Just watch the game and enjoy!