Chapter 2: Organizing with Plug-ins

Back to Table of Contents.

Organizing with Plug-ins

In most Rails books, this chapter comes last. It takes a lot of complicated examples to show why you need to organize your code at all. Examples at the beginning of books are simple enough that organizational strategies aren’t necessary. By the end of a book—if you get that far—it’s often too late for this sort of wisdom to be practical. Most readers, myself included, want to get to the meat of things right from the start, and are actively programming while reading a new book for useful tips and insights. Unfortunately, the result of this pattern is that many Rails projects end up poorly organized. By the time you get to the chapter on code organization, you may already have a functioning application with millions of users (I hope this is your problem).

In this book, I’m going to break with tradition and present some high-level principles of code organization before we even write any code. Because of this “backwards” presentation, some of what follows may seem unnecessary, burdensome, or even purposeless. Rest assured, the ideas that follow have none of these properties. They come from three years of incremental development of a 100% pure Rails web site that, as of this writing, receives approximately two million unique visitors per month.

From this experience, I can tell you honestly that at some point, the amount of code you have begins to strangle you. You may have started with a dozen classes, but a couple years down the road you have hundreds. You start with a simple request-response architecture, but before you know it, you’re running a dozen asynchronous processes to manage that website alongside regular user requests; you’re sending emails, resizing images, or talking to third-party services.

For any new application that I might start today, I would follow these guidelines uncompromisingly. When followed from the start, organizational principles add no burden. It’s trying to organize a live site’s code jumble where the heavy burden lies. In fact, working in a well-organized project is like strolling down a manicured path in an enchanted forest. The opposite is true of a poorly organized code base; it’s like running through a jungle. You never know what might pop out at you—oh look, you just tripped over a rock and a tiger ate you!

Complex organizational refactorings can be exceedingly difficult late in the game. It’s easiest if you start down the right path from the start, but of course, it’s hindsight, not foresight, that’s 20/20. At this stage, your applications may not be complex enough that lack of organization is a critical issue. If that’s the case, feel free to skim this chapter and then come back to it as you read the rest of the book. All of the examples in the book will adhere to the principles laid forth here, so at least a cursory understanding is definitely in your best interest.

Benefits

Plug-ins sound like something you get from third parties, or optional add-on features. While there are a great many plug-ins available from third parties that you can add to your application, writing your own plug-ins is central to your development process. It’s true that writing and releasing plug-ins to the public at large will make you famous and powerful (well, maybe not), but getting in the habit of writing plug-ins has many benefits even if you have no plans to distribute them. A short list of the benefits:

    1. Plug-ins provide a convenient mechanism to separate architectural enhancements from business logic. Intertwining these two can be a quick route to bugs.
    2. Plug-ins can be tested independently from the rest of your application, giving you greater confidence in the overall robustness of your code.
    3. It’s trivial to test plug-ins against new versions of Rails. Similarly, limiting application code to business logic makes applications themselves easier to upgrade when new Rails versions are released.

Plug-ins can be easily shared between multiple applications. In a service-oriented architecture, where different apps serve different business purposes, code sharing is a big win.

  1. If your first business plan fails and you need to scrap your application code, you can take your plug-ins with you to your next big idea.
  2. Releasing plug-ins may make you famous and powerful.

There are two common themes here, and one overriding principle. The first theme is the separation of business logic from code aimed at architectural enhancement. This has great implications for code quality, maintainability, and testing. Think of your application as layers. There’s the Ruby language itself. On top of that is the Rails framework. Instead of dumping a behemoth, unorganized “code layer” on top of the framework, you can gain a lot by splitting that layer in two: architecture enhancements (plug-ins) and business logic.

The second theme relates to isolating functionality into manageable chunks. Doing so lets you test, upgrade, or deploy bits of code without needing to think about all the other pieces. That can save you a lot of time and hassle when you need to further extend or reengineer some crucial chunk of architecture. It’s also a lifesaver when that big upgrade day comes.

The overriding principle here is organization. The more organized you are from day one, the better your results will be. There are a number of ways to organize code in Rails. Plug-ins are just one strategy, but they are an extremely powerful one.

Figure 2-1 illustrates.

Figure 2-1. Make your applications manageable by separate application code into business logic and separate plug-ins for each architectural enhancement

Writing Your Own Plug-ins

To create a plug-in, type this from your application’s root directory:

./script/generate plugin {plugin-name}

The code generator creates a directory under vendor/plugins with the name you specified in the command above. You should see the following output:

ChakBookPro:example chak$ ./script/generate plugin my_plugin
      create  vendor/plugins/my_plugin/lib
      create  vendor/plugins/my_plugin/tasks
      create  vendor/plugins/my_plugin/test
      create  vendor/plugins/my_plugin/README
      create  vendor/plugins/my_plugin/MIT-LICENSE
      create  vendor/plugins/my_plugin/Rakefile
      create  vendor/plugins/my_plugin/init.rb
      create  vendor/plugins/my_plugin/install.rb
      create  vendor/plugins/my_plugin/uninstall.rb
      create  vendor/plugins/my_plugin/lib/my_plugin.rb
      create  vendor/plugins/my_plugin/tasks/my_plugin_tasks.rake
      create  vendor/plugins/my_plugin/test/my_plugin_test.rb

The most important files for 99% of the plug-ins you write are in bold above. Unfortunately, the two most important files, init.rb and the plug-in code itself, in lib/my_plugin.rb (substitute your own plug-in’s name here), are empty. There is no guidance whatsoever on which to base plug-ins, the most important part of your architectural strategy.

In init.rb, the code generator gave us the following:

# Include hook code here
In lib/my_plugin.rb, the start we get is similarly useless:
# MyPlugin

In fact, all of the files generated are essentially empty. But don’t worry; writing plug-ins is easy. The next section gives you some templates to make writing plug-ins a snap.

Core Enhancements

Core enhancements are modifications you make to the layers below you, either Ruby classes or parts of the Rails framework. Ruby has great flexibility in that classes are not static once they are defined. You can re-open them at any point and stuff in more functionality or modify functionality that already exists.

Keep in mind that power comes with a price: modifying the core can be dangerous. It’s not a good idea to change behavior of core classes to do unexpected things. Bad modifications to the core can come back to bite you, and bite you hard. With that in mind, here’s how you do it.

Initialization template

When your Rails application starts up, the init.rb file of every plug-in is loaded. It’s single purpose is to load the rest of your plug-in. Therefore, all you really need is one line that will load the actual plug-in code:

require 'my_plugin.rb'

Of course, you should replace my_plugin.rb with the filename that was generated for you.

Core plug-in template

What do you put in the my_plugin.rb file? Since this is a core enhancement, the assumption is that we are going to re-open a class and make changes there. Therefore, you need to know what you’re opening. If we were enhancing the Hash class, our file would look like this:

class Hash
# our enhancements go here
end

It’s that simple. It’s almost too simple, so let’s look at a concrete example.

Suppose we wanted to modify the Hash class so that it could be manipulated with more object-like syntax. The normal way to access a hash is like so:

>> h = Hash.new
=> {}
>> h['foo'] = 'bar'
=> "bar"
>> h['foo']
=> "bar"
>> h['baz!']
=> nil

Imagine instead we wanted the following syntax to work, too:

>> h = Hash.new
=> {}
>> h.foo = 'bar'
=> "bar"
>> h.foo
=> "bar"
>> h.baz!
=> nil

In effect, we want the values stored within the hash to be accessible just as an object’s properties are accessed, with dot notation. We could do this by defining the method_missing method within the Hash class. When you call a method that is undefined on an object, method_missing is called with two parameters. The first is a symbol for the method you attempted to call. The second is an array of parameters that you tried to pass.

Our plug-in file, my_plugin.rb (actually, we’re going to call the plug-in hash_extension, which would result in a main plug-in file called hash_extension.rb), would look like Example 2-1.

Example 2-1. A core enhancement plug-in: lib/hash_extension.rb
class Hash
  def method_missing(method, *params)
    method_string = method.to_s
    if method_string.last == "="
      self[method_string[0..-2]] = params.first
    else
      self[method_string]
    end
  end
end

First, we convert the symbol to a string. Then we check if the last character of the string is an equals sign. Are we trying to do an assignment? If so, we invoke the normal hash assignment syntax using the string, minus the trailing =, as the key Otherwise, we return whatever value is in the hash keyed on the method name we used.

Testing

The stub test file will be created under your plug-in directory in test/my_plugin_test.rb and will look like this:

require 'test/unit'

class MyPluginTest < Test::Unit::TestCase
  # Replace this with your real tests.
  def test_this_plugin
    flunk
  end
end

As the comment says, you replace the dummy test with your own tests. A test file for our hash extension plug-in is shown in Example 2-2.

Example 2-2. Unit test for the hash_extension plug-in: test/hash_extension.rb
require 'test/unit'
require File.expand_path(
  File.join(File.dirname(__FILE__), '../../../../config/environment')
)

class HashExtensionTest < Test::Unit::TestCase
  def setup
    @h = Hash.new
  end

  def test_missing_value
    assert_nil @h.baz!
  end
  
  def test_assignment
    @h.foo = 'bar'
    assert_equal @h.foo, 'bar'
  end
end

Note the second through fourth lines in this example. We instruct the test code to load the environment.rb file, which was missing from the generated test code. This line boots up the Rails environment for us, and loads our plug-in (and any plug-ins it might depend on). This change allows us to run our test directly from the command line with the following command:

ruby vendor/plugins/hash_extension/test/hash_extension_test.rb

The output should look like this:

Loaded suite vendor/plugins/hash_extension/test/hash_extension_test
Started
..

Finished in 0.000609 seconds.

2 tests, 2 assertions, 0 failures, 0 errors

You can also run your tests with the built in rake command:

rake test:plugins

The problem with this command is that it runs tests for all of your plug-ins, which may not be what you want.

Using a core plug-in

Using a core plug-in is easy. You actually don’t have to do anything at all within your application for the code to take effect. The init.rb file is automatically loaded when the application starts, and the core functionality modification is applied straightaway. The plug-in just being there is all that’s necessary.

Why Extend Hash?

A criticism often leveraged against Ruby on Rails is that it’s slow. It’s true that the Ruby language is slower at some things than other languages used in web development contexts. That just means that you as a developer need to be aware of where Ruby and Rails can eat up valuable processor cycles. Then you can avoid those hot-spots by choosing less processor-hungry alternatives to some of the great Rails sugar when it doesn’t provide justifiable benefits.

Once cause of slowness I’ve found is in converting the results of a SQL query into ActiveRecord objects. A row from a SQL query result set is just a set of key-value pairs—in other words, a hash. ActiveRecord objects are great in that they come bundled up with methods that let you traverse object relationships, and they also contain methods you’ve written into the classes to facilitate custom behavior. But along with all the sugar that ActiveRecord provides comes a heavy overhead of creating the object itself.

Very often, especially when you’re selecting records to display a web page, all you need are the key-value pairs. A hash would suffice, and it turns out getting your results out of the database as hashes is much faster—more than 50% faster than requesting ActiveRecord objects.

Whereas a regular query that retrieves ActiveRecord objects looks like this:

MyObject.find(:all)

A query that returns hashes looks like this:

MyObject.connection.select_all(“select * from my_objects”)

True, you resort to SQL here, but in slow pages where you need to eke out that last bit of render-time performance, the trade-off can be worth it. In a test of retrieving 40 thousand objects on a MacBook Pro 2.33 Ghz Intel Core 2 Duo, the ActiveRecord approach took 7 seconds of Ruby time, while the hash method took 3.

The other caveat worth noting when replacing ActiveRecord queries with hash queries is that objects and hashes are accessed differently. You access objects with dot notation: my_object.foo versus my_object[‘foo’]. But that’s exactly the problem taken care of by our core Hash extension! Using this extension, you can cherry-pick slow ActiveRecord queries that aren’t using all the “extras” given to you by the ActiveRecord object itself, and swap out the query to boost performance.

Custom Extensions

Whereas core extensions are intended to modify classes that already exist, custom extensions allow you to modify the behavior of classes that have yet to be written. It’s best to think of this type of extension in two separate ways, based on levels of complexity and on how you intend to use it.

The first level is essentially nothing more than making a plug-in out of a mixin. A mixin is a module containing methods that gets “mixed in” to a class with the include keyword. All of the methods in the module become available to the class. These might be utility functions, or an interface to a remote system.

The second level builds on the first. The plug-in still contains a mixin module, but this mixin defines declarative class methods that further modify a class’s behavior when they are called. An example of this sort of plug-in is the well-known acts_as_list, which gives an ActiveRecord class list-like properties. When the acts_as_list plug-in is added to your Rails application, it is mixed in to ActiveRecord, giving your classes the ability to call the method acts_as_list. Upon doing so, that calling class is bestowed with a number of add-ons and modifications that give it its list-like behavior.

We’ll cover the more complicated case here. We’ll create a mix-in that defines a declarative method acts_as_animal, which when applied to a class, creates a noise method within the class. When we call it, it will print out the appropriate noise. The plug-in will also define a generic animal method that acts identically for any class that acts_as_animal: that is, to poop. The first step is to run:

./script/generate plugin acts_as_animal

Initialization template

The initialization template follows the same pattern as for our core extension plug-in: require ‘acts_as_animal’ Extension plug-in template The template for our mixin looks like this: module MyModule module ClassMethods # class methods and instance method generators go here def acts_as_my_module(params) class_eval do <<-DELIM def some_method # remember, string substitution # is OK here. e.g., #{params[:foo]} end DELIM end end module InstanceMethods # generic instance methods go here end def self.included(base) base.extend ClassMethods base.class_eval do include InstanceMethods end end end

There are three main sections. The first section defines class methods we want to add to classes that are extended by this module. These can be generic class methods, or declarative-style generator methods that are defined based on some parameters passed in.

Well-known generator methods are the acts_as variety, which add additional instance methods to the class. Above is a boilerplate for defining these sorts of methods. The trick is to invoke class_eval inside a class method. Inside this class_eval block, you can define dynamic methods that are based on the parameters passed to the acts_as method. Inside the method, we pass a string to class_eval so that we can perform string substitution.

Note that in order to avoid having to escape string delimiters, we use a special, multi-line string syntax. Text between <<-DELIM … DELIM is treated as a plain old string. In this case, our string is Ruby code intended to be evaluated by class_eval.

The second section defines the module InstanceMethods. There we place any generic instance methods that we want to add to a class that decides to act like our module.

You don’t need to modify the last section at all. Its purpose is to include the other two sections—one for class methods and one for instance methods—in the correct way. The method self.included(base) is hook code that runs when the module is actually included by another class.

The first thing we do is extend that class with the class methods we defined. Then, using class_eval, we include the instance methods module as if we had written that statement within the class itself.

Here is the extension plug-in template expanded for our animal example:

module Animal

  def self.included(base)
    base.extend ClassMethods
    base.class_eval do
      include InstanceMethods
    end
  end

  module ClassMethods
    def acts_as_animal(params)
      class_eval <<-STUFF
        def noise
          '#{params[:noise]}!'
        end
      STUFF
    end
  end
  
  module InstanceMethods
    def generic_animal_thing
      'poop'
    end
  end 
end

First, in the InstanceMethods module, we define a method generic_animal_thing, which when called, should return ‘poop’.

Then, in the class level code, we define the method acts_as_animal, which is at the heart of our plug-in. This method invokes class_eval again, to create a custom method in the class from which it is invoked.

Using a custom extension

Unlike core extensions, which are automatically loaded, to use a custom extension, you must first include it somewhere using the include keyword. Depending on what you are trying to extend, you might do this in a model class, a controller, or even environment.rb. The syntax is simple:

include Animal

The class methods defined will be available in the class that had scope where you placed this line. The instance methods will be available in all instances of that class.

Of course, you can also combine the two types of plug-ins covered here, core and custom, to add custom generator extensions to core functionality like ActiveRecord. This is how acts_as_list and other well known ActiveRecord behavior modifications work. To do so, just re-open the class and issue the include statement.

Testing

To test a custom extension module independently of application code, you must do two things. First, include the module again. Second, define a class in your test that will actually do the “acting as.” Below we define a Pig class and create an instance variable of that type in the setup method. We then test that the pig can make a noise and do the generic animal thing, that is, to poop:

require 'test/unit'
require File.expand_path(
 File.join(File.dirname(__FILE__), '../../../../config/environment')
)

include Animal

class Pig
  acts_as_animal :noise => 'oink'
end

class ActsAsAnimalTest < Test::Unit::TestCase
  def setup
    @p = Pig.new
  end

  def test_makes_proper_noise
    assert_equal @p.noise, 'oink!'
  end

  def test_generic_animal_thing
    assert_equal @p.generic_animal_thing, 'poop'
  end
end

We run the tests with:

ruby vendor/plugins/acts_as_animal/test/acts_as_animal_test.rb
And they both pass:

Finished in 0.000307 seconds.

2 tests, 2 assertions, 0 failures, 0 errors

Deployment

Creating the plug-in is the first step toward componentization of code and achieving the goals set forth at the beginning of this chapter. However, if you don’t have a way to share plug-ins between applications, and you resort to copying the plug-in directory from one project to the next, you aren’t getting the full benefit of a shared library.

There are many methods to share plug-ins across applications. One is to package the plug-in up as a gem, and install it on the systems where you want to use the plug-in. If you’re using subversion, another good method requiring very little setup is to use subversion’s “externals” property. This technique is covered below.

svn:externals

If you are using subversion to manage your source code, you can put your plug-ins in some standard location in your repository and then create an svn:externals reference in your application’s vendor/plugins directory that references the plug-in’s location.

For example, suppose we checked our hash_extension plug-in directory into subversion under the path plugins/acts_as_hash.

In any application that we want to have access to the plug-in, we modify the svn:externals property of the vendor/plugins directory. We do this like so:

cd vendor/plugins
svn propedit svn:externals .

This will bring up the editor defined in your EDITOR environment variable. The contents of the editor session will contain any externals properties already set; in this case, probably none. The format is:

directoryname [-rev] svnpath

So for our hash_extension plug-in, it might look like this:

hash_extension svn://rubyforge.org/var/svn/hash_extension/trunk

After saving and quitting, the following two commands will check out the files and set up the link in the repository:

svn update
svn commit
Chapter 1 : The Big Picture
Chapter 3 : Organizing with Modules