Chapter 16: Refactoring to Services

Back to Table of Contents.

Refactoring to Services

In the previous chapter, we created a basic XML-RPC service. All it could do was return a movie object. However, even with just this simple example behind us, by now you should have a solid understanding of what services are for, what considerations should go into their design, and how to build a simple one. In this chapter, we’ll take all of that foundation and use it to connect three applications together: two back-end services that talk to each other, and our integration test framework, which in this chapter will simulate a front-end talking to those services.

Figure 17-1 shows what our architecture will look like by the end of this chapter. First, we’ll build a simple orders service, which allows us to add new product to the product database and place orders. Then we’ll integrate our movies service with the new orders service. Whenever we add a showtime to the system, we’ll register that showtime as a product in the orders service so that tickets can be sold. We won’t actually create a front-end application, but we’ll simulate doing so by building new tests in our integration test framework, which tests services the way a real client would access them.

er_1701

Figure 17-1. Three applications connected in a service-oriented architecture

Our main goal is to learn how to connect systems together in an efficient and natural way. While Figure 17-1 provides a picture of how these pieces link together, Figure 17-2 shows a more accurate representation of how we physically lay out such an architecture in a production environment, complete with redundancy, load balancers, and database failover.

er_1702

Figure 17-2. The three applications from Figure 17-1, shown with redundancy, failover, and caching servers

An Orders Service

We begin with our new orders service. Previously, the tables relating to movie showtimes (movies, theaters, ratings, and the showtimes themselves) were in the same database as the tables corresponding to ordering (orders, ticket purchases, and a variety of payment processing tables). Even our simple example quickly grew to a large number of tables, and in the real world, the count would explode quickly and continuously.

Our orders tables also took on some domain knowledge of the movie-related schema. The table for order line items was called ticket_purchases, because we were selling movie tickets. But some day, we might want to use the same system for selling sodas and popcorn. Tightly coupling the movie-related tables with the order tables can block future paths, or make them more difficult to implement.

If we move our orders-related tables out into their own service database, with a separate service application that knows nothing about movies, we are more likely to design a system that will be amenable to future extensions, be it selling sodas, action-hero figurines, or flight reservations. Developers who are working on the orders service can also work more adeptly in their own domain, unhindered by unrelated concerns—both tables and application code—and address their primary focus of improving or extending the orders service.

Figure 17-3 shows the tables of the orders service, extracted from our original application. There are a few changes to note. First, we have added a product table. Because we don’t have access to any particular product table—in our application our product was contained within the movie_showtimes table—we need a place to keep track of products offered by external applications. This table allows a description to be stored, which can help debug problems. There’s also a column to store a quantity of items available.

Our purchased_tickets table has also been renamed to make it more generic. It is now called line_items, which has meaning regardless of what is being sold, and particularly to developers who are working on the orders service in isolation. The foreign key pointing to the products table—previously the movie_showtimes table—has also been moved from the orders table to the line_items table. Whereas in our movies application it made sense to purchase tickets for one movie at a time in any given order, in a generic ordering application, it doesn’t make sense to require that each type of item be placed in its own order. Instead, an order can have line items for many types of items, and each line item can have an associated quantity.

er_1703

Figure 17-3. The schema for our orders service

Example 17-1 shows the DDL for our new schema. It is much like the schema we built up in previous chapters; however, because we simplified the problem, we also simplified the solution. Now that we have a simple products table, which is domain-independent, our primary key is simply the id column.

Example 17-1. The DDL for the OrdersService schema

create language plpgsql;

create sequence products_id_seq;
create table products (
  id integer not null 
    default nextval('products_id_seq'),
  description text,
  price_cents integer not null,
  quantity integer not null,
  created_at timestamp with time zone
);

create table zip_codes (
  zip varchar(16) not null,
  city varchar(255) not null,
  state_abbreviation varchar(2) not null,
  county varchar(255) not null,
  latitude numeric not null,
  longitude numeric not null,
  primary key(zip)
);

create table addresses (
  line_1 varchar(256) not null
    check (length(line_1) > 0),
  line_2 varchar(256),
  city varchar(128) not null
    check (length(city) > 0),
  state varchar(2) not null
    check (length(state) = 2),
  zip_code varchar(9) not null
    references zip_codes(zip)
);

-- index on zip code, a common search criteria
create index address_zip_code_idx on addresses(zip_code);

create sequence credit_card_payments_id_seq;
create table credit_card_payments (
  id integer not null
    default nextval('credit_card_payments_id_seq'),
  card_number varchar(16) not null,
  type varchar(32) not null
   check (type in ('AmericanExpress', 'Visa', 'MasterCard')),
  expiration_month integer not null
   check (expiration_month > 0 and expiration_month <= 12),
  expiration_year integer not null
   check (expiration_year > 2008),
  primary key (id)
) inherits (addresses);

create sequence promotional_payment_id_seq;
create table promotional_payments (
  id integer not null
    default nextval('promotional_payment_id_seq'),
  promotion_id varchar(32) not null,
  created_at timestamp with time zone,
  primary key (id)
);

create sequence paypal_payment_id_seq;
create table paypal_payments (
  id integer not null
    default nextval('paypal_payment_id_seq'),
  email varchar(128) not null,
  auth_response text,
  created_at timestamp with time zone,
  primary key (id)
);

create sequence orders_id_seq;
create table orders (
  confirmation_code varchar(16) not null
    check (length(confirmation_code) > 0),
  credit_card_payment_id integer
    references credit_card_payments(id),
  promotional_payment_id integer
    references promotional_payments(id),
  paypal_payment_id integer
    references paypal_payments(id),
  primary key (confirmation_code)
);

alter table orders add constraint payment_xor check(
  (case when credit_card_payment_id is not null then 1 else 0 end + 
   case when paypal_payment_id      is not null then 1 else 0 end + 
   case when promotional_payment_id is not null then 1 else 0 end) = 1
);

create table line_items (
  order_confirmation_code varchar(16) not null
    references orders(confirmation_code),
  product_id integer not null,
  quantity integer
    check (quanitity > 0),
  position integer not null,
  purchase_price_cents integer not null
    check (purchase_price_cents >= 0),
  primary key (order_confirmation_code, position)
);

In our implementation of the OrdersService tables, we have no column to store an ID corresponding to the primary key of a showtime, our purchasable unit. Because our service is intended to be generic, it assigns its own IDs to each product in the products table. Clients of the OrderService add new products as they need to via the remote API, and obtain an ID from the OrdersService as a result. They then refer to products by that ID in future calls.

There are may benefits to having clients maintain the foreign key from the ProductDB among them:

  1. Multiple clients can add products of any type to the OrdersService and there will be no ID collisions.
  2. The OrdersService can remain neutral and does not need changes if new product types or new clients are added.

Figure 17-4 shows the sequence of how we’ll add products to the OrdersService, as initiated from an administrative control panel in the MoviesService application. Upon adding a new showtime, a service call is made to the OrdersService for the add_product method. The OrdersService creates a new record in the products table and returns a product_id. The MoviesService writes this value to the movie_showtimes table so it knows how to refer to it later when communicating with the OrdersService. A page is then returned to the administrative user, just as it would have been without an OrdersService in the picture. To the user of the system, nothing has changed at all.

er_1704

Figure 17-4. Adding a product to the OrdersService

To facilitate this interaction, we need to define an API for the OrdersService. First we need to create a plugin, orders_service_shared, as we did in the last chapter for the movies service:

./script/generate plugin orders_service_shared

Next we define the API. Example 17-2 shows our orders_api.rb file, which would be placed in the lib/ directory of the client plugin under vendor/plugins. Although an orders service has the potential to have a large number of callable methods, we’ll define two that suffice for our application. The first, add_product, is called from the movies service when a new showtime is added. It is the API method used in the example in Figure 17-4. The second method, place_order, is called by a front-end during a user’s checkout process.

Note that in our API for placing an order, we just pass the line items. We don’t need to create an explicit object for the order itself to place the order. It’s implicit in the method name that what we’re doing is creating an order out of our line items. This is in contrast to a REST approach, where we would be required to define a resource for an “order” so that we could access it at a particular RESTful URI and PUT or POST the order there. However, we clearly don’t need the overhead of such an interface, which would complicate the processing to be done on both eh client and the server. Here, we have an array of line items, each of which references a product_id, as returned from the add_product API method, and a quantity. The array itself is the order, and the return value of the method invocation is an object containing the order confirmation code, and a final price.

Example 17-2. {plugin}//lib/orders_api.rb

class OrdersApi < ActionWebService::API::Base
   
   api_method(:add_product,
              :expects => [{:description => :string},
                           {:quantity => :int},
                           {:price_cents => :int}],
              :returns => [:product_id => :string])

   api_method(:place_order,
              :expects => [{:items => [Logical::LineItem]},
                           {:payment => Logical::Payment}],
              :returns => [Logical::OrderPlaced])
  
end

The next step is to define the logical model for the OrdersService. Example 17-3 shows our classes: LineItem, Order, Address, CreditCard, Payment, and OrderPlaced. Interestingly, at this stage we have no need for a Product class. One might be necessary given a richer API, but at the moment, the context of our methods themselves can make some logical models unnecessary.

Note that previously, our sense of quantity was implicit in the number of rows in the tickets_purchased table, because each order could consist tickets for at most one showtime. However, our generic OrdersService allows for more flexible orders made up of varying products. Therefore, we’ve added a quantity attribute to the LineItem class. There will only be one line item per product, but more than one of each product can be purchased and recorded in that line item record.

Note that although the line_items table contains a reference to the orders table, the relationship is reversed in our logical model. Here, the Order class contains an array of LineItem objects. It is frequently the case that relationships such as this one are reversed between the logical and physical models; the former is object-oriented and designed to be a natural representation of the world, while the latter is designed to eliminate duplicate data and maintain referential integrity.

Most of our other classes are fairly straightforward and based closely on the physical models underlying them. One exception is the OrderPlaced object, which is the result of calling place_order. It contains the confirmation number, which could be used to retrieve an order after it has been placed. We have also eliminated the subclasses of Payment and CreditCard here, opting instead to differentiate the payment and credit cards types based on class constants. The logical object model gains simplicity, while the physical model still contains the complexity necessary to actually process the payments and record the data correctly in the database.

Example 17-3. {plugin}/lib/orders_service.rb

module Logical
  
  class LineItem < ActionWebService::Struct
    member :product_id,            :integer
    member :quantity,              :integer
  end

  class Order < ActionWebService::Struct
    member :line_items,            [LineItem]  
  end
  
  class Address < ActionWebService::Struct
    member :line_1,                :string
    member :line_2,                :string
    member :city,                  :string
    member :state,                 :string
    member :zip_code,              :string
  end

  class CreditCard < ActionWebService::Struct
    AMERICAN_EXPRESS = 'american_express'
    MASTERCARD = 'master_card'
    VISA = 'visa'

    member :type,                  :string
    member :card_number,           :string
    member :expiration_month,      :integer
    member :expiration_year,       :integer
  end
  
  class Payment < ActionWebService::Struct
    PAYPAL = 'paypal'
    PROMOTIONAL = 'promotional'
    CREDIT_CARD = 'credit_card'

    member :address,               Address
    member :type,                  :string
    member :credit_card,           CreditCard
  end    
    
  class OrderPlaced < ActionWebService::Struct
    member :confirmation,          :string
    member :price,                 :int
  end

end

With the API declared and our objects defined, we can now implement the API. Example 17-4 shows the orders_service.rb file, which we place in app/controllers/services/.

First, we declare which API declaration class we are implementing, here OrdersApi from Example 17-3. Next we define the add_product method. This method just passes parameters through to the physical model Product class, and returns the resulting product ID.

The place_order method is interesting because the service API does not naturally match the physical layer data types. An array of line items and an object describing the payment are passed in. These must be translated into a payment subclass as returned from the Payment factory constructor method, and an Order object, consisting of LimeItem objects that require looking up Product objects to determine the correct purchase price.

Example 17-4. controllers/services/orders_service.rb

class OrdersService < ActionWebService::Base
  web_service_api OrdersApi
  
  def add_product(description, quantity, price_cents)
    p = Physical::Product.create!(
      :description => description,
      :quantity => quantity,
      :price_cents => price_cents)
    return p.id
  end
  
  def place_order(items, payment)
    c = Physical::Payment.new_payment(payment.type, payment.credit_card.type)
    c.card_number = payment.credit_card.card_number
    c.expiration_month = payment.credit_card.expiration_month
    c.expiration_year = payment.credit_card.expiration_year
    line_items = items.collect {|li|
      p = Physical::Product.find(li.product_id)
      Physical::LineItem.new(
        :product => p,
        :quantity => 1,
        :purchase_price_cents => p.price_cents)
    }
    o = Physical::Order.create(:payment => c, :line_items => line_items)
    Logical::OrderPlaced.new(:confirmation => o.confirmation_code, :price => o.order_total)
  end
end

While this may seem onerous, the translation layer is actually a boon as your site and business grow. Often, a young designer’s first service API will exactly match the physical models’ API. Remember that the physical models and data layer are geared toward ensuring referential integrity and data correctness, but the logical models and service API are intended to feel natural to clients. If the two are the same, either your physical models are too natural, and are not likely to be in domain-key normal-form, or the service API is too rigid, reproducing the fine-grained ActiveRecord API through to clients, and pushing too much business logic up through to the client layer.

Even if your logical and physical models are initially very similar, over time they will diverge, and having a translation layer like the one shown in Example 17-4 gives you a natural place to translate one to the other. This is crucial if either your data model is changing, but you don’t want clients to be aware of the change, or vice versa.

Next, we must define a standard Rails controller intended to pass control along to our service when a request is made to /orders_service/api. The OrdersServiceController, which implements this behavior, is shown in Example 17-5. Just as in the previous chapter, we create a scaffolding for testing, available at /orders_service/invoke.

Example 17-5. app/controllers/orders_service_controller.rb

class OrdersServiceController < ApplicationController
  web_service_dispatching_mode :layered
  web_service_scaffold :invoke
  
  web_service :orders, OrdersService.new
end

Also similar to our last service, we create a client class that we’ll use to access the OrdersService from other applications. The client library is shown in Example 17-6. It requires that a configuration initializer be placed in config/initializers. This file would take the same form as the initializer file created in the previous chapter for MoviesService, but it should define the location of the OrdersService API as the constant OrdersServiceClient::ENDPOINT_URL.

Example 17-6. {plugin}/lib/orders_service_client.rb

require 'singleton'
class OrdersServiceClient
  include Singleton

  def initialize
    # URL and TIMEOUT_SECONDS are defined in 
    # config/initializers/orders_service_client_config.rb
    @client = ActionWebService::Client::XmlRpc.new(
      OrdersApi, ENDPOINT_URL,
      {:handler_name => 'orders', :timeout => TIMEOUT_SECONDS}
    )
  end
  
  def self.method_missing(method, *args)
    self.instance.send(method, *args)
  end

  def method_missing(method, *args)
    @client.send(method, *args)
  end
end

In Example 17-7, code from the previous chapter to “help” the Rails auto-loader is reproduced. This code is necessary for Rails to find our logical model classes, as well as our service implementation classes. The pertinent lines are in bold.

Example 17-7. application.rb autoloader overrides

Dir["#{RAILS_ROOT}/app/models/physical/*.rb"].each { |file|
  require_dependency "physical/#{file[file.rindex('/') + 1...-3]}"
}

Dir["#{RAILS_ROOT}/app/models/logical/*.rb"].each { |file|
  require_dependency "logical/#{file[file.rindex('/') + 1...-3]}"
}

Dir["#{RAILS_ROOT}/app/controllers/service/*.rb"].each { |file|
  require_dependency "service/#{file[file.rindex('/') + 1...-3]}"
}

We now have all of the files necessary for our second service, OrdersService. The next step is to test the two API methods we created. Example 17-8 shows two tests, one for each method. In the first method, we simply test adding a product using the add_product API method. After adding the “My test product” product, we assert that a valid product ID was returned.

The second method tests the process of placing an order. First, we add a product for which we will place an order. Then we create a line item corresponding to a purchase of the product. We create objects for all of the classes necessary to create a payment—an address, a credit card, and a payment object—and finally we call place_order with our line item repeated four times. We assert that the order succeeded, and that the purchase price returned is what we expect.

Example 17-8. Integration tests for OrdersService

require File.dirname(__FILE__) + '/../test_helper'

class OrdersServiceTestCase < Test::Unit::TestCase

  def test_add_product
    new_id = OrdersServiceClient.add_product(
      "My test product",
      50,
      1000)
    assert new_id
  end
  
  def test_place_order
    p = OrdersServiceClient.add_product(
      "My test product",
      50,
      1000)
    li = Logical::LineItem.new(
      :product_id => p,
      :quantity => 1
    )
    ad = Logical::Address.new(
      :line_1 => '123 Foobar Lane',
      :city => 'Cambridge',
      :state => 'MA',
      :zip_code => '02139'
    )
    cc = Logical::CreditCard.new(
      :card_number => '55555555555555',
      :expiration_month => '12',
      :expiration_year => '2015',
      :type => Logical::CreditCard::AMERICAN_EXPRESS
    )
    payment = Logical::Payment.new(
      :address => ad,
      :type => Logical::Payment::CREDIT_CARD,
      :credit_card => cc
    )
    result = OrdersServiceClient.place_order(
      [li, li, li, li], payment
    )
    assert result.confirmation
    assert result.price == 4000
  end

end

The results of our integration tests are shown in Example 17-9. Our tests pass.

Example 17-9. Result of running the OrdersService integration tests

chakbookpro: chak$ ruby test/integration/orders_service_test_case.rb 
Loaded suite test/integration/orders_service_test_case
Started
..
Finished in 0.344523 seconds.

2 tests, 3 assertions, 0 failures, 0 errors

Integrating with the Movies Service

Because we’ve ripped our application in two, we need some way to link the data in the two applications back together. I’ve already alluded to the idea that a MovieShowtime object is the “product” of the MoviesService, and that the MoviesService needs to obtain and track product IDs returned from the OrdersService. To accomplish this, we need to add a product_id column to the movie_showtimes table, as shown in Example 17-10. Note that the column type is text, rather than integer. Just as OrdersService strives to maintain independence from its clients, so too should clients add a layer of abstraction between themselves and the services they consume. Although the product IDs are integers today, they might not be forever. Since there is no explicit reference to maintain within the movies database, the schema can be built to be generic enough to support today’s as well as tomorrow’s needs.

Example 17-10. Modifications to the movie_showtimes table to support an orders service

create table movie_showtimes (
  id integer not null 
    default nextval('movie_showtimes_id_seq'),
  movie_id integer not null
    references movies(id),
  theatre_id integer not null
    references theatres(id),
  room varchar(64) not null,
  start_time timestamp with time zone not null,
  primary key (id),
  product_id text,
  unique(movie_id, theatre_id, room, start_time),
  foreign key (theatre_id, room)
    references auditoriums(theatre_id, room) initially deferred
);

We also need to instrument the registration of a product and the retrieval of a product ID whenever a movie showtime is added. Example 17-11 shows how we hook into the ActiveRecord observer before_save to call the remote add_product method before saving a showtime, storing the returned product ID for later use.

Example 17-11. physical/movie.rb

module Physical
  class MovieShowtime < ActiveRecord::Base
    def before_save
      self.product_id = OrdersServiceClient.add_product(
        self.movie.name,
        self.auditorium.seats_available,
        1000)
    end
  end
end

Before moving on, we should test that saving a showtime does in fact register the showtime as a product and retrieve a product ID. Normally this would seem like a unit test, but since we must contact a running OrdersService to complete the test, we instead write it as an integration test within the MoviesService application, in the test/integration directory.

Example 17-12 shows our test. The portion of the test that creates a movie, theater, and auditorium so that we can create a showtime is portioned off in the setup method. Our test just creates a showtime, and asserts that the saved showtime has product_id, even though none was ever set explicitly in the test.

Example 17-12. Integration test at test/integration/showtime_create_test_case.rb

require File.dirname(__FILE__) + '/../test_helper'

module Physical
  class ShowtimeCreateTestCase < Test::Unit::TestCase
  
    def setup
      @m = Movie.create!(
        :name => 'Casablanca',
        :length_minutes => 120,
        :rating => Rating::PG13)
      @t = Theatre.create!(
        :name => 'Kendall Cinema',
        :phone_number => '5555555555')
      @a = Auditorium.create!(
        :theatre => @t,
        :room => '1',
        :seats_available => 100)
    end

    def test_getting_product_id
      ms = MovieShowtime.create(
        :movie_id => @m.id,
        :theatre_id => @t.id,
        :room => '1',
        :start_time => Time.new)
      assert ms.product_id
    end
  
  end
end

Other Considerations

Here we’ve constructed a simple interconnection between two services. Since we have split our database, it’s no longer possible for the MoviesService to easily determine if a showtime is sold out. We could deal with this in a number of ways. One possibility would be to add a quantity_remaining API method to the OrdersService, which could take as a parameter an array of product IDs and return, for each one, the number of seats left for sale. Although this solution is simple and straightforward, it also requires two chained service calls to display movie showtimes on our front-end website, which is contrary to our SOA guidelines. It also requires the call be made every time movie showtimes are requested. Figure 17-5 shows the steps involved.

er_1705

Figure 17-5. Chained requests to find available movie showtimes

Another possibility is to create a callback mechanism such that whenever a product sells out, the OrdersService will make an XML-RPC call back to the product provider, in this case the MoviesService, to notify it that a product has sold out. To facilitate this, the MoviesService would need to maintain a sold_out Boolean in the database so that it can filter out sold old showtimes when servicing requests to the front-end. The OrdersService would need to know something about the product providers, so that it could make an appropriate callback. Figure 17-6 shows the entire process, with a new table, providers, containing the callback URLs product provider XML-RPC services.

This push rather than poll interface dramatically cuts down on the number of requests served over the lifetime of the app. Instead of a quantity_remaining requests for every page view (steps 3–5 in Figure 17-5), we instead have one sold_out request per sell-out event, which is rare compared to page views. In Figure 17-6, the place order process is labeled as times A through D. This is because they do not occur in sequence with the numbered steps. And further, if a product does not sell out as a result of the sale, steps C and D are skipped. For a product with a quantity of N units available, the sold_out method will only be called once if the product is sold N times.

Figure 17-7 shows a sales funnel depicting the progressively smaller and smaller numbers of users who trigger each successive event. Compared with page views, there must be fewer purchases. Compared with purchases, the number of sell-out events must also be smaller. In fact, a real sales funnel would likely be much wider, and have most steps squashed down in the bottom of the funnel. Although these sorts of metrics are highly dependent on the business, drop-off rates frequently result in one or two orders of magnitude of decline in numbers at each step. Business types use this sort of analysis to predict revenue and make business decisions. You can also use the same analysis to find the right places in the application to focus optimization efforts.

er_1706

Figure 17-6. Marking showtimes as sold out via a callback

In a generic product service, where we are servicing multiple product providers, we are likely to want both mechanisms. Internally, the callback solution is the most elegant, but for external clients it is challenging to implement due to firewalls, security policies, and business goals. For the rest of this chapter, we’ll assume that the callback mechanism and the showtimes returned from the MoviesService are, in fact, available.

er_1707

Figure 17-7. A sales funnel: progressively fewer users trigger each successive event

MoviesService Object Model

We’ve now connected our two back-end services together. As we add showtimes to the MoviesService, we automatically add the showtimes as products in the ProductsService. And we have devised a mechanism for the ProductsService to inform the MoviesService when a showtime is out of seats.

The last step is to provide an API for our thin, user-facing front-end website to retrieve movies, theaters, and showtimes for display. But designing an API can be tricky. Before rushing forward, let’s discuss some of the factors that will affect our design decisions.

In Chapter 16, we defined four best practices for designing an API. The first was “Send everything you need.” We could interpret this literally, and simply create one service API method per page type in our application, and literally have the back-end construct a data structure containing all needed data, in the perfect format, for every page.

While this would certainly work, it ties the front and back-ends together a bit too tightly. The same reasoning we discovered in the discussion of module dependencies in Chapter 5 applies here as well. If the front-end depends on the back-end services, and not vice versa, then the back-end services should be as blind to the front-end as possible. It should be possible to create another, differently behaving front-end site without resorting to creating an entirely new API on the back-end to support it.

The other extreme would be to create a finely grained API, from which it would be possible to retrieve all data from the back-end in small pieces and then put it together in the client in whatever structure we desire. This moves toward the approach taken by REST web services, where bits of data are moved from service to client, and all application logic taken place in the client. While this is flexible, it means that each client must re-implement the business logic. It also implies a large degree of overhead in making tens or hundreds of service calls rather than one.

Most good designs are a balancing act, and this one will be no different. We can compromise between the two extremes and come up with something in the middle, where the API is flexible enough so that a handful of methods can be used to generate a variety of different front-end pages, but not so fine-grained that the front-end must do all of the heavy lifting. We’ll seek to move obvious functionality into the service layer, so that we do not have to repeatedly implement that functionality in each front-end we might write. But we won’t go so far as to move all logic into the back-end, leading to an ultimately thin front-end, but a rigid back-end. We’ll hope that our API satisfies the 80/20 rule: 80% of future desires will be immediately implementable against our API, and the other 20% are still possible, possibly requiring tweaks to the API.

To come up with our API, we need to imagine the types of pages that will be required within our front-end. Usually, this is an easy task, because the product—the website—will already have been defined by a product team. As engineers, it’s our team’s job to translate those functional requirements into a technical design, including the so-called nonfunctional requirements, which inform the trade-offs we will make when designing a contract between a back-end service and a front-end website. Although users of our site and members of the product team will never directly see the API contract, they may feel it as sluggish or blazing site performance, or in the product team’s case, in follow-on rollouts occurring quickly or at the speed of molasses.

Let’s play the part of the product team now, and define the pages we’ll have on our site. We’ll have:

  1. A home page, listing the current movies, filterable by zip code.
  2. A page for a current movie, listing showtimes in a requested area, by theater.
  3. A page for a theater, listing the movies playing there, with showtimes.

The first page, the home page, could be created with an API called current_movies, which returns an array of Movie objects.

The second and third pages return similar data—showtimes—but in different groupings. The second page is for a single movie, grouped by theater, whereas the third page is for a single theater, grouped by movie. We could certainly take the easy route and lay out a separate object model for each grouping, or we can come up with a more generic object model that could satisfy both pages, and hopefully other future needs as well.

Again, we recognize that the main unit of data we are requesting is the showtime. In addition to a different grouping, our two page types showing showtimes also filter differently. So we have two challenges to overcome: first, parameterized filtering; and second, customizable grouping. Because these are two separate challenges, we’ll deal with them separately, and, as it happens, we’ll solve each problem in a unique way.

The first problem, parameterized filtering, is the easier of the two. To define a forward-looking, generic API, we should provide a way to filter on any data we have: movie, theater, or location, and let the client decide how generic or specific of a query to request. This would allow not only the two pages we have thought up so far to be created, but almost any page focused around movie showtimes.

Our API declaration might look something like this:

api_method(:get_showtimes,
           :expects => [{:zip_code => :string},
                        {:theatre_id => [:int]},
                        {:movie_id => [:int]}],
           :returns => [:showtimes => Logical::ShowtimesResult])

Here we accept one zip code, an array of theater IDs, and an array of movie IDs. We allow any of the parameters to be empty, in which case that parameter does not contribute to the filter set.

Now we are left to define the Logical::ShowtimesResult object. How do we structure this object to be both generic enough for the two pages that we know about today, and generic enough for the pages of tomorrow that we haven’t heard about yet?

We have a problem here. Not only is it challenging to create such a generic object, but also, we have said time and time again that the purpose of the logical model is not to be overly generic, but to be specific to the application domain, hiding highly normalized database implementations from the view of the front-end application developer. So at we want a generic return type to support future needs we don’t know about yet, and at the same time we want custom, nonspecific return types to ease application development. Seems like we are in quite a bind.

To solve both problems at once, we’ll use a technique that may seem like out-and-out trickery. Recall that our service client plugin is a single package of code, which is available on both the back-end service side and on the front-end client side. So far, our plugin has only contained descriptions of the service—all implementation resides within the service code of the back-end application. If we want to find a place to add code that the back-end does not really know about, but which the front-end believes is a natural part of the back-end, we can add this code in the plugin. In the plugin, we can add façade columns that were not in the original XML response, as long as those façade columns can be made up from data that was actually returned in the XML. We can add entirely new methods to the API, as long as those new methods are composed of existing methods. We can even create new return types that were not ever returned by the back-end service.

Since we now know customizing a generic object will be possible, we’ll start by defining the generic object. Example 17-13 shows our initial additions to the MoviesService logical model, which will make up our generic ShowtimesReturn object.

First, we define our Showtime object. It contains a Movie, a Theatre, and start_time and auditorium fields. The first two are objects defined elsewhere in the header file, and the last two are standard datatypes.

Next, we define a “lighter” version of this object, called ShowtimeLight. This version returns the same data, but rather than return entire Movie and Theatre objects, it instead returns the IDs of these objects, by which they can be retrieved in some other way.

This then leads to our ShowtimeResult object declaration. This object consists of an array of ShowtimeLight objects, and separate arrays for the Movie and Theatre objects referenced by the showtimes. In essence, we have normalized the return value. We certainly could have returned full-fledged Showtime objects, but this approach has two benefits:

  1. The amount of data to be serialized, sent over the network, and deserialized is much smaller, which will result in faster user-perceived performance.
  2. Because we have arrays of the Movie and Theatre objects in play, we don’t have to process the array of Showtime objects to “discover” this information.

Thus, each movie and theater in the result set will only be serialized to XML once, and in other places, where those objects would have appeared if full Showtime objects had been used, their IDs will appear instead.

Example 17-13. Additional MoviesService logical model definitions

module Logical
  class Showtime < ActionWebService::Struct
    member :movie,                 Movie
    member :theatre,               Theatre
    member :start_time,            :datetime
    member :auditorium,            :string
  end

  class ShowtimeLight < ActionWebService::Struct
    member :movie_id,              :int
    member :theatre_id,            :int
    member :start_time,            :datetime
    member :auditorium,            :string
  end
  
  class ShowtimesResult < ActionWebService::Struct
    member :movies,                [Movie]
    member :theatres,              [Theatre]
    member :showtimes_light,       [ShowtimeLight]
  end
end

With these base datatypes out of the way, we can now turn our attention to defining the custom methods that we’ll layer on top of get_showtimes. Example 17-14 shows two methods we might like to have in our API, but which we can instead base on the generic get_showtimes method. Each is tailored to the pages we need to display. The first, get_movie_showtimes_by_movie_and_location, returns the showtimes results grouped by theater. The second, get_movie_showtimes_by_theatre, returns showtimes grouped by movie. The API declarations for these methods might look something like what is shown in Example 17-15. In fact, we can add these declarations as comments, so someone reading the declarations file will know these additional methods exist as well.

Example 17-14. API wrapper methods in the MoviesService client plugin

class MoviesServiceClient
  def get_movie_showtimes_by_movie_and_location(movie_id, zip_code)
    result = self.get_showtimes(zip_code, [], movie_id)
    result.group_by_theatre
  end
  
  def get_movie_showtimes_by_theatre(theatre_id)
    result = self.get_showtimes([], theatre_id, '')
    result.group_by_movie
  end
end

From the code in Examples 17-14 and 17-15 it should be clear that we have additional methods to define and logical models to declare. We need to define the methods group_by_theatre and group_by_movie in the client code. Each of these will return data in a new format, as defined by the ShowtimesByTheatre and ShowtimesByMovie logical model types. We still need to declare them, pending our decision of their structure.

Example 17-15. API declarations for our new methods

 #  commented because these methods are implemented in the client
 #  api_method(:get_showtimes_for_theatre,
 #             :expects => [{:theatre_id => [:int]}],
 #             :returns => [:showtimes => Logical::MovieShowtimes])

 #  api_method(:get_showtimes_for_movie_and_location,
 #             :expects => [{:zip_code => :string},
 #                          {:movie_id => [:int]}],
 #             :returns => [:showtimes => Logical::TheatreShowtimes])

Example 17-16 shows the two new classes we define to support our two client-side API methods. The first, MovieShowtimes, contains a Movie object and an array of ShowTime objects. Note that the get_showtimes_for_theatre method returns an array of these objects. In essence, an array of these objects simulates a hash structure (although without O(1) random access). We do this because hashes do not have direct support in ActionWebService. We might have more naturally returned a hash where the keys were Movie objects and the values were arrays of ShowTime objects, but this does the trick. This supports our method get_showtimes_for_theatre; each instance of the array contains a Movie object and all of its showtimes.

The TheatreShowtimes class builds on this object; TheatreShowtimes is appropriate when we expect more than one theater in our results. In the same way as MovieShowtimes, an array of these simulates a hash with Theatre objects as the key and an array of MovieShowtimes as the value. Thus, an array of objects of this type returns showtimes grouped first by theater, then by movie. Our API method, get_showtimes_for_movie_and_location, would return one TheatreShowtimes object per theater. Each would contain the appropriate Theatre object, and a single MovieShowtimes object; because the movie has already been constrained, there is only one element in the array.

Example 17-16. Logical model class definitions for client-side use

module Logical
  # showtimes for a movie in a single theatre
  class MovieShowtimes < ActionWebService::Struct
    member :movie,                 Movie
    member :showtimes,             [Showtime]
  end
  
  # showtimes in a theatre, grouped by movie
  class TheatreShowtimes < ActionWebService::Struct
    member :theatre,               Movie
    member :movie_showtimes,      [MovieShowtimes]
  end
end

Now that we have a complete understanding of what our API looks like, including the structure of return types, we can turn to implementation. We’ll start at the bottom and work our way back up to the API layer.

Example 17-17 shows a new method, get_by_zip_theatre_movie, in the logical Showtime class. This method accepts a zip code (which can be an empty string), and arrays of movie and theater IDs, each of which can be empty. If the parameters are not empty, they are added to the conditions of the find SQL we will execute. Note that we take care to separate the text of our SQL from the values to be inserted as bind variables. The helper method bind_for_array helps us accomplish this.

After retrieving the records from the CurrentMovieShowtimes view-backed model class, we reformat the ActiveRecord data into the structure of our return type, a ShowtimesResult object.

Example 17-17. Logical model method to return a list of current showtimes, by zip, theater, or movie combination; all parameters can be empty

module Logical
  class Showtime < ActionWebService::Struct
    def self.get_by_zip_theatre_movie(zip_code, theatre_ids, movie_ids)
      conditions_sql = Array.new
      conditions_vars = Array.new
      if !zip_code.empty?
        conditions_sql << "miles_between_lat_long(
          (select latitude from zip_codes where zip = ?),
          (select longitude from zip_codes where zip = ?),
          latitude, longitude) < 15"
        conditions_vars.concat [zip_code]*2
      end
      if !theatre_ids.empty?
        conditions_sql << "theatre_id in (#{bind_for_array(theatre_ids)})"
        conditions_vars.concat theatre_ids
      end
      if !movie_ids.empty?
        conditions_sql << "movie_id in (#{bind_for_array(movie_ids)})"
        conditions_vars.concat movie_ids
      end
      conditions_sql << "current is true and sold_out is false"
      psts = Physical::MovieShowtimeWithCurrentAndSoldOut.find(:all,
        :select => [:id, :movie_id, :theatre_id, :latitude, :longtitude],
        :include => [:movie, :theatre],
        :conditions => [conditions_sql.join(" and "), *conditions_vars])
      
      m_hash = Hash.new
      t_hash = Hash.new
      st_array = Array.new
      for pst in psts do
        m_hash[pst.movie_id] ||= Movie.get(pst.movie_id)
        t_hash[pst.theatre_id] ||= Theatre.get(pst.theatre_id)
        st_array << ShowtimeLight.new(
          :movie_id => pst.movie_id,
          :theatre_id => pst.theatre_id,
          :start_time => pst.start_time,
          :auditorium => pst.room
          
        )
      end
      ShowtimesResult.new(
        :movies => m_hash.values,
        :theatres => t_hash.values,
        :showtimes_light => st_array
      )
    end
    
    def self.bind_for_array(array)
      (['?']*array.size).join(",")
    end
    
  end
end

In the service implementation class shown in Example 17-18, we define the get_showtimes method that is part of our external API. This method simply calls the method we defined in Example 17-17 within the Showtime class. Continuing with the idea of “skinny controllers, fat models,” we keep the implementation details within the logical model class, rather than the API implementation class, which is really nothing more than a controller.

Example 17-18. The implementation of get_showtimes

class MoviesService < ActionWebService::Base
  web_service_api MoviesApi
 
  def get_showtimes(zip_code, theatre_id, movie_id)
    Logical::Showtime.get_by_zip_theatre_movie(zip_code, theatre_id, movie_id)
  end  
end

With this much code in place, now we are able to call the get_showtimes API method. Example 17-19 shows our integration test, written in the integration test framework application. For this test, we again assume the presence of some test data in our running service’s database. Therefore, we simply call the method, then verify the structure of the result. A more rigorous test would first insert the data into the database to ensure we get back exactly what we put in.

Example 17-19. Integration test for the get_showtimes API method

require File.dirname(__FILE__) + '/../test_helper'
class MovieServiceGetShowtimesTestCase < Test::Unit::TestCase
  def test_get_showtimes
    result = MoviesServiceClient.get_showtimes('02139', [7,12], [17,20,64])
    assert result.class == Logical::ShowtimesResult
    assert result.movies.class == Array
    assert result.theatres.class == Array
    assert result.showtimes_light.class == Array
    assert result.movies.size > 0
    assert result.theatres.size > 0
    assert result.showtimes_light.size > 0
    for movie in result.movies do 
      assert movie.class == Logical::Movie
    end
    for theatre in result.theatres do
      assert theatre.class == Logical::Theatre
    end
    for showtime in result.showtimes_light do
      assert showtime.class == Logical::ShowtimeLight
    end
  end
end

The test passes, as shown below. Note that the number of assertions shown will be different depending on the test data you have in your database. Also note that the test will fail until the service call actually returns data:

ChakBookPro: chak$ ruby test/integration/movies_svc_test_case.rb
Loaded suite test/integration/movies_svc_test_case
Started
.
Finished in 1.442291 seconds.

1 tests, 64 assertions, 0 failures, 0 errors

With this method working, we can now write our wrapper methods in the client plugin on top of it. Although get_showtimes is extremely generic, these wrapper methods are free to restrict the inputs—and do. While generic methods can be very powerful, they can also be confusing, because the interface is often more complex than it needs to be for any given task. Our two methods, then, restrict the parameter list only to relevant data, and return the output grouped in the way that makes the most sense for the inputs.

Because the procedure is the same, but the code is long, we’ll only go through the process of creating and testing one of our wrapper methods: get_movie_showtimes_by_theatre. It’s the fact that we can have other wrapper methods, and many more as well, that’s the important takeaway. Example 17-20 shows the implementation of the wrapper method, within the MoviesServiceClient class.

The get_movie_showtimes_by_theatre method has one function, and that is to restrict the parameter list to a single theater ID. Within the method, the parameter list is re-expanded and passed to get_movie_showtimes. The result is passed on to group_by_movie, where the bulk of our implementation takes place.

Recall that the purpose of this method is to retrieve movies for a single theater, so we group showtimes within the scope of that theater by movie. The structure of this method is analogous to that of the get_by_zip_theatre_movie method within the logical Showtime class. There, an array of ActiveRecord objects was transformed into a different logical model structure appropriate for the get_showtimes method. Here, we transform that result into one more suitable for our new wrapper API method.

Example 17-20. Client plugin wrappers for get_showtimes

class MoviesServiceClient
  def get_movie_showtimes_by_theatre(theatre_id)
    result = self.get_showtimes('', [theatre_id], [])
    group_by_movie(result)
  end

  protected
  
  # assumes only one theatre, if multiple, not
  # organized by theatre in any way
  def group_by_movie(showtimes_result)
    movies_hash = Hash.new
    theatres_hash = Hash.new
    showtimes_hash = Hash.new
    for movie in showtimes_result.movies do
      movies_hash[movie.id] = movie
    end
    for theatre in showtimes_result.theatres do
      theatres_hash[theatre.id] = theatre
    end
    result = Array.new
    for showtime in showtimes_result.showtimes_light do 
      showtimes_hash[showtime.movie_id] ||= Array.new
      showtimes_hash[showtime.movie_id] << Logical::Showtime.new(
        :movie => movies_hash[showtime.movie_id],
        :theatre => theatres_hash[showtime.theatre_id],
        :start_time => showtime.start_time,
        :auditorium => showtime.auditorium
      )
    end
    showtimes_hash.collect{|movie_id, showtimes|
      Logical::MovieShowtimes.new(
        :movie => movies_hash[movie_id],
        :showtimes => showtimes
      )
    }
  end
end

You should be able to trace this code to see how we transform one structure to another. Figure 17-8 illustrates the resulting structure. It’s important to note that even though we appear to be repeating theater and movie information frequently within our resulting data structure, in fact, we are not. Rather than duplicating that data, we are duplicating references to single locations of our data. Over time, we may decide to move wrapper API methods from the client into the server. However, in doing so, we lose the ability to duplicate references instead of data. While the benefit of having the method defined on the server is that nonRails clients can have access to the same logic through a single API method name, the trade-off is that the amount of data to be serialized, transferred, and deserialized increases for all client types.

er_1708

Figure 17-8. The MovieShowtimes array result structure

We can now write a test for our wrapper method, which we’ll place in the integration test framework’s test suite. Example 17-21 shows our unit test. For this test, we again assume the presence of some test data in our running service’s database, and we only verify the structure of the result.

Example 17-21. Integration test for the get_movie_showtimes_by_theatre wrapper method

require File.dirname(__FILE__) + '/../test_helper'
class MovieServiceGetShowtimesByTheatreTestCase < Test::Unit::TestCase
  def test_get_showtimes
    result = MoviesServiceClient.get_movie_showtimes_by_theatre(7)
    assert result.class == Array, "Result is not an array"
    assert result.size > 0, "Result is empty"
    for by_movie in result
      assert by_movie.class == Logical::MovieShowtimes,
        "Array elements are not MovieShowtimes"
      assert by_movie.movie.class == Logical::Movie, "movie was not a Movie"
      RAILS_DEFAULT_LOGGER.debug("class is: #{by_movie.showtimes.class}")
      assert by_movie.showtimes.class == Array, "showtimes was not an array"
      assert by_movie.showtimes.size > 0, "showtimes array was empty"
      for showtime in by_movie.showtimes do
        assert showtime.class == Logical::Showtime, "Showtime was not a Showtime"
      end
    end
  end
end

The tests pass:

ChakBookPro:integration_test_framework chak$ ruby test/integration/movies_service_get_showtimes_by_theatre_test_case.rb 
Loaded suite test/integration/movies_service_get_showtimes_by_theatre_test_case
Started
.
Finished in 0.19624 seconds.

1 tests, 15 assertions, 0 failures, 0 errors

Putting It All Together

In this chapter, we created an orders service, in which products could be registered for purchase, and then subsequently purchased. We also connected a physical model in the movies service to our new service, so that new showtimes were automatically registered as purchasable products. We also expanded the API of our movies service API to support requests that would lead logically to the assembly of pages on a front-end website. Using our integration test framework, we tested all of these connections.

At this point, everything we need to write a front-end application that can consume multiple back-end services is in our hands. In fact, our integration test framework is just such an application.

Chapter 15: An XML-RPC Service Chapter 17: REST Primer
Advertisements