KeywordSearch Released

Posted 2007-01-09…

In the wild

Along with archival and a good UI, the thing that really strikes me about gmail—what makes it really useful—is the search feature. Obviously we expect good search capabilities in anything Google puts out, but the keyword-based searching is extra handy. I find myself doing things like this pretty often:

  presentation "rails edge" from:chad after:2006/08/16

This searches for emails in exactly as it reads; it will return emails from chad containing the words presentation and rails edge that were sent after the 16th of August, 2006. Nifty… and quick.

Google has developed a list of keywords, and in some cases, expected values for those keywords, that are translated into an advanced query. This is an interesting alternative to a large and unwieldy advanced search form.

Over the last couple of weeks I’ve become increasingly interested in adding a similar flexible, easy-to-extend search feature to a Rails application that I’m developing. I’ve wanted to play around with Dhaka since it came out, and this presented the perfect opportunity.

How it works

The library is generic—it’s not Rails-specific by any means— so you could use it for any Ruby project, but for the purposes of this example, let’s go ahead and put together a controller example.

As you might imagine, installation is as easy as:

sudo gem install keyword_search -y

Now let’s require it. In Rails, this means putting the following in your environment.rb:

require 'keyword_search'

For the example, let’s duplicate he concept of searching for messages, in messages_controller.rb. Let’s work inside an index action (or search, if you prefer), starting out by defining a couple variables we’ll use to accumulate the conditions we’ll need to use in the call to Message.find. There’s better ways to do this (one of which is on my list to release), but for now lets just use a few arrays:

clauses = []
arguments = []
includes = []

The entry point for the library is KeywordSearch.search, which takes a string to parse and a block. Let’s go ahead and parse params[:q], in Googlesque fashion:

KeywordSearch.search(params[:q]) do |with|

The method yields an new instance of KeywordSearch::Definition, which is just a handy container to define the keywords we want to allow. In our example, we’re calling the block variable it gets assigned to with… just because it will sound nice later.

First, we should handle the “keywordless” terms that are just floating around— these belong to the default keyword, which we can set with:

  with.default_keyword :text

Lets add a handler for the text keyword, too. In production, you’d probably want to use something like Ferret, or at least fulltext searches, but for this simple example let’s just use SQL like. Keyword handlers are always passed an array of values—keywords might want to allow more than one value, like the text keyword does:

  with.keyword :text do |values|
    # Check subject and body for each, allowing any match (OR)
    clauses << ["(subject like ? or body like ?)"] * values.size).join(' OR ')
    # Add 2 of each value to the arguments, fitted out with <tt>like</tt> globs
    arguments += values.map{|val| ["%#{val}%"] * 2 }.flatten
  end

Keyword handlers use the magic of closures) to give you the flexibility to do whatever you’d like with any variable in scope—in our case, we add a clause and two arguments for each value that’s encountered.

Now let’s deal with from, which is a bit tricker, from a query standpoint, requiring a join. We’ll allow multiple possible values here too:

with.keyword :from do |values|
  includes << :author unless includes.include?(:author) # just a good habit
  # Assuming the author is a User
  clauses << (['users.name like ?'] * values.size).join(' OR ')
  # Glob for <tt>like</tt> comparison
  arguments += values.map{|value| "%#value%" }
end

Let’s only support one since value, though:

with.keyword :since do |values|
  date = Date.parse(values.first) # Skipped exception handling
  clauses << 'created_on >= ?'
  arguments << date
end

Now that we’re done with keywords, let’s just close up the KeywordSearch.search block:

end

… and put everything together for the call to Message.find, building the conditions array as Rails expects:

conditions = [clauses.map{|c| "(#{c})"}.join(' AND '), *arguments]
@messages = Message.find(:all, :conditions=>conditions, :include=>includes)

Of course, you can always paginate the results—or do anything else you’d like with them.

A time and a place

It might be worth noting that Google, in the documentation for the GMail search feature, calls it “Advanced Search.” This feature truly is more advanced than an unwieldy, but in-your-face search form, requiring some effort and memorization from your users. For that reason, this approach might not be the best fit for many applications.

Although the keyword method does take an optional description argument, at the moment generating documentation for the supported keywords isn’t really built into the API, so use in in-house applications is a more likely scenario.

As to the stability and/or usefulness of the library for its intended purpose at the moment… it’s an initial release

From here

There are a number of things that still need to be worked on in the library.

You can read the API docs here