The Loop

Admitidally, every time i’ve ever looked at the Lift codebase in TextMate, i’ve always wondered what the hell the “lift-amqp” module was for. Well, curiosity may have killed the cat but it was not enough of deterrent to me finding out and building a sample AMQP / RabbitMQ application!

AMQP you say?

Yup, thats right. AMQP – Advanced Message Queuing Protocol to the lay’ man not familiar with the acronym. Essentially AMQP is an open standard for enterprise messaging. There are several implementations of the standard, namely, Open AMQ and RabbitMQ – the latter is newer and generally considered to be the defacto implementation. Rabbit MQ is written in ERLang and has clients for Java, .NET, Ruby etc etc.

So where does Lift fit in?

Strictly speaking, it doesn’t. It just so happens that Steve J was working on the Lift team some time ago and wrote a nifty Scala wrapper around the standard (and highly mutable) Java client implementation of Rabbit MQ. To all intense purpose, “lift-amqp” can be used in any Scala application and has no other dependencies within the Lift framework.

Hand-wavy overview

Lets get down to business… I spent quite a lot of time just looking at the source code and scala-docs for lift-amqp and for the longest while couldn’t figure out what went where. Being a fairly visual person, I’ve drawn up a diagram that explains the implementation and how you (yes, you reading this) would implement your application:

A working example?

So rather than try to post lots of pictures – I thought just putting up a screen-cast explaining the components would make more sense :-)

The client listener

So from the diagram above that explains the code layout – you can see that one of the first things you need to do is subclass AMQPDispatcher. The below is my example code that connects to the RabbitMQ broker and firstly declares a queue with the appropriate parameters. If you wondering why when the 2nd client connected in my example things didn’t explode and complain about more than one exchange existing with the same name, thats because RabbitMQ is clever enough to know that if the queue / exchange already exists then it just Noop’s the requests. Its a nice feature.

class DemonstrationSerializedAMQPDispatcher[T](
  queueName: String, factory: ConnectionFactory, 
  host: String, port: Int) extends AMQPDispatcher[T](
    factory, host, port) {
  override def configure(channel: Channel) {
    val ticket = channel.accessRequest("/data")
    channel.exchangeDeclare(ticket, "mult", "fanout")
    channel.queueDeclare(ticket, queueName)
    channel.queueBind(ticket, queueName, "mult", "example.*")
    channel.basicConsume(ticket, queueName, false, 
      new SerializedConsumer(channel, this))
  }
}

So, your probably thinking that this was not the class that I created an instance of during the screencast – and you’d be right. Essentially we still need a wrapper class to mask our dispatcher; the listener proper.

The purpose of the listener is to establish a connection to the broker, pass the right credentials etc. Within the listener we create an instance of the dispatcher and start its actor. We also then just have an inner class that we use for example purposes (another actor) that prints out the messages you saw in the terminal window. In reality, this would likely be passing messages to some other, external actor which handled the business process. For a full code listing see the end of this article.

The postman…

So now that you know how the example listens for messages, lets see the posting code…

  val params = new ConnectionParameters
  params.setUsername("guest")
  params.setPassword("guest")
  params.setVirtualHost("/")
  params.setRequestedHeartbeat(0)
  val factory = new ConnectionFactory(params)
  // Create a new instance of the string sender.
  // This sender will send messages to the "mult" exchange with a 
  // routing key of "routeroute" 
  val amqp = new StringAMQPSender(
    factory, "macbookpro", 5672, "mult", "example.demo" 
  )
  amqp.start

  /**
   * Salute the rabbit!
   */
  def salute = amqp ! AMQPMessage("hey there!")
}

Little explanation needed here – essentially the string sender is just an actor that knows how to respond to AMQPMessage…. it couldn’t be simpler! I hope this has proved helpful and a good overview of the lift-amqp module.

If you would like to download the source code, you can get it from here

Its been a long time since I posted my original application for iPhone – iDashboard – to control XMPie uProduce on the move. I’ve been busy with a bunch of other cool stuff and in the run up to the annual XMPie Users Group meeting in Las Vegas decided to make Marketing Console for iPhone!

The application syndicates reporting and analysis information about your campaigns and lets you see the latest up-to-date charting / figures wherever you are. Currently, the application is using an internal R&D build of Marketing Console; so right now you’ll have to wait before you can monitor your own campaigns on the move!

If you have feedback, just leave a comment at the bottom of the article. Enjoy…

Im currently exploring Rabbit MQ and had a few issues getting up and running reliably on Mac OSX. The problem wasted so much of my own free time that I thought it would be a good idea to post about it and perhaps it might help others in the future.

The Problem

The broken boots normally as the rabbitmq user defined in the system – however, when trying to connect to it using rabbitmqctl you get the following error (repeatedly):

macbookpro:~ timperrett$ sudo rabbitmqctl status
Status of node rabbit@macbookpro ...
{badrpc,nodedown}
...done.

The Solution

After many hours dabbling, and checking, checking again, rechecking my user and permissions setup, I found that it was actually to do with the way in which ERlang networks. Essentially, I was running the broker on:

rabbit@macbookpro

However, whilst I could ping the host “macbookpro” from terminal, it appears that Rabbit MQ needed it defined in the /etc/hosts file in order to work correctly.

Both strange and annoying, perhaps this will save someone some time!

With my on-going effort to write more documentation and articles for Lift I’ve decided to write a walk through of Lifts dispatching and rewriting mechanisms.

Before we start this discussion, its important that you know (and understand) the importance of partial functions in scala. If your not familiar, check out this article – it should fill you in on all the particulars.

Application Boot

If your not familiar with Lift or are new, you should understand that anything of consequence that changes the application environment must in someway hook into the boot-up cycle – the default looks something like this:

class Boot {
  def boot {
    // where lift will look for snippets
    LiftRules.addToPackages("eu.getintheloop.tutorial")
  }
}

Ok, so not a great deal going on there… this is however a bare-bones lift boot class. In our case, we want to add a rewrite so that the following mappings take place:

/product/some-product-link

/** maps onto **/

./webapp/product-display.html

Its important to note that rewriting is just that, its not used for redirects or any other such activity – its 100% for URI translation and interpretation.

Adding a Rewrite

Pretty much all of application configuration within Lift is done through the LiftRules object – its the central place for configuration PFs and operation vars. So, how do we use it? Well, first add the following to the top of your Boot.scala file.

import _root_.net.liftweb.http.LiftRules

Next, add the following code below the snippet package definition:

LiftRules.rewrite.prepend(NamedPF("ProductExampleRewrite") {
  case RewriteRequest(
      ParsePath("product" :: product :: Nil, _, _,_), _, _) => 
    RewriteResponse(
      "product-display" :: Nil, Map("product" -> product)
  )
})

Rewrite matching in detail

So lets step through this part by part… we already know about the LiftRules object and what its for, and one of its var properties is “rewrite” – a RuleSeq – and has the notion of both prepending values, and appending them. In this case, we prepend this rewrite rule, meaning that it will execute before any other rewrite rules previously added to the rewrite var.

List[String] is used to match the incoming path – lets take a closer look at our request object and the parameters we pass. From the scaladocs, we can see that RewriteRequest object has the following signiture:

/**
 * options for RewriteRequest
 */
case class RewriteRequest(
  val path : ParsePath, 
  val requestType : RequestType, 
  val httpRequest : HttpServletRequest
) 

/**
 * options for ParsePath
 */
case class ParsePath(
  val partPath : List[String], 
  val suffix : String, 
  val absolute : Boolean, 
  val endSlash : Boolean
) 

/**
 * options for requestType object
 */

GetRequest
PostRequest
PutRequest
DeleteRequest

The first argument in the RewriteRequest is a ParsePath, this is one of the primary URI matching mechanisms and enables you to define detailed paramaters on which request to match and which to ignore. Lets take a look at some various RewriteRequest examples:

/**
 * example 1.
 * matches: GET /some/demo/path
 */

RewriteRequest(
  ParsePath("some" :: "demo" :: "path" :: Nil, "", true, false), 
  GetRequest, _
)

/**
 * example 2.
 * matches: PUT /some/image.png
 */

RewriteRequest(
  ParsePath("some" :: "image" :: Nil, "png", true, false), 
  PutRequest, _
)

/**
 * example 3.
 * matches: * /some/demo/
 */

RewriteRequest(
  ParsePath("some" :: "demo" :: Nil, "", true, true), _, _
)

/**
 * example 4.
 * matches: GET /product/<item>/details
 */

RewriteRequest(
  ParsePath("product" :: item :: "details" :: Nil, "", true, true), 
    GetRequest, _
)

So far we’ve seen how to configure the incoming request – to complete the picture, lets now take our request handling and couple that up with matching the response and mapping parameters so they are then available in the rest of our code via S.param.

The RewriteRequest object has several overload apply methods to keep the verbosity of your code to a minimum – namely, these overloads are:

def apply(path : ParsePath, params : Map[String, String])
def apply(path : List[String])
def apply(path : List[String], suffix : String)
def apply(path : List[String], params : Map[String, String])

We can see that this gives us a bunch of flexibility depending on our needs – straight rewrite, rewrite with params etc etc. Lets take a look at some examples:

/**
 * rewrite to a html file in webapp/show.html with no params
 */

RewriteResponse("show" :: Nil)

/**
 * rewrite to a html file in webapp/show.html with 
 * a paramater called "product" from
 * a parameter placeholder called product
 */

RewriteResponse("show" :: Nil, Map("product" -> product))

/**
 * rewrite to a html file in webapp/example.pdf with 
 * no params, but a pdf suffix
 */

RewriteResponse("example" :: Nil, "pdf")

We have now looked at both requests and response – so moving back to our original example I hope you can see how it now works. One thing we have no explored is how to access the various parameters you might configure in your snippets / other application code. The answer my friends, is simple:

S.param("product").openOr("fail over product")

By now I presume you are a rewriting guru!... that might be a bit overboard, but I hope you found this guide informative and useful.

Overview

This stuff crops up time after time again, and people are always asking about how bind() works and what they should actually be doing with it within there lift apps. I’ve seen some fairly ugly abuse of scala.xml._ within Lift snippets… however, not only does this tightly couple your HTML design with the snippet code behind – and thus turn into a nightmare for your designers – your missing out on some of lifts best features!

All the code snippets below presume the following imports:

import _root_.scala.xml.{NodeSeq,Text,Node,Elem}
import _root_.net.liftweb.util.{Box,Full,Empty,Helpers,Log}
import _root_.net.liftweb.util.Helpers._
import _root_.net.liftweb.http.{S,SHtml}

Also, all the methods are a part of the snippet class “Demo”.

Basic Binding

So, if we want to deal with a view control on the server side, we need to bind it to a namespace in the markup. In our first example, we’ll just create a text input and populate its value.

Scala Snippet

var tempValue: String = "" 

def exampleOne(xhtml: NodeSeq): NodeSeq = bind("example", xhtml,
  "item_one" -> SHtml.text(
   tempValue, // the "read" value
   tempValue = _), // the "write" function
  "submit" -> SHtml.submit("Submit", () => Log.info(tempValue))
)

So, lets just review quickly. We’ve made a string variable that were going to assign the value of our text field when the form is submitted. Furthermore, if you watch the console window you see that upon submission the value you entered is output to the console.

XHTML Code

<lift:demo.example_one form="post">
  <p>Sample input: <example:item_one example:id="demo_id" /></p>
  <p><example:submit /></p>
</lift:demo.example_one>

You can see how the prefix we specified in the bind function (“example”) is now used in the markup body of the snippet. Also, note how the actual snippet call uses snake_case (“example_one”) when our method is called “exampleOne” – Lift is clever enough to know what method you are trying to use… this ensures your designers dont moan about camel cased markup!

Binding++

The previous example was illustrative, but pretty usless in real applications. Lets take a look at something that is a bit more feature full – namely, a small form that updates a database table using the Mapper persistence framework.

Before we begin lets assume that we have a model class called “User”, and it has the fields:

• first_name – MappedString
• last_name – MappedString
• email – MappedEmail
• display_email – MappedBoolean

In this fictional form, users just enter their details and a new database user row is added.

Scala Snippet

def exampleTwo(xhtml: NodeSeq): NodeSeq = {
  var user: User = new User
  def submitHandler() = {
    if(user.saveMe){
      S.redirectTo("thank-you")
    }
  }
  bind("example",xhtml,
    "first_name" -> SHtml.text(user.first_name, user.first_name(_)),
    "last_name" -> SHtml.text(user.last_name, user.last_name(_))
    "users_email" -> SHtml.text(user.email, user.email(_)),
    "display_email" -> SHtml.checkbox(false, user.display_email(_)),
    "submit" -> SHtml.submit("Submit", submitHandler _)
  )
}

Here we have something slightly more complex – the principals are exactly the same but with more fields and a nested method to handle the submit button. In this case, the submit button just saves a new record to the database and then if successful redirects the user to a thank-you page.

XHTML Code

<lift:demo.example_two form="post">
  <p>
    <label for="first_name">
      User name <example:first_name example:id="first_name" />
    </label>
  </p>
  <p>
    <label for="last_name">
      Last name <example:last_name example:id="last_name" />
    </label>
  </p>
  <p>
    <label for="users_email">
      Email <example:users_email example:id="users_email" />
    </label>
  </p>
  <p>
    Display your email? <example:display_email />
  </p>
  <p><example:submit /></p>
</lift:demo.example_two>

You can see how we have neatly wrapped our inputs in labels (a largely view centric concern) without any changes or specific ties to the server side logic that does the database insert. This example also shows another common mistake that new-commers make… there is no crazy magic binding going on between the names of the bind placeholders and what you put in them… its purely a convention thing / coincidence that they are called the same. You are free to call them whatever you feel appropriate.

Advanced Binding

Its when you need advanced binding that lift really comes into play – lets take the oh-so common idiom of one article having many “tags”; as you are reading this on my blog i’ll assume you are familiar with the notion.

So, assuming we have a Article model object with the appropriate fields, and we have a Tag model object that has a MappedForeignKey field relating back to the Article model object. We’ll assume we have a utility method in the article object that grabs all the tags for that article – this will allow us to do something like:

 article.tags.flatMap(tag => ....)

Lets get down to business:

Scala Snippet

def exampleThree(xhtml: NodeSeq): NodeSeq = { 
  // load article based on a fictional url parameter
  val article = Article.find(By(Article.link, S.param("permalink")))
  article.flatMap(a => bind("a", xhtml,
    "body" -> Text(a.body), // outputs the article body
    "tags" -> a.tags.flatMap(t => 
      bind("t", chooseTemplate("tag","list", xhtml),
        "name" -> Text(t.name)
      )
    )
  ))
}

After loading a single article based on the URL parameter passed to the snippet by some routing or query string parameter (doing this is out of scope for this article – check on the lift wiki) – we can the bind the contents of that model object and also iterate over the list of tags associated with that article.

The new stuff here is both the nested bind, and the chooseTemplate(...) method – this actually lets us tell the binding that for this nested bind, were going to be using a set template within the passed / input XHTML. Very cool – you’ll see this working in the view code below.

XHTML Code

<lift:demo.example_three>
  <p><a:body /></p>
  <p>Tagged with: <a:tags /></p>
  <tag:list>
    | <t:name /> |
  </tag:list>
</lift:demo.example_three>

You can see how we output the normal dynamic information just as we would any other bind item, wrapping the body in a scala.xml.Text node. We then posistion the tags placeholder of the outer bind in the correct place. Following that we then define the tag template and the content inside the tag template using the “t” prefix we specified in the nested bind function within the exampleThree method def. This should be fairly self evident that the chooseTemplate method is selecting the tag:list template and using that for each itteration of the nested bind – this is way cool and extremely usfull in most applications.

This concludes the tutorial – I hope this has been useful for someone :-)

I was just looking at the Lift’s StreamingResponse and was a bit bemused by the structural type being used for the first parameter. After some fiddling around I realized that:

data: {def read(buf: Array[Byte]): Int}

This will actually make the method more flexible and not tied to a particular hierarchy of classes (and super classes). So, if your looking at Lift’s StreamingResponse and thinking “what the hell”, all you need to remember is that you can pass any thing into the first parameter as long as it implements the read method with the above signature. This, IMO, is majority cool. The default thing that implements this signature is java.io.InputStream, but you could of course pass anything – even your own custom classes!

A sample implementation might look like:

var data: Array[Byte] = // get your data here
val headers = 
    ("Content-type" -> "application/pdf") :: 
    ("Content-length" -> data.length.toString) :: 
    ("Content-disposition" -> "attachment; filname=download.pdf") 
    :: Nil
Full(StreamingResponse(
  new java.io.ByteArrayInputStream(data),
  () => {},
  data.length, 
  headers, Nil, 200)
)

So, to recap, structural types are awesome!

Recently I’ve noticed more and more iPhone applications being released in the run up to both film releases and real-world product launches for large purchase items such as cars. So, what do the marketing chaps want to achieve with these adventures into the world of mobile computing?

Can we summarize that the marketeers see the iPhone as a lot more than just a mobile device? I think so, otherwise they simply wouldnt bother right? The thing I simply cannot get my head around, is how, or indeed if, the marketeers look to get any ROI on what must be a sizable amount of time and investment to create these iPhone apps which are usually quite sophisticated games.

Rhino Ball

For example, lets take Rhino Ball – a pretty sophisticated 3D game in which you control a hamster in a rolling ball. Pretty amusing, but I cant say it really compelled me to go and see the film. So why bother? In times of recession and tight budgets, I simply cant fathom why they are spending money on this rather niche medium to touch a small group of people with a weak message.

Perhaps this is just a prime example of marketing depts. blowing budgets for the simple reason “its cool”.

I just committed new functionality into lift master for handling the abstraction of i18n ResourceBundles’ that are powered by custom sources – for example, database driven localization, or hooking into some translation service.

Now, you can do:

    LiftRules.resourceBundleFactories.prepend { 
      case (_, locale) if locale.getISO3Language == "eng" 
         => new MyResources
      case (_, locale) if locale.getISO3Language == "swe" 
         => new MyResources_sv
      case (_, locale) => new DBResourceLoader(locale)
    }

In this example we assume that you have fictional java.util.ResourceBundle subclasses called “MyResources” and “MyResources_sv”. Furthermore, if the ISO 639-3 code is not either “eng” or “swe”, then it attempts to load it from another fictional ResourceBundle subclass “DBResourceLoader”.

Long term, I will probably provide a ProtoDBResourceBundle or something similar, but for now, your on your own to implement the clever string and translation loading :-)

NB: What follows is pure opinion and based on no actual metric

As we enter the final stages of the Lift 1.0, I cant help but reflect on open source, the enterprise, and how actions taken by communities that create great projects like lift ultimately have a real and tangible benefit to companies.

For a lot of organizations, open source software is generally seen as too much of a “risk” as their is no official support for products blah blah blah. However, it appears to me at least, that more commercial grade systems are having their open APIs integrated into OSS by the communities that power them, we are indeed seeing OSS adopting enterprise, rather than the other way around. This trend appears to be growing at a pretty quick rate with tools like SAP, Oracle and Salesforce.com (all mainstays of enterprise systems) being integrated in lots of different environments and OSS – for instance, Rails integration for salesforce, oracle middleware drivers for PHP etc etc etc.

I then got to thinking – where the hell did all this integration come from? More than likely, it came from guys and girls who were hacking this stuff out with tools they would usually have access to in a enterprise environment (at the workplace). That to me appears to be a fairly sensible rational, now for the “why”...

This I fear is a little more complicated – perhaps those hackers wanted to play with it? Perhaps it was a pet project (a la google private projects)? Or maybe something else entirely? Whatever the reason, the fact remains those tools exist. Now, the really interesting thing I believe to be about this is that the next time those same developers (and perhaps their co-workers) need to complete a project, what tools might they choose to use? Something that they need to write from scratch…? or something they already made in their spare time for that OSS framework or such?

With platforms such as the JVM welcoming technologies like Scala, that still run on the same infrastructure yet can have a totally different code design, I think we might see more and more OSS “leaking in through the backdoor” into the enterprise? OK, yes, its true that most large dev environments have equally large teams to manage all the accompanying bureaucracy, but I think this type of backdoor behavior might well find traction in the SME arena and give birth to a new erea of OSS coding in the enterprise.

Whatever happens – I for one really do hope that SME and Corporate companies open their eyes to what comunity powered development has to offer.

Stop seeing the risk, and start appreciating the possibilities :-)

Over and out.

I have recently been working on some API automation for XMPie uProduce and after some head scratching, realized that the username paramater has now changed format.

Previously, you only needed to provide a username and password on api methods, however, in the new version it is important to include your customer name in a windows AD format. For example, if your credentials were:

Customer Name: ABC
User Name: badger
Password: 123456

You would need to present the username in your API call like so:

ABC\badger

I hope this save someone, somewhere, some time!

Yesterday Jorge Ortiz invited me to join the team at scala-blogs.org – obviously I was delighted, and now I will be sharing my adventures in scala both through this blog, and on scala-blogs.

For my first article I’ll be writing about the PayPal integration I created with David Pollak for Lift.

Watch this space boys and girls, and dont forget to check out scala-blogs.org

If your using the scala-maven plugin and need to enable some of the cool extra functionality then check out these pom.xml snippets:

To enable launchers which let you do

mvn scala:run

use an XML snippet like:

<configuration>
  <scalaVersion>${scala.version}</scalaVersion>
  <args>
    <arg>-target:jvm-1.5</arg>
    <arg>-unchecked</arg>
   </args>
   <launchers>
     <launcher>
       <id>main-launcher</id>
       <mainClass>com.myproj.TheMainClass</mainClass>
       <args>
         <arg>/some/intial/argument</arg>
       </args>
     </launcher>
  </launchers>
</configuration>

Also note the “-unchecked” compiler argument… this will then prompt you if you have any deprection warnings and suggest how you could go about fixing them.

Any one who has ever implemented a SOAP consuming client service will appreciate that the tooling for various languages is a very welcome feather in SOAP’s cap. Without the WSDL tooling that web service developers are accustomed to, a large portion of those developers would probably struggle to use the said service as the sher verbosity of the SOAP entity payload is simply overwhelming.

REST: A idealist view of the web or serious technology?

When REST came onto the scene it appeared to be met with two very different view points. The open source communities and web purists rejoiced… a resource orientated architecture which allowed them to scrap there verbose SOAP and XML-RPC service flow. However, from what I saw first-hand, the reaction from enterprise was somewhat different; namely, one of “So now we have to implement clients by hand? Sod that!”.

For a number of developers, including myself, we could see that REST was a much better architectural style for distributed computing on the web, but with ever decreasing budgets and squeezed time scales the added overhead of creating totally bespoke services – and therefore clients – was one that proved somewhat of a bitter pill to swallow.

WADL: Here to save the day?

Recently, WADL is coming to the fore as a structured way for providing description and automated development workflow for using REST services. WADL is based around similar principles to WSDL in that it provides a contract of service between service resource and the client.

If WADL can provide a robust framework for automated tooling with REST, I really think that we’ll see REST pick up a second wind of uptake in the enterprise setting. With players like Yahoo! leading by example, how long will it be before others follow suit?

Only time will tell…

I’ve recently been looking at how best to make sensitive user data encrpytable within JPA models when in use with lift. I came accross JASYPT and it seems to be a really nice encryption tool. If you want to add it to your scala JPA models, do something like this:

package eu.getintheloop.bloglite.model

import javax.persistence._
import java.util.Date
import org.jasypt.util.password.BasicPasswordEncryptor

@Entity
@Table(){val name="users"}
class User extends BaseEntity {
  @Id
  @GeneratedValue(){val strategy = GenerationType.IDENTITY}
  @Column(){val insertable = false, val unique = true}
  var id: Long = _

  @Column{val nullable = true}
  var first_name: String = "" 

  @Column{val nullable = true}
  var last_name: String = "" 

  @Column{val unique = true, val nullable = false}
  var username: String = "" 

  @Column{val nullable = false}
  var password_hash: String = "" 

  @Column{val unique = false, val nullable = false}
  var email: String = "" 

  @Column{val unique = false, val nullable = false}
  var is_active: Boolean = false

  def password: String = this.password_hash

  def password_=(in: String) = this.password_hash = encrypt(in)

  def authenticate(in: String): Boolean = {
    new BasicPasswordEncryptor().checkPassword(in, password) 
      && is_active
  }

  private def encrypt(in: String): String = 
    new BasicPasswordEncryptor().encryptPassword(in)

}

As you might know, I use the lift framework a lot – I recently came across a strange issue where I couldnt set my own doctype in the layout template. Maybe this nugget of information will be usefull for someone:

ResponseInfo.docType = { 
      case _ if S.getDocType._1 => S.getDocType._2 
      case _ => Full(DocType.xhtmlStrict) 
    }

/*
Avalible options are:

xhtmlTransitional
xhtmlStrict
xhtmlFrameset
xhtml11
xhtmlMobile

*/