Hacks


1
Aug 11

System Scripting with Scala

For the longest time I have used Ruby as my pocket knife for system scripting… you know, just knocking up small little executable files that run helpful tasks or automate yawnful processes. I like Ruby for this kind of thing and it works. Recently a coworker in marketing wanted some automation for something relatively simple and instead of using Ruby I thought i’d just knock it together with Scala and in doing so I came across a neat little thing with the Scala scripting support.

Assuming you have Scala installed, you can execute bash statements and pass them directly to your Scala code. This can be pretty handy as there are certain things that are particularly annoying to do from Scala (and more broadly with Java) like finding out where exactly you are executing too on the file system. Often if you use the good ol’ protection domain trick it will give you a location in /var/tmp, as opposed to the real location of the script. Consider the following:


#!/bin/sh
SCRIPT="$(cd "${0%/*}" 2>/dev/null; echo "$PWD"/"${0##*/}")"
DIR=`dirname "${SCRIPT}"}`
exec scala $0 $DIR $SCRIPT
::!#

import java.io.File

object App {
  def main(args: Array[String]): Unit = {
    val Array(directory,script) = args.map(new File(_).getAbsolutePath)
    println("Executing '%s' in directory '%s'".format(script, directory))
  }
}

Notice the base code between #!/bin/sh and ::!#. This allows you to execute bash script (or whatever script you want) before evaluating this file as a Scala script. This can be pretty handy for certain tasks when doing system scripting :-)

Assume you saved this file as “thing”, you can then execute it like any other script: ./thing

Enjoy.


23
Nov 10

Implicit REST Services with Lift

Do you ever have those days, when you wake up and become unhappy with the code you wrote last year ? Today was one of those days (technically speaking it was yesterday, as I went to bed thinking this, but still). There simply must be a better way. So, im going to blame this post on Ross and Jon-Anders . Those two are truly to blame for my interest in type-classes and compiler voodoo… anyway, I digress…

DISCLAIMER: The following post makes heavy use of Scala implicit conversions and type system foo. Its not big, or clever, but it is fun. Before someone points it out, yes, there is a far simpler route to implement something similar. This was ultimately an experiment :-)

The Goal

I write a lot of services. These services do a lot of different things and more often than not need to be published in a bunch of formats. In this way, its often easy to end up with one of two things:

  • Accidental complexity
  • Code duplication or blurred lines between logic process and media representation

In this post i’m going to show you one possible method of avoiding the second problem, but, subjectively, the first problem may seem better or worse depending on your familiarity with the Scala type system.

Plan of Attack

Ok, so how are we going to do this? Well, as you may or may not know, Scala has this awesomeness called implicit conversions – these can be used to get the compiler to convert type A to type B when it only has A, but requires B. This is useful because the Scala compiler will use the appropriate implicit conversion that it finds within the given lexical scope; essentially meaning that you can “inject” different behaviour at different points during compile time. With this in mind, its a concept that matches quite closes to multi-format services. That is, I have type A from my logic or domain process and I need type B for the representation and delivery of a service.

We will be making a very small sample application that

The Book Store

Before we get into the utter type-madness, let us define the basic domain classes we’ll be using:

I think you’ll agree this is super basic. Basically its just a fake bookstore that has a super limited stock of only Harry potter and Scala books. Strange, but true. Anyway…

Define the Service Types

We’ll need some traits to glue this stuff together and provide the right syntax. ReturnAs[A, B] – This structure is basically used to mediate each format from its input to its output; in and of itself it does not do a lot. The companion object here is actually where the good stuff is: the implicit method f2ReturnAs takes a function A => B and converts it to a ReturnAs type. This is going to be handy when it comes to writing the implementation as we’ll be able to say Book => LiftResponse etc.

The next structure is the Return[A] type and its nested type As[B]. The Return type will be used as an explicit helper within the service implementations, so the API would be Return() via the companion object apply method. In addition to the single parameter, this method also has an implicit parameter of: implicit f: ReturnAs[A, B] which will be made available to use from within the implementation call via Scala context bounds. This context bound will be expanded to plain type parameter T, together with an implicit parameter so that we can “recapture” the correct return type within the implementation.

Format Conversions

With the plumbing in place, one needs to actually provide implementation of the various A => B type conversions. In this exceedingly small sample, im only going to be working with List[Book], so to keep this sample as simple as possible these are the implicit conversions for making an XML and plain text service:

I wont labour the details too much here as this is pretty standard LiftResponse stuff that you can find in my other articles. That being said, there is an interesting point of note, and that is the type signatures:

Return[List[Book]]#As[XmlResponse]

Here we use a type projection to specify the return type – this is pretty sick I think you’ll agree.

Service Implementation

With the media type conversions in place, the only thing that is actually left is within the service creation element is the writing of the method implementations.

So this is important. This is where we use the context bound that we talked about earlier comes into play in expanding to a type parameter and an implicit parameter which allows us to write these rather nice looking methods wrapped in a the Return objects apply method without needing to specify any specific types.

HTTP wire-up

Last but certainly not least is writing this awesomeness up into the HTTP implementation. To do that we are simply going to use Lift’s REST DSL that needs a return type that is some subtype of LiftResponse. Of course, here we have those so we get a rather nice looking implementation:

To test this out, all you need to do is visit /bookshop/books to get the text representation and /bookshop/books.xml to get the XML version. Hope you’ve enjoyed this article, you can find full source available to download from here

Find more information like this within my book, Lift in Action


31
Aug 09

Replacing Checkpoint SecureClient with IPSecuritas on Snow Leopard

So its that time again – another awesome Apple update to the best operating system in the world; for most, its a joyous time, one of new beginnings and wonderment…. for some, alas this is just an ideal as all there proprietary software comes crumbling down under a new kernel.

Friday was one of these momentous days, and yes, my world came crumbling down. For those who don’t read my blog often I work remotely 85% of my working weeks as the company I work for are in another country so having a secure and speedy VPN is critical to actually getting paid and doing some work. After upgrading OSX 10.5 to Snow Leopard my Checkpoint SecureClient completely stopped working – this appears to be the plight of many users out in the interweb so I thought id write up this guide how to use IPSecuritas (as it rocks) which is infinitely better than the default checkpoint client (which sucks major ass!).

Why should I care about IPSecuritas?

This is simple – basically (as above) it rocks and has the following great features:

  • Its speedy.
  • Automatic connection recovery
  • Password persistance
  • Can talk to a bunch of different Firewall types… no more vendor tie-ins
  • Great OSX integration
  • Automatic connection upon login (instantly connected to VPN!)
  • Oh, did I mention its fast?

Removing Checkpoint SecureClient

OK, so now we’ve established that SecureClient is evil, lets remove it. Helpfully, checkpoint took the time to provde a shell script in the install directory to do just this. Open a Terminal window (Applications > Utilities > Terminal) and type the following:



# i'm not 100% sure on the names, as im writing this from memory, 
# but just have a poke around and you'll find what I mean.
timperrett$ cd /opt/C (press tab for auto-complete then return key)
timperrett$ open Uninstall.command

Type “yes” when prompted – the script will then go about removing all the various components. If your thinking of skipping this step, your free to, but remember than this installation is completely broken and wont ever run under Snow Leopard as it appears to be tied to the 9.x Kernel present in 10.5… so you might as well clean up and keep tidy.

Setting up IPSecuritas

If you havent already, download IPSecuritas from here – open the DMG and drag the application to your Applications directory. Once there, double click the application to load it for the first time – you’ll need to enter your Administrator password then the application will install a daemon onto your system and configure itself. Once completed, take my advice and reboot your system – upon reboot you should see a new menu item that looks like a broken wire (below, dont worry about “XMPie”, thats just what I decided to call my profile)…

Choose the “Open IPSecuritas” menu item – and up should spring the main GUI. You now need to configure a connection – this is what you will use to connect to your VPN endpoint (clue’s in the name!). Making this connection is however a rather technical process for most users so im going to post screen shots of my configuration at every stage so that you can make something similar (yours may not be identical – it really depends on the setup implemented on the firewall; however, what I detail uses common place defaults).

So the two boxes in red are the important ones. For Remote IPSec Device you need to fill in the domain name or IP address of your firewall / vpn endpoint; this is organisation specific so i’ve removed mine. Secondly, Network address – this is the base IP range to which you want to connect to; again, organisation specific and yours will likely be different – if you don’t know, check with an IT administrator.

These are standard security options needed to work with the checkpoint vpn – because I work long hours connected to the VPN, i’ve set it to timeout after 10 hours (essentially it never cuts me off and im in charge… sweet).

Again, this is standard checkpoint stuff so just copy the configurations as is – you don’t need to know whats what.

So this is an interested panel – the two boxes ive marked in red need to be filled with your username for the VPN connection; you should have this already with that which you were using with the Checkpoint SecureClient system. In this example (and the checkpoint default) its using Hybrid RSA, but a lot of organisations use XAuth RSA etc that involve certificates etc… if you need this, just use the certificate manager and configure accordingly. I’ve also set it to remember my password so that I dont have to keep entering it – depending on your outlook, this is a good/bad thing. Personally, I think its a great timesaver!

Depending on your setup, you may want to configure specilized DNS servers – you might want this if you have servers that you wish you access with a UNC style such as:

\\somefileshare

Without specifying those DNS, your VPN will try to use external DNS and you just wont get what you want (or expect). I’ve removed mine for security reasons, but it should be fairly simple to figure out what you need to enter.

So this is the killer panel that confuses most people – you have some fairly finite control over the connection parameters – if your using Checkpoint VPN-1, just do as I have (unless your using another authorisation mechanism) and you should succeed!

Once you have all that, your good to go – just close the window, and click “Start” on the main IPSeceritas window then provided all went well you should get a green light next to the connection name – in my instance, i called the connection “office”. If you are having issues, seek help from your IT administrator as it might be a configuration issue – with VPN‘s your client much EXACTLY match what the endpoint is configured to… any miss-match at all will result in failure; however if you wish to debug the issue yourself, bring up the connection log from the top bar menu and you’ll be able to see exactly what is going on under the hood (if you need a boat load of wire information, set the logging level in preferences to DEBUG and then restart/reboot IPSecuritas)

Enjoy, and good luck.


17
May 09

Running Rabbit MQ on Mac OSX – Solving {badrpc,nodedown}

Im currently exploring Rabbit MQ and had a few issues getting up and running reliably on Mac OSX. The problem wasted so much of my own free time that I thought it would be a good idea to post about it and perhaps it might help others in the future.

The Problem

The broken boots normally as the rabbitmq user defined in the system – however, when trying to connect to it using rabbitmqctl you get the following error (repeatedly):


macbookpro:~ timperrett$ sudo rabbitmqctl status
Status of node rabbit@macbookpro ...
{badrpc,nodedown}
...done.

The Solution

After many hours dabbling, and checking, checking again, rechecking my user and permissions setup, I found that it was actually to do with the way in which ERlang networks. Essentially, I was running the broker on:


rabbit@macbookpro

However, whilst I could ping the host “macbookpro” from terminal, it appears that Rabbit MQ needed it defined in the /etc/hosts file in order to work correctly.

Both strange and annoying, perhaps this will save someone some time!


19
Dec 08

Enabling launchers and warnings with scala-maven-plugin

If your using the scala-maven plugin and need to enable some of the cool extra functionality then check out these pom.xml snippets:

To enable launchers which let you do

mvn scala:run

use an XML snippet like:



<configuration>
  <scalaVersion>${scala.version}</scalaVersion>
  <args>
    <arg>-target:jvm-1.5</arg>
    <arg>-unchecked</arg>
   </args>
   <launchers>
     <launcher>
       <id>main-launcher</id>
       <mainClass>com.myproj.TheMainClass</mainClass>
       <args>
         <arg>/some/intial/argument</arg>
       </args>
     </launcher>
  </launchers>
</configuration>

Also note the “-unchecked” compiler argument… this will then prompt you if you have any deprection warnings and suggest how you could go about fixing them.


24
Aug 08

Running Resin Alongside Microsoft IIS

I recently thought that it would be good to run a Servlet container alongside IIS which is so well embedded in the clients that I deal with as they are mainly .NET types…

Anyone who has read my blog before will now that I have somewhat of a penchant for Resin – so once again, this was my choice of servlet container. Luckily for me, Caucho are awesome enough to supply an ISAPI filter which drops right into your IIS system, making Resin just an other ISAPI filter! Very sweet.

If you too would like to have some Java sweetness on your M$ box, then check out this walkthrough


28
Jul 08

Programatically adding Applications to the Login Items

Its taking some kicking around, but here is the fruit of my labour… using core foundation over the RC bridge.



require 'osx/cocoa'

loginItems = OSX::CFPreferencesCopyValue(
      "AutoLaunchedApplicationDictionary", 
      "loginwindow", 
      OSX::KCFPreferencesCurrentUser, 
      OSX::KCFPreferencesAnyHost)

application_path = File.expand_path("~/path/to/your.app")
application_hidden = false

loginItems &lt;&lt; OSX::NSDictionary.dictionaryWithObjects_forKeys(
    [ application_path, application_hidden ], 
    [ <img src='http://blog.getintheloop.eu/wp-includes/images/smilies/icon_razz.gif' alt=':P' class='wp-smiley' /> ath, :Hide ])

# puts loginItems

OSX::CFPreferencesSetValue(
    "AutoLaunchedApplicationDictionary", 
    loginItems, 
    "loginwindow", 
    OSX::KCFPreferencesCurrentUser, 
    OSX::KCFPreferencesAnyHost)

OSX::CFPreferencesSynchronize("loginwindow", 
        OSX::KCFPreferencesCurrentUser, 
        OSX::KCFPreferencesAnyHost)

With any luck this might help someone :-)


6
Jun 08

How To Fix Generating XMPie API Java Stubs With Metro

As a number of you are aware, I have spent quite a bit of time wrestling with XMPie uProduce SOAP API with the CXF service framework in Java. I finally (and with quite a lot of disappointment on my part) gave up trying to get that to work. CXF just does not want to play nice and I cant seem to make the XJB bindings work correctly with it and the mish-mash WSDL coming from uProduce

Anyway… with that rant over, I tried to be objective and switched to using Metro – ok, out of the box I still had a whole heap of problems, exactly as I did with CXF. However, I have managed to wangle it!

Step 1

You will need access to an XMPie server with the API‘s installed and working. For the sake of this example i just wget’d the WSDL file so the generation code was not so bloated.

Step 2

You’ll need an XSD XJB binding file which looks like this:


<?xml version="1.0" encoding="UTF-8"?>
<bindings xmlns="http://java.sun.com/xml/ns/jaxb" 
          xmlns:xsd="http://www.w3.org/2001/XMLSchema" 
          xmlns:xjc="http://java.sun.com/xml/ns/jaxb/xjc" 
          version="2.0">

  <globalBindings>
    <xjc:simple />
  </globalBindings>

  <bindings scd="~xsd:complexType">
    <class name="ComplexTypeType"/>
  </bindings>

  <bindings scd="~xsd:simpleType">
    <class name="SimpleTypeType"/>
  </bindings>

  <bindings scd="~xsd:group">
    <class name="GroupType"/>
  </bindings>

  <bindings scd="~xsd:attributeGroup">
    <class name="AttributeGroupType"/>
  </bindings>

  <bindings scd="~xsd:element">
    <class name="ElementType"/>
  </bindings>

  <bindings scd="~xsd:attribute">
    <class name="attributeType"/>
  </bindings>
</bindings>

Save this as xsd.xjb (or whatever you like, just make sure the name is reflected in the command below)

Step 3

Actually generate the code! Your command might look different, but heres mine:



  wsimport -b http://www.w3.org/2001/XMLSchema.xsd \
    -b src/xjb/xsd.xjb \
    -keep \
    -s src/java \
    -d target \
    -p com.xmpie.wsapi.icp \
    src/wsdl/InteractiveCampaign_SSP.wsdl

You should then see something like the following in your finder (or other OS file browser if not on mac):

These are the class files created for the InteractiveCampaign WSDL – all 128 of them!

Anyway, I have run out of time for today, but will try and get another blog up soon about how to use this in a client application from a java CLI.


23
Apr 08

Accessing OSX Network Interfaces From Wireshark X11

Just a short post for people who might not be sure how to get Wireshark to read the interfaces from your mac…

I had to run the applications as root:wheel to let the X11 interface read the network interface devices. To do the same, just run:

sudo /Applications/Wireshark.app/Contents/MacOS/Wireshark

Then, when prompted, just enter your password (provided you are a machine administrator that is). Wireshark X11 should then boot up and work without problem.

If your on 10.4 you will need to install X11 beforehand


9
Apr 08

How to use rails 2.0.2 without a database

Step 1

Run the rake task to freeze rails 2.0.2 into the project so your not reading from gems:

rake rails:freeze:gems

Step 2

Remove the database.yml file from RAILS_ROOT/config/

Step 3

Change line 21 in RAILS_ROOT/config/enviroment.rb to:

config.frameworks -= [ :active_record ]

Bobs your uncle, that should be it!