Thursday, August 25, 2011

Playing with JMX and Scala : MBean Creation & "Remote" access

** Update (2011-12-07) : scala JMX API project just created... I've started refactoring an older JMX scala API, and I'll release the new API as an open source project. The project link : janalyse-jmx : scala JMX API

An example (download corresponding sbt project) of how to create a MBean in Scala. We define the MBean interface using scala trait, and make our class implements the trait capabilities. No need to define the getter thanks to the "@BeanProperty" annotation, scala does the job for us. The MBean registration is directly done within Supervisor class instanciation, the jmxRegister method and with an implicit conversion to automatically convert a String to an ObjectName.

package jmxsandbox

import scala.actors.Actor
import scala.reflect.BeanProperty
import java.lang.management.ManagementFactory
import javax.management.ObjectName

// Some definitions to simplify the code
private object JMXHelpers {
    implicit def string2objectName(name:String):ObjectName = new ObjectName(name)
    def jmxRegister(ob:Object, obname:ObjectName) = 
      ManagementFactory.getPlatformMBeanServer.registerMBean(ob, obname)
}
import JMXHelpers._

// Some messages managed by the Supervisor actor
sealed abstract class Message
case object MAlive extends Message
case object MDead  extends Message
case object MExit  extends Message
case object MGet   extends Message

// The defined MBean Interface
trait SupervisorMBean {
  def getAlive():Int
}

// The class with JMX MBean
class Supervisor extends Actor with SupervisorMBean {
  jmxRegister(this, "JMXSandbox:name=Supervisor")
  @BeanProperty
  var alive=0
  def act() {
    loop { 
      react {
        case MAlive => alive+=1
        case MDead  => alive-=1
        case MExit  => exit
        case MGet   => reply(alive)
      }
    }
  }
}


And now the test case, showing how to test within the same JVM our Supervisor MBean. The test case creates an internal MBean server, which will be used by Supervison internal MBean registration. So we take benefits of local and remote-like access from jmx for our test case.

package jmxsandbox

import java.rmi.registry.LocateRegistry
import java.lang.management.ManagementFactory
import javax.management.remote.JMXConnectorServerFactory
import javax.management.remote.JMXConnectorFactory
import javax.management.remote.JMXServiceURL

import org.scalatest.FunSuite
import org.scalatest.matchers.ShouldMatchers

import JMXHelpers._

class SelfTest extends FunSuite with ShouldMatchers {
  // Let's create and start a local JMX service
  LocateRegistry.createRegistry(4500)
  val url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://localhost:4500/jmxapitestrmi")
  val mbs = ManagementFactory.getPlatformMBeanServer()
  val cs = JMXConnectorServerFactory.newJMXConnectorServer(url, null, mbs)
  cs.start
  

  test("Checking supervisor state with standard access and from JMX interface") {
    // Create and start the supervisor actor
    val supervisor = new Supervisor {start}
    
    // Send a MAlive message to the supervisor
    supervisor ! MAlive
    
    // get the current alive value from the actor
    val stateDirect = (supervisor !? MGet) match {
      case e:Int => e
      case _ => -1
    }
    
    // The check
    stateDirect should equal (1)

    // Do the same but using Supervisor MBean to read the alive value
    val mbserver = JMXConnectorFactory.connect(url).getMBeanServerConnection
    val stateViaJMX = mbserver.getAttribute("JMXSandbox:name=Supervisor","Alive").asInstanceOf[Int]
    
    // The check
    stateViaJMX should equal (stateDirect)
    
    // Ask the actor to exit
    supervisor ! MExit
  }
}


CONTEXT : Linux Gentoo / Scala 2.9.0 / Java 1.6.0_26 / SBT 0.10 / ScalaTest 1.6.1

Monday, August 22, 2011

Simple SBT use case

Simple Build Tool (sbt) is an impressive tool, building is really fast, and configuration quite simple in particular with small projects. No need to learn a new language, as sbt configuration is made using scala ! Of course automatic dependencies resolution is available !

Consider the following simple project, naturalsort.tar.gz, whose file hierarchy is the following :

naturalsort/
naturalsort/build.sbt
naturalsort/src/main/scala/naturalsort/NaturalSort.scala
naturalsort/src/test/scala/naturalsort/NaturalSortTest.scala


"build.sbt" just contains :

name := "NaturalSort"

version := "1.0"

scalaVersion := "2.9.0"

libraryDependencies += "org.scalatest" % "scalatest_2.9.0" % "1.6.1" % "test"


Once sbt is setup, just enter "naturalsort" directory, and run sbt; it will resolve dependencies, and make the sbt console available for you :

$ cd naturalsort/
$ sbt
>  compile
[info] Updating {file:/home/dcr/dev/naturalsort/}default-76224f...
[info] Done updating.
[info] Compiling 1 Scala source to /home/dcr/dev/naturalsort/target/scala-2.9.0.final/classes...
[success] Total time: 6 s, completed 22 août 2011 22:21:14
> test
[info] Compiling 1 Scala source to /home/dcr/dev/naturalsort/target/scala-2.9.0.final/test-classes...
[info] NaturalSortTest:
[info] - basic tests
[info] - advanced tests
[info] - special cases tests *** FAILED ***
[info]   List(1.1, 1.002, 1.3, 1.010) did not equal List(1.001, 1.002, 1.010, 1.02, 1.1, 1.3) (NaturalSortTest.scala:37)
[error] Failed: : Total 3, Failed 1, Errors 0, Passed 2, Skipped 0
[error] Failed tests:
[error]  naturalsort.NaturalSortTest
[error] {file:/home/dcr/dev/naturalsort/}default-76224f/test:test: Tests unsuccessful
[error] Total time: 4 s, completed 22 août 2011 22:21:35
> ~compile
[success] Total time: 0 s, completed 22 août 2011 22:23:02
1. Waiting for source changes... (press enter to interrupt)
> ~ test-only naturalsort.NaturalSortTest
[info] Compiling 1 Scala source to /home/dcr/dev/naturalsort/target/scala-2.9.0.final/test-classes...
[info] NaturalSortTest:
[info] - extreme tests
...
> package
[info] Packaging /home/dcr/dev/naturalsort/target/scala-2.9.0.final/naturalsort_2.9.0-1.0.jar ...
[info] Done packaging.
[success] Total time: 0 s, completed 4 sept. 2011 10:55:24


CONTEXT : Linux Gentoo / Scala 2.9.0 / Java 1.6.0_26 / SBT 0.10 / ScalaTest 1.6.1


Sunday, August 21, 2011

Simple hack to get response time

To use for example from the scala console in order to quickly get a function response time :

  def now=System.currentTimeMillis
  def duration[T](proc : =>T) = {
    val started = now
    val result = proc
    (now - started, result)
  }


Use case : Ordering a 100000 list of integer, built like as follow :

  val lst = (1 to 1000000).reverse.toList


Get / view response time :

  duration {lst.sortBy{x=>x}}


Or store the response time result :

  val (responsetime, _) = duration {lst.sortBy{x=>x}}


Since we're using a Java Virtual Machine, one measurement is not enough, we must make several iterations in order for the jvm to reach its steady state, and masks GarbageCollector border side effects on response times. We can use such approach :

  def howlong[T](proc : =>T, howmany:Long=5) {
    val durs = (1L to howmany) map {i =>
      val (dur,_) = duration {proc}
      dur
    }
    println("Duration : avg=%d min=%d max=%d all=%s".
         format(durs.sum/durs.size, durs.min, durs.max, durs))
  }


It gives, in milliseconds :

  scala> howlong {lst.sortBy{x=>x}}
  Duration : avg=670 min=524 max=903 all=Vector(816, 573, 524, 903, 534)


CONTEXT : Linux Gentoo / Scala 2.9.0 / Java 1.6.0_26

Friday, August 19, 2011

Natural Sorting...

A natural sort implementation using scala :

implicit val ord = new Ordering[String] {
  def groupIt(str:String) = 
       if (str.nonEmpty && str.head.isDigit) str.takeWhile(_.isDigit)
       else str.takeWhile(!_.isDigit)
  val dec="""(\d+)"""r
  def compare(str1: String, str2: String) = {
    (groupIt(str1), groupIt(str2)) match {
      case ("","") => 0
      case (dec(x),dec(y)) if (x.toInt==y.toInt) =>  
         compare(str1.substring(x.size), str2.substring(y.size))
      case (dec(x),dec(y)) => (x.toInt - y.toInt)
      case (x,y) if (x == y) =>
         compare(str1.substring(x.size), str2.substring(y.size))
      case (x,y) => x compareTo y
    }
  }
}


And now a usage case :

test("basic tests") {
  import scala.collection.immutable.TreeSet

  val t0 = new TreeSet[String]() ++ List("10", "5", "1")
  t0.toList should equal (List("1", "5", "10"))

  val t1 = new TreeSet[String]() ++ List("a-100","a-10", "a-5", "a-1")
  t1.toList should equal (List("a-1", "a-5", "a-10", "a-100"))
}


Of course it doesn't manage floating numbers, "1.01", will be sorted as "1.1"... or "a-2-050" will appear after "a-2-25".

CONTEXT : Linux Gentoo / Scala 2.9.0 / Java 1.6.0_26

Wednesday, August 17, 2011

Scala script to merge and archive PDF Files...

Just configure for example thunar action, in order to be able to select N pdf files, right-click and then launch the script. It will merge the pdf file into a single one with a new name starting with a timestamp. Previous files are backup into a Trash directory, each file being renamed with its path. Example :

bill.pdf check.pdf order.pdf   ==> 20110810-bill.pdf

#!/bin/sh
exec scala -nocompdaemon -savecompiled "$0" "$@"
!#
import sys.process._
if (args.size>0) {
  def now     = new java.util.Date()
  val defname = args(0).split("/").last
  val sdf     = new java.text.SimpleDateFormat("yyyyMMdd")
  val dest    = "%s-%s".format(sdf.format(now),defname)
  val res = Process("/usr/bin/pdftk"::args.toList:::"cat"::"output"::dest::Nil) !!
  val trash = scala.util.Properties.userHome+"/"+"Trash"+"/"
  Process("mkdir"::"-p"::trash::Nil)!
  
  for (file <- args) {
    Process("mv"::file::(trash+file.replace("/","++")+"-"+sdf.format(now))::Nil)!
  }
}


Of course this script relies on "app-text/pdftk-1.44" (gentoo package name), for more information see here : pdftk-the-pdf-toolkit.
CONTEXT : Linux Gentoo / Scala 2.9.0 / Java 1.6.0_26 / XFCE 4.8.0 / Thunar 1.2.1

Using scala as a script system admin language is quite cool...

No comment is necessary ! (Using Linux and scala 2.9.x)

import scala.sys.process._
"date"!
"date" #>> new java.io.File("dates.tmp") !
"cat dates.tmp" !

val files="ls"!!
val filteredFiles="ls" #| "grep .tar.gz" !!
val filesArray=("ls"!!).trim.split("\\n")

Using this language and its advanced features make system scripting quite easy and safer. No more problem with space in file names or directories, and so on ... sh coding is really a nightmare.

CONTEXT : Linux Gentoo / Scala 2.9.0 / Java 1.6.0_26