Thursday, April 26, 2012

sbt scala drools expert system skeleton project

Just released a simple scala drools project skeleton. The Knowledge base is used to evaluate drools-mvel / scala interaction.
The Knowledge Base :
package dummy;

dialect "mvel"

import java.util.LinkedList;

// --------------------------------------------------------------

rule "who is young"
  when
    Someone($name:name, age < 30)
  then
    System.out.println($name+" is very young")
end

// --------------------------------------------------------------

rule "people cars"
  when
    $someone:Someone($name:name)
    $car : Car(someone == $someone)
  then
    System.out.println("One of "+$name+"'s car is "+$car.color.name+" "+$car.model)
end

// --------------------------------------------------------------

rule "who own at least one red car"
  when
    $someone:Someone($name:name)
    exists Car(someone == $someone, color == Color.red)
  then
    System.out.println($name+" has at least one red car")
end

// --------------------------------------------------------------

rule "who doesn't have a home"
  when
    $someone:Someone($name:name)
    not Home(someone == $someone)
  then
    insert(new InformationRequest($someone, $name+"'s home is unknown"))
end

// --------------------------------------------------------------

rule "who doesn't have an address"
  when
    $someone:Someone($name:name)
    Home(someone == $someone, !address.isDefined)
  then
    insert(new InformationRequest($someone, $name+" address is unknown"))
end

// --------------------------------------------------------------

rule "process information requests"
  when
    InformationRequest($who:someone, $what:message)
  then
    System.out.println("INFORMATION REQUESTED FOR "+$who+" : "+$what)
end

// --------------------------------------------------------------

rule "who owns more than 1 car"
  when
    $someone : Someone($name:name)
    $cars    : LinkedList( size>1 ) from collect(Car(someone == $someone))
  then
    System.out.println($name+" has "+$cars.size+" cars")  
end

// --------------------------------------------------------------

And the startup code :
package dummy

// ----------------------------------------------------------------
// DOMAIN MODEL

case class Someone(name:String, age:Int)

case class Car(someone:Someone, model:String, year:Int, color:Color)

case class Color(name:String)

object Color {
  val red = Color("red")
  val blue = Color("blue")
  val green = Color("green")
  val black = Color("black")
}

case class Address(street:String, town:String, country:String)

case class Home(someone:Someone, address:Option[Address]) 

case class InformationRequest(someone:Someone, message:String)




// ----------------------------------------------------------------


import java.io.FileInputStream
import java.io.InputStreamReader
import org.drools.RuleBaseFactory
import org.drools.audit.WorkingMemoryFileLogger
import org.drools.compiler.PackageBuilder


object Dummy {

  def main(args: Array[String]) {
    System.setProperty("drools.dialect.mvel.strict", "false")
    
    val rulesfilename = "src/main/resources/KBExpertise.drl"
    val source = new InputStreamReader(new FileInputStream(rulesfilename))
    
    val builder = new PackageBuilder()
    builder.addPackageFromDrl(source)
    if (builder.hasErrors()) {
      System.out.println(builder.getErrors().toString())
      throw new RuntimeException("Unable to compile " + rulesfilename + ".")
    }

    val pkg = builder.getPackage()
    val ruleBase = RuleBaseFactory.newRuleBase()
    ruleBase.addPackage(pkg)

    val session = ruleBase.newStatefulSession()
    //session.addEventListener(new org.drools.event.DebugAgendaEventListener())
    //session.addEventListener(new org.drools.event.DebugWorkingMemoryEventListener())
    
    // setup the audit logging
    val logger:WorkingMemoryFileLogger = new WorkingMemoryFileLogger(session)
    logger.setFileName("drools")
    logger.writeToDisk()

    val martine = Someone(name="Martine", age=30)
    val martin  = Someone(name="Martin", age=40)
    val jack    = Someone(name="Jack", age=12)
    val martineCar = Car(martine, "Ford", 2010, Color.blue)
    val martinCar  = Car(martin, "GM", 2010, Color.black)
    val martinCar2 = Car(martin, "Ferrari", 2012, Color.red)
    val martinCar3 = Car(martin, "Porshe", 2011, Color.red)
    
    val martinHome = Home(martin, None)
    val jackHome   = Home(jack, Some(Address("221B Baker Street", "London", "England")))
    
    List(martine, martin, jack, 
        martineCar, martinCar, martinCar2, martinCar3,
        martinHome, jackHome
    ).foreach(session.insert(_)) 

    session.fireAllRules()

    session.dispose()
  }
}
To finish, the build.sbt file, to build and run the project :
import AssemblyKeys._

seq(assemblySettings: _*)

name := "ScalaDroolsDummyProject"

version := "0.0.1"

scalaVersion := "2.9.2"

mainClass in assembly := Some("dummy.Dummy")

jarName in assembly := "dummy.jar"



libraryDependencies += "org.drools" % "drools-compiler" % "5.3.1.Final"

libraryDependencies += "org.drools" % "drools-core" % "5.3.1.Final"

libraryDependencies += "org.drools" % "drools-jsr94"  % "5.3.1.Final"

libraryDependencies += "org.drools" % "drools-decisiontables"  % "5.3.1.Final"

libraryDependencies += "org.drools" % "knowledge-api"  % "5.3.1.Final"



libraryDependencies += "com.thoughtworks.xstream" % "xstream" % "1.4.2"
            


libraryDependencies += "org.scalatest" %% "scalatest" % "1.7.2" % "test"

libraryDependencies += "junit" % "junit" % "4.10" % "test"

resolvers += "JBoss third party releases repository" at "https://repository.jboss.org/nexus/content/repositories/thirdparty-releases"

Test it by your self :
$ sbt run
INFORMATION REQUESTED FOR Someone(Martin,40) : Martin address is unknown
One of Martin's car is red Porshe
Martin has 3 cars
Martin has at least one red car
One of Martin's car is red Ferrari
One of Martin's car is black GM
One of Martine's car is blue Ford
Jack is very young
INFORMATION REQUESTED FOR Someone(Martine,30) : Martine's home is unknown
[success] Total time: 6 s, completed 27 avr. 2012 22:35:37
And for those who don't know about SBT or GIT :
  • Download SBT jar file
    • You'll get a java jar executable named sbt-launch.jar
  • Download the scala drools skeleton zip archive
    • You'll get a file which will look like : "dacr-scala-drools-dummy-project-6b43143.zip"
    • unzip the content and rename the unzipped directory to scala-drools-dummy-project
test@localhost /tmp/test $ unzip dacr-scala-drools-dummy-project-6b43143.zip 
Archive:  dacr-scala-drools-dummy-project-6b43143.zip
6b43143f369bda03361ca2c5f2376d51ba7d5524
   creating: dacr-scala-drools-dummy-project-6b43143/
  inflating: dacr-scala-drools-dummy-project-6b43143/.gitignore  
  inflating: dacr-scala-drools-dummy-project-6b43143/README  
 extracting: dacr-scala-drools-dummy-project-6b43143/RELEASE-NOTES  
  inflating: dacr-scala-drools-dummy-project-6b43143/build.sbt  
  inflating: dacr-scala-drools-dummy-project-6b43143/cleanup.sh  
   creating: dacr-scala-drools-dummy-project-6b43143/project/
  inflating: dacr-scala-drools-dummy-project-6b43143/project/plugins.sbt  
   creating: dacr-scala-drools-dummy-project-6b43143/src/
   creating: dacr-scala-drools-dummy-project-6b43143/src/main/
   creating: dacr-scala-drools-dummy-project-6b43143/src/main/resources/
  inflating: dacr-scala-drools-dummy-project-6b43143/src/main/resources/KBExpertise.drl  
   creating: dacr-scala-drools-dummy-project-6b43143/src/main/scala/
   creating: dacr-scala-drools-dummy-project-6b43143/src/main/scala/dummy/
  inflating: dacr-scala-drools-dummy-project-6b43143/src/main/scala/dummy/Dummy.scala  
   creating: dacr-scala-drools-dummy-project-6b43143/src/test/
   creating: dacr-scala-drools-dummy-project-6b43143/src/test/scala/
   creating: dacr-scala-drools-dummy-project-6b43143/src/test/scala/dummy/
  inflating: dacr-scala-drools-dummy-project-6b43143/src/test/scala/dummy/DummyTest.scala  
test@localhost /tmp/test $ mv dacr-scala-drools-dummy-project-6b43143 scala-drools-dummy-project
test@localhost /tmp/test $ cd scala-drools-dummy-project
test@localhost /tmp/test/scala-drools-dummy-project $ java -jar ~/Downloads/sbt-launch.jar run
[info] Loading project definition from /tmp/test/scala-drools-dummy-project/project
[info] Set current project to ScalaDroolsDummyProject (in build file:/tmp/test/scala-drools-dummy-project/)
[info] Updating {file:/tmp/test/scala-drools-dummy-project/}default-d556eb...
[info] Resolving org.scala-lang#scala-library;2.9.2 ...
[info] Resolving org.drools#drools-compiler;5.3.1.Final ...
...
[info] Compiling 1 Scala source to /tmp/test/scala-drools-dummy-project/target/scala-2.9.2/classes...
[info] Running dummy.Dummy 
INFORMATION REQUESTED FOR Someone(Martin,40) : Martin address is unknown
One of Martin's car is red Porshe
Martin has 3 cars
Martin has at least one red car
One of Martin's car is red Ferrari
One of Martin's car is black GM
One of Martine's car is blue Ford
Jack is very young
INFORMATION REQUESTED FOR Someone(Martine,30) : Martine's home is unknown
[success] Total time: 8 s, completed 30 avr. 2012 12:59:17

Friday, April 20, 2012

JAJMX scala script to list remote JVM threads using JMX...

A new example script using JAJMX API. This script, named lsthreads, connects to a remote JVM using just the host adress and the used JMX port, and then list all active threads and theirs current states. Some java options (JAVA_OPTS) have been added to allow using this script against itself.
#!/bin/sh
JAVA_OPTS=""
JAVA_OPTS=$JAVA_OPTS" -Dcom.sun.management.jmxremote"
JAVA_OPTS=$JAVA_OPTS" -Dcom.sun.management.jmxremote.port=9999"
JAVA_OPTS=$JAVA_OPTS" -Dcom.sun.management.jmxremote.authenticate=false"
JAVA_OPTS=$JAVA_OPTS" -Dcom.sun.management.jmxremote.ssl=false"
exec java $JAVA_OPTS -jar jajmx.jar -nocompdaemon -usejavacp -savecompiled "$0" "$@"
!#
import jajmx._
import javax.management.openmbean.CompositeData

val host = if (args.size>=1) args(0) else "localhost" 
val port = if (args.size>=2) args(1).toInt else 9999

case class ThreadLock(name:String, ownerId:Option[Long], ownerName:Option[String])

case class ThreadInfo(name:String, id:Long, state:String,lock:Option[ThreadLock],
                      blockedCount:Long, waitedCount:Long)

object ThreadInfo {
  def apply(ti:CompositeData) :ThreadInfo = {
   val id           = ti.get("threadId").toString.toLong
   val name         = ti.get("threadName").toString
   val state        = ti.get("threadState").toString
   val blockedCount = ti.get("blockedCount").toString.toLong
   val waitedCount  = ti.get("waitedCount").toString.toLong
   val lock = Option(ti.get("lockName")) map {_.toString} filter {_.size>0} collect {
                case lockName =>
                    val lockOwnerId = Option(ti.get("lockOwnerId")) map {_.asInstanceOf[Long]} filterNot {_ == -1}
                    val lockOwnerName = Option(ti.get("lockOwnerName")) map {_.toString.trim}
                    ThreadLock(lockName, lockOwnerId, lockOwnerName)
              }
   ThreadInfo(name, id, state, lock, blockedCount, waitedCount)
  }
}

JMX.once(host,port) { jmx =>
  for (threading <- jmx.threading) {
    val ids = threading.get[Array[Long]]("AllThreadIds")
    val rawThreadsInfos = threading.call[Array[CompositeData]]("getThreadInfo", ids).get
    val threadsInfos = rawThreadsInfos map {ThreadInfo(_)} toList
    val countByState = threadsInfos groupBy {_.state} map { case (s,l) => s -> l.size}
    val countByStateStr = countByState map {case (s,l) => s+":"+l} mkString " "
    println("Total %d  %s".format(threadsInfos.size, countByStateStr))
    for ( ti <- threadsInfos sortBy {_.id } ) {
      println("%d - %s - %s".format(ti.id, ti.state, ti.name) )
    }
  }
}

Usage examples, with itself and then with a remote apache tomcat:
$ lsthreads
Total 13  TIMED_WAITING:4 WAITING:2 RUNNABLE:7
1 - RUNNABLE - main
2 - WAITING - Reference Handler
3 - WAITING - Finalizer
4 - RUNNABLE - Signal Dispatcher
9 - RUNNABLE - RMI TCP Accept-0
10 - RUNNABLE - RMI TCP Accept-9999
11 - RUNNABLE - RMI TCP Accept-0
12 - RUNNABLE - RMI TCP Connection(1)-127.0.0.1
13 - TIMED_WAITING - RMI Scheduler(0)
14 - TIMED_WAITING - RMI RenewClean-[127.0.0.1:45376]
15 - TIMED_WAITING - GC Daemon
16 - RUNNABLE - RMI TCP Connection(2)-127.0.0.1
17 - TIMED_WAITING - JMX server connection timeout 17

$ lsthreads 192.168.1.10 1099
Total 17  TIMED_WAITING:6 WAITING:2 RUNNABLE:9
1 - RUNNABLE - main
2 - WAITING - Reference Handler
3 - WAITING - Finalizer
4 - RUNNABLE - Signal Dispatcher
10 - RUNNABLE - RMI TCP Accept-0
11 - RUNNABLE - RMI TCP Accept-1099
12 - RUNNABLE - RMI TCP Accept-0
13 - TIMED_WAITING - GC Daemon
16 - TIMED_WAITING - ContainerBackgroundProcessor[StandardEngine[Catalina]]
17 - RUNNABLE - http-bio-8080-Acceptor-0
18 - TIMED_WAITING - http-bio-8080-AsyncTimeout
19 - RUNNABLE - ajp-bio-8009-Acceptor-0
20 - TIMED_WAITING - ajp-bio-8009-AsyncTimeout
22 - RUNNABLE - RMI TCP Connection(5)-127.0.0.1
23 - TIMED_WAITING - RMI Scheduler(0)
24 - RUNNABLE - RMI TCP Connection(6)-127.0.0.1
27 - TIMED_WAITING - JMX server connection timeout 27

Wednesday, April 18, 2012

First nasdaq stock query using scala and JASeries

The following script is a first attempt to make a stock query over nasdaq using JASeries (scala time series API).
#!/bin/sh
exec java -jar jaseries.jar -nocompdaemon -usejavacp -savecompiled "$0" "$@"
!#

import io.Source
import fr.janalyse.series._

// Get nasdaq codes
val codes = 
  Source.fromURL("http://www.nasdaqtrader.com/dynamic/SymDir/nasdaqlisted.txt")
    .getLines
    .map(_.split("[|]",2).head)
    .toList.tail.init //.take(10)

// Get nasdaq stocks history
val stocks =
  codes.flatMap(code =>
    try {
     Some(code -> CSV2Series.fromURL("http://ichart.finance.yahoo.com/table.csv?s="+code))
    } catch {
      case x:java.io.FileNotFoundException => 
        println("Could'nt find %s stock data".format(code))
        None
    }
  )

// get close series
val closes = stocks flatMap { case (code, allSeries) => 
  allSeries.get("Close").map(_.rename(code+" Close"))
}

val top10incr =
  closes
    .sortBy( _.stat.linearApproximation.slope)
    .take(20)
    
// print highest slope top20, but remember that all history data is taken into
// account, so the last value may not be quiter higher than the first one.
for(stock <- top10incr) {
  println("%s : %f -> %f".format(stock.name, stock.head.value, stock.last.value))
}

This script is the starting point to next JASeries improvements and new features.

Saturday, April 14, 2012

JAJMX scala script to list all available JVM numerical JMX values

Using JAJMX it becomes very easy to list all available numerical values of a given JVM JMX plateform :
#!/bin/sh
JAVA_OPTS=""
JAVA_OPTS=$JAVA_OPTS" -Dcom.sun.management.jmxremote"
JAVA_OPTS=$JAVA_OPTS" -Dcom.sun.management.jmxremote.port=9999"
JAVA_OPTS=$JAVA_OPTS" -Dcom.sun.management.jmxremote.authenticate=false"
JAVA_OPTS=$JAVA_OPTS" -Dcom.sun.management.jmxremote.ssl=false"
exec java $JAVA_OPTS -jar jajmx.jar -nocompdaemon -usejavacp -savecompiled "$0" "$@"
!#

import fr.janalyse.jmx._

val host  = args.headOption.getOrElse("localhost")
val port  = if (args.size>1) args.tail.head.toInt else 9999

JMX.once(host, port) { jmx =>
  for(mbean <- jmx.mbeans ;
      attr  <- mbean.attributes collect {case n:RichNumberAttribute => n};
      value <- mbean.getLong(attr)) {
    println("%s - %s = %d".format(mbean.name, attr.name, value))
  }
}
Some jvm parameters (JAVA_OPTS) have been added in order to make possible to test this script against itself. So without any arguments it will give you such results :
$ ./lsnum
java.lang:type=Memory - ObjectPendingFinalizationCount = 0
java.lang:type=MemoryPool,name=PS Eden Space - CollectionUsageThreshold = 0
java.lang:type=MemoryPool,name=PS Eden Space - CollectionUsageThresholdCount = 0
java.lang:type=MemoryPool,name=PS Survivor Space - CollectionUsageThreshold = 0
java.lang:type=MemoryPool,name=PS Survivor Space - CollectionUsageThresholdCount = 0
java.lang:type=MemoryPool,name=Code Cache - UsageThreshold = 0
java.lang:type=MemoryPool,name=Code Cache - UsageThresholdCount = 0
java.lang:type=GarbageCollector,name=PS MarkSweep - CollectionCount = 1
java.lang:type=GarbageCollector,name=PS MarkSweep - CollectionTime = 90
java.lang:type=Runtime - StartTime = 1334354060679
java.lang:type=Runtime - Uptime = 5497
java.lang:type=ClassLoading - LoadedClassCount = 4806
java.lang:type=ClassLoading - UnloadedClassCount = 0
java.lang:type=ClassLoading - TotalLoadedClassCount = 4806
java.lang:type=Threading - DaemonThreadCount = 12
java.lang:type=Threading - PeakThreadCount = 13
java.lang:type=Threading - CurrentThreadCpuTime = 40000000
java.lang:type=Threading - CurrentThreadUserTime = 40000000
java.lang:type=Threading - ThreadCount = 13
java.lang:type=Threading - TotalStartedThreadCount = 13
java.lang:type=Compilation - TotalCompilationTime = 7144
java.lang:type=MemoryPool,name=PS Perm Gen - CollectionUsageThreshold = 0
java.lang:type=MemoryPool,name=PS Perm Gen - CollectionUsageThresholdCount = 0
java.lang:type=MemoryPool,name=PS Perm Gen - UsageThreshold = 0
java.lang:type=MemoryPool,name=PS Perm Gen - UsageThresholdCount = 0
java.lang:type=GarbageCollector,name=PS Scavenge - CollectionCount = 3
java.lang:type=GarbageCollector,name=PS Scavenge - CollectionTime = 57
java.lang:type=OperatingSystem - MaxFileDescriptorCount = 4096
java.lang:type=OperatingSystem - OpenFileDescriptorCount = 19
java.lang:type=OperatingSystem - CommittedVirtualMemorySize = 4598407168
java.lang:type=OperatingSystem - FreePhysicalMemorySize = 10819850240
java.lang:type=OperatingSystem - FreeSwapSpaceSize = 10742177792
java.lang:type=OperatingSystem - ProcessCpuTime = 13030000000
java.lang:type=OperatingSystem - TotalPhysicalMemorySize = 16816541696
java.lang:type=OperatingSystem - TotalSwapSpaceSize = 10742177792
java.lang:type=OperatingSystem - AvailableProcessors = 6
java.lang:type=OperatingSystem - SystemLoadAverage = 0
java.lang:type=MemoryPool,name=PS Old Gen - CollectionUsageThreshold = 0
java.lang:type=MemoryPool,name=PS Old Gen - CollectionUsageThresholdCount = 0
java.lang:type=MemoryPool,name=PS Old Gen - UsageThreshold = 0
java.lang:type=MemoryPool,name=PS Old Gen - UsageThresholdCount = 0

Friday, April 13, 2012

JASeries (JAnalyseSeries) 1.1.0 released

JASeries is a time numeric series operations API. The goal is to make simple, time series summaries generation, using sampling and various kind of cells merging. ( ** project page **)

Latest changes :

  • Statistics : alive added = OpenCell.time - CloseCell.time = series alive period of time
  • realias() without any argument means take the name as the alias.
  • added braces to delta, compact, zeroBased
  • Chart seriesScales method renamed to scales
  • Chart new method : colors which returns a map of [Series[Cell], Color]
  • Added a package object jaseries to define shortcuts to fr.janalyse.series.CSV2Series, fr.janalyse.series.view.Chart classes and objects

Google stock summary catchup scala script

#!/bin/sh
exec java -jar jaseries.jar -nocompdaemon -usejavacp -savecompiled "$0" "$@"
!#

val allSeries = jaseries.CSV2Series.fromURL("http://ichart.finance.yahoo.com/table.csv?s=GOOG")
val closeSeries = allSeries("Close")

println("GOOGLE stock summary")
println("Higher : "+closeSeries.max)
println("Lowest : "+closeSeries.min)
println("Week Trend : "+closeSeries.stat.linearApproximation.daySlope*7)
println("Latest : "+closeSeries.last)

Apple trend chart generation scala script

#!/bin/sh
exec java -jar jaseries.jar -nocompdaemon -usejavacp -savecompiled "$0" "$@"
!#

val allSeries = jaseries.CSV2Series.fromURL("http://ichart.finance.yahoo.com/table.csv?s=AAPL")
val closeSeries = allSeries("Close").realias("Apple stock value")    
jaseries.Chart(closeSeries).toFile("AppleStockTrend.jpg")

JAJMX (JAnalyseJMX) 0.3.0 released

JAJMX is high level scala JMX API. The goal is to simplify to the maximum JMX operations. ( ** project page **)

Latest changes :
  • JMXOptions now contains an extra field "name" which allow user to friendly identify a remote jmx system
  • added a package object jajmx to define shortcuts to fr.janalyse.jmx.JMX class and object
  • JMX.connect renamed to JMX.once
  • scaladoc generation now works fine, follow hack described here : https://github.com/harrah/xsbt/issues/85

jvm force gc script

#!/bin/sh
exec java -jar jajmx.jar -nocompdaemon -usejavacp -savecompiled "$0" "$@"
!#

jajmx.JMX.once("127.0.0.1", 1099) { _.memory map {_ call "gc"} }
 

jmx grep script

#!/bin/sh
exec java -jar jajmx.jar -nocompdaemon -usejavacp -savecompiled "$0" "$@"
!#

import jajmx._

if (args.size < 2) {
  println("Usage   : jmxgrep host port searchMask1 ... searchMaskN")
  System.exit(1)
}
val host  = args(0)
val port  = args(1).toInt
val masks = args.toList.drop(2) map {s=>("(?i)"+s).r}

def truncate(str:String, n:Int=60) = if (str.size>n) str.take(n)+"..." else str

JMX.once(host, port) { jmx =>
  for(mbean <- jmx.mbeans ; attr <- mbean.attributes; value <- mbean.getString(attr)) {
    val found = List(mbean.name, attr.name, value) exists { 
         item => masks exists {_.findFirstIn(item).isDefined }
    }
    if (found) println("%s - %s = %s".format(mbean.name, attr.name, truncate(value)))
  }
}


Thursday, April 12, 2012

JASSH (JAnalyseSSH) 0.7.3 released

JASSH is a high level scala SSH API for easy and fast operations on remote servers. ( ** project page **)

Latest changes :
  • JSCH updated to release 0.1.47
  • SSHOptions now contains an extra field "name" which allow user to friendly identify a remote ssh system
  • SSHOptions password type is now of SSHPassword type instead of String. Implicit conversions are provided from String, Option[String]
  • SSHShell batch method renamed to execute

hello scala script

#!/bin/sh
exec java -jar jassh.jar -nocompdaemon -usejavacp -savecompiled "$0" "$@"
!#

print(jassh.SSH.shell("localhost", "test", "testtest") {_ execute "echo Hello `hostname`" } )
  
  

remote vmstat scala script

#!/bin/sh
exec java -jar jassh.jar -nocompdaemon -usejavacp -savecompiled "$0" "$@"
!#

jassh.SSH.once("localhost", "test", "testtest")  {
  _.run("vmstat 1 10", println(_.getOrElse("")).waitForEnd
}

 

Friday, April 6, 2012

JASSH (JAnalyseSSH) 0.7.2 released

JASSH is a high level scala SSH API for easy and fast operations on remote servers. ( ** project page **)

Latest changes :
  • added a package object jassh to define shortcuts to fr.janalyse.ssh.SSH class and object
  • SSHOptions, host parameter is now in first position !

These small changes enable even simpler scripts :

hello scala script

#!/bin/sh
exec java -jar jassh.jar -nocompdaemon -usejavacp -savecompiled "$0" "$@"
!#

println(jassh.SSH.shell("localhost", "test", Some("testtest")) { _ execute "echo -n Hello `hostname`"})

remote vmstat scala script

#!/bin/sh
exec java -jar jassh.jar -nocompdaemon -usejavacp -savecompiled "$0" "$@"
!#

jassh.SSH.once("localhost", "test", password=Some("testtest"))  {
  _.run("vmstat 1 10", println(_.getOrElse("")).waitForEnd
}

Sunday, April 1, 2012

Leave it crash, akka will restart it for you !!!

Akka is an awesome tool, even using default configuration, you do not have anymore to worry about failure management and recovery details. I've been bluffed to discover that my new monitoring system prototype (akka/scala based) was able to recovery by itself from errors, while I didn't yet implement such features !!

So In order to illustrate this feature, I've written a simple example that illustrates this behavior. The RobustActor actor keeps a ssh connection to a remote system (using janalyse-ssh scala ssh api), and execute/print every 5s the "date" command :
package dummy

import akka.actor.ActorSystem
import akka.actor.Actor
import akka.util.duration._
import com.typesafe.config.ConfigFactory
import akka.actor.Props

import fr.janalyse.ssh._

object DummySSH {
 def main(args:Array[String]) {
      val system=ActorSystem("DummySSHSystem",ConfigFactory.load.getConfig("dummySSH"))
      system.actorOf(
        Props(new RobustActor(system)),
        name="RobustActor")
 }
}


class RobustActor(system:ActorSystem) extends Actor {
  
  val sh = SSH(host="localhost", username="test", password=Some("testtest")).newShell
  
  override def preStart() {
    system.scheduler.schedule(1 seconds, 5 seconds, self, "doit")
  }
  
  override def postStop() {
    sh.close()
  }
    
  def receive = {
    case "doit" => print(sh execute "date")
  }
}
If we start this example code, we wait a little, we identify the sshd process that manages the established ssh connection and then we kill the sshd process, akka will intercept RobustActor internal exception and restart this actor for us !
$ sbt run
[info] Loading project definition from /home/dcr/dev-new/akka-sandbox/project
[info] Set current project to AkkaSandbox (in build file:/home/dcr/dev-new/akka-sandbox/)
[info] Compiling 1 Scala source to /home/dcr/dev-new/akka-sandbox/target/scala-2.9.1/classes...

Multiple main classes detected, select one to run:

 [1] dummy.Dummy
 [2] dummy.DummySSH

Enter number: 2

[info] Running dummy.DummySSH 
dim. avril  1 22:15:32 CEST 2012
dim. avril  1 22:15:37 CEST 2012
dim. avril  1 22:15:42 CEST 2012
dim. avril  1 22:15:47 CEST 2012
[ERROR] [04/01/2012 22:15:53.92] [DummySSHSystem-akka.actor.default-dispatcher-3] [akka://DummySSHSystem/user/RobustActor] Pipe closed
java.io.IOException: Pipe closed
 at java.io.PipedInputStream.checkStateForReceive(PipedInputStream.java:244)
 at java.io.PipedInputStream.receive(PipedInputStream.java:210)
 at java.io.PipedOutputStream.write(PipedOutputStream.java:132)
 at java.io.OutputStream.write(OutputStream.java:58)
 at fr.janalyse.ssh.SSHShell$Producer.sendCommand(SSHAPI.scala:480)
 at fr.janalyse.ssh.SSHShell.sendCommand(SSHAPI.scala:475)
 at fr.janalyse.ssh.SSHShell.execute(SSHAPI.scala:448)
 at dummy.RobustActor$$anonfun$receive$1.apply(DummySSH.scala:34)
 at dummy.RobustActor$$anonfun$receive$1.apply(DummySSH.scala:33)
 at akka.actor.Actor$class.apply(Actor.scala:290)
 at dummy.RobustActor.apply(DummySSH.scala:21)
 at akka.actor.ActorCell.invoke(ActorCell.scala:617)
 at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:179)
 at akka.dispatch.Mailbox.run(Mailbox.scala:161)
 at akka.dispatch.ForkJoinExecutorConfigurator$MailboxExecutionTask.exec(AbstractDispatcher.scala:505)
 at akka.jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:259)
 at akka.jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:997)
 at akka.jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1495)
 at akka.jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)

dim. avril  1 22:15:54 CEST 2012
dim. avril  1 22:15:58 CEST 2012


To identify the process which manages our example ssh connection, just use the following commands, list all sshd processes which belongs to test user, then take the process that do not use your current test shell session.
test@lanfeust ~ $ ps -ef | grep "^test"
test      8824  8815  0 11:20 ?        00:00:00 sshd: test@pts/7 
test      8825  8824  0 11:20 pts/7    00:00:00 -bash
test     32468 32458  0 22:15 ?        00:00:00 sshd: test@pts/8 
test     32469 32468  0 22:15 pts/8    00:00:00 -bash
test     32474  8825  0 22:15 pts/7    00:00:00 ps -ef
test     32475  8825  0 22:15 pts/7    00:00:00 grep --colour=auto ^test
test@lanfeust ~ $ tty
/dev/pts/7
test@lanfeust ~ $ kill -9 32468
test@lanfeust ~ $ 

This example source code is available here : akka-sandbox

JASSH (JAnalyseSSH) 0.7.0 released

JAnalyseSSH is a jsch based scala SSH API which aims to simplify SSH operations.

The hello world script...

(It requires a local user named "test" with password "testtest", remember that you can remove the password, if your public key has been added in authorized_keys file of the test user)
#!/bin/sh
exec java -jar jassh.jar -usejavacp "$0" "$@"
!#
import fr.janalyse.ssh._
SSH.shell(host="localhost", username="test", password=Some("testtest")) { sh =>
  println(sh.execute("""echo -n "Hello World from `hostname`" """))
}

... and the remote vmstat script

#!/bin/sh
exec java -jar jassh.jar -usejavacp "$0" "$@"
!#
import fr.janalyse.ssh._
SSH.once(host="localhost", username="test", password=Some("testtest"))  { ssh =>
  val executor = ssh.run("vmstat 1 10", (l)=> l foreach {println(_)})
  executor.waitForEnd
}


Releases notes for 0.7.0

  • Added new method to SSH : newShell & newSftp for user to manage themselves shell and sftp session
  • Some internal changes to SSHExec class, in order to try to remove actor dependency. Mixing actors systems looks problematic
  • SSHShell new implementation, no more actors used, better performances and behavior, ... (throughput : 504 cmd/s using persistency)
  • SSHExec last result line is no longer lost
  • SSHOptions : new parameter : "prompt" to enable custom shell or console command to be use.
    prompt provide to SSHShell the way to separate command results
  • SSHOptions : connectionTimeout renamed into timeout
  • Various cleanup and enhancements
  • Tests : compare performances persistent SSHShell versus SSHExec commands throughputs
  • SSH : Add an execute immediate method which rely on SSHExec, not SSHShell ! (throughput : 93cmd/s)
    execOnce & execOnceAndTrim
  • SSHExec : Do not rely on DaemonActor/Actor anymore
  • SSHShell : Removed init Thread.sleep => Better performances (throughput : 37 cmd/s instead 1cmd/s)
  • SSH.connect becomes SSH.once
  • Removing apply in SSH class as it may encourage bad usage, and close not called