Monday, August 6, 2012

JASERIES (JAnalyseSeries) 1.2.0 released

JASERIES is a scala API for time numeric series operations. ( ** project page **)

Latest changes :
  • now using sbt 0.12
  • now using sbteclipse 2.1.0
  • now using sbt-assembly 0.8.3
  • now using scalatest 1.8
  • Series factories enhancements
  • showLegend boolean added to chartConfig.
  • Add support for QuoteCell (for stock series processing)
  • Various Code cleanup and enhancements.
  • Add date automatic support for google finance historical data (although the API will be shutdown ~ october 2012)
  • CSV2Series now supports direct parsing of Quote Series ( CSV2Series.quoteFromURL )
  • Automatic supports of CSV with all cells in quotes
  • new tests cases (quotes)
  • duration based takeRight and dropRight method added
  • chart classes refactoring
  • stacked chart type added : StackedChart class (BUT still work in progress as TableXYDataSet implies the same number of cells in each series !!!)
  • Series sample method now preserves cell type !! (was previously returning a StatCell!!)
  • statSample method added to Series to allow sampling using StatCell
  • toSeries method added to Series to allow cell type conversions
  • Series count2Rate method renamed to toRate

Apache total accesses counter to hit rate

The following example demonstrates how to convert a global http hit counter, into a hitrate series:
import fr.janalyse.series._
val csv = CSV2Series.fromURL("http://dnld.crosson.org/modstatus-totalaccesses.csv")
val totalaccesses=csv.values.head

val hitrate=
  totalaccesses
    .delta
    .toSeries[AddCell]
    .sample("10m")
    .toRate()
    .rename("http hit rate")

CSV2Series.toFile(hitrate, "hitrate.csv")

JVM processing time to JVM cpu usage

The following example demonstrates how to convert a java process cpu time counter (mbean : java.lang:type=OperatingSystem__ProcessCpuTime), into a cpu usage metric (here I didn't take into account the number of CPU, so max value = cpu count * 100):
import fr.janalyse.series._
val csv = CSV2Series.fromURL("http://dnld.crosson.org/processcputime.csv")
val cputime=csv.values.head

val cpucells = 
   for(Seq(a,b) <- (cputime / 1000d / 1000d).sliding(2).toIterable ) 
   yield a.time->(b.value-a.value)/(b.time-a.time)*100d

val cpuusage = Series[StatCell]("cpu usage percent", "5m") <<< cpucells

CSV2Series.toFile(cpuusage, "cpuusage.csv")

Wednesday, August 1, 2012

JAJMX (JAnalyseJMX) 0.5.0 released

JAJMX is high level scala JMX abstraction layer. The goal is to simplify to the maximum JMX operations. ( ** project page **)

Latest changes :
  • cleanup, refactoring & general enhancements
  • Support for both generic host/port and jmxservice url approaches (although hidden by default)
  • automatic jmxurl lookups using known formats (external jars may be required, as for example jboss-client.jar)
  • application server type detection (whoami method) : jboss, tomcat, jonas, webmethod, jetty taken into account
  • straightforward threads dumps method added (with automatic support of dumpAllThreads starting from Java 6)
  • JMXOptions can now be built from a JMXServiceURL + credentials
  • Performance enhancements (mbeaninfos cached)
  • README file added
  • JMX.apply now with optional username & password as arguments
  • new methods & fields for RichMBean : "attributeNames():List[String]" and "domain"

jvm force gc script

#!/bin/sh
JAVA_OPTS=""
JAVA_OPTS=$JAVA_OPTS" -Dcom.sun.management.jmxremote"
JAVA_OPTS=$JAVA_OPTS" -Dcom.sun.management.jmxremote.port=9999"
JAVA_OPTS=$JAVA_OPTS" -Dcom.sun.management.jmxremote.authenticate=false"
JAVA_OPTS=$JAVA_OPTS" -Dcom.sun.management.jmxremote.ssl=false"
exec java $JAVA_OPTS -jar jajmx.jar -nocompdaemon -usejavacp -savecompiled "$0" "$@"
!#

jajmx.JMX.once("127.0.0.1", 9999) { jmx =>
  jmx.memory map { _ set("Verbose", true)}  // Activate Verbose GC, to see on stdout next call effect 
  jmx.memory map {_ call "gc"}              // Force GC
}
usage example :
$ ./gcforce
Usage   : jmxgrep host port
  no args given so now let's connecting to myself, and force a gc...
[GC 33835K->800K(245952K), 0.0036500 secs]
[Full GC 800K->673K(245952K), 0.0443690 secs]
[GC 7455K->929K(245952K), 0.0007100 secs]
[Full GC 929K->774K(245952K), 0.0448880 secs]

lsthreads script

#!/bin/sh
JAVA_OPTS=""
JAVA_OPTS=$JAVA_OPTS" -Dcom.sun.management.jmxremote"
JAVA_OPTS=$JAVA_OPTS" -Dcom.sun.management.jmxremote.port=9999"
JAVA_OPTS=$JAVA_OPTS" -Dcom.sun.management.jmxremote.authenticate=false"
JAVA_OPTS=$JAVA_OPTS" -Dcom.sun.management.jmxremote.ssl=false"
exec java $JAVA_OPTS -jar jajmx.jar -nocompdaemon -usejavacp -savecompiled "$0" "$@"
!#
import jajmx._

val host = if (args.size>=1) args(0) else "localhost" 
val port = if (args.size>=2) args(1).toInt else 9999

JMX.once(host,port) { jmx =>
  for (dump <- jmx.threadsDump(0)) {
    val threads = dump.threads.toList
    val countByState = threads groupBy {_.status} map { case (s,l) => s -> l.size}
    val countByStateStr = countByState map {case (s,l) => s+":"+l} mkString " "
    println("Total %d  %s".format(threads.size, countByStateStr))
    for ( ti <- threads sortBy {_.id } ) {
      println("%d - %s - %s".format(ti.id, ti.status, ti.name) )
    }
  }
}
usage example :
$ ./lsthreads 
Total 14  TIMED_WAITING:5 WAITING:2 RUNNABLE:7
1 - RUNNABLE - main
2 - WAITING - Reference Handler
3 - WAITING - Finalizer
4 - RUNNABLE - Signal Dispatcher
9 - RUNNABLE - RMI TCP Accept-0
10 - RUNNABLE - RMI TCP Accept-9999
11 - RUNNABLE - RMI TCP Accept-0
13 - RUNNABLE - RMI TCP Connection(1)-127.0.0.1
14 - TIMED_WAITING - RMI Scheduler(0)
15 - TIMED_WAITING - RMI RenewClean-[127.0.0.1:40993]
16 - TIMED_WAITING - GC Daemon
17 - RUNNABLE - RMI TCP Connection(2)-127.0.0.1
20 - TIMED_WAITING - JMX server connection timeout 20
21 - TIMED_WAITING - Thread-4

jmx grep script

#!/bin/sh
JAVA_OPTS=""
JAVA_OPTS=$JAVA_OPTS" -Dcom.sun.management.jmxremote"
JAVA_OPTS=$JAVA_OPTS" -Dcom.sun.management.jmxremote.port=9999"
JAVA_OPTS=$JAVA_OPTS" -Dcom.sun.management.jmxremote.authenticate=false"
JAVA_OPTS=$JAVA_OPTS" -Dcom.sun.management.jmxremote.ssl=false"
exec java $JAVA_OPTS -jar jajmx.jar -nocompdaemon -usejavacp -savecompiled "$0" "$@"
!#

import jajmx._

if (args.size == 0) {
  println("Usage   : jmxgrep host port searchMask1 ... searchMaskN")
  println("  no args given so now let's connecting to myself, and list my mbeans...") 
}
val host  = if (args.size>0) args(0) else "localhost"
val port  = if (args.size>1) args(1).toInt else 9999
val masks = if (args.size>2) args.toList.drop(2) map {s=>("(?i)"+s).r} else List.empty[util.matching.Regex]


def truncate(str:String, n:Int=60) = {
  val nonl=str.replaceAll("\n", " ").replaceAll("\r", "")
  if (nonl.size>n) nonl.take(n)+"..." else nonl
}

JMX.once(host, port) { jmx =>
  for(mbean <- jmx.mbeans ; attr <- mbean.attributes; value <- mbean.getString(attr)) {
    val found = List(mbean.name, attr.name, value) exists { item => masks exists {_.findFirstIn(item).isDefined } }
    if (masks.isEmpty || found) println("%s - %s = %s".format(mbean.name, attr.name, truncate(value)))
  }
}
usage example :
$ ./jmxgrep localhost 9999 version
java.lang:type=Runtime - SpecVersion = 1.0
java.lang:type=Runtime - ManagementSpecVersion = 1.2
java.lang:type=Runtime - VmVersion = 20.8-b03
java.lang:type=Runtime - SystemProperties = javax.management.openmbean.TabularDataSupport(tabularType=ja...
java.lang:type=OperatingSystem - Version = 3.2.1-gentoo-r2
JMImplementation:type=MBeanServerDelegate - ImplementationVersion = 1.6.0_33-b03
JMImplementation:type=MBeanServerDelegate - SpecificationVersion = 1.4