Wednesday, July 31, 2013

Generic collection compact functions : compact and compactBy

Truly generic collection compaction functions, with the second one you can even provide a custom merge. By compaction I mean merging consecutive "identical" values into a single one.

Monday, July 29, 2013

Serialization example using json4s & and a small github project for tests purposes

Once aware of json4s serialization limitations, it is quite simple to use :

{
  "receipts":[
    {
      "amount":15.0,
      "currency":{
        "jsonClass":"Euro"
      },
      "when":"2013-07-29T20:45:44.121Z",
      "where":{
        "jsonClass":"Address",
        "address":"XIII",
        "town":"Paris"
      },
      "what":"meal",
      "keywords":[
        "food",
        "4work"
      ]
    },
    {
      "amount":1.0,
      "currency":{
        "jsonClass":"Euro"
      },
      "when":"2013-07-29T20:45:44.121Z",
      "where":{
        "jsonClass":"Address",
        "address":"I",
        "town":"Paris"
      },
      "what":"bread",
      "keywords":[
        "food"
      ]
    }
  ],
  "description":"2013 archives"
}

Saturday, July 27, 2013

A scala script that counts the number of lines in source code files

The only dependencies of this script are :

  • "com.github.scala-incubator.io" %% "scala-io-core" % "0.4.2"
  • "com.github.scala-incubator.io" %% "scala-io-file" % "0.4.2"

scala-io provides really a great file operation abstraction layer for scala language.

On some of my projects, this script will print

$ time ./count-lines janalyse-*
----------------------------
All directories
global lines count : 9148 (in 101 files)
lines count for scala : 8875 (in 84 files)
lines count for sbt : 258 (in 14 files)
lines count for sh : 15 (in 3 files)
----------------------------
For directory : janalyse-jmx
global lines count : 1697 (in 21 files)
lines count for scala : 1607 (in 17 files)
lines count for sbt : 90 (in 4 files)
----------------------------
For directory : janalyse-ssh
global lines count : 3130 (in 34 files)
lines count for scala : 3064 (in 29 files)
lines count for sbt : 61 (in 4 files)
lines count for sh : 5 (in 1 files)
----------------------------
For directory : janalyse-snmp
global lines count : 342 (in 7 files)
lines count for scala : 308 (in 4 files)
lines count for sbt : 29 (in 2 files)
lines count for sh : 5 (in 1 files)
----------------------------
For directory : janalyse-series
global lines count : 3979 (in 39 files)
lines count for scala : 3896 (in 34 files)
lines count for sbt : 78 (in 4 files)
lines count for sh : 5 (in 1 files)

real 0m8.969s
user 0m14.058s
sys 0m0.819s

Tuesday, July 23, 2013

Something to know if you try to save heap memory during some processing...

I was used to use code block {} in order to separate logics, for example in report generator code while chaining report sections. But recently I noticed that my application was using a lot of heap memory while processing 6Gb of numerical series, and after some investigations I've found that all loaded series was kept in memory even once we exit the block. Thanks to google group and contributors.

The code example which fails with an OutOfMemoryError (using locally{} locally{} locally{} will give the same problem)

The simple work around :

Monday, July 22, 2013

Playing with generic numbers and collections through examples

Copy paste this code to a scala worksheet to view the results.

The final asquares method is fully generic both with collections and number types, you'll get this result :

  asquares(Seq(1,2,3))               //> res1: Seq[Int] = List(1, 4, 9)
  asquares(List(1,2,3))              //> res2: List[Int] = List(1, 4, 9)
  asquares(Vector(BigDecimal(1),BigDecimal(2),BigDecimal(3)))
                                     //> res3: Vector[BigDecimal] = Vector(1, 4, 9)
  asquares(Array(1,2,3))             //> res4: Array[Int] = Array(1.0, 4.0, 9.0)
  asquares(Set(2,4))                 //> res5: Set[Int] = Set(4, 16)

The right types are always returned !

Small hack to check if a remote HTTP server is accessible or not. Relying on just DNS resolution test is not enough, it is mandatory to try to get the content.