Saturday, November 12, 2011

IO Read binary file with Stream continually helper method

A straightforward approach to read a binary stream in Scala. Using continually helper Stream method gives to the code a functional taste.
#!/bin/sh
exec scala -J-Xmx1g -J-Xms512m -savecompiled "$0" "$@"
!#

import java.io._
import util.Properties.{userHome}
import java.io.File.{separatorChar=> sep, pathSeparatorChar=>pathSep}

def readBinaryFile(input:InputStream):Array[Byte] = {
  val fos = new ByteArrayOutputStream(65535)
  val bis = new BufferedInputStream(input)
  val buf = new Array[Byte](1024)
  Stream.continually(bis.read(buf))
      .takeWhile(_ != -1)
      .foreach(fos.write(buf, 0, _))
  fos.toByteArray
}

// Read 191Mo file
val fname=userHome+sep+"LibO_3.4.4_Win_x86_install_multi.exe"
val fb = readBinaryFile(new FileInputStream(fname))
println("%s, %d bytes".format(fname, fb.size))
This is quite better than writing :
  var c=0
  do {
    c = bis.read(buf)
    if (c > 0) fos.write(buf,0,c)
  } while(c > -1)

3 comments:

  1. yes, but is it perhaps a lot slower? Did you test it?

    ReplyDelete
  2. Didn't notice any performance problem with this way of writing IO. I'm used to read input streams this way in many different situation; the benefits is that you don't have any var.

    ReplyDelete
  3. Thanks for this example. You helped me understand how to use Stream.continually with takeWhile and foreach as I parse an absolutely enormous JSON file.

    ReplyDelete