Thursday, February 23, 2012

Revisiting Git/SVN Update script... and let's try to make it faster

The following script is an enhancement of the one shown in the previous post. It now detects all git or svn projects in the current directory, and then update and compile them.

It could have been straightforward to make it faster by just turning "project" collection into a parallel one (projects.par) but unfortunately, it won't work because of the use of a global mutable variable used to store the current worker directory (Classical trap) ! So more work is mandatory to make this faster.
#!/bin/sh
exec scala -deprecation -savecompiled "$0" "$@"
!#
import java.io.File
import sys.process.Process
import sys.process.ProcessBuilder._

// ======================================================================
case class CurDir(cwd:File)
def file(f:String)=new File(f)
def file(d:File, f:String)=new File(d, f)
implicit def stringToCurDir(d:String) = CurDir(file(d))
implicit def fileToCurDir(d:File) = CurDir(d)
implicit def stringToFile(f:String) = file(f)
implicit def stringToProcess(cmd: String)(implicit curDir:CurDir) = Process(cmd, curDir.cwd)
implicit var cwd:CurDir=file(".").getPath
def cd(dir:File=file(util.Properties.userDir)) = cwd=CurDir(dir)
// ======================================================================

trait Project {
  val dir:File
}
case class GitProject(dir:File) extends Project
case class SvnProject(dir:File) extends Project

// Find GIT or SVN project directories
val projects = file(".").listFiles filter { _.isDirectory} flatMap { d =>
  if (file(d, ".git").exists) Some(GitProject(d))
  else if (file(d, ".svn").exists) Some(SvnProject(d))
  else None
}

// Update compile project (update also eclipse conf files generated by sbteclipse plugin)
for(project<- projects) {
  println(project)
  cd(project.dir)
  project match {
    case GitProject(_) => "git pull" !
    case SvnProject(_) => "svn update" !
  }
  "sbt eclipse compile" !
}
Let's now revisit the current directory hack to make it compatible with parallel processing and avoid border side effect.
Therefore, we have to modify the "cd" command hack in order to remove the mutable cwd variable :
#!/bin/sh
exec scala -deprecation -savecompiled "$0" "$@"
!#
import java.io.File
import sys.process.Process
import sys.process.ProcessBuilder._

// ======================================================================
case class CurDir(cwd:File)
def file(f:String)=new File(f)
def file(d:File, f:String)=new File(d, f)
implicit def stringToCurDir(d:String) = CurDir(file(d))
implicit def fileToCurDir(d:File) = CurDir(d)
implicit def stringToFile(f:String) = file(f)
implicit def stringToProcess(cmd: String)(implicit curDir:CurDir) = Process(cmd, curDir.cwd)
def cd[A](dir:File=file(util.Properties.userDir)) (indir:(CurDir)=>A) = indir(CurDir(dir))
// ======================================================================

trait Project {
  val dir:File
}
case class GitProject(dir:File) extends Project
case class SvnProject(dir:File) extends Project

// Find GIT or SVN project directories
val projects = file(".").listFiles filter { _.isDirectory} flatMap { d =>
  if (file(d, ".git").exists) Some(GitProject(d))
  else if (file(d, ".svn").exists) Some(SvnProject(d))
  else None
}

// Update compile project (update also eclipse conf files generated by sbteclipse plugin)
for(project<- projects.par) {
  println(project)
  cd(project.dir) { implicit cwd =>
    project match {
      case GitProject(_) => "git pull" !
      case SvnProject(_) => "svn update" !
    }
    "sbt eclipse compile" !
  }
}

println("%d project updated".format(projects.size))

So we couln't keep the classical shell cd behavior, because it implies a mutable state, so in this script we turn the cd "classical" definition into a new one, with a new argument, a function which will take as argument, the given working directory to use.

In my case (7 projects), using "projects.par", the script executes in 10s instead of 33s !

CONTEXT : Scala 2.9.1

No comments:

Post a Comment