Converting huge pdf to images using apache pdfbox

Recently I ran into issue where we wanted to convert pdf documents to series of images.

Ran into this stackoverflow article. This uses pdf-render to accomplish the task. However there were two issues with program

  • Code is little verbose
  • Code works well for small sized pdf. However once you have large pdfs ( more than 100 pages / 30-40 MB) you start running into Out of Memory error

So in search of better solution , I stumbled upon Apache PDFBox and it works beautifully addressing both the shortcomings listed above.

Below is snippet of code (with relevant imports)

import java.awt.image.BufferedImage;
import java.util.ArrayList;
import java.util.List;
import javax.imageio.ImageIO;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.pdmodel.PDPage;
import org.apache.pdfbox.pdmodel.PDPageNode;

public List<String> generateImages(String pdfFile) throws IOException  {
     String imagePath = "/Users/$user/pdfimages/";
     List <String> fileNames = new ArrayList<String>();
	  document = PDDocument.load(pdfFile);	//// load pdf	 
	  node = document.getDocumentCatalog().getPages(); ///// get pages
	  List<PDPage> kids = node.getKids();
	  int count=0;
	  for(PDPage page : kids) {   ///// iterate
	       BufferedImage img = page.convertToImage(BufferedImage.TYPE_INT_RGB,128);
	       File imageFile = new File(imagePath+ count++ + ".jpg");
               ImageIO.write(img, "jpg", imageFile);
           return fileNames;	

Fairly Simple, Straightforward and has low memory foot print

Covariance and Contravariance in scala

In this post I want to give an overview of Covariance and Contravariance in scala


Consider diagram below which conveys the concept

If you have 2 classes A,B such that A is subtype of B, and
if you have 2 parameterized types C[A],C[B] such that C[A] is subtype of C[B] then C is said to be covariant

Immutable List and Queue are examples of covariant Types in scala libraries.

 sealed abstract class List[+A] extends .........

 class Queue[+A] extends ........

Covariance is indicated by notation + on type parameter passed

Immutablity and Covariance

If you notice in api-docs covariance is generally associated with immutable types , eg scala.collection.immutable.Queue, scala.collection.immutable.List

There is a very good reason for this. From designers perspective, there were 2 reasons where covariance relation could fail

  • Re-Assignment of values. Classic example being java array reassignment problem, resulting in ArrayStore exception
  • Passing generic type as a parameter in method.

Thus to avoid above pitfalls , covariant types are immutable

If we try second scenario above ( passing type as parameter to method), we get compilation error.

scala> class MyGenericExample[+T] {
     |  def doSomeThing(t:T) = {}
     | }

:10: error: covariant type T occurs in contravariant position in type T of value t
        def doSomeThing(t:T) = {}

However there is a work around

scala> class MyGenericExample[+T] {
     |    def doSomeThing[U >:T] (t:U) = {}
     | }
defined class MyGenericExample

Notice how we are passing in [U >:T], where >: signifies lower bound. This declaration lets you pass anything that is supertype of T.

Below are some api’s from List collection where lower bounds is specified

   ////// cons method. expanded form
  def ::[B >: A](x: B): List[B]
  def reduce[A1 >: A](op: (A1, A1) ⇒ A1): A1

Also in case of Immutable collections,  the type is bumped up when you pass in a super type  as parameter to existing list . Consider below

scala> val list= List("Java","Scala","Ruby")
list: List[String] = List(Java, Scala, Ruby)

scala> 1::list
res1: List[Any] = List(1, Java, Scala, Ruby)

Things to note

  • Notice type of list in line 2 – List[String]
  • We appended 1 (type Int) to list in line 3
  • We got a new List of Type – List[Any] .
    • If you use mutable List ( which is invariant) , this wont work.
    • return type is bumped up to List[Any] which is common super type of Int and String


Consider diagram below which conveys the concept

If you have 2 classes A, B such that A is subtype of B, and
if you have 2 parametrized Types C[A], C[B] such that C[B] is subtype of C[A] then C is said to be contravariant

OuputChannel trait from scala api is example of contravariant type

 trait OutputChannel[-Msg] extends AnyRef

Contravariance is indicated by notation – on type parameter passed. 

Example below illustrates contravariance and covariance
Key points to note

  • We have simple class hierarchy represented by Data, Xml Json, Html
  • Request and Response are higher types which accept other types as parameter. ( In this example Data or one of its subtype)
  • Request is Contravariant . Any method that accepts Request[Xml] can accept Request[Data] as well. ( left hand side in picture)
  • Response is Covariant . Any method that returns Response[Data] can return any sub-types of Response[Data] as well
  • Contravariant types are used as arguments and covariant types are used as return types

Real- world example of Contravariance is scala’s Function api.

   trait Function1[-T1, +R] extends AnyRef


  • T1 – arguments to function. (contravariant)
  • R – return type from function. (covariant)

That completes quick overview of covariance and contravariance. Great reference for above topic is Programming In Scala, chapter 19.

scala concepts through api

In this post I would like discuss some api’s from scala docs and highlight important concepts associated with them.

Below are apis (from List Collection ) I would like to discuss.

  • map
  • collect
  • count
  • collectFirst

Below are the concepts which will be illustrated

  • Higher Order Functions , Anonymous Functions
  • Currying
  • Implicit Parameters
  • BuilderFactory ( CanBuildFrom)
  • Partial Functions, Predicate Functions
  • Option Type


Map is a transformation function. When applied on a collection transforms elements of collection.
Below is the signature of method

     def map[B](f: (A) ⇒ B): List [B]    //////// Default

Understanding Parameters

  • [B] : Generic type parameters – Int, String etc…./li>
  • f: (A) ⇒ B : function to be applied. Input to function is A and output is B
  • List [B] : return type

Below is an example

   scala> val myList = List(1,2,3,4)
   myList: List[Int] = List(1, 2, 3, 4)

   scala> def f(x:Int):String = x.toString
   f: (x: Int)String

   res0: List[String] = List(1, 2, 3, 4)

Mapping parameters in example to api-docs

  • [B] : [String] Inferred by compiler, line 8
  • f: (A) ⇒ B : f , function converting Int to String, line 4
  • List [B] : List[String] , line 8

Below is expanded form of api

def map[B, That](f: (A) ⇒ B)(implicit bf: CanBuildFrom [List[A], B, That]): That   ////// Expanded form

Understanding parameters

  • [B, That] : That represents type collection returned (Set,List… etc) , B represents Type of elements in collection ( Int, String….)
  • (f: (A) ⇒ B) : Transformation function
  • implicit bf: CanBuildFrom [List[A], B, That] : Implicit BuilderFactory. This facilitates creation of Collection That, with elements B, when input is of Type List[A]
  • That: Type of collection returned

Lets look at example

scala> import scala.collection.mutable.{Builder, ListBuffer}
import scala.collection.mutable.{Builder, ListBuffer}

scala> import scala.collection.generic.{GenericSetTemplate, GenericCompanion, CanBuildFrom}
import scala.collection.generic.{GenericSetTemplate, GenericCompanion, CanBuildFrom}

scala> def newListBuilder[A]: Builder[A, List[A]] = new ListBuffer[A]
newListBuilder: [A]=> scala.collection.mutable.Builder[A,List[A]]

scala> def canBuildFrom = new CanBuildFrom[List[Int] , String , List[String]] {
     | def apply (from:List[Int]) = newListBuilder
     | def apply() = newListBuilder
     | }
canBuildFrom: scala.collection.generic.CanBuildFrom[List[Int],String,List[String]]

res1: List[String] = List(1, 2, 3, 4)

Mapping parameters in example to api-docs

  • [B, That] : [String,List[String]] , line 16
  • (f: (A) ⇒ B) : _toString , line 16
  • implicit bf : CanBuildFrom [List[A], B, That] – canBuildFrom[List[Int],String,List[String]], line 14
    • Our custom Builderfactory – To create List[String] from List[Int]
  • That : List[String] , line 17

Comparing 2 apis, whats the difference

  • In case of Example 1, we don’t specify BuilderFactory. Since parameter is marked Implicit, compiler will select most appropriate one based on scope. In this case it will look for same in companion object associated with List Type.
  • In case of Example 2, we pass all the parameters explicitly.( Builder Factory and return Types).
  • For most cases default (Example 1) will serve the purpose. You don’t need to pass Builderfactory or specify Types.

Scala Concepts from above api

  • Higher Order Functions – These functions can be passed around as parameters. In this example our function is f:Int=>String . You can read about them here
  • Anonymous Functions – In Example 2 we passed .toString function to map function directly without declaring it. This is example of anonymous function
  • Currying – Concept in functional programming where you can pass multiple parameters list to a function. In Example 2 , parameters f and bf are curried. Look at example below
    • Regular function call – f(a,b)
    • Curried Function – f(a)(b)

    You can read more about them here

  • Implicit Parameters and Builder Factory- Builder Factory is Generic factory class which facilitates creation of one type of collections from another. In scala this represented by trait CanBuildFrom. You can read more about Implicit Parameters here and BuilderFactory here.( Look at section Factoring out common operations)


When applied on Collection, this api returns a new collection by applying partial function passed

Below is the signature from docs ( for brevity sake I shall consider default signature)

   def collect[B](pf: PartialFunction[A, B]): List[B]

Understanding parameters

  • [B] – Type of elements in List to be returned
  • pf: PartialFunction[A, B] – PartialFunction to be applied. This function takes A and returns B
  • List[B] : return type

Lets look at example.
In this example we have Customer with some attributes (totalPurchases,avgPurchaseCost,etc…) and we want to able to collect all customers based below criteria

  • all customers who have made atleast 10 purchases or
  • if their average purchasePrice is greater than 100

scala> case class Customer(id:Int, totalPurchases:Int, avgPurchasePrice:Double)
defined class Customer

scala> val customerList = List(Customer(1,20,1000.0), Customer(2,10,100.0), Customer(3,0,0.0), Customer(4,1,20.0), Customer(5,4,50.0))
customerList: List[Customer] = List(Customer(1,20,1000.0), Customer(2,10,100.0), Customer(3,0,0.0), Customer(4,1,20.0), Customer(5,4,50.0))

scala> def valuedCustomers:PartialFunction[Customer,Int] =  {
     | case Customer(id,_,avgPurchasePrice) if avgPurchasePrice >=100.0 => id
     | case Customer(id,totalPurchases,_) if totalPurchases >=10 => id
     | }
valuedCustomers: PartialFunction[Customer,Int]

scala> customerList.collect(valuedCustomers)
res12: List[Int] = List(1, 2)

Mapping parameters in example to api-docs

  • pf: PartialFunction[A, B] – valuedCustomers: PartialFunction[Customer,Int] -line 11
  • List[B] – List[Int] – line 14

Scala Concepts from above api

  • Partial Functions – These functions are used along with case classes to facilitate pattern matching. You can read about case classes here and partial functions here


This function returns count based on Predicate function passed.

Below is signature

  def count(p: (A) ⇒ Boolean): Int

Understanding parameters

  • p: (A) ⇒ Boolean – Predicate function with A as input and Boolean return type
  • Int : Return type , Int

Lets look at example.

In this case we are filtering customers who have made at-least 1 purchase. We are passing predicate function to filter customers

scala> def purchasePredicate(customer:Customer):Boolean = if (customer.totalPurchases > 0) true else false
purchasePredicate: (customer: Customer)Boolean

scala> customerList.count(purchasePredicate)
res16: Int = 4

Mapping parameters in example to api-docs

  • p: (A) ⇒ Boolean – purchasePredicate: (customer: Customer)Boolean -line 2
  • Int : Return type , Int

Scala Concepts from above api

  • Predicate Functions – Functions which return true or false


This api can be used to collect first value which matches the Partial Function that is passed

Below is signature

    def collectFirst[B](pf: PartialFunction[A, B]): Option[B]

Understanding parameters

  • (pf: PartialFunction[A, B]) : PartialFunction to be applied. This function takes A and returns B
  • Option[B] : Return type – Option[B] , Option Container of type B

Lets look at example.
In this scenario we want to able to get first customer which matches the partial function we are passing.There could be scenario where there is no match. Hence in this case we are using Option to store values

scala> customerList.collectFirst(valuedCustomers)
res18: Option[Int] = Some(1)

Mapping parameters in example to api-docs

  • pf: PartialFunction[A, B] : valuedCustomers: PartialFunction[Customer,Int]
  • Option[B] : Return Type : Some[1] , Some Container with value of ID 1 , first match to our Partial Function

Scala Concepts from above api

  • Option Type – Container to handle values that could be potentially null. It uses Some ( if value is present ) or None is no value is present. You can read about them here


We looked at some api’s and underlying concepts associated with them. In next post I shall cover some more apis,
Idea being to understand api’s better which in turn will facilitate writing better scala code.

Actor Based system with Akka

This post gives an overview on how to build simple Actor Based system using Akka.

To demonstrate use case , I develop a application which facilitates streaming of Logs

I had developed similar prototype using Java some-time back. You look it up here. However since am learning Scala, decided to port the code to scala

In this POC I use Akka . Akka provides high level abstractions to build asynchoronous/reactive/multi threaded systems without getting to low-level semantics like threads, locks etc. Conceptually it consists of Actors and these Actors communicate by passing Messages.

If you are not familiar with Akka take a look at this example.That should get you started.

Overview of Application

Concept can be depicted in pictorically below

Scala Actors

Below are key components

  • StreamingLogs – Scala Object to Bootstrap application
  • Logwatcher – Actor which watches logs and sends updates to Coordinator
  • TCPServer – Actor which accepts http connections and streams updates to all its clients
  • Coordinator- Actor (parent) which co-ordinates messaging with 2 actors – LogWatcher and TCPServer
    • Starts individual actors
    • Receives updates from LogWatcher when logs are updated
    • Sends mesage to TCPServer regards updates to Logs
  • LogWriter  – Actor which simulates real-world logging

Lets look at important aspects of each component


This class facilitates boot-strap of application.

  val coordinator = system.actorOf(Props(new Coordinator(logFileName)), name = "co-ordinator")
  val logWriter = system.actorOf(Props(new LogWriter(logFileName)), name = "log-writer")

  logWriter ! StartWriting()
  coordinator ! StartCoordinator()

  case class StartWriting()
  case class StartCoordinator()
  case class StopCoordinator()
  case class LogMessages(ls: List[String])
  case class Watch(msg: String)
  case class StartServer(port: Int)
  case class StopServer()

Key points to note

  • Co-Ordinator is initialized – line 1
  • Case classes ( StartWriting, StartCoordinator, LogMessages etc….)are defined. These represent messages which are passed around actors
  • Message sent to LogWriter to get started. This actor simulates logging – line 4.
  • Message sent to Co-ordinator to get started – line 5


This actor performs task of co-ordination, by passing messages across various actors.

class Coordinator(logFileName: String) extends Actor {
    val tcpServer = system.actorOf(Props(new TCPServer()), name = "tcpServer")
    val logWatcher = system.actorOf(Props(new LogWatcher(logFileName)), name = "logWatcher")

    def receive = {

      case StartCoordinator() => {
        println("Starting the coordinator.....")
        tcpServer ! StartServer(8080)
        logWatcher ! Watch("start watching")

      case StopCoordinator() => {
        println("Stopping the coordinator...")

      case LogMessages(ls: List[String]) => {
        tcpServer ! new LogMessages(ls)


Key points to note

  • Actors are initialized – line 2, line 3
  • When this actor recieves message (StartCoordinator), it in turns sends message to LogWatcher, TCPServer to get started- line 9 , line 10
  • Message received from LogWatcher is sent to TCPServer – line 18


This represents simple HttpServer which accepts connections from clients and streams log messages.

class TCPServer() extends Actor {

  val socketConnections = new ArrayList[ReadHandle]()

  def receive = {
    case IO.NewClient(server) => {

    case IO.Read(rHandle, bytes) => {
      println("Added new client.....")

    case StartServer(port: Int) => {
      IOManager(context.system).listen(new InetSocketAddress(port))
      println("Server started on port " + port)

    case LogMessages(ls) => {

      for (i <- 0 until socketConnections.size()) {     
        val handle = socketConnections.get(i)
        println("sending streams to client.....")
        for (line <- ls) {       
          val byteString = ByteString(line)


Key points to note

  • Server started – line 16
  • New clients are added – line 11
  • LogMessages sent to client – line 22


This Actor keeps watching log files and sends messages to co-ordinator as file gets updated
Below is section of code associated with receive method of this actor

 def receive = {
    case Watch(msg) => {
      println("Msg recieved to watch.....")
      var bytesToSkip = 0
      while (keepRunning) {
        //println("original length=" + originalLength + "current length" + fileName.length())
        if (fileName.length() > originalLength) {
          val lines = getNewLinesFromFile(bytesToSkip)
          sender ! new LogMessages(lines)
          bytesToSkip = bytesToSkip + getBytesToSkip(lines)
          originalLength = fileName.length()

Key Points to note

  • This actor gets started when it receives Watch message – line 2
  • If contents of file is changed sends message back to Coordinator – line 9

Complete code base can be downloaded at github

Running The Application

  • Checkout the code and import it in eclipse
  • Ensure scala,akka libraries are present in buildpath
  • Start app by running ScalaStreamingLogs ( update location of logfile in line 3)
  • Open browser and type – http://localhost:8080. After about 30 secs you should see log messages

Overall Actor based model is much simpler,modular and easier to understand than the original Java version.
Further you don’t have to worry about Threads, Locks, Queues etc. Akka takes of low-level plumbing details

Implicits in scala

In this post I want to give an overview of Implicit Conversions in Scala.
These can be used in 3 places

  • Convert from one Type to another
  • Conversions of receivers – to facilitate calling of methods which are not defined for particular object
  • Conversions of Parameters

Lets look at each of them in detail

Convert from one Type to another

This technique can be used when you want to convert from one type to another. Consider a scenario where you have a function which does some processing on List. At some point later you need to be able to apply same function on Map, then you could use this technique. Below example illustrates the point

scala> val list = List(1,2,3,4,5)
list: List[Int] = List(1, 2, 3, 4, 5)

scala> def process(mylist:List[Int]):List[Int] = mylist.filter( _ <  5)
process: (mylist: List[Int])List[Int]

scala> process(list)
res15: List[Int] = List(1, 2, 3, 4)

scala> val myMap = Map ("key1"->1,"key2"->2,"key3"->3,"key4"->4,"key5"->5)
myMap: scala.collection.immutable.Map[String,Int] = Map(key4 -> 4, key5 -> 5, key1 -> 1, key2 -> 2, key3 -> 3)

scala> process(myMap)
<console>:25: error: type mismatch;
 found   : scala.collection.immutable.Map[String,Int]
 required: List[Int]
scala> import scala.language.implicitConversions
import scala.language.implicitConversions

scala> implicit def mapToList(myMap:Map[String,Int]):List[Int]= myMap.valuesIterator.toList
mapToList: (myMap: Map[String,Int])List[Int]

scala> process(myMap)
res17: List[Int] = List(4, 1, 2, 3)

scala> implicit def mapToList(myMap:Map[String,Int]):List[Int]= myMap.valuesIterator.toList.sorted
mapToList: (myMap: Map[String,Int])List[Int]

scala> process(myMap)
res18: List[Int] = List(1, 2, 3, 4)

In above example, we are

  • creating a list — line 1
  • applying function (process) on it. This returns a new list — line 4
  • creating a map and applying function (process) on it. This results in error — line 13
  • import implicitConversions — line 19
  • created implicit def mapToList which converts Map to List — line 22
  • function (process) is applied on map and this time it works fine –line 25

Conversions of receivers

This type of conversion can be used when you want to apply new behavior on existing objects.
Lets say you have Phone Object which supports following methods

  • makeCall
  • sendSMS

and a Tablet Object which supports following method

  • browseInternet

then by using this conversion – you can enable browseInternet method on Phone and makeCall,sendSMS method on Tablet
Below code illustrates the same.

scala > class Phone (n:String) {
         var name:String = n
         def makeCall()  = println("Making call with "+name)
         def sendSMS() = println("Sending sms with"+name)

scala >defined class Phone

scala > class Tablet (n:String) {
 var name :String = n;
 def browse() = println("Browsing with "+name);

scala > defined class Tablet

scala> var phone =  new Phone("iPhone-5c")
phone: Phone = Phone@70d18a80

scala> phone.makeCall();
Making call with iPhone-5c

scala> phone.browse
<console>:11: error: value browse is not a member of Phone

scala> var tablet = new Tablet("iPad");
tablet: Tablet = Tablet@4efade06

scala> tablet.browse
Browsing with iPad

scala> tablet.makeCall
<console>:11: error: value makeCall is not a member of Tablet

scala> implicit def phoneToTablet(x:Phone) = new Tablet (
phoneToTablet: (x: Phone)Tablet

scala> implicit def tabletToPhone(x:Tablet)   = new Phone (
tabletToPhone: (x: Tablet)Phone

scala> tablet.makeCall
Making call with iPad

scala> phone.browse
Browsing with iPhone-5c

Things to note

  • we defined classes Phone and Tablet — line 1, line 9
  • we get error message when we do phone.browse — line 22
  • we get error message when we do tablet.makeCall — line 32
  • we added implicit conversions — line 36, line 39
  • phone.browse and tablet.makeCall works fine — line 42, line 46

A wonderful explanation of these type of conversions ( View As Conversions) can be read here

Conversions for parameters

This type of conversion is used when you want to pass in some default parameters to your functions.
Consider a scenario where you want to be able to display Home Page for user based on particular Theme selected by him/her. However you also want to provide sensible defaults. By using this particular technique you can achieve the same

scala> class Theme(val name:String)
defined class Theme

scala> object Controller {
  def displayHomePage(name:String)(implicit theme:Theme) {
    println("Displaying home page for = "+name);
    println("Using theme = ";

scala> defined module Controller

scala> val simpleTheme = new Theme("simple");
simpleTheme: Theme = Theme@218f5a04

scala> Controller.displayHomePage("user1")(simpleTheme)
Displaying home page for = user1
Using theme = simple

scala> Controller.displayHomePage("user1")
<console>:20: error: could not find implicit value for parameter theme: Theme
scala> object ThemePrefs {
     |    implicit val theme = new Theme("default")
     | }
defined module ThemePrefs

scala> import ThemePrefs._
import ThemePrefs._

scala> Controller.displayHomePage("user1")
Displaying home page for = user1
Using theme = default

Things to note

  • Theme and Controller is defined — line 1, line 4
  • displayHomePage is called with Theme — line 16
  • displayHomePage being called without theme results in error — line 20
  • defaultTheme is defined and imported — line 24 , line 29
  • displayHomePage being called without theme works fine — line 32

That completes a quick overview on Implicits in Scala. For more thorough treatment refer to links below

Java Convert Date to a specific Timezone

As many would agree Java dates are messy. In this post I want to give overview on how you can convert dates in java from one Timezone to another using Joda Time library

Lets say you want to display dates to users based on their timezone. Below can be a static method to do the same

 public static Date getDateInTimeZone(Date currentDate, String timeZoneId) {
        TimeZone tz = TimeZone.getTimeZone(timeZoneId);
        Calendar mbCal = new GregorianCalendar(TimeZone.getTimeZone(timeZoneId));
        Calendar cal = Calendar.getInstance();
        cal.set(Calendar.YEAR, mbCal.get(Calendar.YEAR));
        cal.set(Calendar.MONTH, mbCal.get(Calendar.MONTH));
        cal.set(Calendar.DAY_OF_MONTH, mbCal.get(Calendar.DAY_OF_MONTH));
        cal.set(Calendar.HOUR_OF_DAY, mbCal.get(Calendar.HOUR_OF_DAY));
        cal.set(Calendar.MINUTE, mbCal.get(Calendar.MINUTE));
        cal.set(Calendar.SECOND, mbCal.get(Calendar.SECOND));
        cal.set(Calendar.MILLISECOND, mbCal.get(Calendar.MILLISECOND));
        return cal.getTime();

Now lets call above method and print the dates which we get

     Date currDate = new Date();
     Date dateIndia = DateUtil.getDateInTimeZone(currDate,"Asia/Kolkata");
     System.out.println("India date      :"+dateIndia);
     System.out.println("India time unix :"+dateIndia.getTime());

     Date dateUtc = DateUtil.getDateInTimeZone(currDate,"UTC");
     System.out.println("Utc date :"+dateUtc)
     System.out.println("Utc time unix :"+dateUtc.getTime());

Below will be output when you run it

     India date       :Sun Dec 15 12:09:06 IST 2013
     India time unix  :1387089546262

     Utc  date      :Sun Dec 15 6:39:06 IST 2013
     UTC time unix  :1387089546262

Notice how time and date is correct but timezone is messed up. Always shows IST (in my computer)

Lets try to do the same using Timezone and offset. Below is piece of code

public static Date getDateInTimeZone(Date currentDate, String timeZoneId) {
        TimeZone tz = TimeZone.getTimeZone(timeZoneId);
        Date convDate = new Date(currentDate.getTime()+ tz.getOffset(currentDate.getTime()));
        return convDate;

Now lets run it using below snippet

     Date dateIndia = DateUtil.getDateInTimeZone(new Date(1387089546262l),"Asia/Kolkata");
     System.out.println("India date      :"+dateIndia);
     System.out.println("India time unix :"+dateIndia.getTime());

     Date dateUtc = DateUtil.getDateInTimeZone(new Date(1387089546262l),"UTC");
     System.out.println("Utc date :"+dateUtc)
     System.out.println("Utc time unix :"+dateUtc.getTime());

Below is the output

     India date       :Sun Dec 15 12:09:06 IST 2013
     India time unix  :1387089546262

     Utc  date      :Sun Dec 15 6:39:06 IST 2013
     UTC time unix  :1387089546262

Unfortunately same thing. It prints date and time correctly but timezone is not correct. (Internally it uses you settings associated with time zone computer you are running the code).

After looking at above i resorted to Joda Time (DateTime object) and things worked like charm. Take a look at example below

    public static String getDateForTimeZoneAsString(long dbDate,String timezoneId){
        DateTimeFormatter fmt = DateTimeFormat.forPattern("E,MMM d yyyy HH:mm:ss zz");
        DateTime origDate = new DateTime(dbDate);
        DateTime dtTz = origDate.withZone(DateTimeZone.forID(timezoneId));
        String dateForTz = dtTz.toString(fmt);
        return dateForTz;

Below is snippet of code to run it

     String dateIndia = DateUtil.getDateInTimeZone(new Date(1387089546262l),"Asia/Kolkata");
     System.out.println("india date :"+dateIndia)
     String dateUtc = DateUtil.getDateInTimeZone(new Date(1387089546262l),"UTC");
      System.out.println("Utc date :"+dateUtc)

and below is the output

     India date       :Sun Dec 15 2013 12:09:06 IST
     Utc  date        :Sun Dec 15 2013 6:39:06 UTC

Things work like charm. Look at timezone. It prints correctly .Indeed very simple and elegant.

Functions in Scala

In this post I want to briefly talk about functions in scala.

In scala functions are first class citizens. They can be assigned to variables and passed around. Look at example below.

scala> (x:Int) => x * 2
res0: Int => Int = <function1>
scala> res0(2)
res1: Int = 4

Things to note

  • Line 1 illustrates example of declaring a simple function.
  • Line 2 shows that function is of name res0, parameter is Int  and return type is Int
  • Line 3 shows how function is called

Now lets assign function to a variable. Once we do that function can be used in expression. Consider example below

scala> var multiply = (x:Int) => x * 2
multiply: Int => Int = <function1>
scala> multiply(2)
res2: Int = 4
scala> multiply(multiply(2))
res3: Int = 8

Things to note

  • we assigned function to variable multiply  – Line 1
  • we can use multiply in any expression – Line 3, Line 5

This is the fundamental premise of functional programming. Functions can be passed around as variables in expression and each function performs some action and returns value. It does not have any side-effect

Consider example below. Further illustrates concept of passing functions as variables

scala> val myList = List(1,2,3,4,5)
myList: List[Int] = List(1, 2, 3, 4, 5)
scala> for(x:Int <- myList) yield multiply (x)
res6: List[Int] = List(2, 4, 6, 8, 10)

Things to note

  • In line 3, we are passing multiply function to for loop and using it with yield to return

In scala functions which take other functions are argument are called Higher Order Functions.
Consider example below.

scala> var multiply = (x:Int) => x * 2
scala> var add  = (x:Int) => x +2
scala> val myList = List(1,2,3,4,5)

scala> def transFormList(list:List[Int], myfunc:Int=>Int ) =  for(x:Int <- list) yield myfunc (x) 
transFormList: (list: List[Int], myfunc: Int => Int)List[Int]

scala> transFormList(myList,add)
res13: List[Int] = List(3, 4, 5, 6, 7)

scala> transFormList(myList,multiply)
res14: List[Int] = List(2, 4, 6, 8, 10)
  •  In line 1 we define function and assign it to multiply
  • In line 2 we define function and assign it to add
  • In line 5 we define a transformation function ( This is Higher order function)
    • parameters to function are – list, myfunc( a function).
    • It applies myfunc on list and returns new list
  • Lines 9 and 12 illustrates the result

Hope these examples provide an overview of how functions work in scala

Scala concepts for Beginners

In this post I would like to discuss about important concepts in scala when starting out. These were questions I asked when I started working on scala but had to dig around to find answers. My hope is this article can be help and shorten the learning curve.
Below are topics I would talk about

  • Type Inference
  • Immutability
  • Arrays,Collections and Tuples
  • Loops and Expressions
  • Functions
  • Classes and Companion Object

Type Inference

Scala as language does type inferencing. So you don’t need to specify the type when you declare any variables. Lets look at example below.   Open scala REPL and type below ( All examples mentioned in this article are tried in REPL)

scala> val number1:Int =5
number1: Int = 5

scala> val number2=5
number2: Int = 5

In example above in first case we declared variable number1 and defined its Type as Int ( after colon)
In second case we didnt specify Type and scala inferenced it as Int.
Also in Scala you dont need to specify semi-colon. It is inferenced . Thus you need to type less


Scala as language promotes immutability, aligned with principles of functional programming. So if you don’t specify any modifier, by default val will be used. If you use var then value associated with variable can be changed . An exception to rule is while using REPL, where if you declare variable as val, then you can change the value

Arrays , Collections and Tuples

Below are couple of examples of Arrays in Scala. They are self explanatory. Take a look at type inferencing( String for first case and Int for second)

scala> val names = Array("John","Ron","Tom")
names: Array[String] = Array(John, Ron, Tom)

scala> val numbers = Array(1,2,3)
numbers: Array[Int] = Array(1, 2, 3)

To access specific element of array just specify the index in parenthesis.

scala> numbers(1)
res0: Int = 2

Another interesting function is mkString which you can use to concatenate elements of array or collections for that matter

scala> names.mkString(",")
res6: String = John,Ron,Tom

Scala supports both mutable and immutable collections. Default is immutable.  Take a look at below diagram which shows set hierarchy. Similar hierarchy exists for Map

ref: Talks by Bill Venners and Dick Wall at Parleys

ref: Talk by Bill Venners and Dick Wall at Parleys

By default when you create a set, immutable set is created. However if you want to create mutable set, you can use import scala.collection.mutable.Set.   Look at example below

scala> val citySet = Set("LA","NY","MUM")
citySet: scala.collection.immutable.Set[String] = Set(LA, NY, MUM)

scala> citySet+="SFO"
:9: error: value += is not a member of scala.collection.immutable.Set[String]

scala> import scala.collection.mutable.Set
import scala.collection.mutable.Set

scala> val citySetNew=Set("LA","NY","MUM")
citySetNew: scala.collection.mutable.Set[String] = Set(MUM, NY, LA)

scala> citySetNew+="SFO"
res3: citySetNew.type = Set(MUM, SFO, NY, LA)

scala> citySetNew
res4: scala.collection.mutable.Set[String] = Set(MUM, SFO, NY, LA)

Rules for Map is similar to set above. Look at examples below

scala> val cityMap = Map(1->"Mum",2->"NY",3->"LA")
cityMap: scala.collection.immutable.Map[Int,String] = Map(1 -> Mum, 2 -> NY, 3 -> LA)

scala> cityMap += (4->"SFO")
:10: error: value += is not a member of scala.collection.immutable.Map[Int,String]
              cityMap += (4->"SFO")

scala> import scala.collection.mutable.Map
import scala.collection.mutable.Map

scala> val cityMap2 =  Map(1->"Mum",2->"NY",3->"LA")
cityMap2: scala.collection.mutable.Map[Int,String] = Map(2 -> NY, 1 -> Mum, 3 -> LA)

scala> cityMap2 +=(4->"SFO")
res6: cityMap2.type = Map(2 -> NY, 4 -> SFO, 1 -> Mum, 3 -> LA)

Below are examples for list. Note – If list is declared as val , you cannot add new element to it, however if you declare it as var , you can add element

scala> val list1 = List(1,2,3,4)
list1: List[Int] = List(1, 2, 3, 4)

scala> list1 = 5::list1
:8: error: reassignment to val
       list1 = 5::list1

scala> var list2 = List(1,2,3,4)
list2: List[Int] = List(1, 2, 3, 4)

scala> list2 = 5::list2
list2: List[Int] = List(5, 1, 2, 3, 4)

scala> list2.size
res0: Int = 5

You can use tuples when you want to return multiple values/types from a function.

scala> val tup1 = (4,"SFO")
tup1: (Int, String) = (4,SFO)

Expressions  and Loops

In scala every statement can be an expression (Any statement that returns a value is considered an expression).
If Loop

Below is example of if loop in scala

scala> val num1 = 4
num1: Int = 4

scala> val num2 =5
num2: Int = 5

scala> val finalNum = if(num1 > num2) num1 else num2
finalNum: Int = 5

scala> println(finalNum)

Result of if/else expression is assigned to finalNum. This is very much similar to Ternary Operator in Java

For Loop
Below is example of simple for loop in scala.

scala> val array1 = Array(1,2,3)
array1: Array[Int] = Array(1, 2, 3)
scala> for(x:Int <- array1) print (x)

Above for loop prints value not idiomatic scala way to do it as it has a side-effect.
Ideally in purely functional terms loop should return some value which can be used futher down the line. Look at example below. It uses yield to return value .

scala> array1
res10: Array[Int] = Array(1, 2, 3)
scala> for(x:Int<-array1) | yield(x*2)  
res11: Array[Int] = Array(2, 4, 6)  

In above case a new array (res11) is returned


Below is an example of declaring a function in scala

scala> def max(a:Int,b:Int):Int ={
          if (a > b) a
          else b
max: (a: Int, b: Int)Int

scala> max(3,4)
res12: Int = 4

Things to note

  •  function definition should begin with def keyword
  • parameters are specified along with their types( as always type should be specified after name)
  • Return type of function specified after parameters . In above case it is Int
  • Function definition is specified after equals (=) inside braces

A streamlined version of above can be specified as below, which is more idiomatic scala

scala> def maxoneline(a:Int,b:Int):Int = if(a > b) a else b
maxoneline: (a: Int, b: Int)Int

Classes and Companion Object

Below is example of  class in Scala.

scala> class Product(n:String, p:Int) {
      var name :String= n
      var price:Int=p
defined class Product
scala> val p = new Product("Product1",100)
p: Product = Product@17d9d565

scala> print(
scala> print(p.price)

Rather very concise and simple. The variables are public by default hence you can change them. checkout this article for example of private variables and getters/setters in scala

Above also illustrates example of Primary constructor (where values to variable name and price is assigned)

In scala class anything which is not included inside def will be executed as part of constructor.

Below is example of Auxiliary constructor

scala> class Product(n:String, p:Int) {
        var name :String= n
        var price:Int=p
        def this(n:String) = this(n , 200)
        println("name="+name+" , price="+price)
defined class Product
scala> val p2 = new Product("product2")
name=product2 , price=200
p2: Product = Product@7a64f150

res4: String = product2

scala> p2.price
res5: Int = 200

As you can see from above Auxilary constructor uses Primary constructor internally. Below example illustrates use to overriding toString in scala

scala> class Product(n:String, p:Int) {
       var name :String= n
       var price:Int=p
       def this(n:String) = this(n , 200)
       override def toString = ( "name ="+name+", price="+price)
defined class Product
scala> val p7 = new Product("product7",700)
p7: Product = name =product7, price=700

Companion Object

Scala doesnt have concept of static. It however has something called Companion object. A companion Object has same name as class name. You can invoke methods on them directly without creating instance

class Product(n:String, p:Int) {
   var name :String= n
   var price:Int=p
   def this(n:String) = this(n , 200)
   override def toString = ( "name ="+name+", price="+price)
object Product {
    def computeTax(price:Int):Double = {
defined class Product
defined module Product

scala> val p8 = new Product("product8",800)
p8: Product = name =product8, price=800

scala> val tax = Product.computeTax(p8.price)
tax: Double = 8800.0

In above example computeTax can be called on product class without creating an instance.
As you have public static void main in Java, you need to define main method inside Companion Object in Scala

That completes some basic concepts in scala which can help you get started. I would recommend to look at below fantastic videos which covers above in more details

  1. First steps with Scala – part 1
  2. First steps with Scala – part 2

Nginx configurations

In this article I would like to write about some basic Nginx configurations.   These configurations can be made in default file which is usually located in /etc/nginx/sites-available folder in nginx. I would talk about below topics

  • Creating a proxy pass
  • Url re-write
  • Setting up ssl
  • Configuring php

Creating a Proxy Pass

Often when developing applications , UI layer would need access to api layer which could be deployed in another domain. To avoid cross-domain issue you need to set up proxy pass in Nginx. Below is the example configuration

 upstream _tomcat {
    server :8080;
 ##### urls which you want request to be sent to app server
 location ~ /url1* {
     proxy_pass http://_tomcat;
 location ~ /url2* {
     proxy_pass http://_tomcat;

 Url re-write

At times we run into issue where we want to be able to do redirect old urls to new urls ( without changing url in browser bar). Consider a case where you have old url of form

/products which needs to be redirected to  /products/listNew.html
/products/productName   which needs to be redirected to /products/listProduct.php?name=productName. Below is example configuration for same

  rewrite ^/products/$    /pr/listNew.html redirect;
  rewrite ^/products/(.*)  /pr/listProduct.php?slug=$1 break;

In above configs pr  is folder in nginx where all the static/phps are located

Setting Up SSL

Configuring nginx with ssl is super simple. First thing you need to do is get certificate for your domain from some certificate providers. You would need following to enable ssl

  • key  : key with which you will sign your certficate , ex yourdomain.key
  • crt   : certficiates which you can download from your provider after you sign up , ex

You can place both of these in ssl folder in nginx.  After that to enable ssl for you domain add below configuration

   listen 80;
   listen 443 default_server ssl;
   ssl_certificate      /etc/nginx/ssl/;
   ssl_certificate_key  /etc/nginx/ssl/yourdomain.key;

In case of above both http and https will work. If however you want all traffic to be served via https, add below configuration

server {
   listen 80;
    return 301 https://$host$request_uri;

Configuring Php

To be able to configure php in nginx, you need to install below packages

  • php5-common
  • php5-curl
  • php5-fpm

Below is nginx config

 location ~ .php$ {
   try_files $uri =404;
   fastcgi_split_path_info ^(.+\.php)(/.+)$;
   fastcgi_pass unix:/var/run/php5-fpm.sock;
   fastcgi_index index.php;
   include fastcgi_params;

In addition make sure below line is un-commented /etc/php5/fpm/pool.d/www.conf

  listen = /var/run/php5-fpm.sock

That completes some basic nginx configurations which can come in handy. Good reference about same can be found here

Hibernate Search – Many to Many

In this article I want to give quick overview on Hibernate Search and Luke

Hibernate search is indeed very simple to implement. Internally it uses Lucene to index the domain objects.   To be able to search an entity you need do the following

  • add annotation @Indexed to index and search domains
  • add annotation @Field to index and search on fields associated with domain
  • add annotation @IndexEmbeded to to index and search on collections associated with domain

Consider a case where you have product and its associated with tags.  Relationship between Product and Tag would be Many to Many

Below is section of Product and Tag domain with appropriate annotations

@Table(name = "products")
public class Product {
    private String name;

    private String description;

    private String keywords;

    @JoinTable(name = "products_tags", joinColumns = { @JoinColumn(name = "product_id") },
    inverseJoinColumns = { @JoinColumn(name = "tag_id") })
    @Cache(usage = CacheConcurrencyStrategy.READ_WRITE)
    private List tags = Lists.newArrayList();

    /////// getters and setters here
@Table(name = "tags")
public class Tag {

    private String label;

    private String type;

     /////// getters and setters here

If you want to search Product based on name, keyword, description  and tags , below would be section of code for same

       QueryBuilder qb = getFullTextEM().getSearchFactory()

        Query query = qb
            .onFields("name", "description", "keywords", "tags.label")

        int count = getFullTextEM()
            .createFullTextQuery(query, Product.class)

        List products = getFullTextEM()
            .createFullTextQuery(query, Product.class)

When specifying query in addition to name,keywords, description also note tags.label has been specified
Also ensure that you annotate @Field on fields in tag domain.

Above also works for  Many to One relationships

One thing which I found really useful when working with Hibernate search is to use Luke.  Luke provides a neat UI to view the indexes generate on domain objects. If results of search are not reflected correctly it would be good to check if indexes are being generated for fields you are searching on

You can read more on luke here


Get every new post delivered to your Inbox.