Saturday, June 11, 2016

Grails (Hibernate) Batch Insert/Update

To prevent memory issue and increase the performance of batch insert or update it can be divided into the smaller bunch data, e.g. 1000 records, and flush and clear the transaction after saving each bunch.

def batch = []  // temp batch list
dataList.each { d ->
    d.value = 'blah blah blah'
    batch.add(d)
    if(batch.size() > 1000)
       runBatch(batch)
}


And the runBatch can be a piece of code like this:
        log.debug("Start Data batch saving")
        Data.withTransaction {
            for(Data d in batch){
                log.debug "Saving datum: ${d} ${d.isDirty('value')}"
                // d.save(flush: true) // not required to flush here, since session will flush before clear
                d.save()
            }
        }
        batch.clear()
        def session = sessionFactory.getCurrentSession()
        session.flush()
        session.clear()
        log.debug("Batch saving transaction & clearing hbn session finished")

The hibernate details can be found here.


No comments: