RSS Feed for This PostCurrent Article

Open Source.NET Batch Framework ? Do we have one ?

I was looking at Spring Batch Framework, and was trying to find a .NET compatible. For those of you who do know heard of it, “Spring Batch provides a technical framework and programming model to support long-running processes that perform a given set of tasks repetitively. A typical batch program generally reads a large number of records from a database, file, or queue, processes the data in some fashion, and then writes back data in a modified form. Spring Batch automates this basic batch iteration, providing the capability to process similar transactions as a set, typically in an offline environment without any user interaction.”

Batch processing is definitely unavoidable in large enterprises. There are lots of web application frameworks for Java and .NET, and new ones are emerging now and then. Interestingly, no many are developing batch framework for either .NET or Java. The only one I knew of for Java is Spring Batch, and for .NET, there is none that I know of.

I have used SQL Server Integration Service before in one of the projects to do some batch processing to copy data from flat file/Oracle database to SQL Server database. SQL Server Integration Service provide a GUI to do the normal ETL. However, it is not free, and also when I tried to benchmark it, the results are not really satisfactory.

I have developed then some batch processing routines using C#, and as a comparison with Spring Batch, certain features are the same. Maybe it will be interesting to develop a open source batch framework.

  • Commit batch process periodically
  • Concurrent batch processing: parallel processing of a job
  • Staged, enterprise message-driven processing
  • Massively parallel batch processing
  • Manual or scheduled restart after failure
  • Sequential processing of dependent steps (with extensions to workflow-driven batches)
  • Partial processing: skip records (e.g. on rollback)
  • Whole-batch transaction: for cases with a simple enough data model or a small batch size

If anyone of you know of any open source .NET batch framework, please let me know.


Trackback URL


RSS Feed for This Post4 Comment(s)

  1. David | Sep 27, 2007 | Reply

    Hello. I’d like to offer to exchange links. My blog is http://www.portadev.com. If you like my proposition please let me know.

    Popiashvili David

  2. Chris Whelan | Oct 26, 2007 | Reply

    I have used Quartz to handle these types of task in Java land and since starting .net development recently, I discovered that there is a port - Quartz.net http://sourceforge.net/projects/quartznet

  3. admin | Oct 26, 2007 | Reply

    Chris,

    Quartz is a job scheduling system. A batch framework should be more than that. For Java now there is Spring Batch Framework.

    I know that Avanade, the company which contributed to the Microsoft Enterprise Library (which I think originally from its ACA.Net framework) also has a ACA Batch Framework.

  4. uwlan | Mar 27, 2008 | Reply

    I’d like to offer to exchange links. My blog is http://www.gravetown.blogspot.com. If you like my proposition please let me know.

    ulwan

1 Trackback(s)

  1. From Mark SoftWare Top » Open Source.NET Batch Framework ? Do we have one ? | Nov 3, 2007

RSS Feed for This PostPost a Comment