Open Source.NET Batch Framework ? Do we have one ?
By admin on Sep 26, 2007 in .NET
I was looking at Spring Batch Framework, and was trying to find a .NET compatible. For those of you who do know heard of it, “Spring Batch provides a technical framework and programming model to support long-running processes that perform a given set of tasks repetitively. A typical batch program generally reads a large number of records from a database, file, or queue, processes the data in some fashion, and then writes back data in a modified form. Spring Batch automates this basic batch iteration, providing the capability to process similar transactions as a set, typically in an offline environment without any user interaction.”
Batch processing is definitely unavoidable in large enterprises. There are lots of web application frameworks for Java and .NET, and new ones are emerging now and then. Interestingly, no many are developing batch framework for either .NET or Java. The only one I knew of for Java is Spring Batch, and for .NET, there is none that I know of.
I have used SQL Server Integration Service before in one of the projects to do some batch processing to copy data from flat file/Oracle database to SQL Server database. SQL Server Integration Service provide a GUI to do the normal ETL. However, it is not free, and also when I tried to benchmark it, the results are not really satisfactory.
I have developed then some batch processing routines using C#, and as a comparison with Spring Batch, certain features are the same. Maybe it will be interesting to develop a open source batch framework.
- Commit batch process periodically
- Concurrent batch processing: parallel processing of a job
Staged, enterprise message-driven processing
- Massively parallel batch processing
- Manual or scheduled restart after failure
- Sequential processing of dependent steps (with extensions to workflow-driven batches)
- Partial processing: skip records (e.g. on rollback)
- Whole-batch transaction: for cases with a simple enough data model or a small batch size
If anyone of you know of any open source .NET batch framework, please let me know.
David | Sep 27, 2007 | Reply
Hello. I’d like to offer to exchange links. My blog is http://www.portadev.com. If you like my proposition please let me know.
Chris Whelan | Oct 26, 2007 | Reply
I have used Quartz to handle these types of task in Java land and since starting .net development recently, I discovered that there is a port – Quartz.net http://sourceforge.net/projects/quartznet
admin | Oct 26, 2007 | Reply
Quartz is a job scheduling system. A batch framework should be more than that. For Java now there is Spring Batch Framework.
I know that Avanade, the company which contributed to the Microsoft Enterprise Library (which I think originally from its ACA.Net framework) also has a ACA Batch Framework.
uwlan | Mar 27, 2008 | Reply
I’d like to offer to exchange links. My blog is http://www.gravetown.blogspot.com. If you like my proposition please let me know.
Al | Jul 19, 2009 | Reply
Has anyone found a .NET batch framework ?
Ben Cops | Sep 28, 2009 | Reply
I was looking for one and wrote one in the end. I’ve just codeplexed it up so have a look.
Teun | Feb 10, 2010 | Reply
You may be interested in my new project: BatchFlow. http://code.google.com/p/batchflow/