RSS Feed for This PostCurrent Article

Open Source.NET Batch Framework ? Do we have one ?

I was looking at Spring Batch Framework, and was trying to find a .NET compatible. For those of you who do know heard of it, “Spring Batch provides a technical framework and programming model to support long-running processes that perform a given set of tasks repetitively. A typical batch program generally reads a large number of records from a database, file, or queue, processes the data in some fashion, and then writes back data in a modified form. Spring Batch automates this basic batch iteration, providing the capability to process similar transactions as a set, typically in an offline environment without any user interaction.”

Batch processing is definitely unavoidable in large enterprises. There are lots of web application frameworks for Java and .NET, and new ones are emerging now and then. Interestingly, no many are developing batch framework for either .NET or Java. The only one I knew of for Java is Spring Batch, and for .NET, there is none that I know of.

I have used SQL Server Integration Service before in one of the projects to do some batch processing to copy data from flat file/Oracle database to SQL Server database. SQL Server Integration Service provide a GUI to do the normal ETL. However, it is not free, and also when I tried to benchmark it, the results are not really satisfactory.

I have developed then some batch processing routines using C#, and as a comparison with Spring Batch, certain features are the same. Maybe it will be interesting to develop a open source batch framework.

  • Commit batch process periodically
  • Concurrent batch processing: parallel processing of a job
  • Staged, enterprise message-driven processing
  • Massively parallel batch processing
  • Manual or scheduled restart after failure
  • Sequential processing of dependent steps (with extensions to workflow-driven batches)
  • Partial processing: skip records (e.g. on rollback)
  • Whole-batch transaction: for cases with a simple enough data model or a small batch size

If anyone of you know of any open source .NET batch framework, please let me know.


Trackback URL


RSS Feed for This Post18 Comment(s)

  1. David | Sep 27, 2007 | Reply

    Hello. I’d like to offer to exchange links. My blog is http://www.portadev.com. If you like my proposition please let me know.

    Popiashvili David

  2. Chris Whelan | Oct 26, 2007 | Reply

    I have used Quartz to handle these types of task in Java land and since starting .net development recently, I discovered that there is a port – Quartz.net http://sourceforge.net/projects/quartznet

  3. admin | Oct 26, 2007 | Reply

    Chris,

    Quartz is a job scheduling system. A batch framework should be more than that. For Java now there is Spring Batch Framework.

    I know that Avanade, the company which contributed to the Microsoft Enterprise Library (which I think originally from its ACA.Net framework) also has a ACA Batch Framework.

  4. uwlan | Mar 27, 2008 | Reply

    I’d like to offer to exchange links. My blog is http://www.gravetown.blogspot.com. If you like my proposition please let me know.

    ulwan

  5. Al | Jul 19, 2009 | Reply

    Hi,

    Has anyone found a .NET batch framework ?

    Thanks,
    Al

  6. Ben Cops | Sep 28, 2009 | Reply

    I was looking for one and wrote one in the end. I’ve just codeplexed it up so have a look.

    http://nbatch.codeplex.com/

  7. Teun | Feb 10, 2010 | Reply

    You may be interested in my new project: BatchFlow. http://code.google.com/p/batchflow/

  8. googly | Jan 13, 2011 | Reply

    Hi All,

    Did anyone find any .Net batch framework like spring-batch?

  9. googly | Jan 13, 2011 | Reply

    @Ben – I don’t see anything with the URL http://nbatch.codeplex.com/

  10. Ben Cops | Jan 13, 2011 | Reply

    @googly, I was made to rename it.

    New name is NQueue, the code is the same.

    http://nqueue.codeplex.com/

    Enjoy!

  11. www.youtube.com | May 7, 2013 | Reply

    great points altogether, you just gained a new reader.

    What would you recommend about your post that you made a few days in the past?

    Any sure?

  12. goo.gl | May 29, 2013 | Reply

    Does your blog have a contact page? I’m having a tough time locating it but, I’d like to send you an email.
    I’ve got some creative ideas for your blog you might be interested in hearing. Either way, great site and I look forward to seeing it grow over time.

  13. http://www.nchomewarranties.com/ | Jun 15, 2013 | Reply

    obviously like your web-site however you need to test the spelling on several of your posts.
    Several of them are rife with spelling issues and I to find it very bothersome to tell the reality on the other hand I will surely come back again.

  14. mathan | Aug 8, 2013 | Reply

    Thanks for sharing this useful information.
    Regards..
    Mathan
    DOT NET Training in Chennai

  15. .Net Training in Chennai | Sep 23, 2013 | Reply

    Thanks for the valuable info.

  16. Travis Quinn | Sep 26, 2013 | Reply

    There are so many other web application frameworks for Java and .NET, and new ones are coming always, but nothing able to bit .NET development framework in nearest future.

  17. https://www.Facebook.com/pages/Sumitseojacksonville/369789216497393 | Apr 24, 2014 | Reply

    If its customers are satisfied, a company will go ahead even
    in diffiicult competition. Walla!: It is an Israeli Internet information
    portal as also a lookup motor and an mal service supplier.
    https://www.Facebook.com/pages/Sumitseojacksonville/369789216497393

  18. delhi international movers and packers | Jul 10, 2014 | Reply

    I loved as much as you will receive carried out right here.
    The sketch is attractive, your authored subject matter stylish.
    nonetheless, you command get bought an shakiness over that you
    wish be delivering the following. unwell unquestionably
    come further formerly again since exactly the same nearly a lot
    often inside case you shield this increase.

1 Trackback(s)

  1. From Mark SoftWare Top » Open Source.NET Batch Framework ? Do we have one ? | Nov 3, 2007

RSS Feed for This PostPost a Comment

*