RSS Feed for This PostCurrent Article

Develop a Java File Watcher

Download Source Code

These are some utlities classes that I wrote some time back to monitor file creation, deletion, and modification in a specified directory.The DirectoryWatcher implements AbstractResourceWatcher to monitor a specified directory. It accepts a ResourceListener class, which is FileListener in thi case. Whenever DirectoryWatcher detects any changes, it will raise the onAdd, onChange, or onDelete events in the listener class, passing in the necessary File object.It is quite straightforward, but is useful when you are developing Java backend ETL processes.

public class DirectoryWatcher extends AbstractResourceWatcher {
/**
* The current map of files and their timestamps (String fileName => Long
* lastMod)
*/
private Map currentFiles = new HashMap();

/**
* The directory to watch.
*/
private String directory;

/**
* The map of last recorded files and their timestamps (String fileName =>
* Long lastMod)
*/
private Map prevFiles = new HashMap();

/**
* Constructor that takes the directory to watch.
*
* @param directoryPath the directory to watch
* @param intervalSeconds The interval to use when monitoring this
* directory. I.e., ever x seconds, check this directory to see
* what has changed.
* @throws IllegalArgumentException if the argument does not map to a
* valid directory
*/
public DirectoryWatcher(String directoryPath, int intervalSeconds)
throws IllegalArgumentException {

//Get the common thread interval stuff set up.
super(intervalSeconds, directoryPath + ” interval watcher.”);

//Check that it is indeed a directory.
File theDirectory = new File(directoryPath);

if (theDirectory != null && !theDirectory.isDirectory()) {

//This is bad, so let the caller know
String message = “The path ” + directory +
” does not represent a valid directory.”;
throw new IllegalArgumentException(message);

}

//Else all is well so set this directory and the interval
this.directory = directoryPath;

}

/**
* For testing only.
*
* @param args
*/
public static void main(String[] args) {
// Monitor c:/temp every 5 seconds
DirectoryWatcher dw = new DirectoryWatcher(“c:/temp/”, 5);
dw.addListener(new FileListener());
dw.start();
}

/**
* Start the monitoring of this directory.
*/
public void start() {

//Since we’re going to start monitoring, we want to take a snapshot of the
//current directory to we have something to refer to when stuff changes.
takeSnapshot();

//And start the thread on the given interval
super.start();

//And notify the listeners that monitoring has started
File theDirectory = new File(directory);
monitoringStarted(theDirectory);
}

/**
* Stop the monitoring of this directory.
*/
public void stop() {

//And start the thread on the given interval
super.stop();

//And notify the listeners that monitoring has started
File theDirectory = new File(directory);
monitoringStopped(theDirectory);
}

/**
* Store the file names and the last modified timestamps of all the files
* and directories that exist in the directory at this moment.
*/
private void takeSnapshot() {

//Set the last recorded snap shot to be the current list
prevFiles.clear();
prevFiles.putAll(currentFiles);

//And get a new current state with all the files and directories
currentFiles.clear();

File theDirectory = new File(directory);
File[] children = theDirectory.listFiles();

//Store all the current files and their timestamps
for (int i = 0; i < children.length; i++) {

File file = children[i];
currentFiles.put(file.getAbsolutePath(),
new Long(file.lastModified()));

}

}

/**
* Check this directory for any changes and fire the proper events.
*/
protected void doInterval() {

//Take a snapshot of the current state of the dir for comparisons
takeSnapshot();

//Iterate through the map of current files and compare
//them for differences etc…
Iterator currentIt = currentFiles.keySet().iterator();

while (currentIt.hasNext()) {

String fileName = (String) currentIt.next();
Long lastModified = (Long) currentFiles.get(fileName);

//If this file did not exist before, but it does now, then
//it’s been added
if (!prevFiles.containsKey(fileName)) {
//DirectorySnapshot.addFile(fileName);
resourceAdded(new File(fileName));
}
//If this file did exist before
else if (prevFiles.containsKey(fileName)) {

Long prevModified = (Long) prevFiles.get(fileName);

//If this file existed before and has been modified
if (prevModified.compareTo(lastModified) != 0) {
// 27 June 2006
// Need to check if the file are removed and added
// during the interval
/* if (!DirectorySnapshot.containsFile(fileName)) {
resourceAdded(new File(fileName));
} else {*/
resourceChanged(new File(fileName));
//}
}
}
}

//Now we need to iterate through the list of previous files and
//see if any that existed before don’t exist anymore
Iterator prevIt = prevFiles.keySet().iterator();

while (prevIt.hasNext()) {

String fileName = (String) prevIt.next();

//If this file did exist before, but it does not now, then
//it’s been deleted
if (!currentFiles.containsKey(fileName)) {
//DirectorySnapshot.removeFile(fileName);
resourceDeleted(fileName);
}
}
}
}

public class FileListener extends BaseListener implements IFileListener {

/**
* Connstructor
*/
public FileListener() {
super();
}

public void onStart(Object monitoredResource) {
// On startup
if (monitoredResource instanceof File) {
File resource = (File) monitoredResource;
if (resource.isDirectory()) {

System.out.println(“Start to monitor ” + resource.getAbsolutePath());

/*File[] files = resource.listFiles();
for (int i = 0; i < files.length; i++) {
File f = (File) files[i];
onAdd(f);
}*/
}
}
}

public void onStop(Object notMonitoredResource) {

}

public void onAdd(Object newResource) {
if (newResource instanceof File) {
File file = (File) newResource;
if (file.isFile()) {
System.out.println(file.getAbsolutePath() + ” is added”);
}
}
}

public void onChange(Object changedResource) {
if (changedResource instanceof File) {
File file = (File) changedResource;
if (file.isFile()) {
System.out.println(file.getAbsolutePath() + ” is changed”);
}

}
}

public void onDelete(Object deletedResource) {
if (deletedResource instanceof String) {
String deletedFile = (String) deletedResource;
System.out.println(deletedFile + ” is deleted”);
}
}
}


Trackback URL


RSS Feed for This Post16 Comment(s)

  1. Jason Daggs | Oct 2, 2007 | Reply

    This was the initial approach I tried on several occasions. However, one problem I ran into was that even though the file name shows up in the directory list the file itself may not actually exist on the file system. Let me explain. If I copy a large file from a remote network source to the local directory being monitored, that file will still show up in the directory listing, but before the network copy has completed. If I try to do almost anything non trivial to the file at that moment like move it to another directory or open it for writing, an exception will be thrown because really the file is not yet completely there and the OS still has a write lock on it. So, what I did was perform a test on each new file before I raised the “file ready” event. I opened up a FileIputStream on the file if a FileNotFoundException is thrown then sleep a few seconds and then keep trying( bit forever but you get the picture) If no exception is thrown the the file is ready.(tested on windows not yet on linux) This approach will not solve all issues of course, but did allow us to implement a very robust approach to file system monitoring. Of course, IMHO avoiding directory monitoring whenever possible would be my preferred approach.

  2. Rob Di Marco | Oct 2, 2007 | Reply

    Another implementation is available using the Java Native Access API. You can see an example FileMonitor at:

    https://jna.dev.java.net/source/browse/
    jna/trunk/jnalib/src/com/sun/jna/examples/
    FileMonitor.java?rev=HEAD&view=markup

  3. Tim Wall | Oct 2, 2007 | Reply

    The polling approach has a number of drawbacks that rapidly become apparent as the number of ‘watched’ files increases. It’s better to rely on an OS-provided service for this sort of thing.

    You’ll find that the underlying ‘stat’ triggered by many of java.io.File methods can quickly become a performance bottleneck.

    So polling is not a bad solution if you’re looking at one or two files, but I wouldn’t recommend it for an IDE tracking external changes to a project’s files.

  4. thoughtworks | Oct 2, 2007 | Reply

    Jason,

    I faced the some problem as what you encountered also. I resolved it using 2 ways.

    1. 1st way is similar to what you did, I monitor the file size/modified date and if the file size/modified date are not the same during an interval, then I will keep on waiting.

    2. Another way to resolve the large file issue is to monitor on dummy file with empty content. E.g. copying a large file to directory A and once it is done, create a dummy file in directory B, and monitoring is only only directory B. Of course this depends on the program pushing the file to create another dummy file once the upload/copying is completed.

    Rob,
    I need it on Unix/Linux platform. Let me know if you have a similiar solution on Unix/Linux platform.

    Thanks

  5. Sander | Oct 2, 2007 | Reply

    Why are you calling yourself ThoughtWorks? Do you have anything to do with http://www.thoughtworks.com?

  6. timber | Oct 3, 2007 | Reply

    There is always this project on sourceforge. Not used it though.

    http://jpoller.sourceforge.net/

  7. GuidoLx | Oct 3, 2007 | Reply

    There is just one thing I don’t like in your design.
    Whenever you are doing file-related operations, use the java.io.File object and do not use a java.lang.String!
    My 2c,

    Guido

  8. Alex Miller | Oct 3, 2007 | Reply

    BTW, support for file (or directory) watching is included in JSR 203 (the new new Java I/O API) which should be part of Java 7. More info on my Java 7 page here.

  9. sudhakar | Apr 22, 2008 | Reply

    this DirectoryWatcher applicaton is almost similar to our requirements.with this code it shows only modified or added or deleted filenames when ever you updated the directory. but in our requirement, basically directory contains number of log files and not only monitering the directory i need to parse the content of a modified or new log file and pick the required information from that log file.

    can anybody help me out please? this is urgent requirement…

  10. kyle | May 2, 2008 | Reply

    Instead of a directory, I want to watch a table inside a database for entries, and the call the appropriate methods to handle the records. Basically the requirement is:
    if the timestamp on a particular row is greater then 4 minutes, meaning its not been processed, then it should send a message to group list notifying them of this condition.

  11. vamsi krishna | Jul 16, 2008 | Reply

    hi thanks for this code i am looking for the same functionality in java.

  12. Oljas | Oct 7, 2008 | Reply

    Hi everyone.
    I try to use this program.
    I run it and open TaskManager on XP too.
    So, the resource size this program is using is growing up, each second(or 2-3 seconds) its growing for some bytes … thats because i can’t use it(after some time my computer won’t have any resource to use it). Is there any way to solve this program?

  13. Nitin Gupta | Nov 24, 2008 | Reply

    Hi, Can you please explain me how can i use this code?

    Thanks
    Nitin

  14. Suman K | Dec 9, 2008 | Reply

    Oljas,

    Add the following in the end of the takeSnapshot() function:

    Runtime runtime = Runtime.getRuntime();
    runtime.gc();

    This should solve the memory leak problem.

  15. sam | Jun 5, 2009 | Reply

    Hi,

    nice code, thx!

    However, I was wondering if the java.io.File declaration defining the watched directory could actually watch a “remote” directory (on a different server) or not? If yes, what would then be the syntax of the directory pathname (\\x.x.x.x\…)?

    Thx!

  16. debol | Nov 23, 2009 | Reply

    http://www2.hawaii.edu/~qzhang/FileSystemWatcher.html

1 Trackback(s)

  1. From iLenceel» Blog 存档 » links for 2007-10-09 | Nov 28, 2007

Sorry, comments for this entry are closed at this time.