net.nutch.util
Class NutchRemoteFileSystem

java.lang.Object
  net.nutch.util.NutchGenericFileSystem
      net.nutch.util.NutchRemoteFileSystem

All Implemented Interfaces:

NutchFileSystem

public class NutchRemoteFileSystem

extends NutchGenericFileSystem

NutchRemoteFileSystem implements the NutchFileSystem over machines that can be linked via some set of command-line args. (presumably 'scp').

Author:

Mike Cafarella

Constructor Summary

NutchRemoteFileSystem(File dbRoot, ShareSet shareSet, String cpTemplate, String rmTemplate, String mkdirTemplate)
The NutchRemoteFileSystem takes template-strings for its various needed commands, which may differ among installations.

NutchRemoteFileSystem(File dbRoot, String cpTemplate, String rmTemplate, String mkdirTemplate)
Create the ShareSet automatically, then do regular constructor.

Method Summary

protected void	copyFile(File srcFile, String locationMach, String locationStr, String nutchFileName, boolean overwrite) Copy a file from one place to another.
protected void	deleteFile(String locationMach, String locationStr, String nutchFileName) Remove a file the given location.
protected void	lockFile(String locMach, String locStr, String filename, boolean exclusive) Currently unimplemented
protected void	release(String locMach, String locStr, String filename)
protected void	renameFile(File srcFile, String locMach, String locStr, String filename, boolean overwrite)

Methods inherited from class net.nutch.util.NutchGenericFileSystem

close, completeDir, delete, get, get, getWorkingFile, lock, put, release, renameTo

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

NutchRemoteFileSystem

public NutchRemoteFileSystem(File dbRoot,
                             String cpTemplate,
                             String rmTemplate,
                             String mkdirTemplate)
                      throws IOException

Create the ShareSet automatically, then do regular constructor.

NutchRemoteFileSystem

public NutchRemoteFileSystem(File dbRoot,
                             ShareSet shareSet,
                             String cpTemplate,
                             String rmTemplate,
                             String mkdirTemplate)
                      throws IOException

The NutchRemoteFileSystem takes template-strings for its various needed commands, which may differ among installations. The class will fill in these templates with the necessary args, and then invoke them via System.exec(). We're given the ShareSet here.

Method Detail

copyFile

protected void copyFile(File srcFile,
                        String locationMach,
                        String locationStr,
                        String nutchFileName,
                        boolean overwrite)
                 throws IOException

Copy a file from one place to another. Requires that template-strings be set correctly.

Specified by:

copyFile in class NutchGenericFileSystem

Throws:

IOException

deleteFile

protected void deleteFile(String locationMach,
                          String locationStr,
                          String nutchFileName)
                   throws IOException

Remove a file the given location. Requires that template- strings be set correctly.

Specified by:

deleteFile in class NutchGenericFileSystem

Throws:

IOException

lockFile

protected void lockFile(String locMach,
                        String locStr,
                        String filename,
                        boolean exclusive)
                 throws IOException

Currently unimplemented

Specified by:

lockFile in class NutchGenericFileSystem

Throws:

IOException

release

protected void release(String locMach,
                       String locStr,
                       String filename)
                throws IOException

Specified by:

release in class NutchGenericFileSystem

Throws:

IOException

renameFile

protected void renameFile(File srcFile,
                          String locMach,
                          String locStr,
                          String filename,
                          boolean overwrite)
                   throws IOException

Specified by:

renameFile in class NutchGenericFileSystem

Throws:

IOException