net.nutch.fs
Class FSDataset

java.lang.Object
  net.nutch.fs.FSDataset

All Implemented Interfaces:

FSConstants

public class FSDataset

extends Object

implements FSConstants

FSDataset manages a set of data blocks. Each block has a unique name and an extent on disk.

Author:

Mike Cafarella

Field Summary

Fields inherited from interface net.nutch.fs.FSConstants

BLOCK_SIZE, BLOCKREPORT_INTERVAL, CHUNKED_ENCODING, DATANODE_STARTUP_PERIOD, FILE_COMPLETE_FAILED, FILE_COMPLETE_ONGOING, FILE_COMPLETE_SUCCESS, HEARTBEAT_INTERVAL, OP_ACK, OP_BLOCKRECEIVED, OP_BLOCKREPORT, OP_CLIENT_ADDBLOCK, OP_CLIENT_ADDBLOCK_ACK, OP_CLIENT_COMPLETEFILE, OP_CLIENT_COMPLETEFILE_ACK, OP_CLIENT_DELETE, OP_CLIENT_DELETE_ACK, OP_CLIENT_LISTING, OP_CLIENT_LISTING_ACK, OP_CLIENT_OPEN, OP_CLIENT_OPEN_ACK, OP_CLIENT_RENAMETO, OP_CLIENT_RENAMETO_ACK, OP_CLIENT_STARTFILE, OP_CLIENT_STARTFILE_ACK, OP_CLIENT_TRYAGAIN, OP_ERROR, OP_FAILURE, OP_HEARTBEAT, OP_INVALIDATE_BLOCKS, OP_READ_BLOCK, OP_TRANSFERBLOCKS, OP_TRANSFERDATA, OP_WRITE_BLOCK, RUNLENGTH_ENCODING, SYSTEM_STARTUP_PERIOD

Constructor Summary

FSDataset(File dir, long capacity)
An FSDataset has a directory where it loads its data files.

Method Summary

void	finalizeBlock(Block b) Complete the block write!
InputStream	getBlockData(Block b) Get a stream of data from the indicated block.
Block[]	getBlockReport() Return a table of block data
long	getCapacity() Return total capacity, used and unused
long	getLength(Block b) Find the block's on-disk length
long	getRemaining() Return how many bytes can still be stored in the FSDataset
void	invalidate(Block[] invalidBlks) We're informed that a block is no longer valid.
boolean	isValidBlock(Block b) Check whether the given block is a valid one.
boolean	startBlock(Block b) A Block b will be coming soon!
OutputStream	writeToBlock(Block b) Start writing to a block file

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FSDataset

public FSDataset(File dir,
                 long capacity)
          throws IOException

An FSDataset has a directory where it loads its data files.

Method Detail

getCapacity

public long getCapacity()

Return total capacity, used and unused

getRemaining

public long getRemaining()

Return how many bytes can still be stored in the FSDataset

getLength

public long getLength(Block b)
               throws IOException

Find the block's on-disk length

Throws:

IOException

getBlockData

public InputStream getBlockData(Block b)
                         throws IOException

Get a stream of data from the indicated block.

Throws:

IOException

startBlock

public boolean startBlock(Block b)
                   throws IOException

A Block b will be coming soon!

Throws:

IOException

writeToBlock

public OutputStream writeToBlock(Block b)
                          throws IOException

Start writing to a block file

Throws:

IOException

finalizeBlock

public void finalizeBlock(Block b)
                   throws IOException

Complete the block write!

Throws:

IOException

getBlockReport

public Block[] getBlockReport()

Return a table of block data

isValidBlock

public boolean isValidBlock(Block b)

Check whether the given block is a valid one.

invalidate

public void invalidate(Block[] invalidBlks)
                throws IOException

We're informed that a block is no longer valid. We could lazily garbage-collect the block, but why bother? just get rid of it.

Throws:

IOException