org.apache.derby.impl.store.raw.data
Class RAFContainer4

java.lang.Object
  org.apache.derby.impl.store.raw.data.BaseContainer
      org.apache.derby.impl.store.raw.data.FileContainer
          org.apache.derby.impl.store.raw.data.RAFContainer
              org.apache.derby.impl.store.raw.data.RAFContainer4

All Implemented Interfaces:

java.security.PrivilegedExceptionAction, Cacheable, TypedFormat, Lockable

 class RAFContainer4
extends RAFContainer

RAFContainer4 overrides a few methods in FileContainer/RAFContainer in order to use FileChannel from Java 1.4's New IO framework to issue multiple IO operations to the same file concurrently instead of strictly serializing IO operations using a mutex on the container object. Since we compile with Java 1.4, the override "annotations" are inside the method javadoc headers.

Note that our requests for multiple concurrent IOs may be serialized further down in the IO stack - this is entirely up to the JVM and OS. However, at least in Linux on Sun's 1.4.2_09 JVM we see the desired behavior: The FileChannel.read/write(ByteBuffer buf, long position) calls map to pread/pwrite system calls, which enable efficient IO to the same file descriptor by multiple threads.

This whole class should be merged back into RAFContainer when Derby officially stops supporting Java 1.3.

Significant behavior changes from RAFContainer:

1. Multiple concurrent IOs permitted.
2. State changes to the container (create, open, close) can now happen while IO is in progress due to the lack of locking. Closing a container while IO is in progress will cause IOExceptions in the thread calling readPage or writePage. If this happens something is probably amiss anyway. The iosInProgress variable is used in an attempt to detect this should it happen while running a debug build.

See Also:

FileChannel

Field Summary

private java.lang.Object	channelCleanupMonitor
private ContainerKey	currentIdentity
private boolean	giveUpIO
private java.lang.Object	giveUpIOm
private int	iosInProgress For debugging - will be incremented when an IO is started, decremented when it is done.
private java.nio.channels.FileChannel	ourChannel This channel will be retrieved from RAFContainer's fileData member when fileData is set.
private boolean	restoreChannelInProgress
private int	threadsInPageIO

Fields inherited from class org.apache.derby.impl.store.raw.data.RAFContainer

fileData, needsSync

Fields inherited from class org.apache.derby.impl.store.raw.data.FileContainer

allocCache, canUpdate, CHECKSUM_SIZE, CONTAINER_INFO_SIZE, containerCache, containerInfo, containerVersion, dataFactory, estimatedRowCount, FIRST_ALLOC_PAGE_NUMBER, FIRST_ALLOC_PAGE_OFFSET, firstAllocPageNumber, firstAllocPageOffset, formatIdInteger, initialPages, isDirty, lastLogInstant, minimumRecordSize, pageCache, pageSize, preDirty, SPACE_TRACE, spareSpace

Fields inherited from class org.apache.derby.impl.store.raw.data.BaseContainer

identity, isCommittedDrop, isDropped, isReusableRecordId

Constructor Summary

RAFContainer4(BaseDataFileFactory factory)

Method Summary

private void	awaitRestoreChannel(java.lang.Exception e, boolean stealthMode) Use when seeing an exception during IO and when another thread is presumably doing the recovery.
(package private) void	closeContainer() override of RAFContainer#closeContainer
(package private) void	createContainer(ContainerKey newIdentity) override of RAFContainer#createContainer
private static void	debugTrace(java.lang.String msg)
private java.nio.channels.FileChannel	getChannel() Return the file channel for the current value of the fileData field.
private java.nio.channels.FileChannel	getChannel(StorageRandomAccessFile file) Return the FileChannel for the specified StorageRandomAccessFile if it is a RandomAccessFile.
(package private) byte[]	getEmbryonicPage(StorageRandomAccessFile file, long offset) Read an embryonic page (that is, a section of the first alloc page that is so large that we know all the borrowed space is included in it) from the specified offset in a StorageRandomAccessFile.
private void	handleClosedChannel(java.nio.channels.ClosedChannelException e, boolean stealthMode, int retries) This method handles what to do when, during a NIO operation we receive a ClosedChannelException.
(package private) boolean	openContainer(ContainerKey newIdentity) Open a container.
private void	readFull(java.nio.ByteBuffer dstBuffer, java.nio.channels.FileChannel srcChannel, long position) Attempts to fill buf completely from start until it's full.
protected void	readPage(long pageNumber, byte[] pageData) Read a page into the supplied array.
private void	readPage(long pageNumber, byte[] pageData, long offset) Read a page into the supplied array.
private void	readPage0(long pageNumber, byte[] pageData, long offset)
private boolean	recoverContainerAfterInterrupt(java.lang.String whence, boolean stealthMode) Use this when the thread has received a ClosedByInterruptException (or, prior to JDK 1.7 it may also be AsynchronousCloseException - a bug) exception during IO and its interruped flag is also set.
private void	reopen() When the existing channel (ourChannel) has been closed due to interrupt, we need to reopen the underlying RAF to get a fresh channel so we can resume IO.
(package private) void	writeAtOffset(StorageRandomAccessFile file, byte[] bytes, long offset) Write a sequence of bytes at the given offset in a file.
private void	writeFull(java.nio.ByteBuffer srcBuffer, java.nio.channels.FileChannel dstChannel, long position) Attempts to write buf completely from start until end, at the given position in the destination fileChannel.
protected void	writePage(long pageNumber, byte[] pageData, boolean syncPage) Write a page from the supplied array.
private void	writePage0(long pageNumber, byte[] pageData, boolean syncPage)

Methods inherited from class org.apache.derby.impl.store.raw.data.RAFContainer

backupContainer, clean, encryptOrDecryptContainer, flushAll, getFileName, getRandomAccessFile, isDirty, preAllocate, privGetFileName, removeContainer, removeFile, reopenContainer, run, truncatePages, updatePageArray

Methods inherited from class org.apache.derby.impl.store.raw.data.FileContainer

bumpContainerVersion, canUpdate, clearIdentity, clearPreallocThreshold, compressContainer, createIdent, createIdentity, deallocatePage, decryptPage, doPreAllocatePages, dropContainer, encryptPage, getAllocPage, getAnyPage, getContainerProperties, getContainerVersion, getEmbryonicPage, getEncryptionBuffer, getEstimatedPageCount, getEstimatedRowCount, getFirstHeadPage, getHeadPage, getLastPageNumber, getLatchedPage, getMinimumRecordSize, getNextHeadPage, getPage, getPageForCompress, getPageForInsert, getPageSize, getReusableRecordIdSequenceNumber, getSpaceInfo, getSpareSpace, getTypeFormatId, incrementReusableRecordIdSequenceNumber, initPage, latchPage, letGo, logCreateContainerInfo, newPage, preDirty, prepareForBulkLoad, readHeader, reCreatePageForRedoRecovery, setDirty, setEstimatedRowCount, setIdent, setIdentity, trackUnfilledPage, updateEstimatedRowCount, writeHeader, writeHeader

Methods inherited from class org.apache.derby.impl.store.raw.data.BaseContainer

addPage, compressContainer, fillInIdentity, getAllocPage, getAnyPage, getCommittedDropState, getContainerId, getContainerStatus, getDeallocLock, getDroppedState, getFirstPage, getIdentity, getNextPage, getSegmentId, isReusableRecordId, lockAttributes, lockerAlwaysCompatible, lockEvent, removePage, requestCompatible, setCommittedDropState, setDroppedState, setReusableRecordIdState, truncate, unlockEvent, use

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.derby.iapi.services.cache.Cacheable

getIdentity

Field Detail

ourChannel

private java.nio.channels.FileChannel ourChannel

This channel will be retrieved from RAFContainer's fileData member when fileData is set. We wrap a couple of RAFContainer's methods to accomplish this.

channelCleanupMonitor

private final java.lang.Object channelCleanupMonitor

threadsInPageIO

private volatile int threadsInPageIO

restoreChannelInProgress

private volatile boolean restoreChannelInProgress

giveUpIO

private boolean giveUpIO

giveUpIOm

private final java.lang.Object giveUpIOm

iosInProgress

private int iosInProgress

For debugging - will be incremented when an IO is started, decremented when it is done. Should be == 0 when container state is changed.

currentIdentity

private ContainerKey currentIdentity

Constructor Detail

RAFContainer4

public RAFContainer4(BaseDataFileFactory factory)

Method Detail

getChannel

private java.nio.channels.FileChannel getChannel(StorageRandomAccessFile file)

Return the FileChannel for the specified StorageRandomAccessFile if it is a RandomAccessFile. Otherwise, return null.

Parameters:

file - the file to get the channel for

Returns:

a FileChannel if file is an instance of RandomAccessFile, null otherwise

getChannel

private java.nio.channels.FileChannel getChannel()

Return the file channel for the current value of the fileData field. If fileData doesn't support file channels, return null.

Callers of this method must synchronize on the container object since two shared fields (fileData and ourChannel) are accessed.

Returns:

a FileChannel object, if supported, or null

openContainer

boolean openContainer(ContainerKey newIdentity)
                throws StandardException

Description copied from class: FileContainer

Open a container.

Longer descrption of routine.

Open a container. Open the file that maps to this container, if the file does not exist then we assume the container was never created. If the file exists but we have trouble opening it then we throw some exception.
MT - single thread required - Enforced by cache manager.

Overrides:

openContainer in class RAFContainer

Throws:

StandardException - Standard exception policy.

createContainer

void createContainer(ContainerKey newIdentity)
               throws StandardException

override of RAFContainer#createContainer

Overrides:

createContainer in class RAFContainer

Throws:

StandardException - Derby Standard error policy

reopen

private void reopen()
             throws StandardException

When the existing channel (ourChannel) has been closed due to interrupt, we need to reopen the underlying RAF to get a fresh channel so we can resume IO.

Throws:

StandardException

closeContainer

void closeContainer()

override of RAFContainer#closeContainer

Overrides:

closeContainer in class RAFContainer

readPage

protected void readPage(long pageNumber,
                        byte[] pageData)
                 throws java.io.IOException,
                        StandardException

Read a page into the supplied array.

override of RAFContainer#readPage

MT - thread safe

Overrides:

readPage in class RAFContainer

Throws:

java.io.IOException - exception reading page

StandardException - Standard Derby error policy

readPage

private void readPage(long pageNumber,
                      byte[] pageData,
                      long offset)
               throws java.io.IOException,
                      StandardException

Read a page into the supplied array.

override of RAFContainer#readPage

MT - thread safe

Parameters:

pageNumber - the page number to read data from, or -1 (called from getEmbryonicPage)

pageData - the buffer to read data into

offset - -1 normally (not used since offset is computed from pageNumber), but used if pageNumber == -1 (getEmbryonicPage)

Throws:

java.io.IOException - exception reading page

StandardException - Standard Derby error policy

readPage0

private void readPage0(long pageNumber,
                       byte[] pageData,
                       long offset)
                throws java.io.IOException,
                       StandardException

Throws:

java.io.IOException

StandardException

writePage

protected void writePage(long pageNumber,
                         byte[] pageData,
                         boolean syncPage)
                  throws java.io.IOException,
                         StandardException

Write a page from the supplied array.

override of RAFContainer#writePage

MT - thread safe

Overrides:

writePage in class RAFContainer

Throws:

StandardException - Standard Derby error policy

java.io.IOException - IO error accessing page

handleClosedChannel

private void handleClosedChannel(java.nio.channels.ClosedChannelException e,
                                 boolean stealthMode,
                                 int retries)
                          throws StandardException

This method handles what to do when, during a NIO operation we receive a ClosedChannelException. Note the specialization hierarchy:

ClosedChannelException -> AsynchronousCloseException -> ClosedByInterruptException

If e is a ClosedByInterruptException, we normally start container recovery, i.e. we need to reopen the random access file so we get get a new interruptible channel and continue IO.

If e is a AsynchronousCloseException or a plain ClosedChannelException, the behavior depends of stealthMode:

If stealthMode == false, the method will wait for another thread tp finish recovering the IO channel before returning.

If stealthMode == true, the method throws InterruptDetectedException, allowing retry at a higher level in the code. The reason for this is that we sometimes need to release monitors on objects needed by the recovery thread.

Parameters:

e - Should be an instance of ClosedChannelException.

stealthMode - If true, do retry at a higher level

retries - Give up waiting for another thread to reopen the channel when retries reaches 0. Only applicable if stealthMode == false.

Throws:

InterruptDetectedException - if retry at higher level is required stealthMode == true.

StandardException - standard error policy, incl. when we give up waiting for another thread to reopen channel

awaitRestoreChannel

private void awaitRestoreChannel(java.lang.Exception e,
                                 boolean stealthMode)
                          throws StandardException

Use when seeing an exception during IO and when another thread is presumably doing the recovery.

If stealthMode == false, wait for another thread to recover the container after an interrupt. If stealthMode == true, throw internal exception InterruptDetectedException to do retry from higher in the stack.

If stealthMode == false, maximum wait time for the container to become available again is determined by the product InterruptStatus.MAX_INTERRUPT_RETRIES * InterruptStatus.INTERRUPT_RETRY_SLEEP. There is a chance this thread will not see any recovery occuring (yet), in which case it waits for a bit and just returns, so the caller must retry IO until success.

If for some reason the recovering thread has given up on resurrecting the container, cf #giveUpIO, the method throws FILE_IO_INTERRUPTED.

Parameters:

e - the exception we saw during IO

stealthMode - true if the thread doing IO in stealth mode

Throws:

StandardException - InterruptDetectedException and normal error policy

recoverContainerAfterInterrupt

private boolean recoverContainerAfterInterrupt(java.lang.String whence,
                                               boolean stealthMode)
                                        throws StandardException

Use this when the thread has received a ClosedByInterruptException (or, prior to JDK 1.7 it may also be AsynchronousCloseException - a bug) exception during IO and its interruped flag is also set. This makes this thread a likely candicate to do container recovery, unless another thread started it already, cf. return value.

Parameters:

whence - caller site (debug info)

stealthMode - don't update threadsInPageIO if true

Returns:

true if we did recovery, false if we saw someone else do it and abstained

Throws:

StandardException

writePage0

private void writePage0(long pageNumber,
                        byte[] pageData,
                        boolean syncPage)
                 throws java.io.IOException,
                        StandardException

Throws:

java.io.IOException

StandardException

writeAtOffset

void writeAtOffset(StorageRandomAccessFile file,
                   byte[] bytes,
                   long offset)
             throws java.io.IOException,
                    StandardException

Write a sequence of bytes at the given offset in a file. This method operates in stealth mode, see doc for handleClosedChannel. This presumes that IO retry happens at a higher level, i.e. the caller(s) must be prepared to handle InterruptDetectedException.

This method overrides FileContainer#writeAtOffset.

Overrides:

writeAtOffset in class FileContainer

Parameters:

file - the file to write to

bytes - the bytes to write

offset - the offset to start writing at

Throws:

java.io.IOException - if an I/O error occurs while writing

StandardException - Derby Standard error policy

getEmbryonicPage

byte[] getEmbryonicPage(StorageRandomAccessFile file,
                        long offset)
                  throws java.io.IOException,
                         StandardException

Read an embryonic page (that is, a section of the first alloc page that is so large that we know all the borrowed space is included in it) from the specified offset in a StorageRandomAccessFile.

override of FileContainer#getEmbryonicPage

Overrides:

getEmbryonicPage in class FileContainer

Parameters:

file - the file to read from

offset - where to start reading (normally FileContainer.FIRST_ALLOC_PAGE_OFFSET)

Returns:

a byte array containing the embryonic page

Throws:

java.io.IOException - if an I/O error occurs while reading

StandardException - if thread is interrupted.

readFull

private void readFull(java.nio.ByteBuffer dstBuffer,
                      java.nio.channels.FileChannel srcChannel,
                      long position)
               throws java.io.IOException,
                      StandardException

Attempts to fill buf completely from start until it's full.

FileChannel has no readFull() method, so we roll our own.

Parameters:

dstBuffer - buffer to read into

srcChannel - channel to read from

position - file position from where to read

Throws:

java.io.IOException - if an I/O error occurs while reading

StandardException - If thread is interrupted.

writeFull

private void writeFull(java.nio.ByteBuffer srcBuffer,
                       java.nio.channels.FileChannel dstChannel,
                       long position)
                throws java.io.IOException

Attempts to write buf completely from start until end, at the given position in the destination fileChannel.

FileChannel has no writeFull() method, so we roll our own.

Parameters:

srcBuffer - buffer to write

dstChannel - channel to write to

position - file position to start writing at

Throws:

java.io.IOException - if an I/O error occurs while writing

StandardException - If thread is interrupted.

debugTrace

private static void debugTrace(java.lang.String msg)