com.faunos.skwish.sys
Class BaseSegment

java.lang.Object
  com.faunos.skwish.Segment
      com.faunos.skwish.sys.BaseSegment

All Implemented Interfaces:

Range

Direct Known Subclasses:

UnitDir.Seg

public class BaseSegment
extends Segment

The base implementation that is backed by an offset (index) and a contents file.

Author:

Babak Farhang

See Also:

Implementation Notes: On where files begin

Field Summary

protected FileChannel	entryFile
protected static ChannelUtil<SegmentException>	helper
protected Index	index

Fields inherited from class com.faunos.skwish.Segment

validator

Constructor Summary

BaseSegment(FileChannel offsetFile, FileChannel entryFile)
Reads and returns a new instance using the given offset (index) file and associated entry [contents] file.

BaseSegment(Index index, FileChannel entryFile)
Creates a new instance.

Method Summary

protected ByteBuffer	allocateWorkBuffer() Returns a work buffer for the current thread of execution.
void	append(BaseSegment segment) Appends the given segment to this segment.
void	append(BaseSegment segment, int maxIdGap) Appends the given segment to this segment.
void	close() Closes this segment and its underlying streams.
void	commit() Commits the entry count.
void	delete(long id, int count, boolean purge) Deletes count entries starting at the entry with the specified id.
void	flush() Flushes out the backing storage streams.
protected long	generateNoise() Returns a numeric value used to overwrite entries that are to be purged.
long	getBaseId() Returns the instance's base ID.
long	getContentsSize() Returns an estimate of the byte size of the total entry contents contained in this segment.
void	getEntry(long id, ByteBuffer out, ByteBuffer workBuffer) Returns the contents of the entry with the specified id.
FileChannel	getEntryChannel(long id) Returns the contents of the entry with the given id as a FileChannel, or null, if deleted.
long	getEntryCount() Returns the number of entries in this instance.
FileChannel	getEntryInsertionChannel() Return an output stream (FileChannel) for inserting the next entry.
int	getEntryPart(long id, long position, ByteBuffer out, ByteBuffer workBuffer) Returns the specified part of the entry contents with the specified id.
long	getEntrySize(long id, ByteBuffer workBuffer) Returns the size of the entry with the specified id.
FileChannel	getEntryUpdateChannel(long id)
long	getNextId() Returns the ID the entry will get on the next insertion.
long	getTotalSize() Returns an estimate of the total byte size of this segment, accounting for the index overhead.
long	insertEntry(ByteBuffer entry) Inserts the given entry and returns the entry's ID.
long	insertEntry(ReadableByteChannel entry) Inserts the entry represented by the given channel.
boolean	isAlive() Determines whether the instance is still alive.
boolean	isAutoCommit() Determines whether the instance is in auto-commit mode.
boolean	isReadOnly() Determines whether this instance is read-only (unmodifiable).
long	killNext(int count) Kills the next count entry IDs.
protected SubFileChannel	newSubFileChannelImpl(FileChannel entryFile, long start, long end, boolean update) Returns the implementation-specific SubFileChannel.
void	setAutoCommit(boolean autoCommit) Sets the auto-commit property.
static BaseSegment	writeNewSegment(FileChannel offsetFile, FileChannel entryFile, long baseId, Word type) Writes and returns a new segment to the specified offset (index) and entry [contents] files.

Methods inherited from class com.faunos.skwish.Segment

contains, delete, delete, getEntry, getEntrySize, getEntryStream, hi, isDeleted, isDeleted, killNext, lo, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

index

protected final Index index

entryFile

protected final FileChannel entryFile

helper

protected static final ChannelUtil<SegmentException> helper

Constructor Detail

BaseSegment

public BaseSegment(FileChannel offsetFile,
                   FileChannel entryFile)
            throws IOException

Reads and returns a new instance using the given offset (index) file and associated entry [contents] file.

Parameters:

offsetFile - the offset (index) file. Must be positioned at the start of the index (which need not be at the beginning of the file)

entryFile - the associated file containing the entry contents. The file position doesn't matter.

Throws:

IOException

BaseSegment

public BaseSegment(Index index,
                   FileChannel entryFile)
            throws IOException

Creates a new instance.

Throws:

IOException

See Also:

pseudo-constructor

Method Detail

writeNewSegment

public static BaseSegment writeNewSegment(FileChannel offsetFile,
                                          FileChannel entryFile,
                                          long baseId,
                                          Word type)
                                   throws IOException

Writes and returns a new segment to the specified offset (index) and entry [contents] files. Both files are assumed to be positioned at the start of where the segment will begin (which need not necessarily be at the beginning of the files).

Throws:

IOException

isReadOnly

public boolean isReadOnly()

Description copied from class: Segment

Determines whether this instance is read-only (unmodifiable).

Specified by:

isReadOnly in class Segment

Returns:

false

See Also:

Write operations

getBaseId

public long getBaseId()

Description copied from class: Segment

Returns the instance's base ID. An instances valid entry IDs are in the range [baseId, baseId + entryCount).

Specified by:

getBaseId in class Segment

See Also:

Segment.getEntryCount()

getEntryCount

public long getEntryCount()

Description copied from class: Segment

Returns the number of entries in this instance.

Specified by:

getEntryCount in class Segment

getNextId

public long getNextId()

Description copied from class: Segment

Returns the ID the entry will get on the next insertion.

Specified by:

getNextId in class Segment

See Also:

ID invariants

killNext

public long killNext(int count)
              throws IOException

Description copied from class: Segment

Kills the next count entry IDs. A killed entry is an entry that begins (and ends) life in the deleted state.

Specified by:

killNext in class Segment

Parameters:

count - the number of entries to kill (must be greater than 1)

Returns:

the value that getNextId() returns after this method has completed

Throws:

IOException

getContentsSize

public long getContentsSize()

Returns an estimate of the byte size of the total entry contents contained in this segment. This is an estimate because deleted entries may contribute to this size. Also, the returned size does not include the index overhead per entry (typically 4 bytes).

See Also:

getTotalSize()

getTotalSize

public long getTotalSize()

Returns an estimate of the total byte size of this segment, accounting for the index overhead.

insertEntry

public long insertEntry(ByteBuffer entry)
                 throws IOException

Description copied from class: Segment

Inserts the given entry and returns the entry's ID. The contents of the entry is understood to be the remaining contents in the given buffer.

Specified by:

insertEntry in class Segment

Parameters:

entry - they contents of the entry

Returns:

the ID assigned to the entry

Throws:

IOException

insertEntry

public long insertEntry(ReadableByteChannel entry)
                 throws IOException

Description copied from class: Segment

Inserts the entry represented by the given channel. The contents are of the entry is understood to be the entire remaining contents in the specified channel.

This is the most efficient method for inserting a large-size entry (even one that doesn't fit in memory).

Specified by:

insertEntry in class Segment

Throws:

IOException

getEntryInsertionChannel

public FileChannel getEntryInsertionChannel()
                                     throws IOException

Return an output stream (FileChannel) for inserting the next entry. The next entry is actually created on invoking the close method on the returned FileChannel. If another insert operation (including this one) is invoked on this instance before the returned stream is closed, then an IllegalStateException is raised.

Throws:

IOException

delete

public void delete(long id,
                   int count,
                   boolean purge)
            throws IOException

Description copied from class: Segment

Deletes count entries starting at the entry with the specified id. The contents of the deleted entries may optionally be requested to be purged (overwritten) immediately; o.w. the deleted contents will be purged whenever the segment is compacted or merged with another segment.

Note that purging is not part of any transaction semantic; whether or not a purge immediately takes place on its being signaled is implementation-dependent.

Specified by:

delete in class Segment

Parameters:

id - the ID of the first entry to be deleted. Must be a valid ID in this segment [baseId, baseId + entryCount)

count - the number of entries to be deleted. Specifies a range of IDs to be deleted: [id, id + count). The range of IDs must be valid entries within this segment; o.w. an exception is raised.

purge - if true, the contents of the deleted entries are signaled to be purged (overwritten) immediately; o.w. the contents will be deleted on the next time the segment is compacted.

Throws:

IOException

generateNoise

protected long generateNoise()

Returns a numeric value used to overwrite entries that are to be purged. This base implementation always returns 0. A subclass might use a random number generator to better mask purged entries.

getEntrySize

public long getEntrySize(long id,
                         ByteBuffer workBuffer)
                  throws IOException

Description copied from class: Segment

Returns the size of the entry with the specified id. A work buffer with a minimum size of 16 must be passed in.

Specified by:

getEntrySize in class Segment

Returns:

the size of the entry if present; -1, if deleted.

Throws:

IOException

getEntryPart

public int getEntryPart(long id,
                        long position,
                        ByteBuffer out,
                        ByteBuffer workBuffer)
                 throws IOException

Description copied from class: Segment

Returns the specified part of the entry contents with the specified id. Up to out.remaining() bytes of the contents of the entry are copied into the given out buffer.

Specified by:

getEntryPart in class Segment

position - the starting position into the entry from where data will be copied into the out buffer

out - the out buffer into which data is written

workBuffer - a work buffer with a minimum size of 16

Returns:

the number of bytes read (possibly zero if out has no remaining bytes), or -1, if the position is greater than the size of the entry, or if the entry is deleted.

Throws:

IOException

getEntry

public void getEntry(long id,
                     ByteBuffer out,
                     ByteBuffer workBuffer)
              throws IOException

Description copied from class: Segment

Returns the contents of the entry with the specified id. If the entry is deleted, the method returns without modifying the out buffer.

Specified by:

getEntry in class Segment

out - the buffer into which the contents are written. Its remaining bytes must be large enough; o.w., an exception is raised.

workBuffer - a work buffer used internally to read offsets. The minimum capacity of this work buffer must be 16

Throws:

IOException

getEntryChannel

public FileChannel getEntryChannel(long id)
                            throws IOException

Description copied from class: Segment

Returns the contents of the entry with the given id as a FileChannel, or null, if deleted. The returned channel is positioned at zero and is read-only.

Specified by:

getEntryChannel in class Segment

Throws:

IOException

See Also:

newSubFileChannelImpl(FileChannel, long, long, boolean)

getEntryUpdateChannel

public FileChannel getEntryUpdateChannel(long id)
                                  throws IOException

Throws:

IOException

newSubFileChannelImpl

protected SubFileChannel newSubFileChannelImpl(FileChannel entryFile,
                                               long start,
                                               long end,
                                               boolean update)
                                        throws IOException

Returns the implementation-specific SubFileChannel. This method provides a hook for subclasses.

Throws:

IOException

See Also:

getEntryChannel(long)

close

public void close()
           throws IOException

Closes this segment and its underlying streams. Note if this instance's underlying streams are shared by other instances of this class, then invoking this method will effectively close those instances as well.

It's OK to invoke this method multiple times.

Throws:

IOException

isAlive

public boolean isAlive()

Determines whether the instance is still alive. An instance is no longer alive once closed.

flush

public void flush()
           throws IOException

Flushes out the backing storage streams.

Throws:

IOException

isAutoCommit

public boolean isAutoCommit()

Determines whether the instance is in auto-commit mode. When not in auto-commit mode, entry insertions (and also kills) are not committed to the backing files until commit() is called; deletions are committed right away.

By default, BaseSegments are created in auto-commit mode.

See Also:

commit(), setAutoCommit(boolean)

setAutoCommit

public void setAutoCommit(boolean autoCommit)

Sets the auto-commit property.

commit

public void commit()
            throws IOException

Commits the entry count. This only matters if the instance is not in auto-commit mode.

Throws:

IOException

See Also:

isAutoCommit(), setAutoCommit(boolean)

append

public void append(BaseSegment segment)
            throws IOException

Appends the given segment to this segment. The contents of entries marked "deleted" are not appended to this segment. The segment's base ID is expected to be close to, but ≥ the next ID of this segment. Specifically, the segment's base ID may be no smaller than this segment's next ID, and no larger than next ID + 1024.

This method simply invokes append(segment, 1024).

Throws:

IOException

See Also:

append(BaseSegment, int)

append

public void append(BaseSegment segment,
                   int maxIdGap)
            throws IOException

Appends the given segment to this segment. The contents of entries marked "deleted" are not appended to this segment. The segment's base ID is expected to be close to, but ≥ the next ID of this segment. The maxIdGap parameter specifies minimum requirements for the proximity of IDs in the segment to be appended.

The append method is designed to support merging segments.

Implementation note:

This method changes the state of the underlying entry file. Specifically, on return from this method, the entry stream is positioned at the end of the file; the state of the source segment, together with that of its underlying streams, however, remains unchanged.

Throws:

IOException

See Also:

append(BaseSegment)

allocateWorkBuffer

protected ByteBuffer allocateWorkBuffer()

Returns a work buffer for the current thread of execution. Currently, only used by the append method. The base class returns a newly allocated 1kb direct buffer. A better implementation may involve perhaps the use of larger thread-local buffer. Such refinements are deferred to subclasses.

Returns:

the base implementation return a newly allocated 1kB direct buffer