com.rapidminer.operator.io
Class BytewiseExampleSource

java.lang.Object
  com.rapidminer.tools.AbstractObservable<Operator>
      com.rapidminer.operator.Operator
          com.rapidminer.operator.io.AbstractReader<ExampleSet>
              com.rapidminer.operator.io.AbstractExampleSource
                  com.rapidminer.operator.io.BytewiseExampleSource

All Implemented Interfaces:

ConfigurationListener, PreviewListener, ResourceConsumer, ParameterHandler, LoggingHandler, Observable<Operator>

Direct Known Subclasses:

DasyLabDataReader, SPSSExampleSource, StataExampleSource

public abstract class BytewiseExampleSource
extends AbstractExampleSource

Superclass for file data source operators which read the file byte per byte into a byte array and extract the actual data from that array. This class provides some methods to extract integer and floating point values from such an array.

Author:

Tobias Malbrecht

Nested Class Summary

Nested classes/interfaces inherited from class com.rapidminer.operator.io.AbstractReader

AbstractReader.ReaderDescription

Field Summary

protected static java.lang.String	GENERIC_ERROR_MESSAGE A generic wrong file format error message.
protected static int	LENGTH_BYTE The length of a byte measured in bytes.
protected static int	LENGTH_DOUBLE The length of a double measured in bytes.
protected static int	LENGTH_INT_32 The length of an int measured in bytes.
static java.lang.String	PARAMETER_DATAMANAGEMENT The parameter name for "Determines, how the data is represented internally.
static java.lang.String	PARAMETER_FILENAME The parameter name for "Name of the file to read the data from.
protected static java.lang.String	UNSPECIFIED_ERROR_MESSAGE A even more generic error message.

Constructor Summary

BytewiseExampleSource(OperatorDescription description)

Method Summary

ExampleSet	createExampleSet() Creates (or reads) the ExampleSet that will be returned by Operator.apply().
protected int	extract2ByteInt(byte[] buffer, int offset, boolean reverseEndian) Extracts a 2-byte (short) int from a byte array.
protected double	extractDouble(byte[] value, int offset, boolean reverseEndian) Extracts a double from a byte array.
protected float	extractFloat(byte[] value, int offset, boolean reverseEndian) Extracts a float from a byte array.
protected int	extractInt(byte[] buffer, int offset, boolean reverseEndian) Extracts an int from a byte array.
protected java.lang.String	extractString(byte[] value, int offset, int length) Extracts string from byte array.
protected abstract java.lang.String	getFileSuffix() Returns the suffix of the files which should be read by the input operator.
java.util.List<ParameterType>	getParameterTypes() Returns a list of ParameterTypes describing the parameters of this operator.
protected int	read(java.io.InputStream inputStream, byte[] buffer, char divider) Reads bytes from a given file reader until either a certain character is read, the buffer is completely filled or the end of file is reached.
protected int	read(java.io.InputStream inputStream, byte[] buffer, char[] divider) Reads bytes from a given file reader until either a specified character sequence is read, the buffer is completely filled or the end of file is reached.
protected int	read(java.io.InputStream inputStream, byte[] buffer, int length) Reads a number (specified by length) of bytes from a given file reader into a byte array beginning at index 0.
protected int	read(java.io.InputStream inputStream, byte[] buffer, int offset, int length) Reads a number (specified by length) of bytes from a given file reader into a byte array beginning at the given offset.
protected abstract ExampleSet	readStream(java.io.InputStream inputStream, DataRowFactory dataRowFactory) Reads the given file and constructs an example set from the read data.
protected int	readWithoutLengthCheck(java.io.InputStream inputStream, byte[] buffer, int length) Reads a number (specified by length) of bytes from a given file reader into a byte array beginning at index 0.

Methods inherited from class com.rapidminer.operator.io.AbstractExampleSource

getGeneratedMetaData, read

Methods inherited from class com.rapidminer.operator.io.AbstractReader

addAnnotations, canMakeReaderFor, createReader, doWork, getFileParameterForOperator, isMetaDataCacheable, registerOperator, registerReaderDescription, supportsEncoding

Methods inherited from class com.rapidminer.operator.Operator

acceptsInput, addError, addError, addValue, addWarning, apply, apply, assumePreconditionsSatisfied, checkAll, checkAllExcludingMetaData, checkDeprecations, checkForStop, checkIO, checkProperties, clear, clearErrorList, cloneOperator, collectErrors, createExperimentTree, createExperimentTree, createFromXML, createFromXML, createFromXML, createMarkedExperimentTree, createMarkedProcessTree, createProcessTree, createProcessTree, disconnectPorts, execute, fireUpdate, freeMemory, getAddOnlyAdditionalOutput, getApplyCount, getCompatibilityLevel, getDeliveredOutputClasses, getDeprecationInfo, getDesiredInputClasses, getDOMRepresentation, getEncoding, getErrorList, getExecutionUnit, getExperiment, getIncompatibleVersionChanges, getInput, getInput, getInput, getInputClasses, getInputDescription, getInputPorts, getIODescription, getLog, getLogger, getName, getNumberOfBreakpoints, getOperatorClassName, getOperatorDescription, getOutputClasses, getOutputPorts, getParameter, getParameterAsBoolean, getParameterAsChar, getParameterAsColor, getParameterAsDouble, getParameterAsFile, getParameterAsFile, getParameterAsInputStream, getParameterAsInt, getParameterAsMatrix, getParameterAsRepositoryLocation, getParameterAsString, getParameterHandler, getParameterList, getParameters, getParameterTupel, getParameterType, getParent, getPortOwner, getProcess, getResourceConsumptionEstimator, getRoot, getStartTime, getTransformer, getUserDescription, getValue, getValues, getXML, getXML, getXML, hasBreakpoint, hasBreakpoint, hasInput, inApplyLoop, isDebugMode, isDirty, isEnabled, isExpanded, isParallel, isParameterSet, isRunning, log, log, logError, logNote, logWarning, lookupOperator, makeDirty, makeDirtyOnUpdate, notifyRenaming, performAdditionalChecks, preAutoWire, processFinished, processStarts, producesOutput, propagateDirtyness, register, remove, removeAndKeepConnections, rename, resume, setBreakpoint, setCompatibilityLevel, setEnabled, setEnclosingProcess, setExpanded, setInput, setListParameter, setPairParameter, setParameter, setParameters, setUserDescription, shouldAutoConnect, shouldAutoConnect, shouldStopStandaloneExecution, toString, transformMetaData, unregisterOperator, updateExecutionOrder, walk, writeXML, writeXML

Methods inherited from class com.rapidminer.tools.AbstractObservable

addObserver, addObserverAsFirst, fireUpdate, removeObserver

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

PARAMETER_FILENAME

public static final java.lang.String PARAMETER_FILENAME

The parameter name for "Name of the file to read the data from."

See Also:

Constant Field Values

PARAMETER_DATAMANAGEMENT

public static final java.lang.String PARAMETER_DATAMANAGEMENT

The parameter name for "Determines, how the data is represented internally."

See Also:

Constant Field Values

GENERIC_ERROR_MESSAGE

protected static final java.lang.String GENERIC_ERROR_MESSAGE

A generic wrong file format error message.

See Also:

Constant Field Values

UNSPECIFIED_ERROR_MESSAGE

protected static final java.lang.String UNSPECIFIED_ERROR_MESSAGE

A even more generic error message.

See Also:

Constant Field Values

LENGTH_BYTE

protected static final int LENGTH_BYTE

The length of a byte measured in bytes.

See Also:

Constant Field Values

LENGTH_INT_32

protected static final int LENGTH_INT_32

The length of an int measured in bytes.

See Also:

Constant Field Values

LENGTH_DOUBLE

protected static final int LENGTH_DOUBLE

The length of a double measured in bytes.

See Also:

Constant Field Values

Constructor Detail

BytewiseExampleSource

public BytewiseExampleSource(OperatorDescription description)

Method Detail

createExampleSet

public ExampleSet createExampleSet()
                            throws OperatorException

Description copied from class: AbstractExampleSource

Creates (or reads) the ExampleSet that will be returned by Operator.apply().

Specified by:

createExampleSet in class AbstractExampleSource

Throws:

OperatorException

getFileSuffix

protected abstract java.lang.String getFileSuffix()

Returns the suffix of the files which should be read by the input operator.

readStream

protected abstract ExampleSet readStream(java.io.InputStream inputStream,
                                         DataRowFactory dataRowFactory)
                                  throws java.io.IOException,
                                         UndefinedParameterError

Reads the given file and constructs an example set from the read data.

Throws:

java.io.IOException

UndefinedParameterError

read

protected int read(java.io.InputStream inputStream,
                   byte[] buffer,
                   int length)
            throws java.io.IOException

Reads a number (specified by length) of bytes from a given file reader into a byte array beginning at index 0.

Throws:

java.io.IOException

read

protected int read(java.io.InputStream inputStream,
                   byte[] buffer,
                   int offset,
                   int length)
            throws java.io.IOException

Reads a number (specified by length) of bytes from a given file reader into a byte array beginning at the given offset.

Throws:

java.io.IOException

readWithoutLengthCheck

protected int readWithoutLengthCheck(java.io.InputStream inputStream,
                                     byte[] buffer,
                                     int length)
                              throws java.io.IOException

Reads a number (specified by length) of bytes from a given file reader into a byte array beginning at index 0. No read length verification is performed.

Throws:

java.io.IOException

read

protected int read(java.io.InputStream inputStream,
                   byte[] buffer,
                   char divider)
            throws java.io.IOException

Reads bytes from a given file reader until either a certain character is read, the buffer is completely filled or the end of file is reached.

Throws:

java.io.IOException

read

protected int read(java.io.InputStream inputStream,
                   byte[] buffer,
                   char[] divider)
            throws java.io.IOException

Reads bytes from a given file reader until either a specified character sequence is read, the buffer is completely filled or the end of file is reached.

Throws:

java.io.IOException

extract2ByteInt

protected int extract2ByteInt(byte[] buffer,
                              int offset,
                              boolean reverseEndian)

Extracts a 2-byte (short) int from a byte array.

extractInt

protected int extractInt(byte[] buffer,
                         int offset,
                         boolean reverseEndian)

Extracts an int from a byte array.

extractFloat

protected float extractFloat(byte[] value,
                             int offset,
                             boolean reverseEndian)

Extracts a float from a byte array.

extractDouble

protected double extractDouble(byte[] value,
                               int offset,
                               boolean reverseEndian)

Extracts a double from a byte array.

extractString

protected java.lang.String extractString(byte[] value,
                                         int offset,
                                         int length)

Extracts string from byte array.

getParameterTypes

public java.util.List<ParameterType> getParameterTypes()

Description copied from class: Operator

Returns a list of ParameterTypes describing the parameters of this operator. The default implementation returns an empty list if no input objects can be retained and special parameters for those input objects which can be prevented from being consumed. ATTENTION! This will create new parameterTypes. For calling already existing parameter types use getParameters().getParameterTypes();

Specified by:

getParameterTypes in interface ParameterHandler

Overrides:

getParameterTypes in class AbstractReader<ExampleSet>