com.rapidminer.operator.validation
Class ValidationChain

java.lang.Object
  com.rapidminer.operator.Operator
      com.rapidminer.operator.OperatorChain
          com.rapidminer.operator.validation.ValidationChain

All Implemented Interfaces:

ConfigurationListener, PreviewListener, ParameterHandler, LoggingHandler

Direct Known Subclasses:

AbstractBootstrappingValidation, BatchSlidingWindowValidation, BatchXValidation, FixedSplitValidationChain, RandomSplitValidationChain, SlidingWindowValidation, XValidation

public abstract class ValidationChain
extends OperatorChain

Abstract superclass of operator chains that split an ExampleSet into a training and test set and return a performance vector. The two inner operators must be a learner returning a Model and an operator or operator chain that can apply this model and returns a PerformanceVector. Hence the second inner operator usually is an operator chain containing a model applier and a performance evaluator.

Author:

Ingo Mierswa, Simon Fischer

Field Summary

static java.lang.String

PARAMETER_CREATE_COMPLETE_MODEL The parameter name for "Indicates if a model of the complete data set should be additionally build after estimation.

Constructor Summary

ValidationChain(OperatorDescription description)

Method Summary

IOObject[]	apply() Applies all inner operators.
abstract IOObject[]	estimatePerformance(ExampleSet inputSet) This is the main method of the validation chain and must be implemented to estimate a performance of inner operators on the given example set.
protected IOContainer	evaluate(ExampleSet testSet) Applies the applier and evaluator (= second encapsulated inner operator).
IOContainer	evaluate(ExampleSet testSet, IOContainer learnResult) Applies the applier and evaluator (= second encapsulated inner operator).
InnerOperatorCondition	getInnerOperatorCondition() Must return a condition of the IO behaviour of all desired inner operators.
java.lang.Class<?>[]	getInputClasses() Returns the the classes this operator provides as output.
InputDescription	getInputDescription(java.lang.Class cls) The default implementation returns an input description that consumes the input IOObject without a user parameter.
protected Operator	getLearner() Returns the first encapsulated inner operator (or operator chain), i.e.
int	getMaxNumberOfInnerOperators() Returns the maximum number of innner operators.
int	getMinNumberOfInnerOperators() Returns the minimum number of innner operators.
java.lang.Class<?>[]	getOutputClasses() Returns the the classes this operator expects as input.
java.util.List<ParameterType>	getParameterTypes() Returns a list of ParameterTypes describing the parameters of this operator.
protected IOContainer	learn(ExampleSet trainingSet) Applies the learner (= first encapsulated inner operator).
protected void	setResult(PerformanceVector pv) Can be used by subclasses to set the performance of the example set.

Methods inherited from class com.rapidminer.operator.OperatorChain

addAddListener, addOperator, addOperator, checkDeprecations, checkIO, checkNumberOfInnerOperators, checkProperties, clearErrorList, cloneOperator, createExperimentTree, createProcessTree, getAllInnerOperators, getIndexOfOperator, getInnerOperatorForName, getInnerOperatorsXML, getNumberOfAllOperators, getNumberOfOperators, getOperator, getOperatorFromAll, getOperators, performAdditionalChecks, processFinished, processStarts, registerOperator, removeAddListener, removeOperator, shouldAddNonConsumedInput, shouldReturnInnerOutput, unregisterOperator

Methods inherited from class com.rapidminer.operator.Operator

addError, addValue, addWarning, apply, checkForStop, createExperimentTree, createFromXML, createMarkedExperimentTree, createMarkedProcessTree, createProcessTree, getAddOnlyAdditionalOutput, getApplyCount, getDeliveredOutputClasses, getDeprecationInfo, getDesiredInputClasses, getEncoding, getErrorList, getExperiment, getInput, getInput, getInput, getIOContainerForInApplyLoopBreakpoint, getIODescription, getLog, getName, getOperatorClassName, getOperatorDescription, getParameter, getParameterAsBoolean, getParameterAsColor, getParameterAsDouble, getParameterAsFile, getParameterAsFile, getParameterAsInputStream, getParameterAsInt, getParameterAsMatrix, getParameterAsString, getParameterList, getParameters, getParameterType, getParent, getProcess, getStartTime, getStatus, getUserDescription, getValue, getValues, getXML, hasBreakpoint, hasBreakpoint, hasInput, inApplyLoop, isDebugMode, isEnabled, isExpanded, isParallel, isParameterSet, log, logError, logNote, logWarning, register, remove, rename, resume, setApplyCount, setBreakpoint, setEnabled, setExpanded, setInput, setListParameter, setOperatorParameters, setParameter, setParameters, setParent, setUserDescription, toString, writeXML

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

PARAMETER_CREATE_COMPLETE_MODEL

public static final java.lang.String PARAMETER_CREATE_COMPLETE_MODEL

The parameter name for "Indicates if a model of the complete data set should be additionally build after estimation."

See Also:

Constant Field Values

Constructor Detail

ValidationChain

public ValidationChain(OperatorDescription description)

Method Detail

estimatePerformance

public abstract IOObject[] estimatePerformance(ExampleSet inputSet)
                                        throws OperatorException

This is the main method of the validation chain and must be implemented to estimate a performance of inner operators on the given example set. The implementation can make use of the provided helper methods in this class.

Throws:

OperatorException

getMaxNumberOfInnerOperators

public int getMaxNumberOfInnerOperators()

Returns the maximum number of innner operators.

Specified by:

getMaxNumberOfInnerOperators in class OperatorChain

getMinNumberOfInnerOperators

public int getMinNumberOfInnerOperators()

Returns the minimum number of innner operators.

Specified by:

getMinNumberOfInnerOperators in class OperatorChain

getInputDescription

public InputDescription getInputDescription(java.lang.Class cls)

Description copied from class: Operator

The default implementation returns an input description that consumes the input IOObject without a user parameter. Subclasses may override this method to allow other input handling behaviors.

Overrides:

getInputDescription in class Operator

getInputClasses

public java.lang.Class<?>[] getInputClasses()

Returns the the classes this operator provides as output.

Specified by:

getInputClasses in class Operator

getOutputClasses

public java.lang.Class<?>[] getOutputClasses()

Returns the the classes this operator expects as input.

Specified by:

getOutputClasses in class Operator

getInnerOperatorCondition

public InnerOperatorCondition getInnerOperatorCondition()

Description copied from class: OperatorChain

Must return a condition of the IO behaviour of all desired inner operators. If there are no "special" conditions and the chain works similar to a simple operator chain this method should at least return a SimpleChainInnerOperatorCondition. More than one condition should be combined with help of the class CombinedInnerOperatorCondition.

Specified by:

getInnerOperatorCondition in class OperatorChain

getLearner

protected Operator getLearner()

Returns the first encapsulated inner operator (or operator chain), i.e. the learning operator (chain).

setResult

protected final void setResult(PerformanceVector pv)

Can be used by subclasses to set the performance of the example set.

apply

public IOObject[] apply()
                 throws OperatorException

Description copied from class: OperatorChain

Applies all inner operators. The input to this operator becomes the input of the first inner operator. The latter's output is passed to the second inner operator and so on. Note to subclassers: If subclasses (for example wrappers) want to make use of this method remember to call exactly this method (super.apply()) and do not call super.apply(IOContainer) erroneously which will result in an infinite loop.

Overrides:

apply in class OperatorChain

Returns:

the last inner operator's output or the input itself if the chain is empty.

Throws:

OperatorException

learn

protected IOContainer learn(ExampleSet trainingSet)
                     throws OperatorException

Applies the learner (= first encapsulated inner operator).

Throws:

OperatorException

evaluate

public IOContainer evaluate(ExampleSet testSet,
                            IOContainer learnResult)
                     throws OperatorException

Applies the applier and evaluator (= second encapsulated inner operator). In order to reuse possibly created predicted label attributes, we do the following: We compare the predicted label of testSet before and after applying the inner operator. If it changed, the predicted label is removed again. No outer operator could ever see it. The same applies for the confidence attributes in case of classification learning.

Throws:

OperatorException

evaluate

protected IOContainer evaluate(ExampleSet testSet)
                        throws OperatorException

Applies the applier and evaluator (= second encapsulated inner operator). In order to reuse possibly created predicted label attributes, we do the following: We compare the predicted label of testSet before and after applying the inner operator. If it changed, the predicted label is removed again. No outer operator could ever see it. The same applies for the confidence attributes in case of classification learning.

Throws:

OperatorException

getParameterTypes

public java.util.List<ParameterType> getParameterTypes()

Description copied from class: Operator

Returns a list of ParameterTypes describing the parameters of this operator. The default implementation returns an empty list if no input objects can be retained and special parameters for those input objects which can be prevented from being consumed.

Specified by:

getParameterTypes in interface ParameterHandler

Overrides:

getParameterTypes in class Operator