com.rapidminer.operator.performance
Class MeasuredPerformance

java.lang.Object
  com.rapidminer.operator.AbstractIOObject
      com.rapidminer.operator.ResultObjectAdapter
          com.rapidminer.tools.math.Averagable
              com.rapidminer.operator.performance.PerformanceCriterion
                  com.rapidminer.operator.performance.MeasuredPerformance

All Implemented Interfaces:

IOObject, ResultObject, Saveable, Readable, Reportable, LoggingHandler, java.io.Serializable, java.lang.Cloneable, java.lang.Comparable<PerformanceCriterion>

Direct Known Subclasses:

AreaUnderCurve, BinaryClassificationPerformance, ClassificationCostCriterion, CorrelationCriterion, CrossEntropy, LogisticLoss, Margin, MDLCriterion, MinMaxCriterion, MultiClassificationPerformance, NormalizedAbsoluteError, PredictionAverage, PredictionTrendAccuracy, RankCorrelation, RootRelativeSquaredError, SimpleCriterion, SoftMarginLoss, WeightedMultiClassPerformance

public abstract class MeasuredPerformance
extends PerformanceCriterion

Superclass for performance citeria that are actually measured (not estimated). These criteria can be calculated by the operator PerformanceEvaluator.

Beside the methods from Averagable and PerformanceCriterion this class must implement some additonal methods. Please note that the actual measurement could be done in either countExample(Example) or in startCounting(ExampleSet) but is does not need to be performed in both. In all cases where your measure needs to see the entire example set at one time, the actual computation of performance is done in startCounting and the countExample method returns without doing anything. For a performance measure that will be computed incrementally, you would do initialization in startCounting and updating in countExample. In all cases where an incremental calculation is possible the calculation and update in countExample(Example) is preferred since this would allow the easy integration of this criterion for incremental scenarios (although this is currently not supported).

IMPORTANT: Please note that your criterion need a public constructor without arguments in order to create this object via reflection. It is also possible to have a one-argument string constructor taking a user defined parameter string. This is of course not necessary if your criterion does not handle any parameters. Another public one-argument constructor taking an object of the same class must also be provided (clone constructor). This constructor is used to copy the internal settings of a different instance of your measure into the current instance. If you were computing mean and the argument to cloneAverageable was an instance with mean nu and sample count n, you would set the mean of the current instance to nu and the sample count of the current instance to n. Don't forget to invoke the super constructor first in your clone constructor.

The method Averagable.getMikroAverage() should return the value computed in startCounting or the current value calculated by countExample or Double.NaN if the value was not yet known, or was not well defined). If you can calculate a value for micro variance you should return this value in Averagable.getMikroVariance(). If this is not possible simply return Double.NaN.

The method PerformanceCriterion.getExampleCount() returns the number of observations used to calculate the measure. This value is usually used for building micro averages (in the method Averagable.buildSingleAverage(Averagable) and might be important for calculating significance tests.

If your measure is averageable, you use Averagable.buildSingleAverage(Averagable) to fold another copy of your measure into the current copy. Suppose you were implementing sample mean. The current instance would have a mean mu and sample count m. If buildSingleAverage were invoked on another instance with mean nu and sample count n, you would update the current instance to mean (m*mu+n*nu)/(m+n) and count m+n. This is actually the computation of the micro average.

In some cases the criterion is better if the value returned by Averagable.getAverage() is smaller. In order to support optimizations, the method PerformanceCriterion.getFitness() should return the result of Averagable.getAverage() (not from Averagable.getMikroAverage() since the macro average is often more stable for this purpose and will be returned by getAverage() if possible) in cases where higher values are better. If smaller values are better, the method should return something like -1 * Averagable.getAverage().

The methods Averagable.getName() and PerformanceCriterion.getDescription() are user by the user interface. Please note that the name should only contain lowercase letters and underscore (RapidMiner parameter format). You might also want to override ResultObjectAdapter.toResultString() or Averagable.getVisualizationComponent(com.rapidminer.operator.IOContainer) in order to provide a nice visualization for the performance criterion but you usually don't have to.

Author:

Ingo Mierswa, Simon Fischer Exp $

See Also:

Serialized Form

Constructor Summary

MeasuredPerformance()

MeasuredPerformance(MeasuredPerformance o)

Method Summary

abstract void	countExample(Example example) Counts a single example, e.g. by summing up errors.
void	startCounting(ExampleSet set) Deprecated. Please use the other start counting method directly
void	startCounting(ExampleSet set, boolean useExampleWeights) Initializes the criterion.

Methods inherited from class com.rapidminer.operator.performance.PerformanceCriterion

compareTo, getDescription, getExampleCount, getFitness, getMaxFitness

Methods inherited from class com.rapidminer.tools.math.Averagable

buildAverage, buildSingleAverage, clone, cloneAveragable, formatPercent, getAverage, getAverageCount, getExtension, getFileDescription, getMakroAverage, getMakroStandardDeviation, getMakroVariance, getMikroAverage, getMikroStandardDeviation, getMikroVariance, getName, getStandardDeviation, getVariance, getVisualizationComponent, setAverageCount, toString

Methods inherited from class com.rapidminer.operator.ResultObjectAdapter

addAction, getActions, getResultIcon, isSavable, log, logError, logNote, logWarning, save, toHTML, toResultString

Methods inherited from class com.rapidminer.operator.AbstractIOObject

copy, getLog, getSource, initWriting, read, setLoggingHandler, setSource, write

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface com.rapidminer.operator.IOObject

copy, getLog, getSource, setLoggingHandler, setSource, write

Constructor Detail

MeasuredPerformance

public MeasuredPerformance()

MeasuredPerformance

public MeasuredPerformance(MeasuredPerformance o)

Method Detail

countExample

public abstract void countExample(Example example)

Counts a single example, e.g. by summing up errors.

startCounting

@Deprecated
public final void startCounting(ExampleSet set)
                         throws OperatorException

Deprecated. Please use the other start counting method directly

Initialized the criterion. The default implementation invokes the initialization method with useExampleWeights set to true.

Throws:

OperatorException

startCounting

public void startCounting(ExampleSet set,
                          boolean useExampleWeights)
                   throws OperatorException

Initializes the criterion. The default implementation does nothing.

Throws:

OperatorException