Pages: [1]
  Print  
Author Topic: Precision/Recall charts  (Read 2697 times)
svm_friend
Newbie
*
Posts: 3


« on: October 17, 2008, 07:41:57 PM »

Is it possible to create 2D-Precision/Recall charts like this on:

http://proteomics.bioengr.uic.edu/malibu/docs/images/evaluate/spr_thumb.png

All I found were ROC-curves
Logged
Sebastian Land
Administrator
Hero Member
*****
Posts: 2426


« Reply #1 on: October 27, 2008, 10:31:50 AM »

Hi,
I'm not quite sure, that this is exactly what you want. But you could log the precision / recall and plot it using the plotter as you plot your data.
Here's a process showing how to log them:
Code:
<operator name="Root" class="Process" expanded="yes">
    <description text="#ylt#p#ygt# Often the different operators have many parameters and it is not clear which parameter values are best for the learning task at hand. The parameter optimization operator helps to find an optimal parameter set for the used operators. #ylt#/p#ygt#  #ylt#p#ygt# The inner crossvalidation estimates the performance for each parameter set. In this experiment two parameters of the SVM are tuned. The result can be plotted in 3D (using gnuplot) or in color mode. #ylt#/p#ygt#  #ylt#p#ygt# Try the following: #ylt#ul#ygt# #ylt#li#ygt#Start the experiment. The result is the best parameter set and the performance which was achieved with this parameter set.#ylt#/li#ygt# #ylt#li#ygt#Edit the parameter list of the ParameterOptimization operator to find another parameter set.#ylt#/li#ygt# #ylt#/ul#ygt# #ylt#/p#ygt# "/>
    <operator name="Input" class="ExampleSource">
        <parameter key="attributes" value="..\data\iris.aml"/>
    </operator>
    <operator name="ParameterOptimization" class="GridParameterOptimization" expanded="yes">
        <list key="parameters">
          <parameter key="Training.C" value="50,100,150,200,250"/>
          <parameter key="Training.degree" value="1,2,3,4,5"/>
        </list>
        <operator name="Validation" class="XValidation" expanded="yes">
            <parameter key="sampling_type" value="shuffled sampling"/>
            <operator name="Training" class="LibSVMLearner">
                <parameter key="C" value="250.0"/>
                <list key="class_weights">
                </list>
                <parameter key="degree" value="5"/>
                <parameter key="epsilon" value="0.01"/>
                <parameter key="kernel_type" value="poly"/>
            </operator>
            <operator name="ApplierChain" class="OperatorChain" expanded="yes">
                <operator name="Test" class="ModelApplier">
                    <list key="application_parameters">
                    </list>
                </operator>
                <operator name="ClassificationPerformance" class="ClassificationPerformance">
                    <list key="class_weights">
                    </list>
                    <parameter key="main_criterion" value="weighted_mean_recall"/>
                    <parameter key="weighted_mean_precision" value="true"/>
                    <parameter key="weighted_mean_recall" value="true"/>
                </operator>
            </operator>
        </operator>
        <operator name="Log" class="ProcessLog">
            <parameter key="filename" value="paraopt.log"/>
            <list key="log">
              <parameter key="C" value="operator.Training.parameter.C"/>
              <parameter key="degree" value="operator.Training.parameter.degree"/>
              <parameter key="recall" value="operator.Validation.value.performance1"/>
              <parameter key="precision" value="operator.Validation.value.performance2"/>
            </list>
        </operator>
    </operator>
</operator>

Greetings,
  Sebastian
Logged
ema
Newbie
*
Posts: 33


« Reply #2 on: March 04, 2009, 11:17:29 AM »

In case of other classifiers like nearest neighbors and others
what parameters should we consider
Logged
Sebastian Land
Administrator
Hero Member
*****
Posts: 2426


« Reply #3 on: March 06, 2009, 11:19:51 AM »

Hi Ema,
do you ask, which parameters you should log into the processLog? I think this depends on your task at hand. Values you never change probably don't give you much information. So include everything you might optimize or change during the process.

Greetings,
  Sebastian
Logged
Pages: [1]
  Print  
 
Jump to: