Pages: [1]
  Print  
Author Topic: Cannot evaluate performance for current parameter combination  (Read 1384 times)
ben2313
Guest
« on: December 21, 2008, 01:09:45 PM »

Hi,

I'm doing some GridParameterOptimization for RandomForests. My model
is pretty much the same as the one given in the samples.

Three parameters are evaluated:

RandomForest.number_of_trees, min=1.0, max=100.0, steps=10, logarithmic
RandomForest.criterion, all possible criterion
and RandomForest.maximal_depth, min=-1.0, max=100.0, steps=10, logarithmic.

Sometimes, the learner terminates with:

[NOTE] OperatorChain: Process stopped.
[Warning] GridParameterOptimization: Cannot evaluate performance for current
parameter combination: Process stopped in OperatorChain
[NOTE] GridParameterOptimization: Process stopped.

Why Is it possible that some parameter combinations are not valid? And
how can I figure out which combination failed?
To avoid such "crashes" in the future, is it possible to tell RapidMiner to
ignore these invalid combinations without terminating?

Thanks,
ben
Logged
Sebastian Land
Administrator
Hero Member
*****
Posts: 2426


« Reply #1 on: December 23, 2008, 02:03:25 PM »

Hi Ben,
could you post your process file here?  Thats probably not the behavior intended Smiley And at the first glance I don't see, why this can happen...

Greetings,
  Sebastian
Logged
ben2313
Guest
« Reply #2 on: December 24, 2008, 12:57:32 PM »

Hi Sebastian,

that's my model:

Code:
<operator name="Root" class="Process" expanded="yes">
    <operator name="CSVExampleSource" class="CSVExampleSource">
        <parameter key="filename" value="examples.csv"/>
        <parameter key="label_name" value="mylabel"/>
    </operator>
    <operator name="GridParameterOptimization" class="GridParameterOptimization" expanded="yes">
        <list key="parameters">
          <parameter key="RandomForest.number_of_trees" value="[1.0;100.0;10;logarithmic]"/>
          <parameter key="RandomForest.criterion" value="gain_ratio,information_gain,gini_index,accuracy"/>
          <parameter key="RandomForest.maximal_depth" value="[-1.0;100.0;10;logarithmic]"/>
        </list>
        <operator name="XValidation" class="XValidation" expanded="yes">
            <parameter key="leave_one_out" value="true"/>
            <parameter key="sampling_type" value="shuffled sampling"/>
            <operator name="RandomForest" class="RandomForest">
                <parameter key="criterion" value="gain_ratio"/>
                <parameter key="maximal_depth" value="-1"/>
                <parameter key="number_of_trees" value="5405"/>
            </operator>
            <operator name="OperatorChain" class="OperatorChain" expanded="yes">
                <operator name="ModelApplier" class="ModelApplier">
                    <list key="application_parameters">
                    </list>
                </operator>
                <operator name="ClassificationPerformance" class="ClassificationPerformance">
                    <parameter key="absolute_error" value="true"/>
                    <parameter key="accuracy" value="true"/>
                    <list key="class_weights">
                    </list>
                    <parameter key="classification_error" value="true"/>
                </operator>
            </operator>
        </operator>
        <operator name="ProcessLog" class="ProcessLog">
            <parameter key="filename" value="/tmp/process.log"/>
            <list key="log">
              <parameter key="iteration" value="operator.XValidation.value.iteration"/>
              <parameter key="time" value="operator.XValidation.value.time"/>
              <parameter key="deviation" value="operator.XValidation.value.deviation"/>
              <parameter key="variance" value="operator.XValidation.value.variance"/>
              <parameter key="accuracy" value="operator.XValidation.value.performance"/>
              <parameter key="classification_error" value="operator.XValidation.value.performance2"/>
              <parameter key="absolute_error" value="operator.XValidation.value.performance3"/>
              <parameter key="num_trees" value="operator.RandomForest.parameter.number_of_trees"/>
              <parameter key="criterion" value="operator.RandomForest.parameter.criterion"/>
              <parameter key="max_depth" value="operator.RandomForest.parameter.maximal_depth"/>
            </list>
            <parameter key="persistent" value="true"/>
        </operator>
    </operator>
    <operator name="PerformanceWriter" class="PerformanceWriter">
        <parameter key="performance_file" value="/tmp/performance.per"/>
    </operator>
    <operator name="ParameterSetWriter" class="ParameterSetWriter">
        <parameter key="parameter_file" value="/tmp/parameters.par"/>
    </operator>
</operator>

Maybe you can find something.

Thank you.

Ben
Logged
Sebastian Land
Administrator
Hero Member
*****
Posts: 2426


« Reply #3 on: January 05, 2009, 11:28:04 AM »

Hi Ben,
it seems to me, that RapidMiner does not have the permission to write into the file you specified. Did you check that?
I'm only able to reproduce the error then.

Greetings,
  Sebastian
Logged
Pages: [1]
  Print  
 
Jump to: