select attributes by aggregation [SOLVED]

[sfmorais]

Hi!


I have a set of examples with one id attribute and several regular integer attributes that contains only binary data (0 or 1).
I wish select only the regular attributes whose sum (by column) is greater than a specific value (defined in a context variable)  


Which the best way to solve it (from original data, select only the regular attributes whose sum is greater than a variable value)  ?

Many thanks in advance

[Marius]

Hi,

try something like the process below. After the aggregation, the aggregated data is filtered (in this case such that the sum of att1 is smaller than -5), then joined with the original data. To understand the process, it might be helpful to define some operator breakpoints and debug through the process.

Best, Marius

Code:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.2.002">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="5.2.002" expanded="true" name="Process">
    <process expanded="true" height="206" width="882">
      <operator activated="true" class="generate_data" compatibility="5.2.002" expanded="true" height="60" name="Generate Data" width="90" x="112" y="30">
        <parameter key="target_function" value="random classification"/>
      </operator>
      <operator activated="true" class="aggregate" compatibility="5.2.002" expanded="true" height="76" name="Aggregate" width="90" x="246" y="120">
        <list key="aggregation_attributes">
          <parameter key="att1" value="sum"/>
        </list>
        <parameter key="group_by_attributes" value="|label"/>
      </operator>
      <operator activated="true" class="filter_examples" compatibility="5.2.002" expanded="true" height="76" name="Filter Examples" width="90" x="380" y="30">
        <parameter key="condition_class" value="attribute_value_filter"/>
        <parameter key="parameter_string" value="sum(att1) &lt; -5"/>
      </operator>
      <operator activated="true" class="rename" compatibility="5.2.002" expanded="true" height="76" name="Rename" width="90" x="514" y="30">
        <parameter key="old_name" value="label"/>
        <parameter key="new_name" value="label2"/>
        <list key="rename_additional_attributes"/>
      </operator>
      <operator activated="true" class="join" compatibility="5.2.002" expanded="true" height="76" name="Join" width="90" x="648" y="120">
        <parameter key="join_type" value="left"/>
        <parameter key="use_id_attribute_as_key" value="false"/>
        <list key="key_attributes">
          <parameter key="label2" value="label"/>
        </list>
      </operator>
      <operator activated="true" class="select_attributes" compatibility="5.2.002" expanded="true" height="76" name="Select Attributes" width="90" x="782" y="120">
        <parameter key="attribute_filter_type" value="subset"/>
        <parameter key="attributes" value="|sum(att1)|label2"/>
        <parameter key="invert_selection" value="true"/>
        <parameter key="include_special_attributes" value="true"/>
      </operator>
      <connect from_op="Generate Data" from_port="output" to_op="Aggregate" to_port="example set input"/>
      <connect from_op="Aggregate" from_port="example set output" to_op="Filter Examples" to_port="example set input"/>
      <connect from_op="Aggregate" from_port="original" to_op="Join" to_port="right"/>
      <connect from_op="Filter Examples" from_port="example set output" to_op="Rename" to_port="example set input"/>
      <connect from_op="Rename" from_port="example set output" to_op="Join" to_port="left"/>
      <connect from_op="Join" from_port="join" to_op="Select Attributes" to_port="example set input"/>
      <connect from_op="Select Attributes" from_port="example set output" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
    </process>
  </operator>
</process>

[sfmorais]

Hi Marius!

Thanks for your reply. But my problem is not very well what was answered (maybe I am not have been clear)

In this link  http://www.lavradeirasarcozelo.com/extra/description.pdf   I have a pdf with images that explain better my problem.


Many many thanks

Regards

[Marius]

Sorry, I read examples for attributes. The process setup is basically the same, the trick is to transpose the example sets, such that rows and columns are interchanged. Then you can do the same as in the process before and then transpose back, et voilà!
Hope this helps

Code:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.2.002">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="5.2.002" expanded="true" name="Process">
    <process expanded="true" height="386" width="1150">
      <operator activated="true" class="generate_data" compatibility="5.2.002" expanded="true" height="60" name="Generate Data" width="90" x="45" y="30">
        <parameter key="target_function" value="random classification"/>
      </operator>
      <operator activated="true" class="aggregate" compatibility="5.2.002" expanded="true" height="76" name="Aggregate" width="90" x="179" y="30">
        <parameter key="use_default_aggregation" value="true"/>
        <parameter key="default_aggregation_function" value="sum"/>
        <list key="aggregation_attributes"/>
      </operator>
      <operator activated="true" class="transpose" compatibility="5.2.002" expanded="true" height="76" name="Transpose (3)" width="90" x="380" y="165"/>
      <operator activated="true" class="transpose" compatibility="5.2.002" expanded="true" height="76" name="Transpose" width="90" x="380" y="30"/>
      <operator activated="true" class="filter_examples" compatibility="5.2.002" expanded="true" height="76" name="Filter Examples" width="90" x="514" y="30">
        <parameter key="condition_class" value="attribute_value_filter"/>
        <parameter key="parameter_string" value="att_1 &gt; 5"/>
      </operator>
      <operator activated="true" class="select_attributes" compatibility="5.2.002" expanded="true" height="76" name="Select Attributes (2)" width="90" x="648" y="30">
        <parameter key="invert_selection" value="true"/>
      </operator>
      <operator activated="true" class="replace" compatibility="5.2.002" expanded="true" height="76" name="Replace" width="90" x="782" y="30">
        <parameter key="attribute_filter_type" value="single"/>
        <parameter key="attribute" value="id"/>
        <parameter key="include_special_attributes" value="true"/>
        <parameter key="replace_what" value="sum\((.*)\)"/>
        <parameter key="replace_by" value="$1"/>
      </operator>
      <operator activated="true" class="join" compatibility="5.2.002" expanded="true" height="76" name="Join" width="90" x="782" y="165">
        <parameter key="join_type" value="left"/>
        <list key="key_attributes"/>
      </operator>
      <operator activated="true" class="transpose" compatibility="5.2.002" expanded="true" height="76" name="Transpose (2)" width="90" x="916" y="30"/>
      <connect from_op="Generate Data" from_port="output" to_op="Aggregate" to_port="example set input"/>
      <connect from_op="Aggregate" from_port="example set output" to_op="Transpose" to_port="example set input"/>
      <connect from_op="Aggregate" from_port="original" to_op="Transpose (3)" to_port="example set input"/>
      <connect from_op="Transpose (3)" from_port="example set output" to_op="Join" to_port="right"/>
      <connect from_op="Transpose" from_port="example set output" to_op="Filter Examples" to_port="example set input"/>
      <connect from_op="Filter Examples" from_port="example set output" to_op="Select Attributes (2)" to_port="example set input"/>
      <connect from_op="Select Attributes (2)" from_port="example set output" to_op="Replace" to_port="example set input"/>
      <connect from_op="Replace" from_port="example set output" to_op="Join" to_port="left"/>
      <connect from_op="Join" from_port="join" to_op="Transpose (2)" to_port="example set input"/>
      <connect from_op="Transpose (2)" from_port="example set output" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
    </process>
  </operator>
</process>

[sfmorais]

Hi Marius!

yes, now it is I wants.

Many thanks