Pages: 1 [2]
  Print  
Author Topic: PaREn Extension  (Read 7503 times)
Christian Kofler
Newbie
*
Posts: 4


WWW
« Reply #15 on: October 07, 2010, 04:43:11 PM »

Hi Sebastian,

yes, we left on Wednesday evening and did not attend the final training day. Special Interest Group sounds interesting, please send me more info or point me to it if already available.

Well, I don't think that a dedicated forum for the PaREn Wizard is needed. Maybe one for third-party contributions? Kind of a pre-roll for the envisioned marketplace.

Regarding the timing predictions we would be happy to join forces and exchange insight. Looks like you should do a Rapid-I group excursion to Kaiserslautern Wink

Cheers

Christian
Logged
Sebastian Land
Administrator
Hero Member
*****
Posts: 2425


« Reply #16 on: October 08, 2010, 01:26:58 PM »

Hi Christian,
the mailling lists of the Special Interest Groups are listed on the sourceforge page of RapidMiner. We are still working on a page explaining in more detail topics and aims of each group, but nevertheless they are online. You might join there directly or write me an email and I will put you on the list.

The idea with a third party forum is good. I will add one.

I've never been in Kaiserslautern. Seems to be a good idea to change that now Smiley I will contact you by mail.

Greetings,
  Sebastian
Logged
Ingo Mierswa
Administrator
Hero Member
*****
Posts: 1221



WWW
« Reply #17 on: October 08, 2010, 05:07:51 PM »

Hi,

Quote
A similar (though not identical) feature, very effective indeed, is offered by IBM SPSS Modeler for instance as an automatic modeling operator, via which several models are produced automatically, and the best of them are proposed to the user. Moreover, the models may be combined to produce a kind of voting model, which may have better performance in some occasions than the individual models. See a demo here.

whoa, but that's quite a difference: in SPSS all models are actually tested (which can also be done with the PaREn extension during the evaluation step but is also possible with a simple process for core RapidMiner as Simon has pointed out).

The cool thing about the PaREn extension is that it predicts which model is probably the best even without any testing. This is the first time I have actually see this meta learning approach really working and this is probably the reason why we at Rapid-I and many others love it. Kudos to the Christian and the team of the DFKI for this great extension!

I have also a suggestion: would be great if a k-fold cross validation or even a single split would be selectable instead of the rather time consuming LOO evaluation.

Cheers,
Ingo
Logged

Did you try our new Marketplace? Upload or download new Extensions, add comments, and organize your operators. Have a look at  http://marketplace.rapid-i.com
tobyb
Newbie
*
Posts: 11


« Reply #18 on: October 12, 2010, 08:30:41 PM »

I am getting an error when I perform Step 3 of the Automatic System Construction with the Iris dataset.  It states, "No parameters were specified which should be optimized".  The wizard closes after I click the OK button.  I have followed the instructions exactly on how to use this extension.  Has anyone else had this same problem?


10-27-10
At the time of my post I thought I had the latest update for PaREn.  Once I installed the latest version I am not experiencing this problem any longer.
« Last Edit: October 27, 2010, 04:36:23 PM by tobyb » Logged
Ingo Mierswa
Administrator
Hero Member
*****
Posts: 1221



WWW
« Reply #19 on: October 12, 2010, 11:20:32 PM »

Hi,

as Simon has pointed out: there should be a new version on our update server. Do you really use the latest version available? If yes: What's your OS?

Cheers,
Ingo
Logged

Did you try our new Marketplace? Upload or download new Extensions, add comments, and organize your operators. Have a look at  http://marketplace.rapid-i.com
dan_
Full Member
***
Posts: 114


« Reply #20 on: October 13, 2010, 06:31:18 PM »

Quote
whoa, but that's quite a difference: in SPSS all models are actually tested (which can also be done with the PaREn extension during the evaluation step but is also possible with a simple process for core RapidMiner as Simon has pointed out).

The cool thing about the PaREn extension is that it predicts which model is probably the best even without any testing. This is the first time I have actually see this meta learning approach really working and this is probably the reason why we at Rapid-I and many others love it. Kudos to the Christian and the team of the DFKI for this great extension!

@ Ingo: Note that SPSS Modeler has less, but very carefully chosen and highly optimised algorithms (there where it is possible - example: take C5 as opposed to C4.5 implemented in open source software). Therefore one affords to create models for most classification algorithms available in SPSS and to retain the best ones, in a reasonable amount of time.

Factually speaking  (by the way as a fan of both software - RM and SPSS Modeler), there are obviously similarities and differences in the features we discuss about, and I am afraid that the differences show for now that SPSS Modeler is incomparably much ahead: time of running to build the best models, reliability and performance of models (see my previous posting above regarding unexpectedly suboptimal optimised PaREn models), the combination of best models in an overall model to use, etc. On the other hand, the estimated accuracies in PaREn were quite far from the actual accuracies in most of my experiments, but the idea is interesting.

@ Christian et al. : I would have an additional suggestion to which I had thought when posting questions earlier in this topic. ROC Analysis can be added to searching the model giving the best accuracy when the output/label attribute is binominal. More precisely, after finding the best parameters for a learner, given a dataset, one can get also the optimised threshold from a ROC curve (as opposed to using the default threshold 0.5), which guarantees the best accuracy.

However, perhaps this suggestion may be useful to consider after the ROC Analysis implemented in Rapid Miner would be revised as it is still unreliable in this package (i.e. AUC calculation needs corrections, as I have shown on the forum http://rapid-i.com/rapidforum/index.php?PHPSESSID=18d6261d2d63b2ca946477f03c2552bc&topic=2237.0
, and Find Threshold operator does not find the best threshold as expected but provides suboptimal solutions - I emailed a complete report to the RM development team, with relevant processes illustrating this).

PaREn is an excellent initiative towards RM's enrichment. However the extension needs to be more practical and more accurate. Indeed, it requires relatively much processing time and models are not as optimised as expected - see postings in this thread, where it is explained that both - an ad hoc model created with no particular setting, and a trivial model that picks up blindly the most frequent class as prediction - are better in accuracy than the optimised, time consuming to build PaREn model. Improvement would be very beneficial and necessary indeed for the extension. Other users of the extension may wish to generate ad hoc models in addition to the PaREn models, and to compare their accuracies - this would be a useful feedback to the development team.

I hope the feedback and suggestions in this thread help and would be useful to PaREn, as part of the community's contribution to improve the open source software.  Good luck!

Regards,
Dan
« Last Edit: October 14, 2010, 05:50:09 AM by dan_agape » Logged
haddock
Hero Member
*****
Posts: 852



WWW
« Reply #21 on: October 14, 2010, 10:35:28 AM »

Greetings to all,

The producers of C5.0 compare it against C4.5 here..

http://www.rulequest.com/see5-comparison.html

So a lot of the difference we already have, moreover C5.0 is closed source and not free.

Quote
AUC calculation needs corrections, as I have shown on the forum

Really? Check out my recent post  http://rapid-i.com/rapidforum/index.php/topic,2237.msg10540.html#msg10540

But what has any of this to do with the PaREn extension? Not much! As Ingo says...

Quote
The cool thing about the PaREn extension is that it predicts which model is probably the best even without any testing.

So it simply misses the point to state that "models are not as optimised as expected".

Toode Pip!



« Last Edit: October 14, 2010, 10:41:43 AM by haddock » Logged

Where is the wisdom we have lost in knowledge?
Where is the knowledge we have lost in information?

T.S.Eliot ~ Choruses from the Rock 1934
crappy_viking
Newbie
*
Posts: 20


« Reply #22 on: March 21, 2011, 09:04:38 PM »

Has Paren extension anything to do with Amine platform ? The formalism they use is said to be compatible with parallel processing (mix of lambda calculus into ontologies)
:http://amine-platform.sourceforge.net/.
Logged
Pages: 1 [2]
  Print  
 
Jump to: