Pages: 1 [2]
  Print  
Author Topic: Logistic Regression  (Read 4474 times)
Sebastian Land
Administrator
Hero Member
*****
Posts: 2426


« Reply #15 on: April 15, 2010, 09:09:39 AM »

Hi,
probably there will be differences in the implementations and I doubt the weights will be the same. But they should either come near to the other weights or at least perform equally.

Greetings,
  Sebastian
Logged
B_Miner
Jr. Member
**
Posts: 72


« Reply #16 on: April 16, 2010, 11:17:35 PM »

Its curious, the weights are not close for RM or WEKA logistic regression (RM was set to dot kernel and WEKA is the Simple Logistic) compared to SAS. They are not close to each other at all. The prediction probabilities for WEKA are close to SAS, RM is far different.

Its curious because logistic regression is used not only for prediction but for inference, from a strictly statistical position, were the exponentiated weights are odds ratios.

I have coefficient from SAS and small data file if interested.
Logged
Ingo Mierswa
Administrator
Hero Member
*****
Posts: 1226



WWW
« Reply #17 on: April 19, 2010, 05:00:54 PM »

Hello,

it is actually not a big surprise that those differences occur. First, in contrast to most other implementations, the logistic regression learner from RapidMiner is basically a support vector machine with a different loss function. The author of this implementation told me once that the whole optimization approach is a bit different from that known from more traditional implementations. While this make some nifty things possible like the integration of kernel function, this might also lead to different results. At least, the predictions should rely a lot on some parameters as "C" and can hardly be directly compared.

The second difference seems to be the way how the confidences are calculated. Due to the kernel based optimization approach they are derived from the predictions based on the lagrange multipliers, the training examples and the kernel function. On those predictions a probability scaling somewhat similar (but much simpler) to Platt scaling is applied. As long as you read the confidences as what they are (as "confidence" instead of "probability") this is usual fine.

Cheers,
Ingo

Logged

Did you try our new Marketplace? Upload or download new Extensions, add comments, and organize your operators. Have a look at  http://marketplace.rapid-i.com
B_Miner
Jr. Member
**
Posts: 72


« Reply #18 on: April 19, 2010, 10:34:58 PM »

Thanks Ingo! If I get a chance, I will test performance of this implementation against the traditional maximum likelihood logistic regression (SAS) and advise if I see anything interesting.

B
Logged
Ingo Mierswa
Administrator
Hero Member
*****
Posts: 1226



WWW
« Reply #19 on: April 20, 2010, 08:57:24 AM »

Yes, please keep me updated if you get the chance. I could imagine that the real strength of the kernel logistic regression lies in cases where classification tasks are non-linear and an appropriate kernel function is used. The traditional logistic regression on the other hand might outperform in the linear case and is definitely better suited if real probabilities are necessary. But maybe I am completely wrong  Cheesy

Don't forget to optimize at least C since without it the kernel logistic regression is not likely to produce good results anyway...

Cheers,
Ingo
Logged

Did you try our new Marketplace? Upload or download new Extensions, add comments, and organize your operators. Have a look at  http://marketplace.rapid-i.com
Pages: 1 [2]
  Print  
 
Jump to: