Pages: [1]
  Print  
Author Topic: fp-growth exceeds memory  (Read 617 times)
help
Newbie
*
Posts: 12


« on: June 11, 2013, 01:41:10 AM »

Hi,

I have a set of 27 attributes (which after transformation to become binomial 128) and 14000 records.
I tried to run the fp-growth algorithm followed by create association rules, but he runs out of memory (my computer has 4GB of RAM and is 64 bits). I'm already using the maximum capacity (already changed the script RapidminerGui.bat).

But what I do not understand is that the same process has been run with 100000 records and double attributes, with a much smaller support.

Has anyone experienced a similar problem?
Have a suggestion to solve this problem?


Thank you!  Wink
Logged
Marcin
Global Moderator
Full Member
*****
Posts: 165


« Reply #1 on: June 13, 2013, 12:25:43 PM »

Has anyone experienced a similar problem?

This is a common problem if you are looking for fequent itemsets. All algorithms are sensitive to the min support and to be honest: 4 GM RAM is not much memory nowadays Wink. So your only way is to carefully increase the min-support or use other constraints as the "max items". Good values depend highly on your dataset. So, even if you have more records and more attributes, the data can contain less frequent itemsets and therefore will not blow up your machine.
Logged
help
Newbie
*
Posts: 12


« Reply #2 on: June 21, 2013, 03:23:51 PM »

Hi,

Thanks for your help ...  Cheesy

I increased min_sup and managed to run the process with this amount of records ... Up to this point everything went right.

However, I decided to do another test. And I have a question  Huh:
My computer is running a process there are 40 hours, did not show the warning that popped the memory capacity. However the runtime process does not go continuously (without moving time is about 3 hours and then updates the time, so I know that is already running for 40 hours).

Does RapidMiner process is running?
Worth leaving more time running?


Thank you!
Logged
Pages: [1]
  Print  
 
Jump to: