Pages: [1]
  Print  
Author Topic: Association Rules with Thousands of Products  (Read 206 times)
a.isabel.marques
Newbie
*
Posts: 3


« on: December 12, 2013, 03:55:04 PM »

Hi,

I am trying to create association rules.
My dataset have 30496 transactions and 18270 products and it is stored in a MySQL database like this:

transaction | product
1 | 0987
1 | 6543
2 | 1234
... | ...

Before use FP-Growth operator I convert data with Nominal to Numerical operator and aggregate the rows of the same transaction. After that I convert data again with Numerical to Binomial operator.

I run the process with RapidAnalytics but it returns the error "java.lang.OutOfMemoryError: GC overhead limit exceeded ". And I used Free Memory operator.

How can I solve this problem? It is possible to create association rules with this number of products?
Other question, it is possible to convert data to a sparse matrix?

I am studing other approach in other database. I installed in a PostgreSQL database the MADlib library. And with the same dataset, MADlib function create the association rules in 30 seconds without any transformation of the data!!!

Best regards.
« Last Edit: December 13, 2013, 12:45:10 PM by a.isabel.marques » Logged
Pages: [1]
  Print  
 
Jump to: