Pages: [1]
Author Topic: Association Rules with Thousands of Products  (Read 643 times)
Posts: 3

« on: December 12, 2013, 03:55:04 PM »


I am trying to create association rules.
My dataset have 30496 transactions and 18270 products and it is stored in a MySQL database like this:

transaction | product
1 | 0987
1 | 6543
2 | 1234
... | ...

Before use FP-Growth operator I convert data with Nominal to Numerical operator and aggregate the rows of the same transaction. After that I convert data again with Numerical to Binomial operator.

I run the process with RapidAnalytics but it returns the error "java.lang.OutOfMemoryError: GC overhead limit exceeded ". And I used Free Memory operator.

How can I solve this problem? It is possible to create association rules with this number of products?
Other question, it is possible to convert data to a sparse matrix?

I am studing other approach in other database. I installed in a PostgreSQL database the MADlib library. And with the same dataset, MADlib function create the association rules in 30 seconds without any transformation of the data!!!

Best regards.
« Last Edit: December 13, 2013, 12:45:10 PM by a.isabel.marques » Logged
Jr. Member
Posts: 70


« Reply #1 on: September 26, 2014, 03:28:23 PM »

That is funny, I have somehow the same problem, but nobody bothered to answer..
Pages: [1]
Jump to: