Pages: [1]
  Print  
Author Topic: Outof Memory Error loading big SPSS file  (Read 4082 times)
Braulio
Newbie
*
Posts: 7


« on: May 26, 2008, 08:20:38 PM »

Hi there,

I am having a problem while loading a big SPSS file in RapidMiner. (pasted below)
The file is about 420 MB (over 1 million lines and 70 columns)

My question: what memory does the error refer to? RAM memory?

Thanks a lot

Braulio

Here is the error:

P May 26, 2008 2:24:18 PM: [NOTE] Process starts
P May 26, 2008 2:24:18 PM: Process:
   Root[0] (Process)
   +- SPSSExampleSource[0] (SPSSExampleSource)
P May 26, 2008 2:29:10 PM: [Fatal] OutOfMemoryError occured in 1st application of SPSSExampleSource (SPSSExampleSource)
P May 26, 2008 2:29:10 PM: [Fatal] Process failed: Java heap space
          Root[1] (Process)
 here ==> +- SPSSExampleSource[1] (SPSSExampleSource)
Logged
Tobias Malbrecht
Global Moderator
Sr. Member
*****
Posts: 293



WWW
« Reply #1 on: May 26, 2008, 10:22:17 PM »

Hello Braulio,

this error occurs when the Java Virtual Machine (JVM) tries to occupy more memory than it has been made available to the JVM. If you use RapidMiner on a Windows system you may set the memory that is made available to the JVM by editing the file RapidMinerGUI.bat in the scripts subdirectory in the RapidMiner program directory.

The line you have to edit appears like:

Code:
if "%MAX_JAVA_MEMORY%"=="" set MAX_JAVA_MEMORY=1024

There you can specify how much memory the JVM may use (in megabyte). If you then start RapidMiner via the RapidMinerGUI.bat the JVM which executes RapidMiner should use the amount of memory you have specified. For a 420mb SPSS file 1024mb of memory however might eventually not be enough since SPSS files may be compressed and therefore the amount of memory used might be actually quite more than the size of the file.

Hope to have helped,
Tobias
Logged

Tobias Malbrecht
Director of Product Marketing
RapidMiner
Pages: [1]
  Print  
 
Jump to: