Memory Issues

From Rapid-I-Wiki

(Difference between revisions)
Jump to: navigation, search
m (1 revision: Content of our first wiki.)
 

Latest revision as of 10:22, 18 October 2010

This article tries to explain what you can do when you run out of memory.

Contents

Java in general

Java has two memory options which you define during startup and which determine the size of the heap (where your objects are stored in memory)

-Xms 
Lower bound
-Xmx 
Upper bound

If you want to set this settings start java as follow:

<source lang="java"> java -Xmx512m </source>

or

<source lang="java"> java -Xms256m -Xmx512m </source>

whereby m indicates Mb.

The JVM resizes the heap adaptively, meaning it will attempt to find the best heap size for your application. -Xms and -Xmx simply specifies the range in which the JVM can operate and resize the heap. If -Xms and -Xmx are the same value, then the JVM's heap size will stay constant at that value.

It's typically best to just set -Xmx and let the JVM find the best heap size, unless there's a specific reason why you need to give the JVM a big heap at JVM launch.

Heap settings in RapidMiner

The default Java settings uses only a small amount of memory (-Xmx 64 or 128 Mb). If you use RapidMiner from a Windows installation the -Xmx option is automatically set to 80% of your memory. Otherwise the RapidMiner scripts (these in the scripts-directory) set the maximum to 512 Mb (-Xmx512m). If you want to increase the used heap you have two options. Either you set a environment variable called MAX_JAVA_MEMORY in your operating system or you change a line in the script you use to start RapidMiner (MAX_JAVA_MEMORY=1024 if you want to use 1Gb). How you can change your environment variable you can read here. Please note that you set the environment variable without the m.

RapidMiner on a 32-bit Windows

On Windows, by default, the lower 2 GB are reserved for user-mode programs and the upper 2 GB are reserved for kernel-mode programs. When you run a 32-bit Windows operating system, regardless of the amount of RAM you have installed, by default, it will only allocate a maximum of 2GB to an application (reserving the other 2GB for the OS). There is a boot-options to use more than 2 GB (/3GB), but this doesn't work with Java (see Why can't I allocate 2GB of heap to the JVM on Windows, Part 2). If you want to use more than 2 GB the best solution is to run RapidMiner on a 64-bit OS.

Java Tuning

If you want to tune your memory settings even more, try this and/or that.

Personal tools