The Text plugin is based on the WVTool which is a flexible Java library for statistical language modeling. It especially supports the creation of word vector representations of text documents in the vector space model (each document is represented by the terms it contains). The vector space model is the point of departure for many text processing applications (e.g. web mining, text classification or information retrieval).

Features

The aim of the Word Vector Tool is to provide a simple to use, simple to extend pure Java library for creating word vectors. It is tightly integrated with the data mining and machine learning environment RapidMiner allowing data mining applications directly on textual data. The Word Vector Tool bridges a gap between highly sophisticated linguistic packages on the one side and specialized, partial solutions on the other side.

The key features are:

• 100% Java implementation
• Very easy to extend
• Flexible choice of processing steps (e.g. language dependent)
• Integrates many preprocessing components, as multi-lingual stemming and stop word lists
• Allows to load documents from various sources (files, URLs ...)
• Integrates directly with the RapidMiner data mining environment
• Integrated Web crawling
• Wordnet support
• Regular expression based dictionaries

Download and Documentation

The following files are available from the RapidMiner Plugins download page: