Pages: [1]
  Print  
Author Topic: Get Page with query parameters  (Read 707 times)
JEdward
Full Member
***
Posts: 145


« on: September 09, 2013, 01:18:25 PM »

Hello I'm trying to use the Get Page operator & finding a problem I haven't done before.  After I put in the query parameters RapidMiner adds an additional '&' after the line when it sends it to the web.  This is causing the REST service to reply back saying that the request is invalid. 

It seems a little strict, but why is it that RM is adding the '&'?  Has anyone else faced a similar issue? 

Below is an example of the process I mean, it would generate the request URL "https://webservice.com/detail/number/site-here?withParameter=true&"

Code:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<process version="5.3.012">
  <context>
    <input/>
    <output/>
    <macros/>
  </context>
  <operator activated="true" class="process" compatibility="5.3.012" expanded="true" name="Process">
    <process expanded="true">
      <operator activated="true" class="web:get_webpage" compatibility="5.3.001" expanded="true" height="60" name="Get Page" width="90" x="112" y="75">
        <parameter key="url" value="https://webservice.com/detail/number/site-here"/>
        <parameter key="follow_redirects" value="false"/>
        <list key="query_parameters">
          <parameter key="withParameter" value="true"/>
        </list>
        <list key="request_properties">
          <parameter key="Content-Type" value="application/xml"/>
          <parameter key="Authorization" value="Basic xxxx=="/>
        </list>
        <parameter key="override_encoding" value="true"/>
        <parameter key="encoding" value="UTF-8"/>
      </operator>
      <connect from_op="Get Page" from_port="output" to_port="result 1"/>
      <portSpacing port="source_input 1" spacing="0"/>
      <portSpacing port="sink_result 1" spacing="0"/>
      <portSpacing port="sink_result 2" spacing="0"/>
    </process>
  </operator>
</process>
Logged
Marius
Administrator
Hero Member
*****
Posts: 1794



WWW
« Reply #1 on: September 09, 2013, 01:30:32 PM »

Hi,

thanks for reporting this issue. I have created an internal ticket for this. Until it is fixed you can add the query parameters manually directly into the url parameter.

Best regards,
Marius
Logged

Please add [SOLVED] to the topic title when your problem has been solved! (do so by editing the first post in the thread and modifying the title)
Please click here before posting.
JEdward
Full Member
***
Posts: 145


« Reply #2 on: September 09, 2013, 01:53:15 PM »

Hi Marius,

I didn't realise it was a bug as it also does the same when I manually entered the parameters as a URL. 
I'll try a few workarounds & post up a working one to this thread. 

Best,
J.
Logged
JEdward
Full Member
***
Posts: 145


« Reply #3 on: September 10, 2013, 05:19:20 PM »

Just to update.  I haven't yet found a workable workaround for this. 
Marius suggestion above is still having the same issue on my machine, passing the URL to the operator as a macro (RM is still adding the additional '&' at the end). 

I'll keep searching & let you all know. 
Logged
JEdward
Full Member
***
Posts: 145


« Reply #4 on: April 22, 2014, 04:46:25 AM »

Just to update: in the version of RapidMiner I use (5.3) this has not been fixed.  Can anyone confirm if it's fixed in RapidMiner 6+? 

Also I now have a work around which involves writing a new version of the Get Page operator using a bunch of macros alongside the Execute Script operator with Groovy HTTPBuilder. 
http://groovy.codehaus.org/modules/http-builder/download.html

It's not pretty and has to be heavily customized for each use case so I'm afraid the process isn't in a fit state to share, however, I wish this post this might help someone who comes across the same problem. Feel free to contact me if you need any assistance or would like me to share some of the 'ugly script'.   
Logged
Marius
Administrator
Hero Member
*****
Posts: 1794



WWW
« Reply #5 on: April 22, 2014, 10:11:09 AM »

Hi J,

I am sorry, this issue is still marked as open in our bug tracker.

Best regards,
Marius
Logged

Please add [SOLVED] to the topic title when your problem has been solved! (do so by editing the first post in the thread and modifying the title)
Please click here before posting.
Pages: [1]
  Print  
 
Jump to: