Pages: [1]
  Print  
Author Topic: Get Pages - Error parsing HTTP headers  (Read 1387 times)
miguelal
Newbie
*
Posts: 23


« on: November 07, 2013, 05:41:30 PM »

Hi,

When a web server sends a header string that violates the cookie specification, the method getHeaderFields() of the HttpURLConnection class throws an IllegalArgumentException which is not handled by RapidMiner and makes the "Get Pages" operator in the Web Mining extension fail. I have added a try/catch around that code, and it seem to be working now.

This the code I modified in line 164 of the GetWebPageOperator:

Code:
try { // El metodo GetHeaderFields falla si hay cookies que no tienen el simbolo = entre el nombre y el valor
for (Entry<String, List<String>> header : connection.getHeaderFields().entrySet()) {
getLogger().info("Response Header:" + header.getKey() + ": " + header.getValue());
}
} catch(IllegalArgumentException ex) {
getLogger().warning("Failed to get HTTP header fields. Error: " + ex.getMessage());
}

I am posting this in case it helps anyone.

Thanks,
Miguel
« Last Edit: November 13, 2013, 04:54:15 PM by miguelal » Logged
Marius
Administrator
Hero Member
*****
Posts: 1793



WWW
« Reply #1 on: November 11, 2013, 03:00:14 PM »

Hi Miguel,

thanks for reporting. Can you give us a link to a page that reproduces this error?
Best regards,
Marius
Logged

Please add [SOLVED] to the topic title when your problem has been solved! (do so by editing the first post in the thread and modifying the title)
Please click here before posting.
miguelal
Newbie
*
Posts: 23


« Reply #2 on: November 13, 2013, 08:17:40 AM »

Hi Marius,

I am sorry, but unfortunately I forgot to keep the URL that was causing this problem. I process lots of different URLs everyday, and tried looking for the one (since I know the problem happened on Nov 6th) but I couldn't find it. The only thing I have is the screenshot of the error in RapidAnalytics, which I know isn't going to be of much help to you.  Cry



Thanks,
Miguel
« Last Edit: November 13, 2013, 08:22:50 AM by miguelal » Logged
Marius
Administrator
Hero Member
*****
Posts: 1793



WWW
« Reply #3 on: November 13, 2013, 09:43:55 AM »

Ok, thanks for searching in any case Smiley
I will forward your error description and the piece of code above to the developers. Let's see if it is of use to them.

Best regards,
Marius
Logged

Please add [SOLVED] to the topic title when your problem has been solved! (do so by editing the first post in the thread and modifying the title)
Please click here before posting.
Pages: [1]
  Print  
 
Jump to: