Pages: [1]
  Print  
Author Topic: Web mining a list of URLs  (Read 1272 times)
adommnich
Newbie
*
Posts: 2


« on: November 14, 2010, 09:00:41 PM »

Hello,

how can I loop over a list of different URLs and extract the content? The operator Crawl Web has no input but the parameter 'URL'. Is there a way to fill it dynamically using a Loop operator?

Thanks for your attention.

Regards
adommnich
Logged
colo
Full Member
***
Posts: 245


« Reply #1 on: November 15, 2010, 11:14:59 AM »

Hi adommnich,

I would use a "Loop Examples" operator for the URL list (if you read it into an example set before). Inside the loop you can easily extract the current URL with "Extract Macro" and use the value for the "Get Page" operator. If you want to crawl and not just retrieve one website you can replace "Get Page" by "Crawl Web".

Regards,
Matthias
Logged
adommnich
Newbie
*
Posts: 2


« Reply #2 on: November 15, 2010, 02:21:12 PM »

Hi Matthias,

thanks a lot for your quick reply. I am afraid I wasn' t aware of the Macro concept.  Embarrassed

It works fine now.

Regards
Alex
Logged
Pages: [1]
  Print  
 
Jump to: