Hi Dear Knime Users :)
I am trying to visit html links and parse that data.
For doing that ,
I created a excel source which has 10.000 url. and then Http Retreiver node ->Html Parser node -> Xpath
here is my question. when i run workflow. my result set is changing all the time, some web sites return that i want but some of them retuning statusCode=500 when i try to visit url on my browser i can get the results.
but it takes more than 2 seconds to getting results. I thing this is the problem. how can i put some delays for every url respons for this node?
i tryed to change and increase Http Retreiver node's parameters but it didnt solve my problem.
Any idea ?
Thans for your interest :) |