Geek asks and answers

Java API for web search or web development

I am looking for a good Java api for creating web clips. I tried the WEB-Harvest api http://web-harvest.sourceforge.net/usage.php , but I think this is a little awkward. Any other suggestions?

+3
source share
3 answers

I used httpunit to complete this task in the production process.

0
source

http://hc.apache.org/httpcomponents-client-ga/

(Maven dependency)

<dependency>
  <groupId>commons-httpclient</groupId> 
  <artifactId>commons-httpclient</artifactId> 
  <version>3.1</version> 
</dependency>
0
source

: https://github.com/subes/invesdwin-webproxy

It supports HttpClient and HtmlUnit (a browser without a browser that supports javascript) and parallelizes it, if required, for a large proxy pool. I can also recommend JSoup for static html processing.

0
source

More articles:


All Articles