Geek asks and answers

Get a fully rendered page using Curl or other ways?

Is there a way to get fully processed html from a page using javascript rendering? If I use curl, it just extracts the basic html, but does not have iframes post-rendering, javascript processing, etc.

What would be the best way to achieve this?

+5
source share
5 answers

As no one else answered (except for the above, but I will come to him later), I will try to help as much as possible.

There is no "simple" answer. PHP cannot handle javascript / navigate the DOM initially, so you need something that can.

Your options, as I see it:

- , , 1, ;)

+2

- GUI, . PhantomJS .

+1
curl [-u login:pass] http://www.a_page.com | w3m -T text/html -dump


curl [-u login:pass] http://www.a_page.com | lynx -stdin -dump
-1

IECapt.exe

-2

It does not support javascript, but it lynxwas useful for me in a situation where I needed to process data from a web page. This way I got the rendering (plaintext) and didn't have to filter the raw html tags, as with curl.

lynx -nonumbers -dump -width=9999999 ${url} | grep ...  et cetera.
-2
source

More articles:


All Articles