Writing a Java script that regularly requests data from a website

Question

Writing a Java script that regularly requests data from a website

I am working on a project that requires me to use the border waiting time information provided by the Canadian Border Patrol on my website to create a visual representation of the distribution of waiting times.

I am trying to find a way to regularly check the Java script website and retrieve information at several different border stations (not all of them). I believe that I will use XPath to get certain stations, but how can I regularly load a web page?

(PS I know that they already have a Twitter account, but they update it once a day, and more specifically I would like to learn how to work with websites and XPATH)

+5

java xmlhttprequest load

Codybugstein Feb 13 '13 at 5:30

source share

4 answers

Use URL in Java. Create a URL and then use its .openConnection () method to start reading from a website.

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;


public class webVisitor {


    public static void main(String[] args) {

        URL url;

        try {

            url = new URL("http://seinfeldaudio.com");
            URLConnection conn = url.openConnection();

            BufferedReader buffRead = new BufferedReader(new InputStreamReader(conn.getInputStream()));

            String inputLine = "";

            while (inputLine != null){
                inputLine = buffRead.readLine();
                System.out.println(inputLine);
            }


        }
        catch (Exception e){

        }

    }

}

Additional information here: http://www.mkyong.com/java/how-to-get-url-content-in-java/

+4

Codybugstein Feb 28 '13 at 0:34

source share

Ajax, setInterval("function()",x)

- jQuery ajax-

node.js, loos -

0

mr mojo risin 13 . '13 5:36

Use DWR (Easy Ajax for Java) , call the DWR method from your java script, setting the time interval as

setInterval(DWR function here , millisec, lang)

In the Java method, use the java.net.URL class to read and parse content as needed.

0

VKPRO Mar 6 '13 at 16:12

source share

Thomas Sofras · Accepted Answer · 2013-02-28T10:16:00+0000

Well, I had some free time at work, and I thought to help and write it for you. Excuse me for any mistakes the first time I took apart the site, I did a little research and decided to use jSoup for this.

Well, this code will analyze a table and a system of three columns with values, you can change the code and build it according to your needs :)

You need to download jsoup jar Download jSoup

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;
import java.util.Iterator;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;


/**
 * 
 */
public class ParseWithJsoup{


    public static void main(String[] args) {

        URL url;

        try {

            url = new URL("http://www.cbsa-asfc.gc.ca/bwt-taf/menu-eng.html");
            URLConnection conn = url.openConnection();

            BufferedReader buffRead = new BufferedReader(new InputStreamReader(conn.getInputStream()));
            StringBuffer buffer = new StringBuffer("");

            String inputLine = "";

            // Append the site in a buffer
            while (inputLine != null){
                inputLine = buffRead.readLine();
                buffer.append(inputLine);
            }

            Document doc = Jsoup.parse(buffer.toString());

            // Parse the table
            Element table = doc.select("table[class=bwt]").first();

            //Office elements iterator
            Iterator<Element> officeElements = table.select("td[headers=Office]").iterator();

            //Commercial Flow iterator
            Iterator<Element> comElements = table.select("td[headers=Com ComCanada]").iterator();

            //Travellers Flow iterator
            Iterator<Element> travElements = table.select("td[headers=Trav TravCanada]").iterator();


            // Iterate all elements through first element row for all columns
            while(officeElements.hasNext()){            
                System.out.println("Office: " + officeElements.next().text());
                System.out.println("Commercial Flow: " + comElements.next().text());
                System.out.println("Travellers Flow: " + travElements.next().text());
            }

        }
        catch (Exception e){
            System.out.println("Exc:"+e.getMessage());
        }
    }


}

`

Writing a Java script that regularly requests data from a website

More articles: