Can Java Programmatically Search Google Without a Dedicated API?

How to Search Google Programmatically Using Java API

This article examines the process of searching Google programmatically, specifically focusing on the availability of a Java API for this purpose.

Using Google's Web Search API

Originally, Google offered a public web search API that returned JSON format data. However, this service has since been deprecated. As of November 2010, the best alternative is to query Google's search engine directly using a user agent and then parse the HTML response using a parser such as Jsoup.

Java Implementation

Firstly, let's set up the variables and establish the search URL:

String google = "http://www.google.com/search?q=";
String search = "stackoverflow";
String charset = "UTF-8";
String userAgent = "ExampleBot 1.0 (+http://example.com/bot)"; // Customize for your bot

Next, we use Jsoup to make the HTTP request and parse the HTML:

Elements links = Jsoup.connect(google + URLEncoder.encode(search, charset)).userAgent(userAgent).get().select(".g>.r>a");

Finally, we iterate over the search results and extract the title and URL:

for (Element link : links) {
    String title = link.text();
    String url = link.absUrl("href");
    url = URLDecoder.decode(url.substring(url.indexOf('=') + 1, url.indexOf('&')), "UTF-8");
    if (!url.startsWith("http")) {
        continue; // Ads/news/etc.
    }

The above is the detailed content of Can Java Programmatically Search Google Without a Dedicated API?. For more information, please follow other related articles on the PHP Chinese website!