Overview

This tutorial is designed to provide a basic overview of how to use HttpClient . When you have completed the tutorial you will have written a simple application that downloads a page using HttpClient .

It is assumed that you have an understanding of how to program in Java and are familiar with the development environment you are using.

Getting Ready

The first thing you need to do is get a copy of HttpClient and it's dependencies. Currently the only required dependency is commons-logging . This tutorial was written for HttpClient 2.0 and at a minimum requires 2.0 Alpha 2. You will also need JDK 1.2.2 or above.

Once you've downloaded HttpClient and commons-logging you'll need to put them on your classpath. There is also an optional dependency on JSSE which is required for HTTPS connections; this is not required for this tutorial.

Concepts

The general process for using HttpClient consists of a number of steps:

1. Create an instance of HttpClient .
2. Create an instance of one of the methods (GetMethod in this case). The URL to connect to is passed in to the the method constructor.
3. Tell HttpClient to execute the method.
4. Read the response.
5. Release the connection.
6. Deal with the response.

We'll cover how to perform each of these steps below. Notice that we go through the entire process re gardless of whether the server returned an error or not. This is important because HTTP 1.1 allows multiple requests to use the same connection by simply sending the requests one after the other. Obviously, if we don't read the entire response to the first request, the left over data will get in the way of the second response. HttpClient tries to handle this but to avoid problems it is important to always read the entire response and release the connection.

Instantiating HttpClient

The no argument constructor for HttpClient provides a good set of defaults for most situations so that is what we'll use.

HttpClient client = new HttpClient();

Creating a Method

The various methods defined by the HTTP specification correspond to the various classes in HttpClient which implement the HttpMethod interface. These classes are all found in the package org.apache.commons.httpclient.methods .

We will be using the Get method which is a simple method that simply takes a URL and gets the document the URL points to.

HttpMethod method = new GetMethod("http://www.apache.org/");

Execute the Method

The actual execution of the method is performed by calling executeMethod on the client and passing in the method to execute. Since networks connections are unreliable, we also need to deal with any errors that occur.

There are two kinds of exceptions that could be thrown by executeMethod, HttpRecoverableException and IOException .

The other useful piece of information is the status code that is returned by the server. This code is returned by executeMethod as an int and can be used to determine if the request was successful or not and can sometimes indicate that further action is required by the client such as providing authentication credentials.

          int statusCode = -1;
          // We will retry up to 3 times.
          for (int attempt = 0; statusCode == -1 && attempt < 3; attempt++) {
              try {
                  // execute the method.
                  statusCode = client.executeMethod(method);
              } catch (HttpRecoverableException e) {
                  System.err.println("A recoverable exception occurred,
                  retrying.  " + e.getMessage());
              } catch (IOException e) {
                  System.err.println("Failed to download file.");
                  e.printStackTrace();
                  System.exit(-1);
              }
          }
          // Check that we didn't run out of retries.
          if (statusCode == -1) {
              System.err.println("Failed to recover from exception.");
              System.exit(-2);
          }
        

Read the Response

It is vital that the response body is always read regardless of the status returned by the server. There are three ways to do this:

• Call method.getResponseBody() . This will return a byte array containing the data in the response body.
• Call method.getResponseBodyAsString() . This will return a String containing the response body. Be warned though that the conversion from bytes to a String is done using the default encoding so this method may not be portable across all platforms.
• Call method.getResponseBodyAsStream() and read the entire contents of the stream then call stream.close() . This method is best if it is possible for a lot of data to be received as it can be buffered to a file or processed as it is read. Be sure to always read the entirety of the data and call close on the stream.

For this tutorial we will use getResponseBody() for simplicity.

byte[] responseBody = method.getResponseBody();

Release the Connection

This is a crucial step to keep things flowing. We must tell HttpClient that we are done with the connection and that it can now be reused. Without doing this HttpClient will wait indefinitely for a connection to free up so that it can be reused.

method.releaseConnection();

Deal with the Repsonse

We've now completed our interaction with HttpClient and can just concentrate on doing what we need to do with the data. In our case, we'll just print it out to the console.

It's worth noting that if you were retrieving the response as a stream and processing it as it is read, this step would actually be combined with reading the connection, and when you'd finished processing all the data, you'd then close the input stream and release the connection.

Note: We should pay attention to character encodings here instead of just using the system default.

System.err.println(new String(responseBody));

Final Source Code

When we put all of that together plus a little bit of glue code we get the program below.

        import org.apache.commons.httpclient.*;
        import org.apache.commons.httpclient.methods.*;
        import java.io.*;

        public class HttpClientTutorial {
          
          private static String url = "http://www.apache.org/";

          public static void main(String[] args) {
            // Create an instance of HttpClient.
            HttpClient client = new HttpClient();

            // Create a method instance.
            HttpMethod method = new GetMethod(url);

            // Execute the method.
            int statusCode = -1;
            int attempt = 0;
            // We will retry up to 3 times.
            while (statusCode == -1 && attempt < 3; attempt++) {
              try {
                // execute the method.
                statusCode = client.executeMethod(method);
              } catch (HttpRecoverableException e) {
                System.err.println(
                  "A recoverable exception occurred, retrying." + 
                  e.getMessage());
              } catch (IOException e) {
                System.err.println("Failed to download file.");
                e.printStackTrace();
                System.exit(-1);
              }
            }
            // Check that we didn't run out of retries.
            if (statusCode == -1) {
              System.err.println("Failed to recover from exception.");
              System.exit(-2);
            }

            // Read the response body.
            byte[] responseBody = method.getResponseBody();

            // Release the connection.
            method.releaseConnection();

            // Deal with the response.
            // Use caution: ensure correct character encoding and is not binary data
            System.err.println(new String(responseBody));
          }
        }