Connecting to the Google Vision API via a Proxy

Images are the second most common and fastest growing content type in Box. Last month we introduced image recognition in Box to help unlock the value of these images to your business. We're applying machine learning to images, making it easier for companies to discover, organize and drive actionable insights from their content.

Now, upon uploading images to Box, any objects and handwritten or typed text in images can be automatically detected and indexed as metadata, bringing actionable context to content. To power this, we're leveraging Google Cloud Vision, a state-of-the-art machine learning tool that automatically catalogues images into thousands of categories.

The Google API Client Library for Java can be used to authenticate with the Google Cloud Platform and to make requests to the Google Vision API. However, it is not entirely straightforward to use the library when behind a proxy, especially one that requires authentication, and when connecting over HTTPS. In this post, we'll go over several different options for connecting via proxy, explaining the pros and cons of each.

JVM Proxy Settings

The easiest way to configure the Google API Client Library to connect via a proxy is by using the following JVM settings:

http.proxyHost
http.proxyPort

If you are connecting over HTTPS, use the following settings:

https.proxyHost
https.proxyPort

These settings can be set on the command line or by calling System.setProperty():

java -Dhttp.proxyHost=proxy.example.com -Dhttp.proxyPort=8080

System.setProperty(“http.proxyHost”, “proxy.example.com”);

While this approach is simple, it has two downsides:

1. The settings are “global,” and may affect other parts of your application or service
2. The settings do not allow you to specify a username / password for your proxy

Global Authenticator

If your proxy requires a username / password, you can use Java’s Authenticator to register a global “hook” that returns a username / password during authentication. In the following example, we only return the username / password when the requestor is a proxy, but you can also filter based on the requestor’s hostname, port, etc., to avoid sending credentials to the wrong server.

Authenticator.setDefault(
  new Authenticator() {
    public PasswordAuthentication getPasswordAuthentication() {
      if (getRequestorType() == RequestorType.PROXY) {
        return new PasswordAuthentication(proxyUser, proxyPassword.toCharArray());
      }

      return null;
    }
  }
);

This solves the problem of connecting to a proxy that requires credentials, but it is still a “global” solution. What if you only want specific connections to use the proxy, such as requests to the Google Cloud Vision API?

ApacheHttpTransport

The Google API Client Library for Java can use several types of HTTP transports: ApacheHttpTransport, NetHttpTransport, UrlFetchTransport, etc. The ApacheHttpTransport is highly configurable and can be setup to use a proxy that requires credentials. Furthermore, it is a non-global solution, meaning that its proxy settings only apply to the connections that it creates. This helps to insulate the rest of your application.

The following example demonstrates how to initialize an ApacheHttpTransport with a proxy host, port, username and password:

HttpHost proxy = new HttpHost(proxyHost, proxyPort);

DefaultHttpClient httpClient = new DefaultHttpClient();

httpClient.getCredentialsProvider().setCredentials(
  new AuthScope(proxy.getHostName(), proxy.getPort()),
  new UsernamePasswordCredentials(proxyUser, proxyPassword));

ApacheHttpTransport httpTransport = new ApacheHttpTransport.Builder()
  .setProxy(proxy)
  .build();

Then httpTransport can then be used to initialize the Vision service as follows:

JsonFactory jsonFactory = JacksonFactory.getDefaultInstance();

GoogleCredential credential = GoogleCredential.fromStream(inputStream, httpTransport, jsonFactory)
  .createScoped(VisionScopes.all);

Vision vision = Vision.Builder(httpTransport, jsonFactory, credential)
  .setApplicationName(“MyApp”)
  .build();

Note that we pass httpTransport to both GoogleCredential.fromStream() and Vision.Builder(). This ensures that the transport (and proxy) is used for oauth (GoogleCredential) and requests to the Vision API.

HTTPS

If you are connecting over HTTPS rather than HTTP, you need to configure ApacheHttpTransport to trust the public certificate of your proxy. You can specify this using ApacheHttpTransport.Builder.trustCertificatesFromStream() as follows:

FileInputStream fin = new FileInputStream("/path/to/cert.crt");

ApacheHttpTransport httpTransport = new ApacheHttpTransport.Builder()
  .trustCertificatesFromStream(fin)
  .setProxy(proxy)
  .build();

DefaultHttpClient httpClient = (DefaultHttpClient) httpTransport.getHttpClient();

…

Stale Connections

ApacheHttpTransport maintains a pool of connections and attempts to re-use them when possible. However, this can lead to problems when the remote host closes the connection: ApacheHttpTransport might attempt to re-use the closed connection and generate an exception (NoHttpResponseException). To fix this, you can enable a check for stale connections as follows:

ApacheHttpTransport.Builder httpTransportBuilder = new ApacheHttpTransport.Builder()
  .trustCertificatesFromStream(fin)
  .setProxy(proxy);

HttpConnectionParams.setStaleCheckingEnabled(httpTransportBuilder.getHttpParams(), true);

ApacheHttpTransport httpTransport = httpTransportBuilder.build();