Web services


So far we've been dealing with the world as it exists on our individual computers. But a lot of exciting problems arise in the context of having multiple machines work together on something. We'll get a taste of such approaches in the upcoming week, starting today with the basic question of how to obtain and process data from web servers. Our goal is to build a simple query interface for Flickr; we'll cover how to send and process queries, and how to use a graphical user interface (GUI) to provide a somewhat nicer interface than we have seen.

We'll be using a number of different Java packages to help out. As usual, Oracle provides a detailed reference. Somewhat more useful to get going, they also provide a set of Java tutorials. I've provided links to relevant ones within the menu below. Note that they cover much more than we do, and are just provided for your reference in case you want supplementary reading.

Outline

All the code files for today: FlickrSearchJSON.java; FlickrSearchXML.java; FlickrSearchCore.java; ProcessXML.java;
json-simple-1.1.1.jar; WWWGet.java; WWWGetTry.java

Note: you need to download json-simple-1.1.1.jar and install it as an external JAR (as you did for the opencv files) to run FlickrSearchJSON.java. Otherwise you can run FlickrSearchXML.java without installing the JAR. The XML version does the same as the JSON version, but exchanges data with Flickr via XML instead of JSON.

Slides from class

Graphical user interface

Let's start by mocking up a little GUI for the photo browser we have in mind: FlickrSearchCore.java. GUI construction is tedious, and best done with a GUI development environment that lets us graphically lay out the various components. So we'll just do something simple that illustrates the use of some of the components (buttons, a text field, a combo box); once you've seen these, you can pick up the rest by following the same principles.

We've gotten away from our good old DrawingGUI, and rolled all the GUI functionality directly here. Thus the class extends JFrame, a Java class for a top-level window.

The constructor creates a "canvas" as a JComponent (generic GUI component) with a method to paint itself. In this method we have to call the superclass paintComponent, and then can do whatever drawing we want to. That will eventually be to display an image, but it's blank for now. (In DrawingGUI, this is where I had it call the draw() method that we filled in, in different ways.) One odd thing about this creation of the canvas is that it calls "new JComponent()" but then has curly braces and a method definition. This is actually the combination of defining a new class that is a subclass of JComponent, and then creating an instance of that class, all at once. But we never bother to give the class a name — an anonymous class, like the anonymous functions we saw before — because we only ever need that one instance. We could have had a separate class definition in another file, say:

public class Canvas extends JComponent {
  public void paintComponent(Graphics g) {
    super.paintComponent(g);
    // will add code here to draw the current image
  }
};

And then new'ed one of these. The advantage of the anonymous class (besides keeping things self-contained) is that it has access to the instance variables of the containing class. So when the whole thing is finished, we'll actually have the canvas directly access the set of images belonging to its "parent" FlickrSearch object; we don't have to pass them in.

After the canvas, there's a bunch of boilerplate to set the size, pack the GUI components, make it visible, etc. If you look back at DrawingGUI, you'll see all that stuff in there. (Aren't you glad I buried it under the rug in week one?)

The other GUI elements are created in a separate method setupGUI(). First we have a "next" button and a "previous" button that step through the photo array. While these are buttons that you press specifically, rather than just general mouse presses within the window, the mechanism is the same. Again, I buried that under the rug in week one, so that the Java mechanism just ended up calling a "handleMousePress" method that we could define ourselves, but this is how that works.

In order to respond to user input, we need a way to represent the input and invoke appropriate methods. The representation is in terms of "events" (e.g., the event that the mouse was pressed), and a "listener" machinery tells the GUI what events to pay attention to, and what to do when they happen. For key presses, Java provides KeyEvent instances. For mouse motion, it provides MouseMotion. And for all other kinds of mouse stuff, Java provides MouseEvents — when a mouse button is pressed, released, or clicked (pressed and released without intevening motion), and when the mouse enters from outside the window or exits from the window. The MouseEvent object contains information about where the event happened, which button, modifiers, etc. For GUI components like buttons here, there's a catch-all ActionEvent.

We have to tell Java what to do when it notices some such event. This is handled by a "listener" object that provides an appropriate method to invoke with the event. One of the cases:

prevB.addActionListener(new AbstractAction() {
  public void actionPerformed(ActionEvent e) {
     // go to the previous image
  }
});

This is again an anonymous class, one that extends AbstractAction and provides a method that responds to an event. Here we will again (in the final version) make use of the fact that the class is "inside" the FlickrSearch object and can access its instance variables. Look back at DrawingGUI to see the other types of events in action, and how they then connect to the methods you wrote.

There's another way to provide an action listener as of Java 8: just directly pass a method (here as an anonymous function):

nextB.addActionListener(e -> {
  // will add code here to move to the next image
  System.out.println("next");
});

The next GUI element is a combo box (a drop-down menu) that allows selection of how the photos should be sorted — relevance, date, or interestingness (ascending or descending). This component likewise has an action event in which we note the selected sort order. The construction is a little tricky, as we have to specify the options to put in the list. I'm using a hard-coded list of Strings.

The textbox for the search text is simple, and the search button is set up the same way as the other buttons. Its action is somewhat more complex, as it invokes our search function loadImages() and has to catch some of the errors it might encounter.

The key thing we have to account for with multiple components is how to lay them out. (That's what's best done graphically.) Java provides a number of different layout managers with different behaviors. We put the buttons in their panel with a "flow layout" that just adds them in rows, left to right, with a new row started when the next component won't fit in the current row. We then put that panel along with the canvas into the main content panel with a "border layout" that allows the user to put one thing at the top (NORTH), one at the bottom (SOUTH), one on the right side (EAST), one on the left side (WEST), and one in the middle (CENTER). The CENTER expands to occupy all space not used by the other four. So the button panel is at the top and the canvas in the center, taking the rest of the space.

Getting stuff from the web

At its heart, network programming is all about how to properly transfer information between computers. As we discussed briefly when we first encountered type declarations in Java, a computer only ever deals with bits, so how is any particular set of 0s and 1s to be interpreted? What does an incoming bunch of bits imply, and how do I send the appropriate outgoing bunch of bits? That's what network protocols are all about. With the internet, there are multiple layers of how to handle bits, to gain increasing reliability (in case of misread bits, dropped bits or entire messages, etc.). You'll have to take the networking course to get into that; here we're just going to look at a high level, provided by web-based protocols.

A key component of the web is HTTP, the HyperText Transfer Protocol, which is a set of rules for how browsers (and other agents on your machine) communicate with web servers. For example, your browser might ask the Dartmouth web server "get /~reg/index.html"; and then the web server responds with the hypertext for that page. Your browser would then notice that there are some images referenced in there, and ask the server for them. Note that the basic process is stateless (doesn't remember you from one request to the next). Cookies, anyone?

The URL (Uniform Resource Locator) is the global identity of the page you want; e.g. http://www.dartmouth.edu/~reg/index.html

Java provides libraries that let us get from a web server, within a program. Here's a little self-referential program that gets this page.

[WWWGet.java]

import java.net.*;
import java.io.*;

public class WWWGet {
	public static void main(String[] args) throws Exception {
		// Open a stream reader for processing the response from the URL
		URL url = new URL("http://www.cs.dartmouth.edu/~tjp/cs10/notes21.html");
		System.out.println("*** getting " + url);
		BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
		// Read lines from the stream
		String line;
		while ((line = in.readLine()) != null) {
			System.out.println(line);
		}
		in.close();
		System.out.println("*** done");
	}
}

The Reader/Stream mechanism for reading data works like we've done previously with files; the difference here is that the file is served up over the web, rather than loaded from your own machine. We create a new InputStreamReader that will get its input from the web server's response for the URL we provide.

As with files, trying to open and read from a URL can raise exceptions. An exception-handling version of our previous buck-passing program: WWWGetTry.java. The handlers for the exceptions from creating the URL and the reader are at the bottom. We then have a try-within-a-try for reading (boy would Yoda be displeased — do or do not, there is no try), because we want to close the stream no matter what. Note that the stream closing itself can throw an exception, which is handled by the same catcher as for opening. Whew.

Web services

Instead of dealing with web pages, we'll be dealing with web services — when information is delivered via this same machinery. In that case, we actively engage with the server, sending it values of parameters with the input, and getting back results specific to that input. It's like a function call (and indeed some of the mechanisms make it feel even more that way). We'll be using a basic, but powerful and common mechanism, called REST (REpresentational State Transfer) that is built directly on top of the same HTTP approach we've just used for web pages.

For example, here's a greeting program using REST: http://cs.dartmouth.edu/~tjp/cs10/hello.php?name=tim. This is just a URL, extended with a parameter "name" whose value here is "tim" (try substituting your own name, of course). Multiple such parameters can be included, separated by ampersands (&); e.g., http://cs.dartmouth.edu/~tjp/cs10/hello.php?name=tim&color=blue. This approach directly uses the "get" machinery already in place for fetching web pages. I'm sure you've seen parameters showing up in URLs in other contexts; can you think of a few? A more powerful, but somewhat more complicated, approach is to leverage the "post" machinery that is commonly used when sending form data (e.g., when ordering something from a web store). Rather than including the name/value pairs in the URL, they are sent directly afterwards via an established connection. We'll stick with the simpler "get" approach.

One thing to note is that not all characters are acceptable in a URL. For example, URLs aren't supposed to include spaces. So we have to "URL encode" the parameter values using an encoding of such "special characters". A simple way to do that is with the URLEncoder class:

URL url = new URL("http://cs.dartmouth.edu/~tjp/cs10/hello.html?name="
         + URLEncoder.encode("tim pierson","UTF-8"));

Processing XML

So now we've handled sending parameters to a web service; how do we get a response back that we can interpret? There are a number of approaches to encoding data in a way that a program could interpret it. One standard way is XML (eXtensible Markup Language), which is related to and much like HTML (HyperText Markup Language) in that they are both languages (sets of syntax and grammar rules) describing how to represent something. For HTML, that's hypertext; for XML, it's basically anything.

XML represents data by wrapping it with tags that give some meaning to each piece. The tags can be nested, with the inner parts being components of the outer parts. A tag can also have a set of name-value attributes that give particular details (without going to the trouble of nesting them, since they're unique). A simple example:

<enrollment>
 <course department="CS" number="1" term="18W">
  <student name="Alice" year="20" />
  <student name="Bob" year="19" />
  <student name="Charlie" year="18" />
 </course>
 <course department="CS" number="10" term="18W">
  <student name="Delilah" year="19" />
  <student name="Elvis" year="00" />
  <student name="Flora" year="20" />
 </course>
</enrollment>

Note that tags are enclosed with angle brackets (< ... >) and must come in properly nested pairs (start tag <name> ... end tag </name>;). A start tag can include attributes. If there's nothing to be nested inside an element, the tag can be self-closing, with a slash at the end of the start tag: <name/>.

To do something with XML, we have to know how to interpret the tags and what their relationships are to each other. That's obviously problem-specific. We're focusing here on a Flickr-based photo search application. We'll give Flickr a query describing the photos we're interested in, and it will return information about the photos that meet our criteria. The Flickr API documentation describes the format for different types of queries; we're doing a standard photo search here.

Fortunately the XML is pretty simple. An example (from a search for Dartmouth images, of course): flickr-dart.xml Actually, that's a hand-edited version of what Flickr gave back, as the original included an ampersand in "Dev & The Cataracs", which is a no-no in XML. It caused me some pain when writing code to process this, and as you'll see, I just did something simple (aka "gross hack") to deal with it.

So what can we do with XML? For our application, we want to actually pull down the images (we could also label them with their titles, etc., but I'll leave that for you to do on your own, if you're so interested). The Flickr URL documentation says that we can access a photo with a URL of this form:

http://farm{farm-id}.staticflickr.com/{server-id}/{id}_{secret}.jpg

So for example, the fourth image in our XML file (Baker) has this URL: http://farm3.staticflickr.com/2622/3840099142_fb08d49e3a.jpg.

Let's write a program to pull the URLs from the XML file. We need an ability to read the XML file into a representation we can handle with a program. Java provides numerous XML parsing libraries. Here I'll use the document object model (DOM) approach, in which we directly build a document structure mirroring the nested structure in the XML file. (If you ever explore your HTML through a power-user tool on your browser, that's the same idea.) The DOM gives us the ability to search for elements with a specific tag name and get their attributes (among many other things; e.g., navigating the nesting structure). So we can find the photos and build the URLs from the attributes as illustrated above.

The code: ProcessXML.java. Save flickr-dart.xml in a folder called "inputs". We start by pulling the XML file into one big String. We read the individual lines, using the same approach we've seen before, and concatenate them together. Along the way, we replace ampersands with pluses, the gross hack I mentioned. Then there's quite a bit of boilerplate to parse the XML string, due to the power and generality of the mechanism. Finally, we piece together the URLs for the photo elements as described. I had to write an extra little piece of code to look up the attribute of a given name and return its value, just by marching down the list of attributes and seeing which one matches.

The finished product

FlickrSearch.java puts together all the pieces we've seen, with just a bit more to drive the search.

The meat of the application is loadImages, which works basically the same way as our ProcessXML.java code, wrapped up in code to construct and execute the query (one such query returned the XML we used directly there) and fetch images from the URLs in the XML.

The query specifies the various parameters to the REST query, as specified in the photo search docs on Flickr (note that you can test out the search interactively there). Here we use the textbox input as the value of "text" (properly URL encoded), provide the specified sort as the value of "sort", and limit the results to 10. To run this yourself, you'll need an API key for Flickr (as given in a final variable). I have obtained one for use in the class and have provided it on Canvas. Please limit your usage of this application to simple and appropriate searches, so that I may maintain my key. If you want to do more extensive searches yourself, it's easy to apply for a key.

We form the image URLs using the technique we already discussed; the "z" at the end of the URLs indicates an image that is max 640 pixels on a side. We then use the standard Java image mechanism to pull in the images and store them in an array.

Java notes

inner class
A class can be defined inside another class, as we saw with the elements of linked lists. There it was useful as a type of information hiding (nobody else needed the elements) and to keep things local. It can be even more useful when the inner class benefits from direct access to the outer class variables and methods.
anonymous class
An instance of an inner class can be created without giving that class a name. The superclass is named, and the extensions (additional methods and instance variables) directly coded within the "new" call. As an inner class, it can use the outer class variables and methods.