================[ More HTTP ]================ http://www.garshol.priv.no/download/text/http-tut.html is another nice tutorial on HTTP. ================[ CGI standard ]================ Read these two short chapters on CGI from the early days of the web, 1996 (you may skip the Perl parts if you're not into Perl; Python was still a niche language back then, not yet at 2.0; perl was the sysadmin & web admin language of choice): http://www.oreilly.com/openbook/cgi/ch04_01.html http://www.oreilly.com/openbook/cgi/ch04_02.html Since then, the CGI standard remains in place, and still defines how GET requests are done (with the REST programming methodology recommending the meaning of the parameters, but keeping their syntax and encoding the same). However, the body of POST requests these days are XML or JSON objects. ======================[ JSON ]====================== http://www.json.org/ As a real example of structured, nested data transferred as JSON, look at how the National Weather service API formats responses for forecast requests for a geographic location. The API takes a GET request (i.e., you can type this request into the address bar of your browser and get exactly the same response as any app that sends similar HTTP headers), which contains GPS coordinates of a point you want to request weather forecasts for. The response contains a JSON object with many JSON objects in it, including a link to the specification of the format, the point requested, and forecasts for several time periods from now. There's a sample in forecasts.txt. We will parse this data on Friday. ================[ Web Programming ]================ Before you jump into Android app network programming, it makes sense to understand the development of web programming. Web programming started simple. Then it scaled, and scaled, and scaled up. In early CGI web programming, it was OK to think of data sent in web requests as merely contents of a form, a bunch of named strings. Then handling flat bags of named strings got too confusing. Programmer confusion leads to application breakage. Similarly, storing data as bags of strings (a.k.a. dictionaries or hashes) worked well for single forms and early webstores' shopping carts. Then having collections of inter-related objects beyond what fits in a hash or a relational database table became a necessity. Instead of treating GET request parameters, POST request bodies, and the respective responses as just free format data, there had to be a discipline to follow as to what they mean. Two very productive ideas were that CGI requests encoded objects and operations on objects held by the server. Some CGI parameters or URL path name parts thus specify which object, others what operation (method) is being called, the rest specify the arguments. REST (Representational State Transfer) codified this interpretation. It makes sense to think of POST request bodies as objects of their own, too. So web programming now deals with transfer of objects and calling object methods remotely. You may have come across the term RPC (Remote Procedure Call), which originated earlier than the web (e.g., the Networked File System, NFS, which we are using in the Unix clusters, is based on RPC; it was pioneered by Sun Microsystems, as much of modern Unix), but has been adopted on the web (say, the XML-RPC standard, JSON-RPC, and similar; look up https://en.wikipedia.org/wiki/XML-RPC). Arguably, the story of web successes is the story of the next steps in handling complexity: eliminating it where not needed, adding it to represent more powerful abstractions (like nesting objects obeying schemas) where necessary. The following note can be read as a story about that: https://plus.google.com/+RipRowan/posts/eVeouesvaVX (if you are in a hurry, scroll to the retelling of Jeff Bezos' Big Mandate, quoted below) -----------------------[ begin quote ]----------------------- 1) All teams will henceforth expose their data and functionality through service interfaces. 2) Teams must communicate with each other through these interfaces. 3) There will be no other form of interprocess communication allowed: no direct linking, no direct reads of another team's data store, no shared-memory model, no back-doors whatsoever. The only communication allowed is via service interface calls over the network. 4) It doesn't matter what technology they use. HTTP, Corba, Pubsub, custom protocols -- doesn't matter. Bezos doesn't care. 5) All service interfaces, without exception, must be designed from the ground up to be externalizable. That is to say, the team must plan and design to be able to expose the interface to developers in the outside world. No exceptions. 6) Anyone who doesn't do this will be fired. -----------------------[ end quote ]----------------------- This Big Mandate may have been the secret weapon for not letting complexity accumulate where it could be destructive and confusing, and for making people fight for complexity where they needed it, thereby improving their designs. This may have been one of the most significant things that proved to be the right approach---at least for this generation of the web. ================[ Android Network Code ]================ Look up the previous lecture notes for examples of using Android's HttpURLConnection to send GET and POST requests. Two examples, examples/Connector and examples/Uploader show how to send simple GET and POST requests that comply with our Lab 2 server protocol (lab2/server-protocol.txt). However, these examples do not react nicely to errors, and do not report their state or progress. This is fine for small requests, but not for noticeable downloads. So Android offers a primer on "how to connect right": https://developer.android.com/training/basics/network-ops/connecting.html This turns out to be a very long primer! You must read it, however, to see what libraries like Volley that encapsulate asynchronous network connections---and they must all be asynchronous on Android, off the main UI tread!---buy you. Pay special attention to https://developer.android.com/training/basics/network-ops/connecting.html#config-changes In a nutshell, https://developer.android.com/training/basics/network-ops/connecting.html describes a 3-part solution to designing a download handler: - an AsyncTask to do the actual connection - a headless Fragment to hold a reference to the task & make callbacks into Activity to return results or errors - an Interface for the Activity to implement so that the Fragment can update its UI - additionally, the task defined a Result class for returning results or exceptions, as a union. This allows the task & fragment to pass exceptions to the Activity (as an algebraic type, i.e., a type whose contents are a logical either-or of two other types; there are also other types constructed via logical operations---the approach that Haskell, ML, OCaml, F#, and many other modern languages take in their type systems). The headless Fragment is meant to be persisted with setRetainInstance(true), to keep a reference to the AsyncTask & avoid lost contexts and memory leaks. This example shows, at least, how Android engineers think in Java. It handles errors nicely, and also reports progress in a set of stages, but it's a nightmare to maintain alongside your code! ================[ Volley ]================ So there are other libraries. For a while, Apache's HttpClient was a leading tool, but it's been deprecated since Android 6.0. You will still find plenty of handy Stackoverflow answers using it, but be aware that it's now deprecated. Google's recommended library appears to be Volley: https://developer.android.com/training/volley/index.html Try https://developer.android.com/training/volley/simple.html or see my example/Volley1 for a GET request against our Lab 2 server. A huge advantage of Volley is that it will parse JSON requests for you, so that you get a valid JSON object, not just a String or InputStream that you must parse on your own risk. See https://developer.android.com/training/volley/request.html On Friday, we will use this library for sending and receiving JSON objects in requests (see examples/UploaderVolley for how Uploader can be redone in Volley). See also this post for more handy examples: http://arnab.ch/blog/2013/08/asynchronous-http-requests-in-android-using-volley/