======================[ Readings ]====================== HTTP: https://danielmiessler.com/study/http/ --this tutorial covers the basics of HTTP we saw in class. JSON format & syntax: http://www.json.org/ Wireshark: download the tool at wireshark.org. See tips about filtering http in "Display Filter", which I used in class: https://wiki.wireshark.org/Hyper_Text_Transfer_Protocol (you can also set the Capture Filter when you start capturing packets, to reduce the screen clutter from the get-go. It's important that display filters and capture filters are two _completely_ different languages). ==================[ Operating Wireshark ]====================== First, you need to select the interface to capture on. This is the network device that is currently connected. On MacOS, it's typically "en0" for a Wireless, or "enX" where X is an integer for other Ethernet or Wi-Fi devices. Filter what gets shown of the packets captured by using a "Display Filter" (e.g., http or ip.addr == 129.170.213.53) in the top bar of the application window. Choose a packet and use "Analyze > Follow > TCP Stream" to see the character streams of a connection containing this packet in raw byte form, "Analyze > Follow > HTTP Stream" to see gzipped parts of HTTP connection expanded, and the "SSL Stream" option to see the steps of the SSL/TLS protocol that uses cryptographic certificates for handshakes and encrypts the rest of the connection. Wireshark is an extremely useful tool. There are many visual tutorials, e.g., https://www.lifewire.com/wireshark-tutorial-4143298 https://www.howtogeek.com/104278/how-to-use-wireshark-to-capture-filter-and-inspect-packets/ and also plenty on Youtube. ======================[ HTTP Basics ]====================== HTTP is a plain-text protocol for web requests and responses. Its prominent features are: - The first line of a *request* (from client to server) specifies the method (GET, POST, PUT, ...) and the URL less the host part, plus the protocol version. E.g.: GET /kitty1880.jpg HTTP/1.1 GET /profile.pl?name=sergey HTTP/1.1 PUT /profile.pl HTTP/1.1 - The first line of a *response* (from server to client) specifies the status code, such as 200 OK or 404 Forbidden: E.g.: HTTP/1.1 200 OK Codes are important, because they indicate if you got what you asked for, or whether it moved to another known address, or if it's gone forever, or whether you are being blocked from seeing it (cf. https://en.wikipedia.org/wiki/HTTP_451) - The first line in both requests and responses is followed by a number of header lines, of the syntax "Header: Value(s)". E.g.: Host: cs65.cs.dartmouth.edu --in request: this is intended for cs65.cs.dartmouth.edu, no matter how many servers may be sharing the same IP address. (Such sharing is very frequent with virtual hosting, and is possible exactly because of his header). Accept-Encoding: identity --in request: do not compress responses to save bandwidth Content-Type: application/json --interpret the body as JSON Content-Type: image/jpg --interpret the body (of a response) as a JPG image Content-Length: --the exact length, in bytes, of the body following the headers, in POST requests and responses. User-Agent: and so on. Google the headers you see in captured requests. - All requests and responses indicate the end of their headers by an empty line. For GET requests, this empty line finishes the request. For POST requests and responses, this empty line separates the headers from the body. It may be the most important part of the HTTP syntax. Note that all newlines in HTTP are \r\n (an old DOS/Windows legacy). All of these headers can be manipulated with Android Java's HttpURLConnection with setRequestProperty(..), e.g., to POST some JSON content: URL url = new URL("http://cs65.cs.dartmouth.edu/profile.pl"); HttpURLConnection conn = (HttpURLConnection) url.openConnection(); conn.setRequestMethod("POST"); conn.setDoOutput(true); // needed by POST with a non-empty body conn.setRequestProperty("Content-Type", "application/json"); // we want to see strings going back and forth, don't compress them conn.setRequestProperty("Accept-Encoding", "identity"); conn.setFixedLengthStreamingMode(req.length()); // sets Content-Length: header OutputStream out = new BufferedOutputStream(conn.getOutputStream()); out.write(req.getBytes()); out.flush(); out.close(); // to read the response into a String InputStream in = new BufferedInputStream(conn.getInputStream()); String res = readStream(in); Sending a GET request is somewhat shorter: URL url = new URL("http://cs65.cs.dartmouth.edu/profile.pl?name=sergey&password=123"); HttpURLConnection conn = (HttpURLConnection) url.openConnection(); conn.setRequestMethod("GET"); // we want to see strings going back and forth, don't compress them conn.setRequestProperty("Accept-Encoding", "identity"); InputStream in = new BufferedInputStream(conn.getInputStream()); String res = readStream(in); Where (so iconic of Java I/O design): private String readStream(InputStream in) throws IOException { BufferedReader r = new BufferedReader(new InputStreamReader(in)); StringBuilder total = new StringBuilder(); String line; while ((line = r.readLine()) != null) { total.append(line).append('\n'); } return total.toString(); } and the above snippets of code must also handle a variety of exceptions, because many things can go wrong with a network connection. ======================[ Code samples ]====================== The following code samples are quick-and-dirty. They may crash if your connections fail or time out. Moreover, they neither check nor request the android.permission.INTERNET in the code, although they do declare (otherwise the app would crash on newer Android devices). Your apps should properly check and request permissions. examples/Connector/ performs a simple fixed GET request, and displays the response body in a TextView examples/Uploader/ performs simple GET and POST requests, taking their bodies from a TextEdit, against my server. NOTE that the formats for GET and POST that my server would accept are very different: GET takes strings like "name=tim&password=1234", whereas POST requires well-formed JSON like {"name":"tim","password":"1234"} . See lab2/server-protocol.txt for details. Just in case it's not sunk in: HTTP is a terrible protocol for private data, and these are terrible terrible passwords!