Figure 1: DartNet Communications Stack
As part of cs60 we will build our very own communications stack called DartNet. The ideas is that protocols remain abstract but neat mechanisms but without hands on experience to reinforce the knowledge few skills are burnt in. We are challenging you to think about the design of a transport and network protocol which functionally are similar to TCP and IP protocols that drive the Internet. We cannot run our code on routers in the Internet so DartNet’s transport - called the Simple Reliable Transport (SRT) and network - called Simple Network Protocol (SNP) layers will run on an overlay that we will also construct. You have already came across overlay networks - P2P networks are overlay networks.
In this lecture we will discuss the designed design of SRT and will leave SNP for a future lecture. As we progress we will move down the DartNet stack.
Much of the information presented in this lecture is important for completing Lab4 - Implementation of the Simple Reliable Transport.
In this lab we will write a client and a server.
We plan to learn the following from these lectures:
OK, let’s get going.
____________________________________________________________________________
The DartNet communications stack shown below in Figure 1 allows clients and servers to create sockets, open connections between hosts for reliable delivery of data. DartNet is a layered architecture with each layer at n-1 offering layer n a service and API. For example, SRT offers applications the reliable delivery of byte streams. SNP offers the transport layer connectivity between end-systems similar to IP. The overlay layer sets up a network of end hosts that run the DartNet stack. We show for correctness that TCP/IP runs under the overlay. Put you can think of SNP+overlay as being the network layer and the lower layers (in this case TCP/IP) as part of the lower layers of the architecture akin to the data link and physical layers.
SRT provides the reliable deliver of in order byte stream just like TCP. However, we would be here all year coding TCP (which would be fun granted) so SRT makes a number of assumptions to simplify the design and implementation of the protocol.
The following mechanisms are implemented as part of SRT:
In Lab4 we do not consider DATA segments. We only consider control (or signaling messaging) used by connection management.
There are a number of APIs between the different layers of DartNet. In n Fig.2 we show the full stack. The SRT API offers the ClientApp and ServerApp a number of function calls - collectively called APIs. Over the course of the DartNet thematic labs you will implement all of these API functions. We discuss these APIs in more detail later in these notes.
For Lab4 we remove SNP to simplify the assignment and focus on the implementation of a restricted set of functions associated with the SRT API for the client and server respectively. We only consider function calls associated with set up and tearing down connections between the client and server. Note, SRT is capable of supporting multiple simultaneous connections like TCP. We will discuss the data structures and Finite State Machines (FSMs) need to support this later in this text. Also note that the SRT sits directly on the “thin” overlay layer when SNP is removed. The overlay in this case simply implements a TCP connection between the client and server SRT transport protocols. The overlay just contains a direct TCP connection between the client and the server. The overlay_start() and overlay_stop() functions create the drect TCP connection between the client and the server. overlay_start() should return the TCP socket descriptor. overlay_stop() closes the TCP connection. The code needed to do this is essentially the C code we developed for the socket programming assignment. Therefore, you already have 90% of the code for the overlay. For lab4, the API functions need to implement are shown in Fig.3. These are a subset of the DartNet API function calls.
You will need to implement snp_sendseg() and snp_recvseg() function calls to send and receive segments using the overlay’s TCP connection. In the case of TCP data is sent as byte stream. Note, in order to send data segments over the overlay TCP connection, we need to add delimiters in the byte stream. In SRT uses the special characters “!&” to indicate the start of a segment and the special characters “!#” to indicate the end of a segment. Two notes on these delimiters. First, these characters should not occur in the data sent between the client and the service. This is a restriction (and the protocol would operate incorrectly if “!&” or “!#” were to occur in the byte stream from the client. So you should make sure your data does not include these pairs of special control characters. Second, delimiters are necessary because the receiving side of a STR packet (i.e., STR header + STR segment) needs to know when it successfully receives an SRT packet. This is one way to do it. Probably the simplest approach.
Because the overlay is built from TCP rather than say UDP we do not have any loss. To allow the overlay to look as if it runs across lossy links and that packets can be lost in the Internet we implement loss() function invoked in snp_recvseg() when an SRT packet is received to discard the segment with probability PROBABILITY_LOSS to simulate the packet loss to force SRT to recover the missing segments which can be SRT DATA packets, SYN, SYNACK, FIN, and FINACK. The SRT protocol must be able to recover correctly from loss of any of these packets.
The flow of the functions are as follows. The app_client.c and app_server.c, the application layer code first start the overlay by creating a direct TCP link between the client and the server in overlay_start(). In Lab4 students implement overlay_start() in the app_client and app_server. The server initializes the SRT server by calling srt_svr_init() and the client initializes the SRT client by calling srt_client_init(). Then the server calls srt_svr_sock() to create a server side socket and srt_svr_accept() to accept a connection request from the client (i.e., control message SYN at the SRT layer). The client creates a socket and connects to the server socket by calling srt_client_sock() and srt_client_connect(), respectively. In Lab4 the client and server establish two connections on different ports to prove that the SRT can cope with multiple connections: the server creates another socket and waits for incoming connection, and the client creates another socket and connects to the new server socket. By doing this, we show that SRT can have multiple connections at the same time. In Lab4, the client disconnects from the server after a wait period by calling srt_client_disconnect() for each connection. Finally, the server closes the socket by calling srt_svr_close() and the client closes the socket by calling srt_client_close(). The app_client and app_server stop the overlay before terminating their processes by calling overlay_end() at the client and server. In Lab4 students implement not only overlay_stop() but all the SRT functions used in srt˙client.c and srt_server.c.
SRT defines a number of important data structures that implement the control and packetization of the data into segments.
An SRT packet comprises an SRT header (see srt_hdr_t below) and SRT segment (see seg_t - data bytes stream goes here). The SRT packet is encapsulated into a SNP packet which we will discuss when we get to the SNP lecture. Notice that the SRT header includes important information associate with the packet include for instance unsigned short int type;.
The type field in the packet header defines the type of packet:
SRT protocol sends and receives data in segments. The segment header format is shown in Fig. 4 and the C language equivalent is shown below:
It is defined in seg.h:
The SRT maintains all the state associated with a connection using a Transport Control Block (TCB). For each connection the client and server side initialize and maintain a TCB. When a connection is closed the TCB is returned to the pool of unused TCB. The figure below shows a server side TCB maintained for each connection. A TCB table is initialized at the client and server on startup when srt_svr_init() and srt_client_init() are called by the server_App and client_App, respectively. Implementation tip: always use malloc()/free() for data structures and tables of data structures when coding the SRT.
For each connection, the SRT client maintains a client and serever TCB.
SRT has a similar setup and teardown of connections as TCP. However SRT simply uses a two-way handshake.
A connection is established when the client sends a SYN and the server responds with a SYNACK as shown in shown in Fig. 5.
But life is not that simple: corruption and loss of control messages get in the way
In class today we discussed the following problems that occur with set up and tear down:
What happens in the SYN is lost or corrupt; client knows it sent it, server has no knowledge. So how is this operation issue solved? Furthermore, what happens if the server received the SYN, send the SYNACK but the SYNACK is lost. The server knows it sent an SYNACK and does not know that the client did not get it.
Solutions to these anomalies is as follows. In the case of the lost SYN the str_client needs to set a SYN_TIMEOUT and if it does not receive a SYN ACK in that period it sends another SYN and set the timer again. It repeats this until it either gets a SYNACK or the SYN_MAX_RETRY is exceeded. For example, the client sends a SYN and moves into a CONNECT state. In that state it is likely to receive SYNs. What should it do? Send a SYNACK and let the client move to CONNECT to. We discuss the correct state transitions later on client and server FSMs.
A connection is taken (tear) down when the client sends a FIN and the server responds accordingly with a FINACK as shown in 6
But we can lose packets here. What happens in a FIN is lost? The client needs to send another FIN after FIN_TIMEOUT up to FIN_MAX_RETRY. What happens if the FINACK is lost? The client side has to wait for retransmissions of FIN_MAX_RETRY in an attempt to get a FINACK. If it doesn’t get the FINACK it transitioned to CLOSE state after a wait period and assumes the connection had time to close in an orderly manner.
The client and server side implement FSMs. FSMs are made up of a limited number of states. The transitions to a state is governed by the state machines and the EVENT/ACTION. Basically, in a state e.g., on the client FSM, e.g., assume it is in state = SYNSENT and the EVENT SYN_TIMEOUT occurs then the appropriate ACTION is send SYN. Check the client FSM to see if this is indeed the case.
The states maintained in the client TCB in the “state” element of the structure is as follows:
When implementing your protocol you will need to consult with the state machines shown here to determine that state transitions for the client and the server due to the exchange of segments during connection establishment and tear down. Here the SRT server state machine is shown in Fig. 8. SRT client state machine is shown in Fig. 7.
The states maintained in the server TCB in the “state” element of the structure is as follows:
With the major data structures defined for the messages (SYN, SYNACK, FIN, FINACK) on the “wire” (i.e., to be sent to the overlay) and the internal TCB used to maintain the “state” information associated with connections (note, that the index into the TCB table is used as the sockfd for the connection on the client and server side); and the message interactions between the client and server defined for setup and teardown; and the detailed description of the client and server FSMs; then, you have most of the information to get rockin and implement the SRT. There is missing information that you can think about and design accordingly, but, this important information is sufficient to implement SRT.
There are a number of files that show examples of the app_client.c and app_server.c that show the SRT Socket API. In addition, we provide a number of header files that define constants used by SRT that you should include in your code.
Clients application - examples of client using SRT Socket API.
Client side data structures and prototypes needed to implement the client side SRT protocol
Server application - examples of server side SRT Socket API.
Server side data structures and prototypes needed to implement the server side SRT protocol
SNP APIs - header file. SRT header and segement data structure, SNP prototypes.
Constants
