Figure 1: Client Transport Control Block (TCB)
Up until this point we have dealt with how the Simple Reliable Transport (SRT) can establish connections over an overlay. We have implemented the transport support for signaling and built a robust SRT signaling system that handles many of the operational issues that can occur over a network, for example, lost of signaling messages.
OK, now it is time to move onto implement the data transfer side of the operation - the essential aspect of the transport.
In this lecture, we will discuss the design design of SRT data transfer which is based on the classic implementation of Go-Back-N.
Much of the information presented in this lecture is important in completing Lab5 - Implementation of the Simple Reliable Transport’s Go-Back-N. So read these notes closely before starting Lab5.
We plan to cover the following in this lecture:
OK, let’s get going.
____________________________________________________________________________
After SRT establishes a connection, a client can send data to a server using the SRT API srt_client_send() call. Similarly, a server can receive data from a client using the SRT API srt_svr_recv() call. The APIs are only valid when the client and server side of the connection are in the FSM state CONNECTED i.e., tcbstate == CONNECTED. These notes are all about the processing in the CONNECTED state.
The srt_client_send() function is a non-blocking function call which returns to the caller after the data is queued to be sent. Where is the data stored or buffered you may ask. In our design of SRT data transfer each client TCB maintains a send buffer to store the data to be sent. SRT uses Go back N data transmission mechanism.
The srt_svr_recv() function is a blocking function call returns once there is the requested number of bytes recieved by the server; then control is passed back to the application. When srt_svr_recv() is called the received data is returned to the application - assuming that the requested amount of data is available. Each server TCB therefore needs a receive buffer to store the received data as it comes “off the wire” (i.e., the network).
In what follows, we discuss the send and receive side buffer design.
The send buffer is a linked list containing all the unAcked DATA segments. The data contained in the segments comes from srt_client_send() function call. The segments contained in send buffer linked list are sent one by one from head to tail to the network.
SRT’s GBN mechanism dictates that at any time there are at most a GBN_WINDOW sent-but-not-Acked segments in send buffer. In SRT the client sends a DATA packet (SRT header + SRT data segment) and the server response with a DATAACK control packet. When a DATAACK is received, the Acked segments are removed from the send buffer.
Each DATA packet in the send buffer linked list is maintained as a sendBuf structure. The structure of sendBuf is defined in srt_client.h; here it is:
Note: each client TCB (transport control block) shown in Fig. 1 maintains a unique send buffer for each end of the connection.
Client TCB structure is defined in srtclient.h and below:
Each client TCB maintains a unique send buffer. When a client TCB is initialized in srt_client_sock(), the send buffer is empty and a mutex structure is dynamically created.
When srt_client_send() is called, the data passed to this function is used to create new segments. These new segments will have type of DATA and be encapsulated in segBuf structures. These new segBufs are then appended to the send buffer. At this point segments should be sent starting from the first unsent segment until the number of sent-but-not-Acked segments reaches GBN_WINDOW, as discussed above. When a segment in send buffer is sent, its sent time is recorded in the sentTime field of its segBuf.
A thread called sendBuf_timer should always be running when the send buffer is not empty. This thread polls the first sent-but-not-Acked segment (which should always be a the head of the send buffer) every SENDBUF_POLLING_INTERVAL time to see if: the current time - the first sent-but-not-Acked segment’s sent time is larger than DATA_TIMEOUT value. If this is the case, a timeout event is triggered and all the sent-but-not-Acked segments in send buffer are re-sent. This thread is started in srt_client_send() if this function finds that the current send buffer is empty (which means there is no sendBuf_timeout thread running). When all the DATA segments in send buffer are Acked and the send buffer is empty the sendBuf_timer thread terminates itself.
When a DATAACK segment is received by the client’s seghandler, the Acked segments are removed from send buffer and freed. Note that the sequence number in DATAACK is the expect sequence number from the server for the next DATA segment it expects; indicating, that all the segments with a smaller sequence number should be freed since the server has the segments.
When the client TCB transitions to CLOSED state in srt_client_disconnect() the send buffer should be cleared and an empty linked list results.
When the client TCB is freed in srt_client_close(), the mutex for send buffer is also freed.
The receive buffer is a byte array with fixed length. It is used to store received data extracted from DATA segment (the segment header is not stored).
Each server TCB maintains a unique receive buffer. The server TCB structure is shown in Fig. 2.
The TCB structure is defined in srt_server.h and also shown below:
When a server TCB is initialized in srt_svr_sock(), the mutex and send buffer itself should be dynamically allocated and initialized. The receive buffer is a fixed size of RECEIVE_BUFFER_SIZE.
When a the server transitions to CONNECTED state, the server TCB sets its expect_seqNum using the sequence number in the received SYN segment.
When the server’s seghandler thread receives a DATA segment and the sequence number is the same as expect_seqNum, the received data should be stored into the receive buffer. The usedBufLen and expect_seqNum are incremented with the data size of received DATA segment. A DATAACK is sent back to the client with the updated expect_seqNum; or with the old expect_seqNum if the received DATA segment’s sequence number is not the same as expect_seqNum - as stated above. If the receive buffer reaches its size limit then the server simply discards the received DATA segment - silently.
When srt_svr_recv() is called, the server checks if the receive buffer contains enough data to be returned. If this is the case the data is returned and removed from receive buffer (usedBufLen is updated and the remaining data is moved to the starting point of the receive data). Otherwise, the server busy waits on the receive buffer - keeps polling the receive buffer every RECEIVE_BUFFER_POLLING_INTERVAL until there is sufficient data in the receive buffer.
When the server TCB’s state transitions to CLOSED after CLOSEWAIT_TIMEOUT, the receive buffer is cleared by setting its usedBufLen as 0.
When the server TCB is freed when srt_svr_close() is called, the mutex and receive buffer is freed.
In what follows, we discuss the FSM that runs inside the CONNECTED state on the client and server side.
Figure 3 shows the Go-Back-N event/actions that are triggered in the CONNECTED state of client FSM. When the client is in CONNECTED state and srt_client_send() function is called, data segments are created using the data passed to srt_client_send(). These data segments are encapsulated in segBuf structures and appended to the end of send buffer. If the send buffer is empty before appending the segBufs, a sendBuf_timer thread is started to poll the send buffer and trigger timeout events. The client then sends the segments from the first unsent segment in send buffer until the sent-but-not-Acked segments’ number reaches GBN_WINDOW.
When the client is in CONNECTED state and the send buffer is not empty, the sendBuf_timer thread keeps polling the first sent-but-not-Acked segment every SENDBUF_POLLING_INTERVAL timeout to check if: the current time - the first sent-but-not-Acked segment’s sent time is larger than DATA_TIMEOUT value. If this is the case, a timeout event is triggered and all the sent-but-not-Acked segments in send buffer are re-sent.
When the client is in CONNECTED state and it receives a DATAACK segment, all the Acked segments are removed from send buffer and freed. The Acked segments are those whose sequence numbers is smaller than the sequence number in the received DATAACK. The sequence number in the DATAACK indicates that the server has received all segments upto but not including the sequence number.
Figure4 shows the Go-Back-N event/actions that are triggered in the CONNECTED state of server FSM. When the server is in CONNECTED state and receives a DATA segment, there are two cases to consider: i) the DATA segment’s sequence number is the same as the server’s expect sequence number; or ii) the DATA segment’s sequence number is different from the server’s expect sequence number. If the DATA segment’s sequence number is different from the server’s expect sequence number, the server responses with a DATAACK segment which contains its original expect sequence number and discards the received DATA segment. By “different” we mean that the DATA segement received has a sequence number which is not the expected sequence number but likely a later segment sent by the client in its GBN_WINDOW but implying that there is a gap in the window: in essence the next DATA segment expected is lost but one of the segments that followed the expected segment from the client arrives at the server - inidicating a gap in the window. Note, that our design of the server does not buffer segments that are out of order or when gaps occur in the window.
If the DATA segment’s sequence number is the same as the server’s expect sequence number, the server extracts the data from DATA segment and stores the data into its receive buffer. The server’s expected sequence number is then increased by the received data size. Finally, the server sends a DATAACK which contains the newly computed expect sequence number back to the client.
There are a number of files needed for the lab5 assignment.
We provide two sets of applicartion client and server files for simple application client server and for stress test application client and server. The simple case will send short text buffers between the client and server application. The stress test case send a plain text file between the client and server; this last app. will test out your transport in a more comprehensive manner.
These are the files you need. Note, that you do not have to write any client or server code in this assignment. This allows you to focus on the implementation of the go-back-n the SRT client and server side code. The basic idea is that you first get your code running with the simple application data transfer code and if that works move to running the stress test against your code base.
Simple client and server code:
Stress test client and server code:
For the stress test you will need two put the following plain text file in the same directory as your app˙stress˙client
Client side data structures and prototypes needed to implement the client side SRT protocol
Server side data structures and prototypes needed to implement the server side SRT protocol
SNP APIs - header file. SRT header and segement data structure, SNP prototypes.
Constants
