Improve latency for TCP by not waiting for Push flag
28 February 2012 by Snakefoot | Comment » | Trackback OffWhen communicating using TCP then the TCP layer attempts to optimize performance by bundling data into chunks. This works well when performing file transfers and other large data transfers. One of the flags in the TCP Header is the Push Flag (PSH bit).
The Push Flag is used by the sending side to mark that it is end of a data chunk (Ex. end of data header or data body being sent). This tells the TCP layer on the receiving side to flush the received data to the receiving application without waiting for the receive buffer to be filled. The setting of the Push Flag is usually not controlled by the sending application, but by the sending TCP layer. Most modern TCP/IP stacks set the PSH bit at the end of the buffer supplied to send() .The Push Flag helps the receiving side to optimize throughput by bundling data into chunks.
But the receiving application might want to get the data from the TCP layer as soon as it arrives, instead of having the data waiting in the TCP layer for buffer optimization.
If in a situation where the TCP layer is forced, and low latency is crucial, then the receiving application should recv() into a small buffer to improve latency. The receive buffer (provided by the receiving application) should not be confused with the receive window (provided by the receiving TCP layer).
The recv() / WSARecv() calls returns when one of the following events occurs:
The sending application can also fix the issue by ensuring that the TCP Socket Send Buffer is at least one byte larger than the User Send Buffer. This is done by calling setsockopt with a socket-buffer-size of 64 KByte, and calling WSASend with a user-buffer-size of 32 KByte. This will ensure that the TCP Push-bit is set for every TCP-send operations. More Info MS KB 823764
If unable to fix the sending application, then one can also use this registry key:
Related TCP_NODELAY disables nagle algorithm and can improve latency d08
The Push Flag is used by the sending side to mark that it is end of a data chunk (Ex. end of data header or data body being sent). This tells the TCP layer on the receiving side to flush the received data to the receiving application without waiting for the receive buffer to be filled. The setting of the Push Flag is usually not controlled by the sending application, but by the sending TCP layer. Most modern TCP/IP stacks set the PSH bit at the end of the buffer supplied to send() .The Push Flag helps the receiving side to optimize throughput by bundling data into chunks.
But the receiving application might want to get the data from the TCP layer as soon as it arrives, instead of having the data waiting in the TCP layer for buffer optimization.
- Streaming movies or music, where the application playing the stream needs the data as soon as possible.
- Low latency application like online games, where the application wants to be closely synchronized with the server as possible.
If in a situation where the TCP layer is forced, and low latency is crucial, then the receiving application should recv() into a small buffer to improve latency. The receive buffer (provided by the receiving application) should not be confused with the receive window (provided by the receiving TCP layer).
The recv() / WSARecv() calls returns when one of the following events occurs:
- Data arrives with the PUSH bit set.
- The user recv buffer is full.
- 0.5 seconds have elapsed since any data has arrived.
More Info MS KB Q126967[HKEY_LOCAL_MACHINE \System \CurrentControlSet \Services \Afd \Parameters]
IgnorePushBitOnReceives = 1
Note this will affect all TCP connections created by any application on the system.
The sending application can also fix the issue by ensuring that the TCP Socket Send Buffer is at least one byte larger than the User Send Buffer. This is done by calling setsockopt with a socket-buffer-size of 64 KByte, and calling WSASend with a user-buffer-size of 32 KByte. This will ensure that the TCP Push-bit is set for every TCP-send operations. More Info MS KB 823764
If unable to fix the sending application, then one can also use this registry key:
[HKEY_LOCAL_MACHINE \System \CurrentControlSet \Services \Afd \Parameters]
NonBlockingSendSpecialBuffering = 1
Note this will affect all TCP connections created by any application on the system.
Related TCP_NODELAY disables nagle algorithm and can improve latency d08
Tags:
Category:
Updated: 7 October 2012