How to receive network messages with minimum delay

[Dave P]

We need to insure a high volume messaging application receives tcp messages
without any delay. The application runs on a machine with 8 hyperthreaded
cores. We would like the application to be able to respond to incoming
messages with sub millisecond response time. Would setting the priority of
the process to real time do it? Does the process priority apply to all
threads within the process? If there are more threads than cores could
windows then become un-responsive?

[m]

As I am sure you are aware, it is not possible to guarantee a
sub-millisecond response to anything on Windows, but assuming that your HW
is correctly sized, there is no interference from other applications, and
your application processing is inherently parallel and brief, you should not
have a problem achieving good performance. Also note that there is no such
thing as a TCP message as TCP is a stream protocol and while messages can be
sent over a stream, the stream itself has not concept of anything except the
bytes that are being sent / received. Thread priority is always set on a
thread by thread basis, but since you are asking this question, I recommend
that you never change it from the values Windows sets for you automatically.

Look at IO completion ports & thread pool functions in MSDN. Two keys for
you will be reducing the number of KM->UM transitions by using block IO and
reducing the number of memory copies by pending read buffers for the network
stack to fill as soon as stream data becomes available.

Post by Dave P
We need to insure a high volume messaging application receives tcp messages
without any delay. The application runs on a machine with 8 hyperthreaded
cores. We would like the application to be able to respond to incoming
messages with sub millisecond response time. Would setting the priority of
the process to real time do it? Does the process priority apply to all
threads within the process? If there are more threads than cores could
windows then become un-responsive?

[mosesvas]

Post by m
As I am sure you are aware, it is not possible to guarantee a
sub-millisecond response to anything on Windows, but assuming that your HW
is correctly sized, there is no interference from other applications, and
your application processing is inherently parallel and brief, you should not
have a problem achieving good performance.  Also note that there is no such
thing as a TCP message as TCP is a stream protocol and while messages can be
sent over a stream, the stream itself has not concept of anything except the
bytes that are being sent / received.  Thread priority is always set on a
thread by thread basis, but since you are asking this question, I recommend
that you never change it from the values Windows sets for you automatically.
Look at IO completion ports & thread pool functions in MSDN.  Two keys for
you will be reducing the number of KM->UM transitions by using block IO and
reducing the number of memory copies by pending read buffers for the network
stack to fill as soon as stream data becomes available.

Post by Dave P
We need to insure a high volume messaging application receives tcp messages
without any delay. The application runs on a machine with 8 hyperthreaded
cores. We would like the application to be able to respond to incoming
messages with sub millisecond response time. Would setting the priority of
the process to real time do it? Does the process priority apply to all
threads  within the process? If there are more threads than cores could
windows then become un-responsive?- Hide quoted text -

- Show quoted text -

Priority boost only gives you lots of cpu cycle for your application.
But you need use it efficiently to get
performance. If you need a more performance, check out Winsock Kernel
it can avoid unnecessary user to kernel mode switching for each &
every sock operation.
http://msdn.microsoft.com/en-us/library/aa504195.aspx

[m]

IMHO, the OP clearly can't handle KM programming and in any case programming
an application in KM should not be done outside of the lab. Besides, the
perf gain from switching from UM to KM is insignificant compared with the
gain from changing to a good design.

Post by mosesvas

Post by m
As I am sure you are aware, it is not possible to guarantee a
sub-millisecond response to anything on Windows, but assuming that your HW
is correctly sized, there is no interference from other applications, and
your application processing is inherently parallel and brief, you should not
have a problem achieving good performance. Also note that there is no such
thing as a TCP message as TCP is a stream protocol and while messages can be
sent over a stream, the stream itself has not concept of anything except the
bytes that are being sent / received. Thread priority is always set on a
thread by thread basis, but since you are asking this question, I recommend
that you never change it from the values Windows sets for you
automatically.
Look at IO completion ports & thread pool functions in MSDN. Two keys for
you will be reducing the number of KM->UM transitions by using block IO and
reducing the number of memory copies by pending read buffers for the network
stack to fill as soon as stream data becomes available.

Post by Dave P
We need to insure a high volume messaging application receives tcp messages
without any delay. The application runs on a machine with 8
hyperthreaded
cores. We would like the application to be able to respond to incoming
messages with sub millisecond response time. Would setting the priority of
the process to real time do it? Does the process priority apply to all
threads within the process? If there are more threads than cores could
windows then become un-responsive?- Hide quoted text -

- Show quoted text -

Priority boost only gives you lots of cpu cycle for your application.
But you need use it efficiently to get
performance. If you need a more performance, check out Winsock Kernel
it can avoid unnecessary user to kernel mode switching for each &
every sock operation.
http://msdn.microsoft.com/en-us/library/aa504195.aspx

[Pavel A.]

Post by Dave P
We need to insure a high volume messaging application receives tcp messages
without any delay.

Well, then, er, insure this.
"Without any delay" obviously is impossible.
Determine where are bottlenecks in processing, maybe run some simulation.
Maybe you'll need load balancing.

-- PA