redis (StackExchange.Redis) publish performance is poor? - performance

Question

I'm trying to use redis pub/sub to transfer data between apps with high velocity (25000 message per second).
I have tested it as below:
topology:
1 publisher, 1 subscriber and redis server. All are hosted in the same pc.
pc hardware:
CPU: Intel(R) Core(TM) I7-4578U CPU#3.00GHz
Memory: 16.0GB
code:
Stopwatch sw = new Stopwatch();
sw.Start();
while (_started)
{
//db.PublishAsync(RawMessagesCapturedMsg.TopicGroupName, redisValue);
db.Publish(RawMessagesCapturedMsg.TopicGroupName, redisValue);
totalRedisMsg++;
if (totalRedisMsg % 10000 == 0)
{
Console.WriteLine("totalRedisMsg: {0} # {1}, time used(ms): {2}",
totalRedisMsg, DateTime.Now, sw.ElapsedMilliseconds);
}
}
sw.Stop();
results:
As shown in the results, it will take about 6 second to publish 10k messages.
I want to confirm that is it the actual performance of redis (or StackExchange.Redis)? Or there is something wrong with my test?
Update:
According to the accepted answer, I found out the reason is my message size is too large (300kB).

Answer 1

A things to check:
What is the CPU load? Is it full? If not, you're probably stuck on bandwidth or latency.
How much is the size of the message? Multiply it by the transfer
rate you see, is it comparable to the bandwidth you (expect to)
have?
What is the ping to the Redis instance? Maybe round trips are taking much time. In that case you can use many threads with many connections to increase throughput.
I had a benchmark at my hand that I used to answer another question. And in Java (lettuce client library) I have this results for 1 thread, local cpu i5-6400, remote cpu E5-2603 v4, 0.180ms ping to remote and the message is "hello".
Benchmark (address) Mode Cnt Score Error Units
LettuceThreads.pooled socket thrpt 5 35699.267 ± 706.946 ops/s
LettuceThreads.pooled localhost thrpt 5 28130.801 ± 9476.584 ops/s
LettuceThreads.pooled remote thrpt 5 3080.115 ± 422.390 ops/s
LettuceThreads.shared socket thrpt 5 41717.332 ± 3559.226 ops/s
LettuceThreads.shared localhost thrpt 5 31092.925 ± 9894.748 ops/s
LettuceThreads.shared remote thrpt 5 3920.260 ± 178.637 ops/s
Compare it to the hardware you have, maybe it will help you evaluate your library performance. Note, how performance drops 10x for remote, even knowing that CPU is slower 2x, that is a lot.
And the following is for 16 threads. So, as you see, bigger number of threads may help at least get the throughput despite the latency.
Benchmark (address) Mode Cnt Score Error Units
LettuceThreads.pooled socket thrpt 5 123846.426 ± 2926.807 ops/s
LettuceThreads.pooled localhost thrpt 5 83997.678 ± 31410.595 ops/s
LettuceThreads.pooled remote thrpt 5 31045.111 ± 2198.065 ops/s
LettuceThreads.shared socket thrpt 5 218331.662 ± 17459.352 ops/s
LettuceThreads.shared localhost thrpt 5 182296.689 ± 52163.154 ops/s
LettuceThreads.shared remote thrpt 5 30803.575 ± 2128.306 ops/s