TPL Performance With Apache Kafka - performance

Question

Halo,
I'm unable to get any improved performance with TPL DataFlow and wondering if I'm using it incorrectly.
The application below does the following:
Pulls message from a Kafka topic
Parses this message into an Foo object with ParseData()
Serializes this Foo into JSON
Then publishes the JSON to a new Kafka topic.
Some single threaded stats:
ParseData can parse strings into Foo at 100 msg/sec (single threaded test)
SerializeMessage can do 200 Foos/sec (single threaded test)
Consuming Kafka messages (skipping all the parsing/serializing) can handle over 2000 msgs/sec
Based on this, I had hopes to leverage TPL for improving throughput. My max throughput should be close to the Kafka limit of 2000 msgs/sec.
However, I'm not seeing any improvements in throughput and I'm running the application on a machine with 12 physical cores (24 w HT). When I print out the size of the queue for each block, the transformBlock is always around 1000, but the others are under 10 which leads me to believe that the transformBlock isn't leveraging multi-core system.
Have I setup TPL DataFlow to leverage paralellism correctly?
app = new App();
await app.Start(new[]{"consume-topic"}, cancelSource);
// App class
async Task Start(IEnumerable<string> topics, CancellationTokenSource cancelSource) {
transformBlock = new TransformBlock<string, Foo>(TransformKafkaMessage,
new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = 8,
BoundedCapacity = 1000,
SingleProducerConstrained = true,
});
serializeBlock = new TransformBlock<Foo, string>(SerializeMessage,
new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = 4,
BoundedCapacity = 1000,
SingleProducerConstrained = true,
});
publishBlock = new ActionBlock<JsonMessage>(PublishJson,
new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = 1,
BoundedCapacity = 1000,
SingleProducerConstrained = true
});
// Setup the pipeline
transformBlock.LinkTo(serializeBlock);
serializeBlock.LinkTo(publishBlock);
// Start Kafka Listener loop
consumer.Subscribe(topics);
while(true) {
var result = consumer.Consume(cancelSource.Token);
await ProcessMessage<Ignore, string>(result);
}
}
// send the content of the kafka message to transform block
async Task ProcessMessage<TKey, TValue>(ConsumeResult<TKey, string> msg) {
var result = await transformBlock.SendAsync(msg.Value);
}
// Convert the raw string data into an object
Foo TransformKafkaMessage(string data) {
// Note this ParseData() function can process about 100 items per sec
// in local single threaded testing
Foo foo = ParseData(data);
return foo;
}
// Serialize the new Foo into JSON
string SerializeMessage(Foo foo) {
// The serializer can process about 200 msgs/sec (single threaded test)
var json = foo.Serialize();
return json;
}
// publish new message back to Kafka
void PublishJson(string json) {
// Create a Confluent.Kafka Message
var kafkaMessage = new Message<Null, string> {
Value = json
};
producer.Produce("produce-topic", kafkaMessage);
}