Masstransit Deserialization Issue With Encryption enabled in Amazon SQS - masstransit

Question

Object deserialization is working without encryption in both RabbitMQ and Amazon SQS in Masstransit and also Object deserialization working fine in case of encryption enabled with RabbitMQ, But getting deserialization error in case of encryption enabled with Amazon SQS.
Getting below error while deserialization :
System.Runtime.Serialization.SerializationException: An exception occurred while deserializing the message envelope
---> System.ArgumentOutOfRangeException: Unexpected BsonType value: 38 (Parameter 'type')
at Newtonsoft.Json.Bson.BsonDataReader.ReadType(BsonType type)
at Newtonsoft.Json.Bson.BsonDataReader.ReadNormal()
at Newtonsoft.Json.Bson.BsonDataReader.Read()
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue)
at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
at MassTransit.Serialization.EncryptedMessageDeserializerV2.Deserialize(MessageBody body, Headers headers, Uri destinationAddress) in /_/src/MassTransit.Newtonsoft/Serialization/EncryptedMessageDeserializerV2.cs:line 47
--- End of inner exception stack trace ---
at MassTransit.Serialization.EncryptedMessageDeserializerV2.Deserialize(MessageBody body, Headers headers, Uri destinationAddress) in /_/src/MassTransit.Newtonsoft/Serialization/EncryptedMessageDeserializerV2.cs:line 62
at MassTransit.Serialization.EncryptedMessageDeserializerV2.Deserialize(ReceiveContext receiveContext) in /_/src/MassTransit.Newtonsoft/Serialization/EncryptedMessageDeserializerV2.cs:line 36
at MassTransit.Middleware.DeserializeFilter.Send(ReceiveContext context, IPipe`1 next) in /_/src/MassTransit/Middleware/DeserializeFilter.cs:line 36
at MassTransit.Middleware.RescueFilter`2.MassTransit.IFilter<TContext>.Send(TContext context, IPipe`1 next) in /_/src/MassTransit/Middleware/RescueFilter.cs:line 43
Below Are Amazon SQS Configuration :
public class Program
{
public static async Task Main(string[] args)
{
await Host.CreateDefaultBuilder(args)
.ConfigureServices((hostContext, services) =>
{
services.AddMassTransit(x =>
{
x.AddConsumers(typeof(SampleCommandConsumer).Assembly);
x.UsingAmazonSqs(
(context, sqsConfig) =>
{
sqsConfig.Host(
Environment.GetEnvironmentVariable("AWS_DEFAULT_REGION"),
h =>
{
var accessKey = Environment.GetEnvironmentVariable("AWS_ACCESS_KEY_ID");
h.AccessKey(accessKey);
var env = Environment.GetEnvironmentVariable("AWS_ENV_PREFIX");
h.Scope(env, true);
var secret = Environment.GetEnvironmentVariable("AWS_SECRET_ACCESS_KEY");
var serviceUrl = Environment.GetEnvironmentVariable("AWS_SQS_URL");
if (!string.IsNullOrEmpty(secret))
{
h.SecretKey(secret);
}
if (!string.IsNullOrEmpty(serviceUrl))
{
h.Config(new AmazonSQSConfig { ServiceURL = serviceUrl });
}
});
byte[] key = { 156, 62, 75, 207, 47, 62, 42, 73, 53, 164, 114, 49, 46, 226, 164, 1, 79, 225, 65, 193, 59, 192, 72, 27, 234, 87, 17, 44, 46, 207, 137, 1 };
sqsConfig.UseJsonSerializer();
sqsConfig.UseJsonDeserializer();
sqsConfig.UseEncryption(key);
var prefix = $"{Environment.GetEnvironmentVariable("ENVIRONMENT")}_{Environment.GetEnvironmentVariable("SERVICE_ACCOUNT_CLIENT_ID")}";
sqsConfig.ConfigureEndpoints(
context,
new KebabCaseEndpointNameFormatter(prefix, false)
);
}
);
});
services.AddHostedService<Publisher>();
})
.Build()
.RunAsync();
}
}
Publish Contract:
public class SampleCommand
{
/// <summary>
/// The correlation id events published as a result of processing this command must include.
/// </summary>
public Guid CorrelationId { get; init; }
/// <summary>
/// The identifer of the Sender that is to be authorized to use the listed message features.
/// </summary>
public long SenderId { get; set; }
}
Consumer:
public class SampleCommandConsumer : IConsumer<SampleCommand>
{
readonly IBus bus;
/// <summary>
/// Initializes a new instance of the <see cref="MessageDeliveredConsumer" /> class.
/// </summary>
public SampleCommandConsumer(IBus bus)
{
this.bus = bus;
}
public async Task Consume(ConsumeContext<SampleCommand> context)
{
await context.Publish(new SampleEvent()
{
CorrelationId = context.CorrelationId.Value,
});
}
}
configured Amazon SQS as per masstransit documentation, but didn't workout

Answer 1

Amazon SQS doesn't allow binary messages, neither does SNS. You should enable encryption at the queue level in SQS if you want message encryption.
You can configure queue encryption using a configuration callback:
x.AddConfigureEndpointCallback((_,cfg) =>
{
if (cfg is IAmazonSqsReceiveEndpointConfigurator sqs)
sqs.QueueAttributes.Add(QueueAttributeName.KmsMasterKeyId, "arn of key");
});