MongDB sink connector: How to upsert deep objects - apache-kafka-connect

Question

I have a usecase of reading messages from kafka topics and loading them into MongoDB. As part of this process, I'm also looking to handle the data update part.
For example, consider this kafka message
{
"_id": "123",
"meta": {
"id": "456",
"name: "abc"
"lastname": "xyz"
}
}
After this is added to MongoDB sink, consider the next message
{
"meta": {
"id": "456",
"lastname": "oxy"
}
}
I'm expecting the sink connector to update only the lastname field without overwriting other fields and the document should look like
{
"_id": "123",
"meta": {
"id": "456",
"name: "abc"
"lastname": "oxy"
}
}
Basically, how to achieve this upsert functionality in MongoDB sink connector write model strategy? Here is the custom strategy and the sink configuration
public class UpsertAsPartOfDocumentStrategy implements WriteModelStrategy, Configurable {
private final static String ID_FIELD_NAME = "_id";
private boolean isPartialId = false;
private static final String CREATE_PREFIX = "%s%s.";
private static final String ELEMENT_NAME_PREFIX = "%s%s";
private static final UpdateOptions UPDATE_OPTIONS = new UpdateOptions().upsert(true);
static final String FIELD_NAME_MODIFIED_TS = "_modifiedTS";
static final String FIELD_NAME_INSERTED_TS = "_insertedTS";
#Override
public WriteModel<BsonDocument> createWriteModel(SinkDocument document) {
BsonDocument vd =
document
.getValueDoc()
.orElseThrow(
() ->
new DataException(
"Could not build the WriteModel,the value document was missing unexpectedly"));
BsonValue idValue = vd.get(ID_FIELD);
if (idValue == null || !idValue.isDocument()) {
throw new DataException(
"Could not build the WriteModel,the value document does not contain an _id field of"
+ " type BsonDocument which holds the business key fields.\n\n If you are including an"
+ " existing `_id` value in the business key then ensure `document.id.strategy.overwrite.existing=true`.");
}
BsonDocument businessKey = idValue.asDocument();
if (isPartialId) {
businessKey = flattenKeys(businessKey);
}
System.out.println("document" + vd);
return new UpdateOneModel<>(businessKey, vd, UPDATE_OPTIONS);
}
Reference: https://github.com/mongodb/mongo-kafka/blob/r1.7.0/src/main/java/com/mongodb/kafka/connect/sink/writemodel/strategy/UpdateOneBusinessKeyTimestampStrategy.java
https://www.mongodb.com/docs/drivers/go/current/fundamentals/crud/write-operations/upsert/
Sink properties
# Connection details
connector.class=com.mongodb.kafka.connect.MongoSinkConnector
connection.uri=mongodb://<connection>
tasks.max=1
topics=topc
database=db
collection=col
# Specific global MongoDB Sink Connector configuration
document.id.strategy.overwrite.existing=true
#writemodel.strategy=com.mongodb.kafka.connect.sink.writemodel.strategy.ReplaceOneBusinessKeyStrategy
writemodel.strategy=custom.writestrategy.UpsertAsPartOfDocumentStrategy
document.id.strategy=com.mongodb.kafka.connect.sink.processor.id.strategy.PartialValueStrategy
document.id.strategy.partial.value.projection.list=meta.id
document.id.strategy.partial.value.projection.type=AllowList
errors.tolerance=all
errors.deadletterqueue.topic.name=error_queue
errors.deadletterqueue.context.headers.enable=true
errors.log.include.messages=true