Elastic Search ttl (time to live) in the dynamic java mapping file - Spring data elastic search - elasticsearch

Question

We use elastic search dynamic mapping and the java file is as follows.
#Document(indexName = "test", type = "test", shards = 1, replicas = 0)
public class ElasticSearchIndexObject {
private #Id
#Indexed
String id;
private #Indexed("name")
String name;
}
We use scheduler that runs at every 60 mins to fetch the data from the DB and to add to the index.
Connection conn = dataSource.getConnection();
stmt = conn.createStatement(java.sql.ResultSet.TYPE_FORWARD_ONLY, java.sql.ResultSet.CONCUR_READ_ONLY);
stmt.setFetchSize(FETCH_SIZE);
rs = stmt.executeQuery(ESEARCH_QUERY);
int i=1;
while (rs.next()) {
ElasticSearchIndexObject indexObj = new ElasticSearchIndexObject();
indexObj.setName(rs.getString("name"));
indexObj.setId(rs.getString("id"));
indexObjects.add(indexObj);
i=i+1;
}
elasticSearchObjectIndexRepository.save(indexObjects);
indexObjects.clear();
}
This scheduler runs every 60 mins and add/update the index.
Add - If the id is not there in the index
Update - If the id is already there in the index
Problem is with the deleted records in the database. These records are not getting deleted from the index and becomes an orphan records.
I came across "ttl" property and looking for a way to add this to the index so that the orphan records will get deleted after the ttl time.
If the ttl is not to add to each index, Should it be at the generic level for all the documents? If so, should i set this for the each schedule run?
Thanks,

Answer 1

Be sure your index type has its "_ttl" : { "enabled" : true } mapping already configured. Then pass the _ttl value for your document in _source. In your POJO add this field:
#JsonInclude(value=Include.NON_EMPTY) //to make it optional
#JsonProperty("_ttl")
private Long ttl;

Answer 2

According to this open issue it doesn't look like the _ttl field is currently supported by Spring Data Elasticsearch.
Another way of doing it is to "soft-delete" records from your database by setting a flag (i.e. a new boolean column). The flag would be true when the record is active and false when the record is deleted. That way when your import process runs, you'd get all records and based on that flag you know you have to delete the documents from Elasticsearch.