Mongo Spark Java Connector Group by - spring-boot

Question

i am storing events on my server from client mobile application, event store is mongodb.
I have mongo-spark connector which fetches list of these events and should display them using rest api. It should be streaming for later, but for now i am trying to display it as a single call.
So far i have written my controller as given below:
#RestController
#RequestMapping("/analytics")
class EventController #Autowired constructor(val eventMongoServiceImpl: EventMongoServiceImpl,
val javaSparkContext: JavaSparkContext) {
#GetMapping("/event")
fun getEvent(): ResponseEntity<EventResponse> {
val customRdd: JavaMongoRDD<Document> = MongoSpark.load(javaSparkContext)
val toDF = customRdd.toDF()
}
}
Please help me filter these result given below for rest api:
[
{
"key": "Event A",
"description": "Event A Description",
"count": 3
},
{
"key": "Event B",
"description": "Event B Description",
"count": 0
}
]
I have a data set as given below:
/* 1 */
{
"_id" : ObjectId("5e61e38eb8425d3b1c7679ea"),
"name" : "Event A",
"description" : "Event A Description",
"date" : ISODate("2020-03-05T18:30:00.000Z"),
"_class" : "x"
}
/* 2 */
{
"_id" : ObjectId("5e61e416b8425d3b1c7679ec"),
"name" : "Event A",
"description" : "Event A Description",
"date" : ISODate("2020-03-05T18:30:00.000Z"),
"_class" : "x"
}
/* 3 */
{
"_id" : ObjectId("5e61e47fb8425d3b1c7679ee"),
"name" : "Event A",
"description" : "Event A Description",
"date" : ISODate("2020-03-05T18:30:00.000Z"),
"_class" : "x"
}

Answer 1

You should be able on the dataframe to make something like this
val aggDf = toDf
.groupBy("name")
.agg(count("name"), max("description"))
Now on the new dataframe aggDf you can do aggDf.toJson and have the results. If columns does not match the output you can tweak them with withColumnRenamed