I have a Http server. When client send a http request to the server, the server side will hold the http connection and send chunked string infinite to the client. I know it will be better using websocket in today, but it is a old project, and I can't change the server side code.
// server.kt
package com.example.long_http
import io.vertx.core.AbstractVerticle
import io.vertx.core.Promise
import io.vertx.core.Vertx
class MainVerticle : AbstractVerticle() {
override fun start(startPromise: Promise<Void>) {
vertx
.createHttpServer()
.requestHandler { req ->
var i = 0
req.response().setChunked(true).putHeader("Content-Type", "text/plain")
val timer = vertx.setPeriodic(2000) {
req.response().write("hello ${System.currentTimeMillis()}")
println("write ${System.currentTimeMillis()}")
}
req.response().closeHandler {
vertx.cancelTimer(timer)
println("close")
}
}
.listen(8888) { http ->
if (http.succeeded()) {
startPromise.complete()
println("HTTP server started on port 8888")
} else {
startPromise.fail(http.cause());
}
}
}
}
fun main() {
Vertx.vertx().deployVerticle(MainVerticle())
}
I try to receive chunked string using okhttp, but it dont work.
// client.kt
package com.example.long_http
import okhttp3.*
import java.io.IOException
fun main() {
val client = OkHttpClient()
val request = Request.Builder().url("http://localhost:8888").build()
client.newCall(request).enqueue(handler())
}
class handler : Callback {
override fun onFailure(call: Call, e: IOException) {
e.printStackTrace()
}
override fun onResponse(call: Call, response: Response) {
println("onResponse")
val stream = response.body!!.byteStream().bufferedReader()
while (true) {
var line = stream.readLine()
println(line)
}
}
}
I am trying to handle closing web socket session in WebSocketHandler. My intuition was to do it in this way:
webSocketClient.execute(
URI.create("some-ws-endpoint")
) { session: WebSocketSession ->
session.receive()
.doOnEach { action(it) }
.then()
.doFinally { session.close() }
}
but I cannot reach doFinally block from Mono<Void> returned by webSocketClient.execute. My full test code for this case is:
fun test() = runBlocking {
val webSocketClient: WebSocketClient = StandardWebSocketClient()
val subscription = webSocketClient.execute(
URI.create("some-ws-endpoint")
) { session: WebSocketSession ->
session.receive()
.doOnEach { println("Message: $it") }
.then()
.doFinally { println("finally") }
}.subscribe()
delay(20000)
subscription.dispose()
delay(5000)
}
from which I have Messages printed, but finally is never shown on my console. From the other hand when I tried to do it on plain reactor-core components, everything works just fine:
runBlocking {
val publisher: Flux<Long> = Flux.interval(Duration.ofSeconds(1))
val subscription = publisher
.doOnEach { println("Value: $it") }
.then()
.doFinally { println("in doFinally") }
.subscribe()
delay(5_000)
subscription.dispose()
delay(1_000)
}
I am new to both WebSockets and Project Reactor, so maybe I am doing some basic mistake. Does anyone see what is wrong with my code?
I'm building a tarball dynamically, and would like to stream it back directly, which should be 100% possible with a .tar.gz.
The below code is the closest thing I could get to a dataBuffer, through lots of googling. Basically, I need something that implements an OutputStream and provides, or publishes, to a Flux<DataBuffer> so that I can return that from my method, and have streaming output, instead of buffering the entire tarball in ram (which I'm pretty sure is what is happening here). I'm using apache Compress-commons, which has a wonderful API, but it's all OutputStream based.
I suppose another way to do it would be to directly write to the response, but I don't think that would be properly reactive? Not sure how to get an OutputStream out of some sort of Response object either.
This is kotlin btw, on Spring Boot 2.0
#GetMapping("/cookbook.tar.gz", "/cookbook")
fun getCookbook(): Mono<DefaultDataBuffer> {
log.info("Creating tarball of cookbooks: ${soloConfig.cookbookPaths}")
val transformation = Mono.just(soloConfig.cookbookPaths.stream()
.toList()
.flatMap {
Files.walk(Paths.get(it)).map(Path::toFile).toList()
})
.map { files ->
//Will make one giant databuffer... but oh well? TODO: maybe use some kind of chunking.
val buffer = DefaultDataBufferFactory().allocateBuffer()
val outputBufferStream = buffer.asOutputStream()
//Transform my list of stuff into an archiveOutputStream
TarArchiveOutputStream(GzipCompressorOutputStream(outputBufferStream)).use { taos ->
taos.setLongFileMode(TarArchiveOutputStream.LONGFILE_GNU)
log.info("files to compress: ${files}")
for (file in files) {
if (file.isFile) {
val entry = "cookbooks/" + file.name
log.info("Adding ${entry} to tarball")
taos.putArchiveEntry(TarArchiveEntry(file, entry))
FileInputStream(file).use { fis ->
fis.copyTo(taos) //Copy that stuff!
}
taos.closeArchiveEntry()
}
}
}
buffer
}
return transformation
}
I puzzled through this, and have an effective solution. You implement an OutputStream and take those bytes and publish them into a stream. Be sure to override close, and send an onComplete. Works great!
#RestController
class SoloController(
val soloConfig: SoloConfig
) {
val log = KotlinLogging.logger { }
#GetMapping("/cookbooks.tar.gz", "/cookbooks")
fun streamCookbook(serverHttpResponse: ServerHttpResponse): Flux<DataBuffer> {
log.info("Creating tarball of cookbooks: ${soloConfig.cookbookPaths}")
val publishingOutputStream = PublishingOutputStream(serverHttpResponse.bufferFactory())
//Needs to set up cookbook path as a parent directory, and then do `cookbooks/$cookbook_path/<all files>` for each cookbook path given
Flux.just(soloConfig.cookbookPaths.stream().toList())
.doOnNext { paths ->
//Transform my list of stuff into an archiveOutputStream
TarArchiveOutputStream(GzipCompressorOutputStream(publishingOutputStream)).use { taos ->
taos.setLongFileMode(TarArchiveOutputStream.LONGFILE_GNU)
paths.forEach { cookbookDir ->
if (Paths.get(cookbookDir).toFile().isDirectory) {
val cookbookDirFile = Paths.get(cookbookDir).toFile()
val directoryName = cookbookDirFile.name
val entryStart = "cookbooks/${directoryName}"
val files = Files.walk(cookbookDirFile.toPath()).map(Path::toFile).toList()
log.info("${files.size} files to compress")
for (file in files) {
if (file.isFile) {
val relativePath = file.toRelativeString(cookbookDirFile)
val entry = "$entryStart/$relativePath"
taos.putArchiveEntry(TarArchiveEntry(file, entry))
FileInputStream(file).use { fis ->
fis.copyTo(taos) //Copy that stuff!
}
taos.closeArchiveEntry()
}
}
}
}
}
}
.subscribeOn(Schedulers.parallel())
.doOnComplete {
publishingOutputStream.close()
}
.subscribe()
return publishingOutputStream.publisher
}
class PublishingOutputStream(bufferFactory: DataBufferFactory) : OutputStream() {
val publisher: UnicastProcessor<DataBuffer> = UnicastProcessor.create(Queues.unbounded<DataBuffer>().get())
private val bufferPublisher: UnicastProcessor<Byte> = UnicastProcessor.create(Queues.unbounded<Byte>().get())
init {
bufferPublisher
.bufferTimeout(4096, Duration.ofMillis(100))
.doOnNext { intList ->
val buffer = bufferFactory.allocateBuffer(intList.size)
buffer.write(intList.toByteArray())
publisher.onNext(buffer)
}
.doOnComplete {
publisher.onComplete()
}
.subscribeOn(Schedulers.newSingle("publisherThread"))
.subscribe()
}
override fun write(b: Int) {
bufferPublisher.onNext(b.toByte())
}
override fun close() {
bufferPublisher.onComplete() //which should trigger the clean up of the whole thing
}
}
}
I know the client and server are connecting because my connect/disconnect events are firing. However, my custom events are not. I am using socket.io java client, and netty-socketio on the server. I usually use the socket.io javascript library which works seamlessly, so I am a bit lost as to why this is happening. I am writing this in Kotlin.
Client-Side
fun connectToServer(ipAddress : String)
{
socket = IO.socket("$ipAddress")
socket!!.on(Socket.EVENT_CONNECT) { obj ->
println("Connected To Server!!!")
}.on(EventNames.signOn) { obj ->
println(EventNames.signOn)
//cast value to string from server, hope for encrypted password
val encryptedPassword = obj[0] as String
when(encryptedPassword)
{
"no user" -> {
}
else -> {
val result = encryptedPassword!!.split("OR")
val isMatch = passwordTextField.text == dataProcessing.Encryption3().decryptValue("decrypt", result[0],result[1])
if(isMatch)
{
}
}
}
println("Encrypted Password: "+encryptedPassword)
}
// socket!!.on(Socket.EVENT_DISCONNECT, object : Emitter.Listener {
//
// override fun call(vararg args: Any) {}
//
// })
socket!!.connect()
// socket!!.open()
// socket!!.emit(Socket.EVENT_CONNECT, "Hello!")
socket!!.send("hey")
socket!!.emit(EventNames.requestClientSignOn, usernameTextField.text)
}
Server-Side
#Throws(InterruptedException::class, UnsupportedEncodingException::class)
fun server()
{
val config = Configuration()
config.setHostname("localhost")
config.setPort(PORT)
server = SocketIOServer(config)
server!!.addConnectListener {
println("Hello World!")
}
server!!.addEventListener(EventNames.requestClientSignOn, String::class.java) { client, data, ackRequest ->
println("Hello from requestClientSignOn..")
}
server!!.addDisconnectListener {
println("Client Disconnecting...")
}
server!!.addConnectListener {
println("client connected!! client: $it")
}
server!!.start()
You cannot use lambda expression in your event listeners, using netty-socketio on the sever.
Using the traditional EventListener solves this problem. I also converted the server to Kotlin, as it was easier to use the demo project as a reference.
server.addEventListener(EventNames.requestClientSignOn, String.class, new DataListener<String>() {
#Override
public void onData(SocketIOClient client, String username, AckRequest ackRequest) {
String isEncryptedPassword = new KOTS_EmployeeManager().getKOTS_User(KOTS_EmployeeManager.kotsUserType.CLIENT, username)
if(isEncryptedPassword != null)
{
//send back ack with encrypted password
ackRequest.sendAckData(isEncryptedPassword);
}else{
//send back ack with no user string
ackRequest.sendAckData("no user");
}
}
});
I'm building an API with Akka that should produce to a Kafka bus. I have been load testing the application using Gatling. Noticed that when more than 1000 users are created in Gatling, the API starts to struggle. On average, about 170 requests per second are handled, which seems like very little to me.
The API's main entry point is this:
import akka.actor.{Props, ActorSystem}
import akka.http.scaladsl.Http
import akka.http.scaladsl.model._
import akka.pattern.ask
import akka.http.scaladsl.server.Directives
import akka.http.scaladsl.unmarshalling.Unmarshaller
import akka.stream.ActorMaterializer
import com.typesafe.config.{Config, ConfigFactory}
import play.api.libs.json.{JsObject, Json}
import scala.concurrent.{Future, ExecutionContext}
import akka.http.scaladsl.server.Directives._
import akka.util.Timeout
import scala.concurrent.duration._
import ExecutionContext.Implicits.global
case class PostMsg(msg:JsObject)
case object PostSuccess
case class PostFailure(msg:String)
class Msgapi(conf:Config) {
implicit val um:Unmarshaller[HttpEntity, JsObject] = {
Unmarshaller.byteStringUnmarshaller.mapWithCharset { (data, charset) =>
Json.parse(data.toArray).asInstanceOf[JsObject]
}
}
implicit val system = ActorSystem("MsgApi")
implicit val timeout = Timeout(5 seconds)
implicit val materializer = ActorMaterializer()
val router = system.actorOf(Props(new RouterActor(conf)))
val route = {
path("msg") {
post {
entity(as[JsObject]) {obj =>
if(!obj.keys.contains("key1") || !obj.keys.contains("key2") || !obj.keys.contains("key3")){
complete{
HttpResponse(status=StatusCodes.BadRequest, entity="Invalid json provided. Required fields: key1, key2, key3 \n")
}
} else {
onSuccess(router ? PostMsg(obj)){
case PostSuccess => {
complete{
Future{
HttpResponse(status = StatusCodes.OK, entity = "Post success")
}
}
}
case PostFailure(msg) =>{
complete{
Future{
HttpResponse(status = StatusCodes.InternalServerError, entity=msg)
}
}
}
case _ => {
complete{
Future{
HttpResponse(status = StatusCodes.InternalServerError, entity = "Unknown Server error occurred.")
}
}
}
}
}
}
}
}
}
def run():Unit = {
Http().bindAndHandle(route, interface = conf.getString("http.host"), port = conf.getInt("http.port"))
}
}
object RunMsgapi {
def main(Args: Array[String]):Unit = {
val conf = ConfigFactory.load()
val api = new Msgapi(conf)
api.run()
}
}
The router actor is as follows:
import akka.actor.{ActorSystem, Props, Actor}
import akka.http.scaladsl.server.RequestContext
import akka.routing.{Router, SmallestMailboxRoutingLogic, ActorRefRoutee}
import com.typesafe.config.Config
import play.api.libs.json.JsObject
class RouterActor(conf:Config) extends Actor{
val router = {
val routees = Vector.tabulate(conf.getInt("kafka.producer-number"))(n => {
val r = context.system.actorOf(Props(new KafkaProducerActor(conf, n )))
ActorRefRoutee(r)
})
Router(SmallestMailboxRoutingLogic(), routees)
}
def receive = {
case PostMsg(msg) => {
router.route(PostMsg(msg), sender())
}
}
}
And finally, the kafka producer actor:
import akka.actor.Actor
import java.util.Properties
import com.typesafe.config.Config
import kafka.message.NoCompressionCodec
import kafka.utils.Logging
import org.apache.kafka.clients.producer._
import play.api.libs.json.JsObject
import scala.concurrent.duration._
import scala.concurrent.{ExecutionContext, Future, Await}
import ExecutionContext.Implicits.global
import scala.concurrent.{Future, Await}
import scala.util.{Failure, Success}
class KafkaProducerActor(conf:Config, id:Int) extends Actor with Logging {
var topic: String = conf.getString("kafka.topic")
val codec = NoCompressionCodec.codec
val props = new Properties()
props.put("bootstrap.servers", conf.getString("kafka.bootstrap-servers"))
props.put("acks", conf.getString("kafka.acks"))
props.put("retries", conf.getString("kafka.retries"))
props.put("batch.size", conf.getString("kafka.batch-size"))
props.put("linger.ms", conf.getString("kafka.linger-ms"))
props.put("buffer.memory", conf.getString("kafka.buffer-memory"))
props.put("key.serializer", conf.getString("kafka.key-serializer"))
props.put("value.serializer", conf.getString("kafka.value-serializer"))
val producer = new KafkaProducer[String, String](props)
def receive = {
case PostMsg(msg) => {
// push the msg to Kafka
try{
val res = Future{
producer.send(new ProducerRecord[String, String](topic, msg.toString()))
}
val result = Await.result(res, 1 second).get()
sender ! PostSuccess
} catch{
case e: Exception => {
println(e.printStackTrace())
sender ! PostFailure("Kafka push error")
}
}
}
}
}
The idea being that in application.conf I can easily specify how many producers there should be, allowing better horizontal scaling.
Now, however, it seems that the api or router is actually the bottleneck. As a test, I disabled the Kafka producing code, and replaced it with a simple: sender ! PostSuccess. With 3000 users in Gatling, I still had 6% of requests failing due to timeouts, which seems like a very long time to me.
The Gatling test I am executing is the following:
import io.gatling.core.Predef._ // 2
import io.gatling.http.Predef._
import scala.concurrent.duration._
class BasicSimulation extends Simulation { // 3
val httpConf = http // 4
.baseURL("http://localhost:8080") // 5
.acceptHeader("text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8") // 6
.doNotTrackHeader("1")
.acceptLanguageHeader("en-US,en;q=0.5")
.acceptEncodingHeader("gzip, deflate")
.userAgentHeader("Mozilla/5.0 (Windows NT 5.1; rv:31.0) Gecko/20100101 Firefox/31.0")
.header("Content-Type", "application/json")
val scn = scenario("MsgLoadTest")
.repeat(100)(
pace(2 seconds)
.exec(http("request_1")
.post("/msg").body(StringBody("""{ "key1":"something", "key2": "somethingElse", "key3":2222}""")).asJSON)
)
setUp( // 11
scn.inject(rampUsers(3000) over (5 seconds)) // 12
).protocols(httpConf) // 13
}
update
Following some pointers from cmbaxter, I tried some things (see discussion in comments), and profiled the application using visualvm during the gatling load test. I don't quite know how to interpret these results though. It seems that a lot of time is spent in the ThreadPoolExecutor, but this might be ok?
Two screenshots from the profiling are below:
To exclude the Kafka producer, I removed the logic from the Actor. I was still getting performance issues. So, as a final test, I reworked the API to simply give a direct answer when a POST came in:
val route = {
path("msg") {
post {
entity(as[String]) { obj =>
complete(
HttpResponse(status = StatusCodes.OK, entity = "OK")
)
}
}
}
}
and I implemented the same route in Spray, to compare performance. The results were clear. Akka HTTP (at least in this current test setup) does not come close to Spray's performance. Perhaps there is some tweaking that can be done for Akka HTTP? I have attached two screenshots of response time graphs for 3000 concurrent users in Gatling, making a post request.
Akka HTTP
Spray
I would eliminate the KafkaProducerActor and router completely and call a Scala wrapped version of producer.send directly. Why create a possible bottleneck if not necessary? I could very well imagine the global execution context or the actor system becoming a bottleneck in your current setup.
Something like this should do the trick:
class KafkaScalaProducer(val producer : KafkaProducer[String, String](props)) {
def send(topic: String, msg : String) : Future[RecordMetadata] = {
val promise = Promise[RecordMetadata]()
try {
producer.send(new ProducerRecord[String, String](topic, msg), new Callback {
override def onCompletion(md : RecordMetadata, e : java.lang.Exception) {
if (md == null) promise.success(md)
else promise.failure(e)
}
})
} catch {
case e : BufferExhaustedException => promise.failure(e)
case e : KafkaException => promise.failure(e)
}
promise.future
}
def close = producer.close
}
(note: I have not actually tried this code. It should be interpreted as pseudo-code)
I would then simply transform the result of the future to a HttpResponse.
After that it's a question of tweaking configuration. Your bottleneck is now either the Kafka Producer or Akka Http.