SSE server sending events in a batch on final close - jersey

I have a Jersey server running locally, it exposes a SSE resource just like the examples here: https://jersey.github.io/documentation/latest/sse.html. I have a local webpack Angular app, that binds to the exposed GET endpoint and listens for data.
On the GET, I start up a thread to send notifications at regular intervals over 6-8 seconds. I don't see anything on the client UNTIL the EventOutput object is closed.
What am I doing wrong, and how can I fix this?
The server code WORKS with just a simple curl, i.e.:
curl http://localhost:8002/api/v1/notify
But on both Chrome and Safari the following code exhibits the behavior
Client (TypeScript):
this.evSource = new EventSource('http://localhost:8002/api/v1/notify');
this.evSource.addEventListener(
'event',
(x => console.log('we have ', x))
);
this.evSource.onmessage = (data => console.log(data));
this.evSource.onopen = (data => console.log(data));
this.evSource.onerror = (data => {
console.log(data);
this.evSource.close();
});
Server (Java):
// cache callback
public void eventCallback(Iterable<CacheEntryEvent<? extends Integer, ? extends Integer>> events) {
for (CacheEntryEvent<? extends Integer, ? extends Integer> x : events) {
LOGGER.info("{} Sending the following value: " + x.getValue(), Thread.currentThread().getId());
final OutboundEvent sseEvent = new OutboundEvent.Builder().name("event")
.data(Integer.class, x.getValue()).build();
this.broadcaster.broadcast(sseEvent);
}
}
#GET
#Produces(SseFeature.SERVER_SENT_EVENTS)
#ApiOperation(value = "Setup SSE pipeline", notes = "Sets up the notification pipeline for clients to access")
#ApiResponses(value = {
#ApiResponse(code = HttpURLConnection.HTTP_UNAUTHORIZED,
message = "Missing, bad or untrusted cookie"),
#ApiResponse(code = HttpURLConnection.HTTP_OK,
message = "Events streamed successfully")
})
#Timed
#ResponseMetered
public EventOutput registerNotificationEvents(
#HeaderParam(SseFeature.LAST_EVENT_ID_HEADER) String lastEventId,
#QueryParam(SseFeature.LAST_EVENT_ID_HEADER) String lastEventIdQuery) {
if (!Strings.isNullOrEmpty(lastEventId) || !Strings.isNullOrEmpty(lastEventIdQuery)) {
LOGGER.info("Found Last-Event-ID header: {}", !Strings.isNullOrEmpty(lastEventId) ? lastEventId : lastEventIdQuery );
}
LOGGER.info("{} Received request", Thread.currentThread().getId());
this.continuation = true;
final EventOutput output = new EventOutput();
broadcaster.add(output);
Random rand = new Random();
IntStream rndStream = IntStream.generate(() -> rand.nextInt(90));
List<Integer> lottery = rndStream.limit(15).boxed().collect(Collectors.toList());
IgniteCache<Integer, Integer> cache = this.ignite.cache(topic_name);
executorService.execute(() -> {
try {
lottery.forEach(value -> {
try {
TimeUnit.MILLISECONDS.sleep(500);
LOGGER.info("{} Sending the following value to Ignite: " + value + " : " + count++, Thread.currentThread().getId());
if (!cache.isClosed()) {
cache.put(1, value);
}
} catch (InterruptedException ex) {
ex.printStackTrace();
}
});
TimeUnit.MILLISECONDS.sleep(500);
continuation = false;
TimeUnit.MILLISECONDS.sleep(500);
if (!output.isClosed()) {
// THIS is where the client sees ALL the data broadcast
// in one shot
output.close();
}
} catch (InterruptedException ex) {
ex.printStackTrace();
} catch (IOException ex) {
ex.printStackTrace();
}
});
LOGGER.info("{} Completing request", Thread.currentThread().getId());
return output;
}
}

Looks like http://github.com/dropwizard/dropwizard/issues/1673 captures the problem. GZip default won't flush even if upper levels ask for it. Solution is something like
((AbstractServerFactory)configuration.getServerFactory()).getGzipFilterFactory().setSyncFlush(true);
will enable flushing to synchronize with GZip if disabling GZip all up is not an option

Related

Incorrect file being produced using websockets in helidon

I am trying to upload a file using websockets in Helidon.I think i am doing it write the right way but the code seems to be flaky in terms of the size of the file produced which is different. The size of the file being produced is different for different runs.
How can i make sure that the file size is same on both ends?
I use a simple protocol for handshake[code below]:
Step1 client sends filesize=11000 buffer=5000
Step2 server sends SENDFILE
Step3 client >> buffer 1 server >> write 1 5000
Step4 client >> buffer 2 server >> write 2 5000
Step5 client >> buffer 3 server >> write 3 1000
Step6 client sends ENDOFFILE server >> session.close
//SERVER side OnOpen session below
session.addMessageHandler(new MessageHandler.Whole<String>() {
#Override
public void onMessage(String message) {
System.out.println("Server >> " + message);
if (message.contains("FILESIZE")) {
session.getBasicRemote().sendText("SENDFILENOW");
}
if(message.contains("ENDOFFILE")) {
System.out.println("Server >> FILE_SIZE=" + FILE_SIZE);
finalFileOutputStream.close();
session.close();
}
}
});
session.addMessageHandler(new MessageHandler.Whole<ByteBuffer>() {
#Override
public void onMessage(ByteBuffer b) {
finalFileOutputStream.write(b.array(), 0, b.array().length);
finalFileOutputStream.flush();
}
});
//CLIENT OnOpen session below
session.getBasicRemote().sendText("FILESIZE=" + FILE_SIZE);
session.addMessageHandler(new MessageHandler.Whole<String>() {
#Override
public void onMessage(String message) {
long M = FILE_SIZE / BUFFER_SIZE;
long R = FILE_SIZE % BUFFER_SIZE;
if(!message.equals("SENDFILENOW"))
return;
try {
System.out.println("Starting File read ... " + path + " " + FILE_SIZE + " " + M + " " +message );
byte[] buffer = new byte[(int) BUFFER_SIZE];
while (M > 0) {
fileInputStream.read(buffer);
ByteBuffer bytebuffer = ByteBuffer.wrap(buffer);
session.getBasicRemote().sendBinary(bytebuffer);
M--;
}
buffer = new byte[(int) R];
fileInputStream.read(buffer, 0, (int) R);
fileInputStream.close();
ByteBuffer bytebuffer = ByteBuffer.wrap(buffer);
session.getBasicRemote().sendBinary(bytebuffer);
session.getBasicRemote().sendText("FILEREADDONE");
session.close();
f.complete(true);
} catch (IOException e) {
fail("Unexpected exception " + e);
}
}
});
Your solution is unnecessarily built on top of several levels of abstraction just to use websockets. Do you really need that? Helidon is very well equipped to handle huge file upload directly and much more efficiently.
public class LargeUpload {
public static void main(String[] args) {
ExecutorService executor = ThreadPoolSupplier.create("upload-thread-pool").get();
WebServer server = WebServer.builder(Routing.builder()
.post("/streamUpload", (req, res) -> req.content()
.map(DataChunk::data)
.flatMapIterable(Arrays::asList)
.to(IoMulti.writeToFile(createFile(req.queryParams().first("fileName").orElse("bigFile.mkv")))
.executor(executor)
.build())
.onError(res::send)
.onComplete(() -> {
res.status(Http.Status.ACCEPTED_202);
res.send();
}).ignoreElement())
.build())
.port(8080)
.build()
.start()
.await(Duration.ofSeconds(10));
// Server started - do upload
//several gigs file
Path file = Path.of("/home/kec/helidon-kafka.mkv");
try (FileInputStream fis = new FileInputStream(file.toFile())) {
WebClient.builder()
.baseUri("http://localhost:8080")
.build()
.post()
.path("/streamUpload")
.queryParam("fileName", "bigFile_" + System.currentTimeMillis() + ".mkv")
.contentType(MediaType.APPLICATION_OCTET_STREAM)
.submit(IoMulti.multiFromByteChannelBuilder(fis.getChannel())
.bufferCapacity(1024 * 1024 * 4)
.build()
.map(DataChunk::create)
)
.await(Duration.ofMinutes(10));
} catch (IOException e) {
throw new RuntimeException(e);
}
executor.shutdown();
server.shutdown()
.await(Duration.ofSeconds(10));
}
static Path createFile(String path) {
try {
Path filePath = Path.of("/home/kec/tmp/" + path);
System.out.println("Creating " + filePath);
return Files.createFile(filePath);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}

Poper way to make a HTTPClient PostAsync and GetAsync?

I want to make a proper HTTPClient request. I have a code but I am always getting so may exceptions like:
Java.IO.IOException: Socket closed
System.OperationCanceledException: The operation was canceled.
Java.Net.SocketException: Connection reset
Java.Net.SocketException: Software caused connection abort
Java.Net.UnknownHostException: android_getaddrinfo failed: EAI_NODATA (No address associated with hostname)
Java.Net.UnknownHostException: Unable to resolve host "tbs.scratchit.ph": No address associated with hostname
Java.IO.IOException: isConnected failed: ETIMEDOUT (Connection timed out)
Java.Net.SocketException: recvfrom failed: ECONNRESET (Connection reset by peer)
I am always getting these kinds of exceptions, errors.
I am starting to wonder how can I create a Post Async and GetAsync properly to avoid these errors in the future?
Here is how I create a HTTP Client:
1. I have a class call Constants, in there I will declare a new HTTP Client so that I only have 1 HTTPClient across my project
public class Constants
{
public static HttpClient client = new HttpClient();
}
2. I have a function(s) that gets data from my server through a PHP API by sending the parameters through JSON.
public async void FirstTimeSyncUser(string host, string database, string contact, string ipaddress)
{
try
{
syncStatus.Text = "Checking internet connection";
string apifile = "first-time-sync-user-api.php";
if (CrossConnectivity.Current.IsConnected)
{
syncStatus.Text = "Initializing first-time user sync";
var db = DependencyService.Get<ISQLiteDB>();
var conn = db.GetConnection();
var getData = conn.QueryAsync<UserTable>("SELECT * FROM tblUser WHERE ContactID = ? AND Deleted != '1'", contact);
var resultCount = getData.Result.Count;
var current_datetime = DateTime.Now.ToString("yyyy-MM-dd HH:mm:ss");
int count = 1;
var settings = new JsonSerializerSettings
{
NullValueHandling = NullValueHandling.Ignore,
MissingMemberHandling = MissingMemberHandling.Ignore
};
if (resultCount == 0)
{
syncStatus.Text = "Getting user data from the server";
var link = "http://" + ipaddress + "/" + Constants.apifolder + "/api/" + apifile;
string contentType = "application/json";
JObject json = new JObject
{
{ "Host", host },
{ "Database", database },
{ "ContactID", contact }
};
Constants.client.DefaultRequestHeaders.ConnectionClose = true;
var response = await Constants.client.PostAsync(link, new StringContent(json.ToString(), Encoding.UTF8, contentType));
if (response.IsSuccessStatusCode)
{
var content = await response.Content.ReadAsStringAsync();
if (!string.IsNullOrEmpty(content))
{
try
{
var dataresult = JsonConvert.DeserializeObject<List<UserData>>(content, settings);
var datacount = dataresult.Count;
for (int i = 0; i < datacount; i++)
{
syncStatus.Text = "Syncing user " + count + " out of " + datacount;
var item = dataresult[i];
var userid = item.UserID;
var usrpassword = item.UsrPassword;
var usertypeid = item.UserTypeID;
var userstatus = item.UserStatus;
var lastsync = DateTime.Parse(current_datetime);
var lastupdated = item.LastUpdated;
var deleted = item.Deleted;
var insertdata = new UserTable
{
UserID = userid,
UsrPassword = usrpassword,
ContactID = contact,
UserTypeID = usertypeid,
UserStatus = userstatus,
LastSync = lastsync,
LastUpdated = lastupdated,
Deleted = deleted
};
await conn.InsertOrReplaceAsync(insertdata);
count++;
}
synccount += "Total synced user: " + count + "\n";
var logType = "App Log";
var log = "Initialized first-time sync (<b>User</b>) <br/>" + "App Version: <b>" + Constants.appversion + "</b><br/> Device ID: <b>" + Constants.deviceID + "</b>";
int logdeleted = 0;
Save_Logs(contact, logType, log, database, logdeleted);
Preferences.Set("userchangeslastcheck", current_datetime, "private_prefs");
FirstTimeSyncSystemSerial(host, database, contact, ipaddress);
}
catch
{
var retry = await DisplayAlert("Application Error", "Syncing failed. Failed to send the data.\n\n Error:\n\n" + content + "\n\n Do you want to retry?", "Yes", "No");
if (retry.Equals(true))
{
FirstTimeSyncUser(host, database, contact, ipaddress);
}
else
{
First_Time_OnSyncFailed();
}
}
}
else
{
Preferences.Set("userchangeslastcheck", current_datetime, "private_prefs");
FirstTimeSyncSystemSerial(host, database, contact, ipaddress);
}
}
else
{
var retry = await DisplayAlert("Application Error", "Syncing failed. Server is unreachable.\n\n Error:\n\n"+ response.StatusCode +" Do you want to retry?", "Yes", "No");
if (retry.Equals(true))
{
FirstTimeSyncUser(host, database, contact, ipaddress);
}
else
{
First_Time_OnSyncFailed();
}
}
}
else
{
SyncUserClientUpdate(host, database, contact, ipaddress);
}
}
else
{
var retry = await DisplayAlert("Application Error", "Syncing failed. Please connect to the internet to sync your data. Do you want to retry?", "Yes", "No");
if (retry.Equals(true))
{
FirstTimeSyncUser(host, database, contact, ipaddress);
}
else
{
First_Time_OnSyncFailed();
}
}
}
catch (Exception ex)
{
Crashes.TrackError(ex);
var retry = await DisplayAlert("Application Error", "Syncing failed. Failed to send the data.\n\n Error:\n\n" + ex.Message.ToString() + "\n\n Do you want to retry?", "Yes", "No");
if (retry.Equals(true))
{
FirstTimeSyncUser(host, database, contact, ipaddress);
}
else
{
First_Time_OnSyncFailed();
}
}
}
3. After getting the data I needed it will execute another function with another POSTASYNC Call. In my code above when I got the user data from my server it will execute the next function which is FirstTimeSyncSystemSerial(host, database, contact, ipaddress);
What am I doing wrong? and How can I improve this so that I can avoid these exceptions?
Debug your code to find out where the exception is thrown.
Put a try catch blocked around that block of code. Then catch all the expected exceptions and try loopback again for a number of time.
You can Make a Generic Custom Service call That Can be Called Anywhere When You Need
public class RestClient : IRestClient
{
private const string TokenHeaderKey = "Any Token Header";
private HttpClient _httpclient = new HttpClient();
public async Task<T> GetAsync<T>(string url, string token) where T : new()
{
var responseContent = await ExecuteRequest(
async () =>
{
try
{
AddTokenToDefaultRequestsHeader(token);
return await _httpclient.GetAsync(url);
}
finally
{
ClearAuthenticationHeader();
}
});
return await Deserialize<T>(responseContent);
}
private void AddTokenToDefaultRequestsHeader(string token)
{
_httpclient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue(TokenHeaderKey, token);
}
private void ClearAuthenticationHeader()
{
_httpclient.DefaultRequestHeaders.Authorization = null;
}
private static async Task<HttpContent> ExecuteRequest(Func<Task<HttpResponseMessage>> requestFunc)
{
HttpResponseMessage response = null;
try
{
response = await requestFunc();
if (!response.IsSuccessStatusCode)
{
var message = $"Executed HTTP request returned status code {response.StatusCode} and reason phrase {response.ReasonPhrase}";
if (response.StatusCode == HttpStatusCode.Unauthorized)
{
throw new Exception(message);
}
throw new Exception(message);
}
return response.Content;
}
catch (Exception exception)
{
if (exception is HttpRequestException || exception is WebException || exception is TaskCanceledException)
{
throw new Exception(
"Could not connect to service.");
}
throw;
}
}
private static async Task<T> Deserialize<T>(HttpContent responseContent) where T : new()
{
try
{
var responseContentString = await responseContent.ReadAsStringAsync();
return JsonConvert.DeserializeObject<T>(responseContentString);
}
catch (Exception exception)
{
if (exception is TaskCanceledException || exception is JsonException)
{
throw new Exception("Could not deserialize response content.", exception);
}
throw;
}
}
And Add an App settings class
public class AppSettings : IAppSettings
{
public string Server => "server url";
public string ServerEndPoint => "End point";
public string Token => "Token If you Have any";
}
Then Call Like this
public class Servicecall
{
private readonly IRestClient _restClient;
private readonly IAppSettings _appSettings;
public PatientService(IRestClient restClient, IAppSettings appSettings)
{
_restClient = restClient;
_appSettings = appSettings;
}
public async Task<IList<PatientViewModel>> GetPatients()
{
var url = _appSettings.Server + _appSettings.EndPoint ;
var token = _appSettings.Token;
return GetPatientList(await _restClient.GetAsync<List<ListModelClass>>(url, token));
}
public IList<Model> GetPatientList(IList<ListModelClass> List)
{
return List.Select(p => new Model(p)).ToList();
}
}
This way You can Call deferent services without typing a lot of boilercodes
This way You can Call Services with real ease

TcpListener handle of multiple clients

I created MyListener which will start listening (using TcpListener) on his own thread upon creation. the TcpListener should handle multiple clients so i am running inside infinte while and handle each client in special task.
this is my code:
public class MyListener
{
public event EventHandler<MessageEventArgs> MessageReceived;
public MyListener()
{
var thread = new Thread(Listen);
thread.Start();
}
private void Listen()
{
TcpListener server = null;
try
{
server = new TcpListener(IPAddress.Any, 8977);
server.Start();
while (true)
{
var client = server.AcceptTcpClient();
Task.Run(() =>
{
try
{
var msg = GetMessageFromClient(client);
MessageReceived?.Invoke(this, new MessageEventArgs { Message = msg });
}
catch (Exception)
{
}
finally
{
client.Close();
}
});
}
}
catch (Exception)
{
}
finally
{
if (server != null)
server.Stop();
}
}
private string GetMessageFromClient(TcpClient client)
{
var bytes = new byte[client.ReceiveBufferSize];
var stream = client.GetStream();
var i = stream.Read(bytes, 0, bytes.Length);
var message = Encoding.UTF8.GetString(bytes, 0, i);
return message;
}
}
here are my questions:
how can i ensure that the task handle the client will use the client i pass to it when i start the task and not different client (becuase after the task start we return to the AcceptTcpClient method and may get new client)
in my example and with multiple clients handled by the same method ("GetMessageFromClient") do i need to put some kind of locking on this
method?

Spring Scheduled Task, strange behavior

I have a Scheduled task which runs every 800 seconds from a server towards its clients to see if they are online. If they are not, it will send me an email, and will send an email when they are back online.
So, this is the scheduled task
#Scheduled(fixedDelay = 800000)
public void pingAllClients() {
logger.debug("Schedule pingClients");
List<Client> clients = clientService.findAllClients();
Iterator<Client> it = clients.iterator();
while (it.hasNext()) {
Client client = it.next();
String ip = client.getCurrentIp();
int idClient = client.getIdClient();
boolean isOnline = client.isOnline();
try {
boolean reachable = reachClient.isReachable(ip);
if (reachable) {
if (!isOnline) {
logger.debug("Client " + idClient + " back online");
client.setOnline(true);
clientService.updateClient(client);
smtp.sendEmail(serverName, ip, true);
}
} else {
logger.debug("Client " + idClient + " not available");
if (isOnline) {
client.setOnline(false);
clientService.updateClient(client);
smtp.sendEmail(serverName, ip, false);
}
}
} catch (Exception e) {
logger.error("Errore", e);
}
}
}
}
reachClient.isReachable(ip) is this method:
public boolean isReachable(String ip){
Socket socket = new Socket();
try {
socket.connect(new InetSocketAddress(ip, 22), 50*1000);
return true;
} catch (IOException e) {
return false;
} finally {
try {
socket.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
So, it should be: every 800 seconds i loop my clients, see if the are reachable or not, and if they change status, i will receive an email. In the email i print the date/time of the request.
It always happens that i get an email from an offline client and soon after (2-10 seconds) an email it is back online, as, for the same client, i'm getting 2 emails in a single iteration, how is it possibile? Each client should be reachable or not for every loop, not both...
The last 2 emails I got are (6 seconds delay)
Client 192.168.42.13 is offline - Date: 11-12-2016 17:14:30
Client 192.168.42.13 back online - Date: 11-12-2016 17:14:36

Twitter ping inside proxy failed but when open from browser it open

I have a problem: when I open twitter.com, it opens, but when use Tweetinvi it doesn’t work.
Therefore, I made this code:
var result = ping.Send("twitter.com");
if (result.Status != System.Net.NetworkInformation.IPStatus.Success)
{
insertlogwithFilename("test ping : 000X" );
}
I setup proxy configuration, but the ping does not work.
TweetinviConfig.CurrentThreadSettings.HttpRequestTimeout = 5000000;
TweetinviConfig.CurrentThreadSettings.UploadTimeout = 9000000;
if (noproxy == "0")
{
TweetinviConfig.CurrentThreadSettings.ProxyURL = "http://" + proxyIP + ":" + proxyPort;
}
try
{
Auth.SetUserCredentials(cuskey, secret, accesstoken, useraccessse);
}
catch (Exception exp)
{
insertlogwithFilename("error in authentication :" + exp.Message);
}
try
{
var authenticatedUser = User.GetAuthenticatedUser();
}
catch (Exception exp)
{
insertlogwithFilename("error in user authentication :" + exp.Message);
}
I test if connection to twitter is connected :
public static void testconn()
{
try
{
using (var client = new WebClient())
{
using (var stream = client.OpenRead("https://www.twitter.com"))
{
insertlogwithFilename("test stream : 8666");
}
}
}
catch
{
insertlogwithFilename("test stream : 8766");
}
}
this block of code return true
public static void testconnping()
{
var ping = new System.Net.NetworkInformation.Ping();
var result = ping.Send("twitter.com");
if (result.Status != System.Net.NetworkInformation.IPStatus.Success)
{
insertlogwithFilename("test ping : 8766" );
//return false;
}
else
{
insertlogwithFilename("test ping : 8666");
//return true;
}
}
this block return false
When you set up TweetinviConfig.CurrentThreadSettings.ProxyURL it does not mean that ping.Send will use this proxy. It means that Tweetinvi will use this proxy.
If you want to check if Tweetinvi is actually accessing the data, you will need to run the GetAuthenticatedUser method as explained in the documentation.
TweetinviConfig.CurrentThreadSettings.ProxyURL = "http://...";
ExceptionHandler.SwallowWebExceptions = false;
Auth.SetUserCredentials("CONSUMER_KEY", "CONSUMER_SECRET", "ACCESS_TOKEN", "ACCESS_TOKEN_SECRET");
var authenticatedUser = User.GetAuthenticatedUser();

Resources