I'm getting secret from Azure KeyVault through rest api. At backend, I'm using the azure-keyvault-client which is using retrofit and okhttp3 at behind. My app has been running well for a long time. Now it shows up the exceptions suddenly. Each time the exceptions happens, I restarts the app. Then the exceptions are gone. Everything looks good. What will be the reason that "stream was cancelled"?
Below is the full stack trace.
I have tried to remote debug the app. I found that the exception is thrown when call FramedStream.closeInternal(). The HttpEngine tries to close but source.finished = false and sink.finished = true.
It could either be a local cancel of the stream, or from the server. You can use HTTP/2 Frame logging so see if it's coming from the server.
https://square.github.io/okhttp/debug_logging/#http2-frame-logging
Does the problem persist until restart? If so it's possible that the server is expecting you or OkHttp to close the connection after the cancel, but that isn't expected in that case. So best to debug the frames and then discuss with the server team.
https://github.com/square/okhttp/blob/master/okhttp-testing-support/src/main/kotlin/okhttp3/OkHttpDebugLogging.kt
fun enableHttp2() = enable(Http2::class)
fun enableTaskRunner() = enable(TaskRunner::class)
fun enable(loggerClass: String) {
val logger = Logger.getLogger(loggerClass)
if (configuredLoggers.add(logger)) {
logger.addHandler(ConsoleHandler().apply {
level = Level.FINE
formatter = object : SimpleFormatter() {
override fun format(record: LogRecord) =
String.format("[%1\$tF %1\$tT] %2\$s %n", record.millis, record.message)
}
})
logger.level = Level.FINEST
}
}
Related
I am using jpos client (In one of the class of java Spring MVC Program) to connect the ISO8585 based server, however due to some reason server is not able to respond back, due to which my program keeps waiting for the response and results in hanging my program. So what is the proper way to implement connection timeout?
My client program look like this:
public FieldsModal sendFundTransfer(FieldsModal field){
try {
JposLogger logger = new JposLogger(ISO_LOG_LOCATION);
org.jpos.iso.ISOPackager customPackager = new GenericPackager(ISO_PACKAGER);
ISOChannel channel = new PostChannel(ISO_SERVER_IP, Integer.parseInt(ISO_SERVER_PORT), customPackager);// live
logger.jposlogconfig(channel);
channel.connect();
log4j.info("Connection established using PostChannel");
ISOMsg m = new ISOMsg();
m.set(0, field.getMti());
//m.set(2, field.getField2());
m.set(3, field.getField3());
m.set(4, field.getField4());
m.set(11, field.getField11());
m.set(12, field.getField12());
m.set(17, field.getField17());
m.set(24, field.getField24());
m.set(32, field.getField32());
m.set(34, field.getField34());
m.set(41, field.getField41());
m.set(43, field.getField43());
m.set(46, field.getField46());
m.set(49, field.getField49());
m.set(102,field.getField102());
m.set(103,field.getField103());
m.set(123, field.getField123());
m.set(125, field.getField125());
m.set(126, field.getField126());
m.set(127, field.getField127());
m.setPackager(customPackager);
System.out.println(ISOUtil.hexdump(m.pack()));
channel.send(m);
log4j.info("Message has been send");
ISOMsg r = channel.receive();
r.setPackager(customPackager);
System.out.println(ISOUtil.hexdump(r.pack()));
channel.disconnect();
}catch (Exception err) {
System.out.println("sendFundTransfer : " + err);
}
return field;
}
Well the real proper way would be to use Q2. Given you don't need a persistent connection you coud just set a timeout for the channel.
PostChannel channel = new PostChannel(ISO_SERVER_IP, Integer.parseInt(ISO_SERVER_PORT), customPackager);// live
channel.setTimeout(timeout); //timeout in millies.
This way channel will autodisconnect if nothing happens during the time specified by timeout , and your call to receive will throw an exception.
The alternative is using Q2 and a mux (see QMUX, for which you need to run Q2, or ISOMUX which is kind of deprecated).
I created an Edge browser extension which uses Native Messaging to a native app running via a Desktop Bridge technology. I used the SecureInput as a sample, which contains the Edge extension, UWP host and a Win32 Desktop Bridge app.
I need the Win32 Desktop Bridge app to connect to a web service using HTTP and WebSocket, so I added an internetClientServer and a privateNetworkClientServer capabilities to the package manifest, beside the already existed runFullTrust one.
The Win32 Desktop Bridge app activates just fine, and it is able to connect to the web server using HTTP. But as soon as it tries to open a WebSocket connection, the BackgroundTaskInstance on the UWP host receives a cancellation request with a BackgroundTaskCancellationReason.SystemPolicy as a reason, and the Desktop Bridge application closes. Unfortunately, the documentation for the BackgroundTaskCancellationReason.SystemPolicy does not explain much about true reasons of the cancellation request.
I tried to use two WebSocket classes: the System.Net.WebSockets.ClientWebSocket and the Windows.Networking.Sockets.MessageWebSocket, with the same result. No fancy code, just regular
var socket = new MessageWebSocket();
...
await socket.ConnectAsync(new Uri("wss://127.0.0.1:9001/myservice"));
The same WebSocket service endpoint is available from other WS clients, so I guess there is no server/firewall/antivirus issue here.
I also played with the CheckNetIsolation tool, adding loopback exemption for the Edge browser and for the package, with no effect. The HTTP works fine without the loopback exemption.
What may be a true reason of the task cancellation, and what can be a possible way to prevent it?
Ok, I resolved the issue. Thanks to this comment by Tom Shane I stumbled upon, I realized that the BackgroundTaskCancellationReason.SystemPolicy tells that the background task is closed by the system to release some system resources, and that in my case it happened because I didn't obtain a deferral in my async event handler. When the event handler yielded without a deferral, the system decided it can shut the task down. Below is a digested version of the code:
static class Program
{
static AppServiceConnection connection = null;
[STAThread]
static void Main(string[] args)
{
Thread appServiceThread = new Thread(new ThreadStart(ThreadProc));
appServiceThread.Start();
Application.Run();
}
static async void ThreadProc()
{
try {
connection = new AppServiceConnection();
connection.AppServiceName = "...";
connection.PackageFamilyName = Windows.ApplicationModel.Package.Current.Id.FamilyName;
connection.RequestReceived += OnRequestReceived;
connection.ServiceClosed += OnServiceClosed;
var status = await connection.OpenAsync();
....
}
catch (Exception e) { ... }
}
private static async void OnRequestReceived(AppServiceConnection sender, AppServiceRequestReceivedEventArgs args)
{
var defer = args.GetDeferral(); // <== that was missing, rookie mistake!
try {
var msg = ParseMessage(args.Request.Message);
if (msg.type.Equals("ws")) {
// this method was truly async
// and every time it yielded the issue was revealed
await HandleWsMessage(request, msg);
}
else if (msg.type.Equals("http")) {
// but this method was actually synchronous despite being marked as "async"
// and it never yielded, masking the issue for HTTP client
await HandleHttpMessage(request, msg);
}
}
catch (Exception e) { ... }
finally {
defer.Complete();
}
}
}
I have an OWIN Selfhosted WebAPI with a Controller that accepts a large file as input. The code works in PC without any problems or errors. However in MONO on OSX, the message stays in hung mode in Chrome. When I supply a small size file using Base64 string, the call gets through. This helped me conclude that my file which is of size 1.5 MB is not an acceptable Base64 message size for POST on MONO with OSX.
However I tried using maxContentSize to a gig on HttpRunTime and Also tried OWIN Middleware implementation with setting MaxRequestSizeInBytes to a Gig. Both of these did not let me POST the file still to Web API Controller and request stays in pending status.
Please let me know, if you have any other ideas for setting up Max Message Size or know if there is something on MONO preventing file of size 1.5 MB.
I found a workaround for MAC using OwinMiddleware. Even if I increased the message size, it didn't work, however below solution let me bypass the message size limitation (65K max).
public override async Task Invoke(IOwinContext context)
{
try
{
IOwinRequest request = context.Request;
if (request.Path.HasValue && request.Path.Value.Contains("METHODNAMEFORLARGEFILE"))
{
//TODO: Remove Workaround for MAC to explicitly download whole stream to avoid issues with message size. PC works without this workaround.
LogHelper.Logger.Info("OWIN METHODNAMEFORLARGEFILE Request");
var stream = request.Body;
string requestBody = await (new StreamReader(stream)).ReadToEndAsync();
var requestData = Encoding.UTF8.GetBytes(requestBody);
context.Request.Body = new MemoryStream(requestData);
}
await Next.Invoke(context);
}
catch(Exception ex)
{
LogHelper.Logger.Error(ex.Message, ex);
throw ex;
}
}
My builds have been failing due to some of the integration tests I've been running. I'm stuck on why it won't work. Here is an example of the output:
I'm using Maven to first build, then it calls the JUnit tests. I'm seeing this 401 Unauthorized message in every single test, and I believe that's what is causing the builds to fail. In my mind, this means there are some permissions / authentication parameters that need to be set. Where would I go about doing this in JUnit?
Edit
#Test
public void testXmlHorsesNonRunners() throws Exception {
String servletUrl = SERVER + "sd/date/2013-01-13/horses/nonrunners";
Document results = issueRequest(servletUrl, APPLICATION_XML, false);
assertNotNull(results);
// debugDocument(results, "NonRunners");
String count = getXPathStringValue(
"string(count(hrdg:data/hrdg:meeting/hrdg:event/hrdg:nonrunner/hrdg:selection))",
results);
assertEquals("non runners", "45", count);
}
If you can, try to ignore the detail. Effectively, this is making a request. This is a sample of a test that uses the issueRequest method. This method is what makes HTTP requests. (This is a big method, which is why I didn't post it originally. I'll try to make it as readable as possible.
logger.info("Sending request: " + servletUrl);
HttpGet httpGet = null;
// InputStream is = null;
DefaultHttpClient httpclient = null;
try {
httpclient = new DefaultHttpClient();
doFormLogin(httpclient, servletUrl, acceptMime, isIrishUser);
httpGet = new HttpGet(servletUrl);
httpGet.addHeader("accept", acceptMime);
// but more importantly now add the user agent header
setUserAgent(httpGet, acceptMime);
logger.info("executing request" + httpGet.getRequestLine());
// Execute the request
HttpResponse response = httpclient.execute(httpGet);
// Examine the response status
StatusLine statusLine = response.getStatusLine();
logger.info(statusLine);
switch (statusLine.getStatusCode()) {
case 401:
throw new HttpResponseException(statusLine.getStatusCode(),
"Unauthorized");
case 403:
throw new HttpResponseException(statusLine.getStatusCode(),
"Forbidden");
case 404:
throw new HttpResponseException(statusLine.getStatusCode(),
"Not Found");
default:
if (300 < statusLine.getStatusCode()) {
throw new HttpResponseException(statusLine.getStatusCode(),
"Unexpected Error");
}
}
// Get hold of the response entity
HttpEntity entity = response.getEntity();
Document doc = null;
if (entity != null) {
InputStream instream = entity.getContent();
try {
// debugContent(instream);
doc = documentBuilder.parse(instream);
} catch (IOException ex) {
// In case of an IOException the connection will be released
// back to the connection manager automatically
throw ex;
} catch (RuntimeException ex) {
// In case of an unexpected exception you may want to abort
// the HTTP request in order to shut down the underlying
// connection and release it back to the connection manager.
httpGet.abort();
throw ex;
} finally {
// Closing the input stream will trigger connection release
instream.close();
}
}
return doc;
} finally {
// Release the connection.
closeConnection(httpclient);
}
I notice that your test output shows HTTP/1.1 500 Internal Server Error a couple of lines before the 401 error. I wonder if the root cause could be hiding in there. If I were you I'd try looking for more details about what error happened on the server at that point in the test, to see if it could be responsible for the authentication problem (maybe the failure is in a login controller of some sort, or is causing a session to be cancelled?)
Alternately: it looks like you're using the Apache HttpClient library to do the request, inside the issueRequest method. If you need to include authentication credentials in the request, that would be the code you'd need to change. Here's an example of doing HTTP Basic authentication in HttpClient, if that helps. (And more examples, if that one doesn't.)
(I'd second the observation that this problem probably isn't specific to JUnit. If you need to do more research, I'd suggest learning more about HttpClient, and about what this app expects the browser to send. One possibility: use something like Chrome Dev Tools to peek at your communications with the server when you do this manually, and see if there's anything important that the test isn't doing, or is doing differently.
Once you've figured out how to login, it might make sense to do it in a #Before method in your JUnit test.)
HTTP permission denied has nothing to do with JUnit. You probably need to set your credentials while making the request in the code itself. Show us some code.
Also, unit testing is not really meant to access the internet. Its purpose is for testing small, concise parts of your code which shouldn't rely on any external factors. Integration tests should cover that.
If you can, try to mock your network requests using EasyMock or PowerMock and make them return a resource you would load from your local resources folder (e.g. test/resources).
I'm looking to increase the performance of a high-throughput producer that I'm writing against ActiveMQ, and according to this useAsyncSend will:
Forces the use of Async Sends which adds a massive performance boost;
but means that the send() method will return immediately whether the
message has been sent or not which could lead to message loss.
However I can't see it making any difference to my simple test case.
Using this very basic application:
const string QueueName = "....";
const string Uri = "....";
static readonly Stopwatch TotalRuntime = new Stopwatch();
static void Main(string[] args)
{
TotalRuntime.Start();
SendMessage();
Console.ReadLine();
}
static void SendMessage()
{
var session = CreateSession();
var destination = session.GetQueue(QueueName);
var producer = session.CreateProducer(destination);
Console.WriteLine("Ready to send 700 messages");
Console.ReadLine();
var body = new byte[600*1024];
Parallel.For(0, 700, i => SendMessage(producer, i, body, session));
}
static void SendMessage(IMessageProducer producer, int i, byte[] body, ISession session)
{
var message = session.CreateBytesMessage(body);
var sw = new Stopwatch();
sw.Start();
producer.Send(message);
sw.Stop();
Console.WriteLine("Running for {0}ms: Sent message {1} blocked for {2}ms",
TotalRuntime.ElapsedMilliseconds,
i,
sw.ElapsedMilliseconds);
}
static ISession CreateSession()
{
var connectionFactory = new ConnectionFactory(Uri)
{
AsyncSend = true,
CopyMessageOnSend = false
};
var connection = connectionFactory.CreateConnection();
connection.Start();
var session = connection.CreateSession(AcknowledgementMode.AutoAcknowledge);
return session;
}
I get the following output:
Ready to send 700 messages
Running for 2430ms: Sent message 696 blocked for 12ms
Running for 4275ms: Sent message 348 blocked for 1858ms
Running for 5106ms: Sent message 609 blocked for 2689ms
Running for 5924ms: Sent message 1 blocked for 2535ms
Running for 6749ms: Sent message 88 blocked for 1860ms
Running for 7537ms: Sent message 610 blocked for 2429ms
Running for 8340ms: Sent message 175 blocked for 2451ms
Running for 9163ms: Sent message 89 blocked for 2413ms
.....
Which shows that each message takes about 800ms to send and the call to session.Send() blocks for about two and a half seconds. Even though the documentation says that
"send() method will return immediately"
Also these number are basically the same if I either change the parallel for to a normal for loop or change the AsyncSend = true to AlwaysSyncSend = true so I don't believe that the async switch is working at all...
Can anyone see what I'm missing here to make the send asynchronous?
After further testing:
According to ANTS performance profiler that vast majority of the runtime is being spent waiting for synchronization. It appears that the issue is that the various transport classes block internally through monitors. In particular I seem to get hung up on the MutexTransport's OneWay method which only allows one thread to access it at a time.
It looks as though the call to Send will block until the previous message has completed, this explains why my output shows that the first message blocked for 12ms, while the next took 1858ms. I can have multiple transports by implementing a connection-per-message pattern which improves matters and makes the message sends work in parallel, but greatly increases the time to send a single message, and uses up so many resources that it doesn't seem like the right solution.
I've retested all of this with 1.5.6 and haven't seen any difference.
As always the best thing to do is update to the latest version (1.5.6 at the time of this writing). A send can block if the broker has producer flow control enabled and you've reached a queue size limit although with async send this shouldn't happen unless you are sending with a producerWindowSize set. One good way to get help is to create a test case and submit it via a Jira issue to the NMS.ActiveMQ site so that we can look into it using your test code. There have been many fixes since 1.5.1 so I'd recommend giving that new version a try as it could already be a non-issue.