I have set up this KinesisMessageDrivenChannelAdapter:
#Bean
public KinesisMessageDrivenChannelAdapter kinesisInboundChannel(AmazonKinesis amazonKinesis, MetadataStore store) {
KinesisMessageDrivenChannelAdapter adapter =
new KinesisMessageDrivenChannelAdapter(amazonKinesis, config.getStreamName());
adapter.setCheckpointMode(CheckpointMode.batch);
adapter.setListenerMode(ListenerMode.batch);
adapter.setStartTimeout(10000);
// Set idle to milliseconds. Max value is 596 before getting an overflow exception.
adapter.setIdleBetweenPolls(config.getPollHours() * 3_600_000);
adapter.setShouldTrack(true);
adapter.setDescribeStreamRetries(5);
adapter.setConcurrency(50);
adapter.setCheckpointStore(store);
adapter.setStreamInitialSequence(KinesisShardOffset.trimHorizon());
adapter.setOutputChannelName("logMessage.input");
adapter.setErrorChannel(errorChannel());
return adapter;
}
Most of the time, it works fine. But from time to time, I get this kind of message:
Exception in thread "kinesisInboundChannel-kinesis-consumer-1" com.amazonaws.services.kinesis.model.AmazonKinesisException: null (Service: AmazonKinesis; Status Code: 500; Error Code: InternalFailure; Request ID: c2f66be9-23f4-b211-9165-ed92383ee673)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1639)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1304)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:743)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:717)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:699)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:667)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:649)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:513)
at com.amazonaws.services.kinesis.AmazonKinesisClient.doInvoke(AmazonKinesisClient.java:2276)
at com.amazonaws.services.kinesis.AmazonKinesisClient.invoke(AmazonKinesisClient.java:2252)
at com.amazonaws.services.kinesis.AmazonKinesisClient.executeGetRecords(AmazonKinesisClient.java:1062)
at com.amazonaws.services.kinesis.AmazonKinesisClient.getRecords(AmazonKinesisClient.java:1038)
at org.springframework.integration.aws.inbound.kinesis.KinesisMessageDrivenChannelAdapter$ShardConsumer.getRecords(KinesisMessageDrivenChannelAdapter.java:853)
at org.springframework.integration.aws.inbound.kinesis.KinesisMessageDrivenChannelAdapter$ShardConsumer.access$3500(KinesisMessageDrivenChannelAdapter.java:688)
at org.springframework.integration.aws.inbound.kinesis.KinesisMessageDrivenChannelAdapter$ShardConsumer$2.run(KinesisMessageDrivenChannelAdapter.java:816)
at org.springframework.integration.aws.inbound.kinesis.KinesisMessageDrivenChannelAdapter$ConsumerInvoker.run(KinesisMessageDrivenChannelAdapter.java:1003)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
After that, the adapter stops working entirely without hanging the app. I specified what error channel to use, and I'd be happy to just restart the application to bring the adapter back online if I need to. But that appears to not be an option.
How do I build error handling into this?
Sounds like a problem has been fixed here: https://github.com/spring-projects/spring-integration-aws/issues/84
You need to consider to use the latest version (2.0.0.M2), or even better 2.0.0.BUILD-SNAPSHOT.
Related
I got "ContainerLaunchException: Timed out waiting for log output matching" when starting testcontainer for elasticserach. How should I fix this issue?
container = new ElasticsearchContainer(ELASTICSEARCH_IMAGE)
.withEnv("discovery.type", "single-node")
.withExposedPorts(9200);
container.start();
12:16:50.370 [main] ERROR 🐳 [docker.elastic.co/elasticsearch/elasticsearch:7.16.3] - Could not start container
org.testcontainers.containers.ContainerLaunchException: Timed out waiting for log output matching '.("message":\s?"started".|] started
$)'
at org.testcontainers.containers.wait.strategy.LogMessageWaitStrategy.waitUntilReady(LogMessageWaitStrategy.java:49)
at org.testcontainers.containers.wait.strategy.AbstractWaitStrategy.waitUntilReady(AbstractWaitStrategy.java:51)
Updated:
I looked into contructor ElasticsearchContainer
public ElasticsearchContainer(DockerImageName dockerImageName) {
super(dockerImageName);
this.caCertAsBytes = Optional.empty();
dockerImageName.assertCompatibleWith(new DockerImageName[]{DEFAULT_IMAGE_NAME, DEFAULT_OSS_IMAGE_NAME});
this.isOss = dockerImageName.isCompatibleWith(DEFAULT_OSS_IMAGE_NAME);
this.logger().info("Starting an elasticsearch container using [{}]", dockerImageName);
this.withNetworkAliases(new String[]{"elasticsearch-" + Base58.randomString(6)});
this.withEnv("discovery.type", "single-node");
this.addExposedPorts(new int[]{9200, 9300});
this.isAtLeastMajorVersion8 = (new ComparableVersion(dockerImageName.getVersionPart())).isGreaterThanOrEqualTo("8.0.0");
String regex = ".*(\"message\":\\s?\"started\".*|] started\n$)";
this.setWaitStrategy((new LogMessageWaitStrategy()).withRegEx(regex));
if (this.isAtLeastMajorVersion8) {
this.withPassword("changeme");
}
}
It uses setWaitStrategy. So I updated my code as below
container.setWaitStrategy((new LogMessageWaitStrategy()).withRegEx(regex).withTimes(1));
But I still get the same error. Here is how far the log messages go.
Updated again: I relized above code change doesn't update any default values.
Here is the new change:
container.setWaitStrategy((new LogMessageWaitStrategy())
.withRegEx(regex)
.withStartupTimeout(Duration.ofSeconds(180L)));
It works with this new change. I have to copy regex from ElasticsearchContainer constructor. I hope it has a better way to override the timeout value.
I'm getting secret from Azure KeyVault through rest api. At backend, I'm using the azure-keyvault-client which is using retrofit and okhttp3 at behind. My app has been running well for a long time. Now it shows up the exceptions suddenly. Each time the exceptions happens, I restarts the app. Then the exceptions are gone. Everything looks good. What will be the reason that "stream was cancelled"?
Below is the full stack trace.
I have tried to remote debug the app. I found that the exception is thrown when call FramedStream.closeInternal(). The HttpEngine tries to close but source.finished = false and sink.finished = true.
It could either be a local cancel of the stream, or from the server. You can use HTTP/2 Frame logging so see if it's coming from the server.
https://square.github.io/okhttp/debug_logging/#http2-frame-logging
Does the problem persist until restart? If so it's possible that the server is expecting you or OkHttp to close the connection after the cancel, but that isn't expected in that case. So best to debug the frames and then discuss with the server team.
https://github.com/square/okhttp/blob/master/okhttp-testing-support/src/main/kotlin/okhttp3/OkHttpDebugLogging.kt
fun enableHttp2() = enable(Http2::class)
fun enableTaskRunner() = enable(TaskRunner::class)
fun enable(loggerClass: String) {
val logger = Logger.getLogger(loggerClass)
if (configuredLoggers.add(logger)) {
logger.addHandler(ConsoleHandler().apply {
level = Level.FINE
formatter = object : SimpleFormatter() {
override fun format(record: LogRecord) =
String.format("[%1\$tF %1\$tT] %2\$s %n", record.millis, record.message)
}
})
logger.level = Level.FINEST
}
}
Currently I use PCF command to delete a Queue on QMANAGER with
PCFMessage message = new PCFMessage( CMQCFC.MQCMD_DELETE_Q );
message.addParameter( CMQC.MQCA_Q_NAME, name);
agent.send( message );
Could I force delete if queue is occupied?
I have tried without succes on QL
#Override
protected PCFMessage getRequestRemove(String objetName,
String qmanagerName,boolean forceQmanager) {
PCFMessage request = new PCFMessage(CMQCFC.MQCMD_DELETE_Q);
request.addParameter( CMQCFC.MQIACF_PURGE, CMQCFC.MQPO_YES );
request.addParameter(CMQC.MQCA_Q_NAME, objetName);
request.addParameter(CMQC.MQIA_Q_TYPE, CMQC.MQQT_LOCAL);
return request;
}
Error code is Caused by: com.ibm.mq.pcf.PCFException: MQJE001: Code achèvement '2', Motif '3014'.
My PCF library is 7.1.0.4
regards
There is no FORCE option on a DELETE queue command. If the queue is currently open by an application for input and they are waiting in an MQGET you can kick them out with the following command.
MQSC
ALTER QLOCAL(q-name) GET(DISABLED)
PCF
PCFMessage message = new PCFMessage (CMQCFC.MQCMD_CHANGE_Q);
message.addParameter(CMQC.MQCA_Q_NAME, name);
message.addParameter(CMQC.MQIA_INHIBIT_GET, CMQC.MQQA_GET_INHIBITED);
agent.send(message);
However if the queue is currently open and the application is not currently in either an MQGET or an MQPUT, then you cannot kick them out in this way, your only option then is to find the application in question using DISPLAY CONN, and then issue a STOP CONN to get them to release the queue.
The mostly likely occupation of a queue is the long MQGET-waiter, and so the above example command will help for most cases.
Morag's answer addresses possible ways to disconnect processes that currently have the queue open, if you also want to remove the queue when messages are on the queue you would need to ask MQ to PURGE the messages:
PCFMessage message = new PCFMessage( CMQCFC.MQCMD_DELETE_Q );
message.addParameter( CMQC.MQCA_Q_NAME, name);
message.addParameter( CMQCFC.MQIACF_PURGE, CMQCFC.MQPO_YES );
agent.send( message );
We are trying to test push notifications, using the latest code from the documentation How to: Set Up a Notification Channel for Windows Phone
public HttpNotificationChannel myChannel;
public void CreatingANotificationChannel()
{
myChannel = HttpNotificationChannel.Find("MyChannel");
if (myChannel == null)
{
myChannel = new HttpNotificationChannel("MyChannel","www.contoso.com");
// An application is expected to send its notification channel URI to its corresponding web service each time it launches.
// The notification channel URI is not guaranteed to be the same as the last time the application ran.
myChannel.ChannelUriUpdated += new EventHandler<NotificationChannelUriEventArgs>(myChannel_ChannelUriUpdated);
myChannel.Open();
}
else // Found an existing notification channel.
{
// The URI that the application sends to its web service.
Debug.WriteLine("Notification channel URI:" + myChannel.ChannelUri.ToString());
}
myChannel.HttpNotificationReceived += new EventHandler<HttpNotificationEventArgs>(myChannel_HttpNotificationReceived);
myChannel.ShellToastNotificationReceived += new EventHandler<NotificationEventArgs>(myChannel_ShellToastNotificationReceived);
myChannel.ErrorOccurred += new EventHandler<NotificationChannelErrorEventArgs>(myChannel_ErrorOccurred);
}
If HttpNotificationChannel.Find() returns null, it opens a new channel, but the ChannelUriUpdated event is never triggered.
If HttpNotificationChannel.Find() returns a channel, the ChannelUri property is null. The sample code crashes here because it assumes the ChannelUri property to be not null.
In neither case is the ErrorOccurred event triggered.
How can i solve this problem? This problem is because of microsoft server or any thing else?
Thnks in advance
EDIT
Waiting for replay,after ten days i am suffering of null uri problem
Can any one tell me how can i solve this problem some time MSPN server give chanalk uri ans some time not i mean some time it give null reference Exception.
What Microsoft doing?
If I don't go wrong, www.contoso.com it's a example URI to demonstrate that you need to put your own server URL address, but in my experience, I never use in that way. I prefer just to put
myChannel = new HttpNotificationChannel("MyChannel");
Look this example (it's in Spanish) but the codes are very clear of what you need to do to set the push notification client and service.
I hope I helped you.
You are testing in what mobile are Emulator,
Do you have developer account subscription for windows phone development,
Had you Developer unlocked your mobile,
Noorul.
I think the problem is that you are using the HttpNotificationChannel constructor of the authenticated web service, according to the documentation.
Instead, you should use the constructor that takes only one parameter, as you can check in this example
/// Holds the push channel that is created or found.
HttpNotificationChannel pushChannel;
// The name of our push channel.
string channelName = "ToastSampleChannel";
// Try to find the push channel.
pushChannel = HttpNotificationChannel.Find(channelName);
// If the channel was not found, then create a new connection to the push service.
if (pushChannel == null)
{
pushChannel = new HttpNotificationChannel(channelName);
...
}
Hope it helps
I followed a tutorial on creating a blob on windows azure. But when I do that, I get an exception error:
Error while creating containerThe server encountered an unknown failure: The remote server returned an error: (300) Ambiguous Redirect.
The code is:
private void SetContainersAndPermission()
{
try
{
// create a container
var CloudAccountStorage = CloudStorageAccount.FromConfigurationSetting("BlobConnectionString");
cloudBlobClient = CloudAccountStorage.CreateCloudBlobClient();
CloudBlobContainer blobContainer = cloudBlobClient.GetContainerReference("documents");
blobContainer.CreateIfNotExist();
// permissions
var containerPermissions = blobContainer.GetPermissions();
containerPermissions.PublicAccess = BlobContainerPublicAccessType.Container;
blobContainer.SetPermissions(containerPermissions);
}
catch(Exception ex)
{
throw new Exception("Error while creating container" + ex.Message);
}
}
Can anyone tell me How to solve this problem....
I would guess the connection string is somehow wrong? Can you share the connection string? (X out your shared key...)
You could also install Fiddler (debugging HTTP proxy) and see what the HTTP request looks like. That may make the issue more obvious.
I also faced the same issue. I am not sure if this is the workaround for it. I modified the container name value in ServiceConfiguration.csfg from "Photograph" to "photograph" and it worked.
I think you can not give upper case letters in queue, table or blob name. The name should have only lower case characters.