I am trying to stop the processor which is in running state using the PUT method in rest api /processors/{id}.
I am able to start the processor by changing the state in the component as follows "state": "RUNNING", and runStatus in the aggregatesnapshot as "runStatus": "Running".
Similarly I tried to stop the processor by changing the state as STOPPED but facing an error as
9204b68d-0159-1000-7d8f-720592b2a2dd is not stopped (409 error conflict nd 400 Badrequest).
Please let me know how to stop the processor.
Thanks in advance.
you can able to stop processor using rest api.
Example:
i having GetFile(ID:9204b68d-0159-1000-7d8f-720592b2a2dd) processor in UI.
RestAPI Url:
http://<host>:<port>/nifi-api/processors/9204b68d-0159-1000-7d8f-720592b2a2dd
Here json content i have passed as PUT Request to stop processor.
{
"status": {
"runStatus": "STOPPED"
},
"component": {
"state": "STOPPED",
"id": "9204b68d-0159-1000-7d8f-720592b2a2dd"
},
"id": "9204b68d-0159-1000-7d8f-720592b2a2dd",
"revision": {
"version": 10,
"clientId": "ab010dd6-0159-1000-615b-f095502a7ceb"
}
}
Revision and Status are most important things in stop the processor from RestAPI.
It works well for me.Try it.
And let me know if not worked.
Related
During developing pipeline which will use Elasticsearch as a source I faced with issue related paging. I am using SQL Elasticsearch API. Basically, I've started to do request in postman and it works well. The body of request looks following:
{
"query":"SELECT Id,name,ownership,modifiedDate FROM \"core\" ORDER BY Id",
"fetch_size": 20,
"cursor" : ""
}
After first run in response body it contains cursor string which is pointer to next page. If in postman I send the request and provide cursor value from previous request it return data for second page and so on. I am trying to archive the same result in Azure Data Factory. For this I using copy activity, which store response to Azure blob. Setup for source is following.
copy activity source configuration
This is expression for body
{
"query": "SELECT Id,name,ownership,modifiedDate FROM \"#{variables('TableName')}\" WHERE ORDER BY Id","fetch_size": #{variables('Rows')}, "cursor": ""
}
I have no idea how to correctly setup pagination rule. The pipeline works properly but only for the first request. I've tried to setup Headers.cursor and expression $.cursor but this setup leads to an infinite loop and pipeline fails with the Elasticsearch restriction.
I've also tried to read document at https://learn.microsoft.com/en-us/azure/data-factory/connector-rest#pagination-support but it seems pretty limited in terms of usage examples and difficult for understanding.
Could somebody help me understand how to build the pipeline with paging abilities utilization?
Responce with the cursor looks like:
{
"columns": [
{
"name": "companyId",
"type": "integer"
},
{
"name": "name",
"type": "text"
},
{
"name": "ownership",
"type": "keyword"
},
{
"name": "modifiedDate",
"type": "datetime"
}
],
"rows": [
[
2,
"mic Inc.",
"manufacture",
"2021-03-31T12:57:51.000Z"
]
],
"cursor": "g/WuAwFaAXNoRG5GMVpYSjVWR2hsYmtabGRHTm9BZ0FBQUFBRUp6VGxGbUpIZWxWaVMzcGhVWEJITUhkbmJsRlhlUzFtWjNjQUFBQUFCQ2MwNWhaaVIzcFZZa3Q2WVZGd1J6QjNaMjVSVjNrdFptZDP/////DwQBZgljb21wYW55SWQBCWNvbXBhbnlJZAEHaW50ZWdlcgAAAAFmBG5hbWUBBG5hbWUBBHRleHQAAAABZglvd25lcnNoaXABCW93bmVyc2hpcAEHa2V5d29yZAEAAAFmDG1vZGlmaWVkRGF0ZQEMbW9kaWZpZWREYXRlAQhkYXRldGltZQEAAAEP"
}
I finally find the solution, hopefully, it will be useful for the community.
Basically, what needs to be done it is split the solution into four steps.
Step 1 Make the first request as in the question description and stage file to blob.
Step 2 Read blob file and get the cursor value, set it to variable
Step 3 Keep requesting data with a changed body
{"cursor" : "#{variables('cursor')}" }
Pipeline looks like this:
pipeline
Configuration of pagination looks following
pagination . It is a workaround as the server ignores this header, but we need to have something which allows sending a request in loop.
I have an Spring Boot application which uses Apache Camel for routing along with ActiveMQ as a message broker. I also use Hawtio for routing monitorization and to stop/start the routes.
Everything is working great except for one single route, the one that has a resequencer used to sort a batch of messages before processing them. I can stop this route from Hawtio, but when I try to start the route again, Hawtio gives me this error:
{
"request": {
"mbean": "org.apache.camel:context=camel,name=\"MAIL_READER\",type=routes",
"attribute": "LastError",
"type": "read"
},
"value": {
"exception": {
"routeId": "MAIL_READER",
"localizedMessage": "Failed to start route MAIL_READER because of Route(MAIL_READER)[From[activemq:queue:MAIL_READER...",
"cause": {
"localizedMessage": "java.lang.IllegalThreadStateException",
"cause": {
"localizedMessage": null,
"cause": null,
"suppressed": [],
"message": null
},
"suppressed": [],
"message": "java.lang.IllegalThreadStateException"
},
"suppressed": [],
"message": "Failed to start route MAIL_READER because of Route(MAIL_READER)[From[activemq:queue:MAIL_READER..."
},
"phase": "START"
},
"timestamp": 1608036365,
"status": 200
}
If I remove the resequencer from the route, the stop/start works fine. I can pause/resume the route with the resequencer though, without any kind of problem. This is the Camel route definition in the Spring Boot application:
from("activemq:queue:mail_reader?transacted=true")
.resequence(simple("${header.priority}")).batch().timeout(30000)
.process(mailProcessor)
.to("activemq:queue:processed_mails");
Seems like the resequencer is trying to use the original threads instead creating new ones when I send the route start order. Is there some kind of known problem with the Camel resequencer and threads? I found this, and it seems to be solved...
Thanks in advance.
I'm working with Google Auto ML Natural Language API.
I have already a trained model.
In the beginning, when I trained the model, it was deployed and everything was fine. According with Google's new rules from 22th January 2019, models that have no prediction traffic for 7 or more weeks will be silently undeployed and archived.
At moment, I can't predict any result with that trained model, because it is undeployed as it was probably unused for 7 weeks.
Also according to Google if id like to continue using the model I should redeploy it using the Deploy API.
https://cloud.google.com/natural-language/automl/docs/models#deploying_or_undeploying_a_model
I try to redeploy the model and I get an error so I can't make any prediction.
How can I deploy a model, without errors, in order to begin predict results?
So, I'll show the steps that I made to try to solve this problem:
Run deploy request with right data.
Run operations request
Wait for deployment to finish
Run operations request again
Show list of models that I have (It's UNDEPLOYED)
1
https://automl.googleapis.com/v1beta1/projects/{project}/locations/{location}/models/{Model ID}:deploy
{
"name": "projects/{project}/locations/{location}/operations/{Model ID}",
"metadata": {
"#type": "type.googleapis.com/google.cloud.automl.v1beta1.OperationMetadata",
"createTime": {Time},
"updateTime": {Time}
}
}
2 and 4
https://automl.googleapis.com/v1beta1/projects/{project}/locations/{location}/operations
"operations": [
{
"name": "projects/{project}/locations/{location}/operations/{Model ID}",
"metadata": {
"#type": "type.googleapis.com/google.cloud.automl.v1beta1.OperationMetadata",
"createTime": {Time},
"updateTime": "{Time},
"progressPercent": 100
},
"done": true,
"error": {
"code": 4
}
}
]
5
https://automl.googleapis.com/v1beta1/projects/{project}/locations/{location}/models
"model": [
{
"name": "projects/{project}/locations/{location}/models/{Model ID}",
"displayName": {name},
"datasetId": {dataset id},
"createTime": {time},
"deploymentState": "UNDEPLOYED",
"updateTime": {time},
"textClassificationModelMetadata": {}
}
]
So, I was expecting 0 errors in operations request, when the model finished the deployment progress, but it shows an error code 4. I have searched this error code 4 in this provided enum: https://github.com/googleapis/googleapis/blob/master/google/rpc/code.proto
For error code 4:
// The deadline expired before the operation could complete. For operations
// that change the state of the system, this error may be returned
// even if the operation has completed successfully. For example, a
// successful response from a server could have been delayed long
// enough for the deadline to expire.
//
// HTTP Mapping: 504 Gateway Timeout
DEADLINE_EXCEEDED = 4;
I don't know why this timeout is happening.
I already search in Quotas Limits, but everything is fine.
This problem should be resolved now. Sorry about the inconvenience, but your model should be deployable now. Please try and write back if you still see an issue.
I am trying out the long running recognize method of the Speech-to-Text API (https://cloud.google.com/speech-to-text/docs/reference/rest/v1p1beta1/speech/longrunningrecognize) and specified all needed parameters such as:
{
"audio":
{
"uri": "gs://xyz/blabla.mp3"
},
"config":
{
"languageCode": "en-US",
"encoding": "AMR_WB",
"sampleRateHertz": 16000
}
}
This returned a name I can use with the get operation (https://cloud.google.com/speech-to-text/docs/reference/rest/v1/operations/get).
The documentation says the "operation" JSON object returned by get would include parameters that I do not see in the response.
For example, there is no "done" node. Instead this is all I get:
{
"name": "xxxxx",
"metadata": {
"#type": "type.googleapis.com/google.cloud.speech.v1.LongRunningRecognizeMetadata",
"progressPercent": 100,
"startTime": "2018-06-08T14:40:54.663240Z",
"lastUpdateTime": "2018-06-08T15:05:01.161911Z"
}
}
Any idea why that is? Should at least return a status and maybe an error (https://cloud.google.com/speech-to-text/docs/reference/rest/v1p1beta1/operations#Operation)?
UPDATE: Now I am getting results. Server issues, however? Is it only a temporary glitch?
{
"name": "xxxxx",
"metadata": {
"#type": "http://type.googleapis.com/google.cloud.speech.v1.LongRunningRecognizeMetadata …",
"progressPercent": 100,
"startTime": "2018-06-08T14:40:54.663240Z",
"lastUpdateTime": "2018-06-08T15:05:01.161911Z"
},
"done": true,
"error": {
"code": 13,
"message": "Server unavailable, please try again later."
}
}
At first sight your request is mixing an unsupported mp3 format versus a supported audio encoding (AMR_WB).
Let's suppose that this mixture is ok. If you receive an empty response (a transcript is not returned and no errors have occurred), it's probably that the encoding in your file is wrong. Check some validation steps in the preceding link to determine if your sound file have troubles, for example Cloud Speech-to-Text service currently supports only one audio channel.
To narrow down your issue, you can convert your sound file following the best practices. It will be enough to transcode your file to lossless FLAC or LINEAR16 encodings with a sampling rate of 16,000 Hz or higher, however for whole recommendations please read the prior link.
The error in your last update it seems to be temporary, do you still face the issue?
If your issue persists with the new file, it could be a good idea to report this situation in their public issue tracker.
Regards!
I am trying to use Google Cloud Logging API to write log entries from a web application I'm developing (happens to be .net).
To do this, I must use the logging.projects.logs.entries.write request. This request dictates that I provide a serviceName argument:
{
"entries": [
{
"textPayload": "test",
"metadata":
{
"serviceName": "compute.googleapis.com"
"projectId": "...",
"region": "us-central1",
"zone": "us-central1-a",
"severity": "DEFAULT",
"timestamp": "2015-01-13T19:17:01Z",
"userId": "",
}
}]
}
Unless I specify "compute.googleapis.com" as the serviceName I get an error 400 response:
{
"error":
{
"code": 400,
"message": "Unsupported service specified",
"status": "INVALID_ARGUMENT"
}
}
For now using "compute.googleapis.com" seems to work but I'm asking - what service name should I give, given that I'm not using Google Compute Engine or Google App Engine here?
The Cloud Logging API currently only officially supports Google resources, so the best course of action is to continue to use "compute.googleapis.com" as the service and supply the labels "compute.googleapis.com/resource_type" and "compute.googleapis.com/resource_id", which are used for indexing and visible in the UI drop-downs.
We also currently permit the service name "custom.googleapis.com" with index labels "custom.googleapis.com/primary_key" and "custom.googleapis.com/secondary_key" but that is not officially supported and subject to change in a future release.