Deploy a AUTOML NL Trainned Model Fails - automl

I'm working with Google Auto ML Natural Language API.
I have already a trained model.
In the beginning, when I trained the model, it was deployed and everything was fine. According with Google's new rules from 22th January 2019, models that have no prediction traffic for 7 or more weeks will be silently undeployed and archived.
At moment, I can't predict any result with that trained model, because it is undeployed as it was probably unused for 7 weeks.
Also according to Google if id like to continue using the model I should redeploy it using the Deploy API.
https://cloud.google.com/natural-language/automl/docs/models#deploying_or_undeploying_a_model
I try to redeploy the model and I get an error so I can't make any prediction.
How can I deploy a model, without errors, in order to begin predict results?
So, I'll show the steps that I made to try to solve this problem:
Run deploy request with right data.
Run operations request
Wait for deployment to finish
Run operations request again
Show list of models that I have (It's UNDEPLOYED)
1
https://automl.googleapis.com/v1beta1/projects/{project}/locations/{location}/models/{Model ID}:deploy
{
"name": "projects/{project}/locations/{location}/operations/{Model ID}",
"metadata": {
"#type": "type.googleapis.com/google.cloud.automl.v1beta1.OperationMetadata",
"createTime": {Time},
"updateTime": {Time}
}
}
2 and 4
https://automl.googleapis.com/v1beta1/projects/{project}/locations/{location}/operations
"operations": [
{
"name": "projects/{project}/locations/{location}/operations/{Model ID}",
"metadata": {
"#type": "type.googleapis.com/google.cloud.automl.v1beta1.OperationMetadata",
"createTime": {Time},
"updateTime": "{Time},
"progressPercent": 100
},
"done": true,
"error": {
"code": 4
}
}
]
5
https://automl.googleapis.com/v1beta1/projects/{project}/locations/{location}/models
"model": [
{
"name": "projects/{project}/locations/{location}/models/{Model ID}",
"displayName": {name},
"datasetId": {dataset id},
"createTime": {time},
"deploymentState": "UNDEPLOYED",
"updateTime": {time},
"textClassificationModelMetadata": {}
}
]
So, I was expecting 0 errors in operations request, when the model finished the deployment progress, but it shows an error code 4. I have searched this error code 4 in this provided enum: https://github.com/googleapis/googleapis/blob/master/google/rpc/code.proto
For error code 4:
// The deadline expired before the operation could complete. For operations
// that change the state of the system, this error may be returned
// even if the operation has completed successfully. For example, a
// successful response from a server could have been delayed long
// enough for the deadline to expire.
//
// HTTP Mapping: 504 Gateway Timeout
DEADLINE_EXCEEDED = 4;
I don't know why this timeout is happening.
I already search in Quotas Limits, but everything is fine.

This problem should be resolved now. Sorry about the inconvenience, but your model should be deployable now. Please try and write back if you still see an issue.

Related

Azure Data Factory REST API paging with Elasticsearch

During developing pipeline which will use Elasticsearch as a source I faced with issue related paging. I am using SQL Elasticsearch API. Basically, I've started to do request in postman and it works well. The body of request looks following:
{
"query":"SELECT Id,name,ownership,modifiedDate FROM \"core\" ORDER BY Id",
"fetch_size": 20,
"cursor" : ""
}
After first run in response body it contains cursor string which is pointer to next page. If in postman I send the request and provide cursor value from previous request it return data for second page and so on. I am trying to archive the same result in Azure Data Factory. For this I using copy activity, which store response to Azure blob. Setup for source is following.
copy activity source configuration
This is expression for body
{
"query": "SELECT Id,name,ownership,modifiedDate FROM \"#{variables('TableName')}\" WHERE ORDER BY Id","fetch_size": #{variables('Rows')}, "cursor": ""
}
I have no idea how to correctly setup pagination rule. The pipeline works properly but only for the first request. I've tried to setup Headers.cursor and expression $.cursor but this setup leads to an infinite loop and pipeline fails with the Elasticsearch restriction.
I've also tried to read document at https://learn.microsoft.com/en-us/azure/data-factory/connector-rest#pagination-support but it seems pretty limited in terms of usage examples and difficult for understanding.
Could somebody help me understand how to build the pipeline with paging abilities utilization?
Responce with the cursor looks like:
{
"columns": [
{
"name": "companyId",
"type": "integer"
},
{
"name": "name",
"type": "text"
},
{
"name": "ownership",
"type": "keyword"
},
{
"name": "modifiedDate",
"type": "datetime"
}
],
"rows": [
[
2,
"mic Inc.",
"manufacture",
"2021-03-31T12:57:51.000Z"
]
],
"cursor": "g/WuAwFaAXNoRG5GMVpYSjVWR2hsYmtabGRHTm9BZ0FBQUFBRUp6VGxGbUpIZWxWaVMzcGhVWEJITUhkbmJsRlhlUzFtWjNjQUFBQUFCQ2MwNWhaaVIzcFZZa3Q2WVZGd1J6QjNaMjVSVjNrdFptZDP/////DwQBZgljb21wYW55SWQBCWNvbXBhbnlJZAEHaW50ZWdlcgAAAAFmBG5hbWUBBG5hbWUBBHRleHQAAAABZglvd25lcnNoaXABCW93bmVyc2hpcAEHa2V5d29yZAEAAAFmDG1vZGlmaWVkRGF0ZQEMbW9kaWZpZWREYXRlAQhkYXRldGltZQEAAAEP"
}
I finally find the solution, hopefully, it will be useful for the community.
Basically, what needs to be done it is split the solution into four steps.
Step 1 Make the first request as in the question description and stage file to blob.
Step 2 Read blob file and get the cursor value, set it to variable
Step 3 Keep requesting data with a changed body
{"cursor" : "#{variables('cursor')}" }
Pipeline looks like this:
pipeline
Configuration of pagination looks following
pagination . It is a workaround as the server ignores this header, but we need to have something which allows sending a request in loop.

Use GitHub GraphQL to retrieve all open milestones and all issues (open and closed) for the milestone

I understand that GitHub's GraphQL-based v4 API is much more efficient than the v3 API.
I would like to use the GraphQL API to retrieve, for a given repo:
All of the open milestones.
For each milestone, its title, description, all of its issues (open and closed)
For each issue, its title, description, status, and all messages.
Is there a straightforward way to do this?
Yes. It is straightforward to do so . The query looks like :
{
repository(owner: "gatsbyjs", name: "gatsby") {
description
url
milestones(states: [OPEN],first:2) {
nodes{
title
description
url
issues(states:[OPEN,CLOSED], first:2){
nodes{
title
state
url
comments(first:2){
nodes{
url
body
createdAt
}
pageInfo{
hasNextPage
endCursor
}
}
}
pageInfo{
endCursor
hasNextPage
}
}
}
pageInfo{
endCursor
hasNextPage
}
}
}
}
Note:
For the repository which the url is https://github.com/gatsbyjs/gatsby , its owner is gatsbyjs and its name is gatsby
Go to its API Explorer to try and fine tune the query.Click Ctrl+Space will auto-suggest the available fields that can be retrieved.
Do the paginating by yourself to loop through all records by adjusting the starting cursor and the number of records to be returned in first , after.
It gives you the following :
{
"data": {
"repository": {
"description": "Build blazing fast, modern apps and websites with React",
"url": "https://github.com/gatsbyjs/gatsby",
"milestones": {
"nodes": [
{
"title": "Next Major",
"description": "Issues that will require a breaking change, and which would constitute being done in the next major version of Gatsby.",
"url": "https://github.com/gatsbyjs/gatsby/milestone/5",
"issues": {
"nodes": [
{
"title": "Make accessibility warnings errors",
"state": "OPEN",
"url": "https://github.com/gatsbyjs/gatsby/issues/19945",
"comments": {
"nodes": [
{
"url": "https://github.com/gatsbyjs/gatsby/issues/19945#issuecomment-568891716",
"body": "Hiya!\n\nThis issue has gone quiet. Spooky quiet. 👻\n\nWe get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.\nIf we missed this issue or if you want to keep it open, please reply here. You can also add the label \"not stale\" to keep this issue open!\nAs a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out [gatsby.dev/contribute](https://www.gatsbyjs.org/contributing/how-to-contribute/) for more information about opening PRs, triaging issues, and contributing!\n\nThanks for being a part of the Gatsby community! 💪💜",
"createdAt": "2019-12-25T12:02:26Z"
},
{
"url": "https://github.com/gatsbyjs/gatsby/issues/19945#issuecomment-570779866",
"body": "Hey again!\n\nIt’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.\nPlease keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m `HUMAN_EMOTION_SORRY`. Please feel free to reopen this issue or create a new one if you need anything else.\nAs a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out [gatsby.dev/contribute](https://www.gatsbyjs.org/contributing/how-to-contribute/) for more information about opening PRs, triaging issues, and contributing!\n\nThanks again for being part of the Gatsby community! 💪💜",
"createdAt": "2020-01-04T12:02:28Z"
}
],
"pageInfo": {
"hasNextPage": false,
"endCursor": "Y3Vyc29yOnYyOpHOIgVo2g=="
}
}
},
{
"title": "Configurable output folder",
"state": "OPEN",
"url": "https://github.com/gatsbyjs/gatsby/issues/1878",
"comments": {
"nodes": [
{
"url": "https://github.com/gatsbyjs/gatsby/issues/1878#issuecomment-324062470",
"body": "Do you have a specific use case in mind? This has been discussed before but no one has come up with a concrete use case that justified adding a new option.\r\n\r\nEvery option we add to Gatsby makes the project more complex which has all sorts of long-term costs so unless something is really valuable, I'd rather people handle this sort of thing themselves e.g. just copy the files to the output directory you want or create a symlink. This could easily be turned into a plugin that people could install, etc.",
"createdAt": "2017-08-22T15:27:41Z"
},
{
"url": "https://github.com/gatsbyjs/gatsby/issues/1878#issuecomment-324074853",
"body": "Yes, I have a use-case. I am going to use Gatsby for a documentation part as a part of complex project. All static files (Gatsby output, plus some others) should be placed into one folder `build`, that will be deployed somehow later. In other words, the Gatsby output is only one subfolder in my setup.\r\n\r\nSo far I have worked this around in `postbuild` step, but it looks hacky:\r\n\r\n```\r\n\"build\": \"gatsby build\",\r\n\"postbuild\": \"mv public build/gatsby-subsite\"\r\n```\r\nAdding configurable output folder will reduce this complexity and will help me not to move files around one more time.",
"createdAt": "2017-08-22T16:08:21Z"
}
],
"pageInfo": {
"hasNextPage": true,
"endCursor": "Y3Vyc29yOnYyOpHOE1D9ZQ=="
}
}
}
],
"pageInfo": {
"endCursor": "Y3Vyc29yOnYyOpLPgAAAAAAAArvODwULXA==",
"hasNextPage": true
}
}
}
],
"pageInfo": {
"endCursor": "Y3Vyc29yOnYyOpHOAEEbsw==",
"hasNextPage": false
}
}
}
}
}

Test ride Speech-to-Text asynchronous operation - no results

I am trying out the long running recognize method of the Speech-to-Text API (https://cloud.google.com/speech-to-text/docs/reference/rest/v1p1beta1/speech/longrunningrecognize) and specified all needed parameters such as:
{
"audio":
{
"uri": "gs://xyz/blabla.mp3"
},
"config":
{
"languageCode": "en-US",
"encoding": "AMR_WB",
"sampleRateHertz": 16000
}
}
This returned a name I can use with the get operation (https://cloud.google.com/speech-to-text/docs/reference/rest/v1/operations/get).
The documentation says the "operation" JSON object returned by get would include parameters that I do not see in the response.
For example, there is no "done" node. Instead this is all I get:
{
"name": "xxxxx",
"metadata": {
"#type": "type.googleapis.com/google.cloud.speech.v1.LongRunningRecognizeMetadata",
"progressPercent": 100,
"startTime": "2018-06-08T14:40:54.663240Z",
"lastUpdateTime": "2018-06-08T15:05:01.161911Z"
}
}
Any idea why that is? Should at least return a status and maybe an error (https://cloud.google.com/speech-to-text/docs/reference/rest/v1p1beta1/operations#Operation)?
UPDATE: Now I am getting results. Server issues, however? Is it only a temporary glitch?
{
"name": "xxxxx",
"metadata": {
"#type": "http://type.googleapis.com/google.cloud.speech.v1.LongRunningRecognizeMetadata …",
"progressPercent": 100,
"startTime": "2018-06-08T14:40:54.663240Z",
"lastUpdateTime": "2018-06-08T15:05:01.161911Z"
},
"done": true,
"error": {
"code": 13,
"message": "Server unavailable, please try again later."
}
}
At first sight your request is mixing an unsupported mp3 format versus a supported audio encoding (AMR_WB).
Let's suppose that this mixture is ok. If you receive an empty response (a transcript is not returned and no errors have occurred), it's probably that the encoding in your file is wrong. Check some validation steps in the preceding link to determine if your sound file have troubles, for example Cloud Speech-to-Text service currently supports only one audio channel.
To narrow down your issue, you can convert your sound file following the best practices. It will be enough to transcode your file to lossless FLAC or LINEAR16 encodings with a sampling rate of 16,000 Hz or higher, however for whole recommendations please read the prior link.
The error in your last update it seems to be temporary, do you still face the issue?
If your issue persists with the new file, it could be a good idea to report this situation in their public issue tracker.
Regards!

How to stop the single processor in nifi1.1.0 using rest api

I am trying to stop the processor which is in running state using the PUT method in rest api /processors/{id}.
I am able to start the processor by changing the state in the component as follows "state": "RUNNING", and runStatus in the aggregatesnapshot as "runStatus": "Running".
Similarly I tried to stop the processor by changing the state as STOPPED but facing an error as
9204b68d-0159-1000-7d8f-720592b2a2dd is not stopped (409 error conflict nd 400 Badrequest).
Please let me know how to stop the processor.
Thanks in advance.
you can able to stop processor using rest api.
Example:
i having GetFile(ID:9204b68d-0159-1000-7d8f-720592b2a2dd) processor in UI.
RestAPI Url:
http://<host>:<port>/nifi-api/processors/9204b68d-0159-1000-7d8f-720592b2a2dd
Here json content i have passed as PUT Request to stop processor.
{
"status": {
"runStatus": "STOPPED"
},
"component": {
"state": "STOPPED",
"id": "9204b68d-0159-1000-7d8f-720592b2a2dd"
},
"id": "9204b68d-0159-1000-7d8f-720592b2a2dd",
"revision": {
"version": 10,
"clientId": "ab010dd6-0159-1000-615b-f095502a7ceb"
}
}
Revision and Status are most important things in stop the processor from RestAPI.
It works well for me.Try it.
And let me know if not worked.

How to handle (failed) validation when creating multiple entities with a REST API

Let's say I have an endpoint user/1/results and I want to upload multiple results at a time.
So I send it JSON like:
{
"data": [
{
"date": "2014-02-14 03:15:41",
"score": 18649,
"time": 42892
},
{
"date": "2013-11-18 09:21:46",
"score": 7856,
"time": 23568.8
}]
}
Let's say time needs to be an integer, so the second entity fails validation.
What's the best thing to do:
Fail both, nothing saves, respond with error message..
Save first entity, respond with error message.
In either case, what would an error message look like? i.e. how/does it specify that it's the second entity that fails validation.
I think you should fail both and respond with an error message because it might be cumbersome again to track the remaining results.
Error message should give the details of failing location. for example if it fails at the second one then specify it in json response.

Resources