I am trying to get a list of all PostgreSQL instances from Alicloud account using Aliyun CLI. But in the output, there are not tags, I would like to filter it through some tags:
aliyun rds DescribeDBInstances
Output (as from aliyun docs):
{
"RequestId": "1AD222E9-E606-4A42-BF6D-8A4442913CEF",
"PageNumber": 1,
"PageRecordCount": 10,
"TotalRecordCount": 100,
"Items": [
{
"VpcId": "vpc-uf6f7l4fg90xxxxxxx",
"DedicatedHostIdForLog": "dh-bpxxxx",
"CreateTime": "2018-11-05T11:26:02Z",
"PayType": "Postpaid",
"DedicatedHostNameForLog": "testlog",
"MutriORsignle": true,
"DedicatedHostGroupName": "testhostgroup",
"EngineVersion": "5.7",
"DedicatedHostGroupId": "dhg-7a9xxxxxxxx",
"VpcName": "test-huadong",
"DedicatedHostZoneIdForMaster": "cn-hangzhou-c",
"ConnectionString": "rm-uf6wjk5xxxxxxx.mysql.rds.aliyuncs.com",
"InstanceNetworkType": "Classic",
"MasterInstanceId": "rm-uf6wjk5xxxxxxxxxx",
"ExpireTime": "2019-02-27T16:00:00Z",
"DestroyTime": "2018-11-05T11:26:02Z",
"GuardDBInstanceId": "rm-uf64zsuxxxxxxxxxx",
"DedicatedHostNameForMaster": "testmaster",
"ZoneId": "cn-hangzhou-a",
"TipsLevel": 1,
"DBInstanceId": "rm-uf6wjk5xxxxxxxxxx",
"DedicatedHostIdForMaster": "dh-bpxxxx",
"TempDBInstanceId": "rm-uf64zsuxxxxxxxxxx",
"DBInstanceStorageType": "ModuleList.4.ModuleCode",
"ConnectionMode": "Standard",
"LockMode": "Unlock",
"GeneralGroupName": "TestGroup",
"VpcCloudInstanceId": "rm-uf6wjk5xxxxxxx",
"DedicatedHostZoneIdForSlave": "cn-hangzhou-d",
"Tips": "一切正常",
"DedicatedHostZoneIdForLog": "cn-hangzhou-b",
"DedicatedHostNameForSlave": "testslave",
"DBInstanceDescription": "测试数据库",
"DBInstanceNetType": "Internet",
"DBInstanceType": "Primary",
"LockReason": "instance_expired",
"DBInstanceStatus": "Running",
"RegionId": "cn-hangzhou",
"VSwitchId": "vsw-uf6adz52c2pxxxxxxx",
"DedicatedHostIdForSlave": "dh-bpxxxx",
"ResourceGroupId": "rg-acfmyxxxxxxx",
"Category": "Basic",
"Engine": "MySQL",
"DBInstanceClass": "rds.mys2.small",
"SwitchWeight": 100,
"ReadOnlyDBInstanceIds": [
{
"DBInstanceId": "rr-uf6wjk5xxxxxxx"
}
],
"DeletionProtection": true
}
],
"NextToken": "o7PORW5o2TJg**********"
}
As per documentation - Tags are given as a request parameter and not in return data. Is there any way to pull tags from db instances and get it with the return data?
Related
In AWS CloudWatch -> X-Ray -> Traces I can find traces like:
{
"Id": "1234",
"Duration": 26.753,
"LimitExceeded": false,
"Segments": [
{
"Id": "1qaz",
"Document": {
"id": "1qaz",
"name": "somesite.io",
"start_time": 221,
"trace_id": "1234",
"end_time": 222,
"parent_id": "qwerty",
"inferred": true,
"http": {
"request": {
"url": "https://somesite.io/api/SUPERID",
"method": "POST"
},
"response": {
"status": 200,
"content_length": 100
}
}
}
}
]
}
I wish to find trace with value SUPERID what is in above JSON in:
Segments[?].Document.http.request.url
To sum up: I'm looking for example query what should be entered to Find input to find above trace by SUPERID value
Refer to X-Ray filter expression
There are 2 ways:
filter by http.url
Add SUPERID in segment annotation, then filter it by annotation.key
I'm still relatively new to Elasticsearch and, currently, I'm attempting to switch from Solr to Elasticsearch and am seeing a huge increase in CPU usage when ES is on our production website. The site sees anywhere from 10,000 to 30,000 requests to ES per second. Solr handles that load just fine with our current hardware.
The books index mapping: https://pastebin.com/bKM9egPS
A query for a book: https://pastebin.com/AdfZ895X
ES is hosted on AWS on an m4.xlarge.elasticsearch instance.
Our cluster is set up as follows (anything not included is default):
"persistent": {
"cluster": {
"routing": {
"allocation": {
"cluster_concurrent_rebalance": "2",
"node_concurrent_recoveries": "2",
"disk": {
"watermark": {
"low": "15.0gb",
"flood_stage": "5.0gb",
"high": "10.0gb"
}
},
"node_initial_primaries_recoveries": "4"
}
}
},
"indices": {
"recovery": {
"max_bytes_per_sec": "60mb"
}
}
Our nodes have the following configuration:
"_nodes": {
"total": 2,
"successful": 2,
"failed": 0
},
"cluster_name": "cluster",
"nodes": {
"####": {
"name": "node1",
"version": "6.3.1",
"build_flavor": "oss",
"build_type": "zip",
"build_hash": "####",
"roles": [
"master",
"data",
"ingest"
]
},
"###": {
"name": "node2",
"version": "6.3.1",
"build_flavor": "oss",
"build_type": "zip",
"build_hash": "###",
"roles": [
"master",
"data",
"ingest"
]
}
}
Can someone please help me figure out what exactly is happening so I can get this deployment finished?
When attempting to index the following document:
{
"branch": "master",
"classes": [
{
"content_count": 2,
"documentation": "",
"extends": [],
"generic": "",
"implements": [],
"line": 10,
"line_count": 36,
"modifiers": [
"public"
],
"name": "removeDuplicateFromString"
}
],
"commit_hash": "e53249ba2381d2f20f3d4493ad70e2da0abb3b05",
"contributors": [
{
"id": "7676016",
"name": "varunu28",
"url": "https://github.com/varunu28"
}
],
"enums": [],
"fields": [],
"filename": "removeDuplicateFromString.java",
"imports": [
{
"name": "java.io.BufferedReader",
"wildcard": false
},
{
"name": "java.io.InputStreamReader",
"wildcard": false
}
],
"interfaces": [],
"license": "",
"methods": [
{
"cyclomatic_complexity": 1,
"documentation": "",
"generic": "",
"line": 11,
"line_count": 9,
"modifiers": [
"public",
"static"
],
"name": "main",
"params": [
{
"name": "args",
"type": "String[]"
}
],
"parent": "removeDuplicateFromString",
"type_": "void"
},
{
"cyclomatic_complexity": 5,
"documentation": "",
"generic": "",
"line": 29,
"line_count": 16,
"modifiers": [
"public",
"static"
],
"name": "removeDuplicate",
"params": [
{
"name": "s",
"type": "String"
}
],
"parent": "removeDuplicateFromString",
"type_": "String"
}
],
"number_forks": 1695,
"number_stars": 4000,
"number_watchs": 394,
"package": "",
"path": "Others",
"repository": "TheAlgorithms/Java"
}
I get the following error:
{"error":{"root_cause":[{"type":"settings_exception","reason":"Failed to load settings from [{\"interfaces\":[],\"imports\":[{\"name\":\"java.io.BufferedReader\",\"wildcard\":false},{\"name\":\"java.io.InputStreamReader\",\"wildcard\":false}],\"package\":\"\",\"methods\":[{\"parent\":\"removeDuplicateFromString\",\"line_count\":9,\"line\":11,\"documentation\":\"\",\"name\":\"main\",\"cyclomatic_complexity\":1,\"modifiers\":[\"public\",\"static\"],\"params\":[{\"name\":\"args\",\"type\":\"String[]\"}],\"type_\":\"void\",\"generic\":\"\"},{\"parent\":\"removeDuplicateFromString\",\"line_count\":16,\"line\":29,\"documentation\":\"\",\"name\":\"removeDuplicate\",\"cyclomatic_complexity\":5,\"modifiers\":[\"public\",\"static\"],\"params\":[{\"name\":\"s\",\"type\":\"String\"}],\"type_\":\"String\",\"generic\":\"\"}],\"number_forks\":1695,\"classes\":[{\"implements\":[],\"line_count\":36,\"extends\":[],\"line\":10,\"documentation\":\"\",\"name\":\"removeDuplicateFromString\",\"content_count\":2,\"modifiers\":[\"public\"],\"generic\":\"\"}],\"repository\":\"TheAlgorithms/Java\",\"branch\":\"master\",\"commit_hash\":\"e53249ba2381d2f20f3d4493ad70e2da0abb3b05\",\"enums\":[],\"path\":\"Others\",\"license\":\"\",\"filename\":\"removeDuplicateFromString.java\",\"number_watchs\":394,\"contributors\":[{\"name\":\"varunu28\",\"id\":\"7676016\",\"url\":\"https://github.com/varunu28\"}],\"fields\":[],\"number_stars\":4000}]"}],"type":"settings_exception","reason":"Failed to load settings from [{\"interfaces\":[],\"imports\":[{\"name\":\"java.io.BufferedReader\",\"wildcard\":false},{\"name\":\"java.io.InputStreamReader\",\"wildcard\":false}],\"package\":\"\",\"methods\":[{\"parent\":\"removeDuplicateFromString\",\"line_count\":9,\"line\":11,\"documentation\":\"\",\"name\":\"main\",\"cyclomatic_complexity\":1,\"modifiers\":[\"public\",\"static\"],\"params\":[{\"name\":\"args\",\"type\":\"String[]\"}],\"type_\":\"void\",\"generic\":\"\"},{\"parent\":\"removeDuplicateFromString\",\"line_count\":16,\"line\":29,\"documentation\":\"\",\"name\":\"removeDuplicate\",\"cyclomatic_complexity\":5,\"modifiers\":[\"public\",\"static\"],\"params\":[{\"name\":\"s\",\"type\":\"String\"}],\"type_\":\"String\",\"generic\":\"\"}],\"number_forks\":1695,\"classes\":[{\"implements\":[],\"line_count\":36,\"extends\":[],\"line\":10,\"documentation\":\"\",\"name\":\"removeDuplicateFromString\",\"content_count\":2,\"modifiers\":[\"public\"],\"generic\":\"\"}],\"repository\":\"TheAlgorithms/Java\",\"branch\":\"master\",\"commit_hash\":\"e53249ba2381d2f20f3d4493ad70e2da0abb3b05\",\"enums\":[],\"path\":\"Others\",\"license\":\"\",\"filename\":\"removeDuplicateFromString.java\",\"number_watchs\":394,\"contributors\":[{\"name\":\"varunu28\",\"id\":\"7676016\",\"url\":\"https://github.com/varunu28\"}],\"fields\":[],\"number_stars\":4000}]","caused_by":{"type":"illegal_state_exception","reason":"only value lists are allowed in serialized settings"}},"status":500}
From which I've gathered that the main issues are either described in the part saying that:
{"type":"illegal_state_exception","reason":"only value lists are allowed in serialized settings"}}
Or:
"error":{"root_cause":[{"type":"settings_exception","reason":"Failed to load settings from [{\"interfaces\":[],\"imports\": ........
But I cannot find any information about this error or what it could be caused by. I've tried indexing both using a predefined index with mappings and to a non-existing index. Nothing seems to work.
Why can't I index this document?
It turns out that, as Farid mentioned in the comments section, I was using the wrong command when indexing from the command line.
The correct command to run is
curl -X POST -H 'Content-Type: application/json' [index location] -d [data]
Where the key is that you use POST and not PUT which is what I was doing.
Adding this for the ones using Kibana Dev Tools.
The key is to use an document type after an index name when adding the document
POST /{index name}/{document type}
{
request body (document) goes here.
}
Rasa NLU version (0.11.3):
Used backend / pipeline (spacy_sklearn):
Operating system (osx):
Issue: I tried to follow the tutorial: https://rasahq.github.io/rasa_nlu/tutorial.html?highlight=project#,
Installed spaCy + sklearn
Created config_spacy.json
Downloaded sample file and train
I've test greeting and goodbye intent and they are work
but when I test with command:
curl -X POST localhost:5000/parse -d '{"q":"I am looking for Mexican food"}' | python -m json.tool
it returns:
{
"intent": {
"name": "None",
"confidence": 1.0
},
"entities": [],
"text": "yes"
}
Content of configuration file (if used & relevant):
{
"project": null,
"fixed_model_name": null,
"config": "config.json",
"data": null,
"emulate": null,
"language": "en",
"log_file": null,
"log_level": "INFO",
"mitie_file": "data/total_word_feature_extractor.dat",
"spacy_model_name": null,
"num_threads": 1,
"max_training_processes": 1,
"path": "/rasa_nlu/projects",
"port": 5000,
"token": null,
"cors_origins": [],
"max_number_of_ngrams": 7,
"pipeline": [],
"response_log": "/rasa_nlu/logs",
"storage": null,
"aws_endpoint_url": null,
"duckling_dimensions": null,
"duckling_http_url": null,
"ner_crf": {
"BILOU_flag": true,
"features": [
[
"low",
"title",
"upper",
"pos",
"pos2"
],
[
"bias",
"low",
"word3",
"word2",
"upper",
"title",
"digit",
"pos",
"pos2",
"pattern"
],
[
"low",
"title",
"upper",
"pos",
"pos2"
]
],
"max_iterations": 50,
"L1_c": 1,
"L2_c": 0.001
},
"intent_classifier_sklearn": {
"C": [
1,
2,
5,
10,
20,
100
],
"kernel": "linear"
}
}
Status:
{
"available_projects": {
"default": {
"status": "ready",
"available_models": [
"fallback"
]
}
}
}
In your config file the pipeline is set to [] but needs to be configured properly. The documentation for the pipeline configuration option can be found here. The available options are discussed here.
The pipeline can either be a pre-configured pipeline like: mitie, spacy_sklearn, or keyword. It can also be a custom pipeline like: ["nlp_spacy", "ner_crf", "ner_synonyms"]. I would recommend setting your pipeline to:
pipeline: "space_sklearn"
Update your configuration file and restart the server. If the server is still running in a console window press Ctrl + c to stop it. Then re-enter the command you used to start it.
I'm trying to use the Amazon EC2 Systems Manager (http://docs.aws.amazon.com/systems-manager/latest/userguide/what-is-systems-manager.html) to create an "Automation" document type to (amongst other things) tag an AMI it just created.
You can create tags in a predetermined manner like this within "mainSteps":
...
{
"name": "CreateTags",
"action": "aws:createTags",
"maxAttempts": 3,
"onFailure": "Abort",
"inputs": {
"ResourceType": "EC2",
"ResourceIds": ["{{ CreateImage.ImageId }}"],
"Tags": [
{
"Key": "Original_AMI_ID",
"Value": "Created from {{ SourceAmiId }}"
}
]
}
},
...
but to tag with a variable number of Tags, I'm assuming the following change is neccessary:
...
{
"name": "CreateTags",
"action": "aws:createTags",
"maxAttempts": 3,
"onFailure": "Abort",
"inputs": {
"ResourceType": "EC2",
"ResourceIds": ["{{ CreateImage.ImageId }}"],
"Tags": {{ Tags }}
}
},
...
with the addition of a new parameter called 'Tags' of type 'MapList':
"parameters": {
"Tags": {
"type": "MapList"
}
}
since running the process was complaining about my using a 'String' type and saying I should use a 'MapList'.
'MapList' is listed as a parameter type of the Amazon EC2 Systems Manager (http://docs.aws.amazon.com/systems-manager/latest/APIReference/top-level.html), but I have not yet found any documentation on how to define this type.
I have guessed at several formats based upon both what I've seen from their 'hardcoded' sample above and other tagging method in their other API's to no avail:
[ { "Key": "Name", "Value": "newAmi" } ]
[ { "Key": "Name", "Values": [ "newAmi" ] } ]
1: { "Key": "Name", "Values": [ "newAmi" ] }
Does anyone know how to define the new parameter types introduced with the Amazon EC2 Systems Manager (specifically, 'MapList')?
Update:
Since the docs are lacking, Amazon Support is asking the automation team how to best tag ami's using this method. I have found how to add a single tag as a parameter value in the console, though:
{ "Key": "TagName", "Value": "TagValue" }
My attempts to add multiple tags will allow the automation to start:
{ "Key": "TagName1", "Value": "TagValue1" }, { "Key": "TagName2", "Value": "TagValue2" }
but ultimately returns this generic error at runtime:
Internal Server Error. Please refer to Automation Service Troubleshooting
Guide for more diagnosis details
It might seem like the [] is missing from around the array, but you seem to get those for free because when I add them I get this error:
Parameter type error. [[ { "Key": "Description", "Value": "Desc" },
{ "Key": "Name", "Value": "Nm" } ]] is defined as MapList.
Thanks for using EC2 Systems Manager, Automation feature. Here's the document I tested, it works.
{
"schemaVersion": "0.3",
"description": "Test tags.",
"assumeRole": "arn:aws:iam::123456789012:role/TestRole",
"parameters": {
"Tags": {
"default": [{
"Key": "TagName1",
"Value": "TagValue1"
},
{
"Key": "TagName2",
"Value": "TagValue2"
}],
"type": "MapList"
}
},
"mainSteps": [
{
"name": "CreateTags",
"action": "aws:createTags",
"maxAttempts": 3,
"onFailure": "Abort",
"inputs": {
"ResourceType": "EC2",
"ResourceIds": [
"i-12345678"
],
"Tags": "{{ Tags }}"
}
}
]
}