why ansible inventory parser my json so slow - performance

I generate a json from public cloud but ansible use this inventory.json so slow.
pls tell me why
my json like this:
{
"angelbeats": {
"hosts": ["10.193.0.24", "10.193.0.23"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"grammy": {
"hosts": ["10.193.2.5", "10.193.0.9", "10.193.10.4", "10.193.8.5"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"cdn": {
"hosts": ["10.193.0.12", "10.193.0.11", "10.193.2.7", "10.193.2.8"],
"vars": {
"ansible_ssh_user": "root"
}
},
"mdianying": {
"hosts": ["10.193.0.31"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"zkmq": {
"hosts": ["10.193.0.26", "10.193.0.27", "10.193.0.25"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"bmovie": {
"hosts": [],
"children": ["angelbeats", "growingpains", "forrestgump", "zkmq", "job", "tm", "inception"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"xxljob": {
"hosts": ["10.193.0.6"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"redis": {
"hosts": ["10.193.0.40", "10.193.2.13"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"tm": {
"hosts": ["10.193.0.13", "10.193.0.14", "10.193.0.16", "10.193.0.17", "10.193.0.15"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"dns": {
"hosts": ["10.193.0.100", "10.193.2.100", "10.193.10.10", "10.193.8.19"],
"vars": {
"ansible_ssh_user": "root"
}
},
"yctask": {
"hosts": ["10.193.8.15"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"cmovie": {
"hosts": [],
"children": ["angelbeats", "growingpains", "forrestgump", "zkmq", "job", "tm", "inception"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"forrestgump": {
"hosts": ["10.193.0.104", "10.193.2.101", "10.193.8.20", "10.193.10.11"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"pwapi": {
"hosts": ["10.193.0.28", "10.193.2.11", "10.193.8.10", "10.193.10.9"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"pwweb": {
"hosts": ["10.193.0.20", "10.193.0.19", "10.193.2.9", "10.193.2.10", "10.193.8.9", "10.193.8.8", "10.193.10.6", "10.193.10.7"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"pwtask": {
"hosts": ["10.193.0.29"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"apishow": {
"hosts": ["10.193.0.32", "10.193.8.11"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"inception": {
"hosts": ["10.193.0.10", "10.193.2.6", "10.193.8.7", "10.193.10.5"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"moviebi": {
"hosts": ["10.193.0.42", "10.193.8.18"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"yanchu": {
"hosts": ["10.193.0.35", "10.193.8.14"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"growingpains": {
"hosts": ["10.193.0.21", "10.193.0.22"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"goods": {
"hosts": ["10.193.0.33", "10.193.8.12"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"urm": {
"hosts": ["10.193.0.39", "10.193.2.12"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"job": {
"hosts": ["10.193.0.105"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
},
"order": {
"hosts": ["10.193.0.34", "10.193.8.13"],
"vars": {
"ansible_ssh_user": "wangluoli"
}
}
}

You need to create a _meta section in your JSON:
https://docs.ansible.com/ansible/latest/dev_guide/developing_inventory.html#tuning-the-external-inventory-script
The stock inventory script system detailed above works for all
versions of Ansible, but calling --host for every host can be rather
inefficient, especially if it involves API calls to a remote
subsystem.
To avoid this inefficiency, if the inventory script returns a top
level element called “_meta”, it is possible to return all of the host
variables in one script execution. When this meta element contains a
value for “hostvars”, the inventory script will not be invoked with
--host for each host. This results in a significant performance increase for large numbers of hosts.

Related

How to use XPATH to return an array of values based on a condition that a specific property exists in a JSON object array

How can I use XPATH to return an array based on the existence of a specific property?
Below is a section of my JSON file. Under "root" there are a number of array objects and SOME of them contain the property "detection". I would like to retrieve the "service_name" of each array object ONLY IF the object array (under root) contains the property "detection".
e.g., "service_name": "IPS" should be returned
but for the example below, the service_name should NOT be returned because property "detection" is not present
Finally, is there a way to combine the above query into one, in order to return an array of values "service_name" and "detection" together, based on the same condition?
My current Power Automate Set Variable command is:
xpath(xml(variables('varProductsRoot')), '//detection | //service_name')
and unfortunately it returns ALL service_names, even if the component they belong to does not contain the "detection" property.
Below is a sample of my JSON file I am trying to parse
{
"root": {
"fg": [
{
"product_name": "fg",
"remediation": {
"type": "package",
"packages": [
{
"service": "ips",
"service_name": "IPS",
"description": "Detects and Blocks attack attempts",
"kill_chain": {
"step": "Exploitation"
},
"link": "https://fgd.fnet.com/updates",
"minimum_version": "22.414"
}
]
},
"detection": {
"attackid": [
51006,
50825
]
}
}
],
"fweb": [
{
"product_name": "fWeb",
"remediation": {
"type": "package",
"packages": [
{
"service": "waf",
"service_name": "Web App Security",
"description": "Detects and Blocks attack attempts",
"kill_chain": {
"step": "Exploitation"
},
"link": "https://fgd.fnet.com/updates",
"minimum_version": "0.00330"
}
]
},
"detection": {
"signature_id": [
"090490119",
"090490117"
]
}
}
],
"fcl": [
{
"product_name": "fcl",
"remediation": {
"type": "package",
"packages": [
{
"service": "vuln",
"service_name": "Vulnerability",
"description": "Detects and Blocks attack attempts",
"kill_chain": {
"step": "Delivery"
},
"link": "https://fgd.fnet.com/updates",
"minimum_version": "1.348"
}
]
},
"detection": {
"vulnid": [
69887,
2711
]
}
},
{
"product_name": "fcl",
"remediation": {
"type": "package",
"packages": [
{
"service": "ob-detect",
"service_name": "ob Detection",
"kill_chain": {
"step": "sm/SOAR"
},
"link": "https://www.fgd.com/services",
"minimum_version": "1.003"
}
]
}
}
],
"fss": [
{
"product_name": "fss",
"remediation": {
"type": "package",
"packages": [
{
"service": "ips",
"service_name": "IPS",
"description": "Detects and Blocks attack attempts",
"kill_chain": {
"step": "Exploitation"
},
"link": "https://fgd.fnet.com/updates",
"minimum_version": "22.414"
}
]
}
}
],
"fadc": [
{
"product_name": "fADC",
"remediation": {
"type": "package",
"packages": [
{
"service": "ips",
"service_name": "IPS",
"description": "Detects and Blocks attack attempts",
"kill_chain": {
"step": "Exploitation"
},
"link": "https://fgd.fnet.com/updates",
"minimum_version": "22.414"
}
]
},
"detection": {
"ips_rulename": [
"Error.Log.Remote.Code.Execution",
"Server.cgi-bin.Path.Traversal"
]
}
},
{
"product_name": "fADC",
"remediation": {
"type": "package",
"packages": [
{
"service": "waf",
"service_name": "Web App Security",
"description": "Detects and Blocks attack attempts",
"kill_chain": {
"step": "Exploitation"
},
"link": "https://fgd.fnet.com/updates",
"minimum_version": "1.00038"
}
]
},
"detection": {
"sigid": [
1002017267,
1002017273
]
}
}
],
"fsm": [
{
"product_name": "fsm",
"remediation": {
"type": "package",
"packages": [
{
"service": "ioc",
"service_name": "IOC",
"kill_chain": {
"step": "sm/SOAR"
},
"link": "https://www.fgd.com/services",
"minimum_version": "0.02355"
}
]
}
}
]
}
}
Thank you in advance,
Nikos
This will work for you. I've broken it up into three steps for ease ...
Step 1
This contains your JSON as you provided. The variable is defined as an Object.
Step 2
Initialise a string variable that contains the following expression ...
xml(variables('JSON'))
... which (as you know) will convert the JSON to XML.
Step 3
This is an Array variable that will extract the values of all service_name elements where the detection element exists, using the following expression ...
xpath(xml(variables('XML')), '//detection/..//service_name/text()')
Result
Voila! You have your values in an array.

how to set gateway for a pod using flannel + containerd

Ive been stuck for a while and not sure what am i possibly doing wrong. For some reason my pod gets the wrong gateway? its suppose to be 10.244.6.2 for the cbr0_ep but it is assigned 10.244.6.1. Is it my flannel config that is wrong?
{
"cniVersion": "0.2.0",
"name": "cbr0",
"type": "flannel",
"capabilities": {
"portMappings": true,
"dns": true
},
"delegate": {
"type": "sdnbridge",
"optionalFlags": {
"forceBridgeGateway": true
},
"AdditionalArgs": [
{
"Name": "EndpointPolicy",
"Value": {
"Type": "OutBoundNAT",
"Settings": {
"Exceptions": [
"10.96.0.0/12",
"10.244.0.0/16",
]
}
}
},
{
"Name": "EndpointPolicy",
"Value": {
"Type": "SDNROUTE",
"Settings": {
"DestinationPrefix": "10.96.0.0/12",
"NeedEncap": true
}
}
},
{
"Name": "EndpointPolicy",
"Value": {
"Type": "SDNROUTE",
"Settings": {
"NeedEncap": true
}
}
}
]
}
}
and my net-conf.json
{
"Network": "10.244.0.0/16",
"Backend": {
"name": "cbr0",
"type": "host-gw"
}
}

Any idea how to do custom supportedCookingModes in Alexa discovery?

I'm trying to return a Discovery Response, but the supportedCookingModes only seems to accept standard values and only in the format of ["OFF","BAKE"], not Custom values as indicated by the documentation. Any idea on how to specify custom values?
{
"event": {
"header": {
"namespace": "Alexa.Discovery",
"name": "Discover.Response",
"payloadVersion": "3",
"messageId": "asdf"
},
"payload": {
"endpoints": [
{
"endpointId": "asdf",
"capabilities": [
{
"type": "AlexaInterface",
"interface": "Alexa.Cooking",
"version": "3",
"properties": {
"supported": [
{
"name": "cookingMode"
}
],
"proactivelyReported": true,
"retrievable": true,
"nonControllable": false
},
"configuration": {
"supportsRemoteStart": true,
"supportedCookingModes": [
{
"value": "OFF"
},
{
"value": "BAKE"
},
{
"value": "CUSTOM",
"customName": "FANCY_NANCY_MODE"
}
]
}
}
]
}
]
}
}
}
Custom cooking modes are brand specific. This functionality is not yet publicly available. I recommend you to choose one of the existing cooking modes:
https://developer.amazon.com/en-US/docs/alexa/device-apis/cooking-property-schemas.html#cooking-mode-values

Multiple searches with with json_query/jmespath filter in Ansible

I'm trying to parse out specifc subnet names in the following piece of json, while using contains_with or starts_with filters in json_query.
It contains two vnets each of which has multiple subnets:
{
"azure_virtualnetworks": [
{
"name": "test-vnet-172-17-0-0-19",
"properties": {
"subnets": [
{
"name": "test-confluent-subnet-172-17-0-0-28",
"properties": {
"addressPrefix": "172.20.88.0/28",
"networkSecurityGroup": {
"id": "/subscriptions/********/resourceGroups/test-confluent-rg/providers/Microsoft.Network/networkSecurityGroups/test-confluent-nsg"
},
"provisioningState": "Succeeded"
}
},
{
"name": "test-test-subnet-172-17-0-32-28",
"properties": {
"addressPrefix": "172.20.88.32/28",
"networkSecurityGroup": {
"id": "/subscriptions/********/resourceGroups/test-test-rg/providers/Microsoft.Network/networkSecurityGroups/test-test-nsg"
},
"provisioningState": "Succeeded"
}
}
]
}
},
{
"name": "test2-vnet-172-17-1-0-19",
"properties": {
"subnets": [
{
"name": "test-confluent-subnet-172-17-1-0-28",
"properties": {
"addressPrefix": "172.20.88.0/28",
"networkSecurityGroup": {
"id": "/subscriptions/********/resourceGroups/test-confluent-rg/providers/Microsoft.Network/networkSecurityGroups/test-confluent-nsg"
},
"provisioningState": "Succeeded"
}
},
{
"name": "test-qatesting-subnet-172-17-1-16-28",
"properties": {
"addressPrefix": "172.20.88.16/28",
"networkSecurityGroup": {
"id": "/subscriptions/********/resourceGroups/test-qatesting-rg/providers/Microsoft.Network/networkSecurityGroups/test-qatesting-nsg"
},
"provisioningState": "Succeeded"
}
}
]
}
}
]
}
I need to search for a subnet name after searching by virtual network name.
I can filter as far down as the list of subnets without problems. e.g
azure_virtualnetworks[?contains(name,`test2-vnet`)].properties.subnets[]
returns:
[
{
"name": "test-confluent-subnet-172-17-1-0-28",
"properties": {
"addressPrefix": "172.20.88.0/28",
"networkSecurityGroup": {
"id": "/subscriptions/********/resourceGroups/test-confluent-rg/providers/Microsoft.Network/networkSecurityGroups/test-confluent-nsg"
},
"provisioningState": "Succeeded"
}
},
{
"name": "test-qatesting-subnet-172-17-1-16-28",
"properties": {
"addressPrefix": "172.20.88.16/28",
"networkSecurityGroup": {
"id": "/subscriptions/********/resourceGroups/test-qatesting-rg/providers/Microsoft.Network/networkSecurityGroups/test-qatesting-nsg"
},
"provisioningState": "Succeeded"
}
}
]
However I'm having problems then searching the subnets. I had thought that some variation on following might work but haven't had any sucess:
azure_virtualnetworks[?contains(name,`test2-vnet`)].properties.subnets[?contains(name,`test-confluent`) ]
I'm struggling to figure out what the correcting syntax is here.
Select required subnets, stop projection with pipe expression, filter required items from the subnets list:
azure_virtualnetworks[?contains(name,`test2-vnet`)].properties.subnets[] | [?contains(name,`test-confluent`)]

Cloudformation stack CREATE_FAILED for ApiGateway::Method with no status reason

If I remove the RestAPI + related resources, the stack succeeds. I'm not sure how to get an error message to see why it is failing with this configuration.
Here's the JSON for the resources that are failing:
{
"RestAPI":{
"Type":"AWS::ApiGateway::RestApi",
"Properties":{
"Name":"FD-API"
}
},
"RestAPIResourceFDImage":{
"Type":"AWS::ApiGateway::Resource",
"Properties":{
"RestApiId":{
"Ref":"RestAPI"
},
"ParentId":{
"Fn::GetAtt":[
"RestAPI",
"RootResourceId"
]
},
"PathPart":"{image}"
},
"DependsOn": ["RestAPI"]
},
"RestAPIMethodGetProperties":{
"Type":"AWS::ApiGateway::Method",
"Properties":{
"AuthorizationType":"NONE",
"ResourceId":{
"Ref":"RestAPIResourceFDImage"
},
"HttpMethod":"GET",
"Integration":{
"Type":"AWS",
"Credentials": {
"Fn::GetAtt": [
"RoleFDApiGateway",
"Arn"
]
},
"IntegrationHttpMethod":"POST",
"PassthroughBehavior": "NEVER",
"Uri":{
"Fn::Join":[
"",
[
"arn:aws:apigateway:",
{
"Ref":"AWS::Region"
},
":lambda:path/2015-03-31/functions/",
{
"Fn::GetAtt":[
"FunctionFDLambda",
"Arn"
]
},
"/invocations"
]
]
},
"RequestTemplates": {
"application/json": {
"Fn::Join": [
"",
[
"{ \"key\": \"$input.params('image')\", \"bucket\": \"",
{
"Ref": "FDBucket"
},
"\" }"
]
]
}
}
}
},
"DependsOn": ["RestAPIResourceFDImage"]
},
"ReatAPIDeploymentFD": {
"Type": "AWS::ApiGateway::Deployment",
"Properties": {
"RestApiId": {"Ref": "RestAPI"},
"StageName": "v1"
},
"DependsOn": ["RestAPIMethodGetProperties"]
}
}

Resources