I'm writing an ansible playbook that creates users in an internal system through a call to an API where the request body contains all of the user information in json format.
Right now my playbook hold all of those variables in a list of dictionaries like this:
userInfoDict: "{{[ {'FirstName': 'John', 'LastName': 'Lennon', 'Number': '', 'email': 'john#example.com'}, {'FirstName': 'Paul', 'LastName': 'McCartney', 'Number': '', 'email': 'paul#example.com'} ]}}"
Where the "Number" field is blank, I have a play to grab available phone numbers from an internal DB, and that list of numbers is held in a variable called numberList
What I want to do, is to iterate through UserInfoDict, and set the Number field to the next value in my list of numbers, But I haven't been able to do that.
Is there any way to do something like this in ansible? Where you access an object at an index of a list?
- set_fact:
finalUserInfo: "{{userInfoDict[i].Number : item}}"
loop: "{{numberList}}
Something like this ^
You can use the zip filter to combine the two lists together. This implies that you have an entry in your numberList for each element in your userInfoDict (side note: which is a misleading var name IMO since it is a list). I created such a list below from what I understood from your question.
You can loop directly on the zipped lists and access their relevant values.
If you absolutely need to create a new list of dicts with the combined info, there are several ways to do so. I used the json_query filter as a demo below (this requires pip install jmespath on the controller).
(Note: in the below playbook, the rather ugly ... to_json | from_json ... is needed to overcome a bug in jmespath <-> ansible communication where each elements of the zipped lists are otherwise interpreted as strings.)
---
- name: Zip demo
hosts: localhost
gather_facts: false
vars:
userInfoDict: [{'FirstName':'John','LastName':'Lennon','Number':'','email':'john#example.com'},{'FirstName':'Paul','LastName':'McCartney','Number':'','email':'paul#example.com'}]
numberList: ["+33123456789", "+33612345678"]
tasks:
- name: Looping over zipped lists directly
vars:
fancy_message: |-
User first name is: {{ item.0.FirstName }}
User last name is: {{ item.0.LastName }}
User number is: {{ item.1 }}
User email is: {{ item.0.email }}
debug:
msg: "{{ fancy_message.split('\n') }}"
loop: "{{ userInfoDict | zip(numberList) | list }}"
loop_control:
label: "{{ item.0.FirstName }} {{ item.0.LastName }}"
- name: Creating a new list of dicts with json_query
vars:
new_dict_query: >-
[*].{
"FirstName": [0].FirstName,
"LastName": [0].LastName,
"Number": [1],
"email": [0].email
}
new_dict_list: >-
{{
userInfoDict
| zip(numberList)
| list
| to_json
| from_json
| json_query(new_dict_query)
}}
debug:
var: new_dict_list
which gives:
PLAY [Zip demo] *****************************************************************************************************************************************************************************************************************************
TASK [Looping over zipped lists directly] ***************************************************************************************************************************************************************************************************
ok: [localhost] => (item=John Lennon) => {
"msg": [
"User first name is: John",
"User last name is: Lennon",
"User number is: +33123456789",
"User email is: john#example.com"
]
}
ok: [localhost] => (item=Paul McCartney) => {
"msg": [
"User first name is: Paul",
"User last name is: McCartney",
"User number is: +33612345678",
"User email is: paul#example.com"
]
}
TASK [Creating a new list of dicts with json_query] *****************************************************************************************************************************************************************************************
ok: [localhost] => {
"new_dict_list": [
{
"FirstName": "John",
"LastName": "Lennon",
"Number": "+33123456789",
"email": "john#example.com"
},
{
"FirstName": "Paul",
"LastName": "McCartney",
"Number": "+33612345678",
"email": "paul#example.com"
}
]
}
PLAY RECAP **********************************************************************************************************************************************************************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Related
I'm looking for a better solution to filter a list of dicts by a key A and return a list of value of key B. To be more concrete - every host has a dict:
infrastructure:
name: "x..."
network:
address: "1..."
There are hosts, where network.address is defined and there are hosts, where network.address is not defined. I need now a list of all infrastructure.name with defined network.address.
- name: "Define Alias fact"
set_fact:
alias: []
- name: "Add Aliases for all hosts with network.address is defined"
set_fact:
alias: "{{ alias + [hostvars[host].infrastructure.name + '-alias'] }}"
when:
- "hostvars[host].network is defined"
- "hostvars[host].network.address is defined"
with_items: "{{ groups['all'] }}"
loop_control:
loop_var: host
That works, but is a little bit messy, because I call set_fact many times and add items to a list.
When I have a look at:
- name: "Define addresses fact"
set_fact:
address: "{{ groups['all'] | map('extract', hostvars) | list | json_query('[*].network.address') }}"
This is much shorter, maybe easier.
I'd like to ask, if I can use map and extract and the "list of dicts" before flatten the list to "filter out" all items where network.address is not defined and use json_query together with some string operation to append the '-alias'. Is there a similar easy way to replace the first script?
In a pure JMESPath way, given the JSON
[
{
"infrastructure": {"name": "x..."},
"network": {"address": "1..."}
},
{
"infrastructure": {"name": "y..."}
},
{
"infrastructure": {"name": "z..."},
"network": {"address": "2..."}
},
{
"infrastructure": {"name": "a..."},
"network": {}
}
]
You can extract the infrastructure.name concatenated with -alias having a network.address set this way:
[?network.address].join('-', [infrastructure.name, 'alias'])
This will yield:
[
"x...-alias",
"z...-alias"
]
The function join is primarily meant to glue array elements together, but it can also be used to concatenate string.
And so for a playbook demonstrating this:
- hosts: all
gather_facts: no
tasks:
- debug:
msg: >-
{{
servers | to_json | from_json |
json_query(
'[?network.address].join(`-`, [infrastructure.name, `alias`])'
)
}}
vars:
servers:
- infrastructure:
name: x...
network:
address: 1...
- infrastructure:
name: y...
- infrastructure:
name: z...
network:
address: 2...
- infrastructure:
name: a...
network:
Note that the odd construct | from_json | to_json is explained in this other answer
This yields:
PLAY [all] ********************************************************************************************************
TASK [debug] ******************************************************************************************************
ok: [localhost] => {
"msg": [
"x...-alias",
"z...-alias"
]
}
PLAY RECAP ********************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
The idea is to add an item into the array if the item does not exist. The select attribute function returns an empty list even though there are items. Therefore the length of an empty is always 0, which in turn will create a new list every time in my case. So the union function in add a new comment section is returning a list with the only new comment erasing all the old ones.
I understand that the issue is with t_array_exists["ansible_facts"]|selectattr("common_motd_qsc_comments_array", "defined")|list|length == 0 conditional statement but I am not sure what I am doing wrong. I have tried many variations of this command. Few of them are commented. Any advice/suggestion is appreciated :)
main.yml:
- name: Get comment array from facts
set_fact:
common_motd_qsc_comments_array: "{{ ansible_local['snps']['motd']['comment_array'] }}"
register: t_array_exists
when:
- ansible_local['snps']['motd'] is defined
- ansible_local['snps']['motd']['comment_array'] is defined
- debug:
var: t_array_exists
- debug:
var: t_array_exists["ansible_facts"]|selectattr("common_motd_qsc_comments_array", "defined")|list
# var: ansible_facts|selectattr("common_motd_qsc_comments_array", "defined")|list
# var: t_array_exists|selectattr("common_motd_qsc_comments_array", "defined")|list
- name: Create an empty array if there is no array
set_fact:
common_motd_qsc_comments_array: []
when:
- t_array_exists["ansible_facts"]|selectattr("common_motd_qsc_comments_array", "defined")|list|length == 0
- debug:
var: t_array_exists["ansible_facts"]|selectattr("common_motd_qsc_comments_array", "defined")|list|length
- name: Deleting a comment if it exists
set_fact:
common_motd_qsc_comments_array: "{{ common_motd_qsc_comments_array | difference([t_new_entry]) }}"
loop: "{{ common_motd_qsc_delete_comment }}"
when: t_new_entry in common_motd_qsc_comments_array
vars:
t_new_entry: "{{ item | trim }}"
- name: Add a new comment if it doesn't exist
set_fact:
common_motd_qsc_comments_array: "{{ common_motd_qsc_comments_array | union([t_new_entry]) }}"
loop: "{{ common_motd_qsc_add_comment }}"
when: t_new_entry not in common_motd_qsc_comments_array
vars:
t_new_entry: "{{ item | trim }}"
- name: Saving comments to snps.fact file
ini_file:
dest: "/etc/ansible/facts.d/snps.fact"
section: 'motd' # header
option: 'comment_array' # key
value: "{{ common_motd_qsc_comments_array }}" # value
Ansible output:
TASK [common/motd_scratch/v1 : Get comment array from facts] ***************************************************************************************
ok: [ansible-poc-cos6]
TASK [common/motd_scratch/v1 : debug] **************************************************************************************************************
ok: [ansible-poc-cos6] => {
"t_array_exists": {
"ansible_facts": {
"common_motd_qsc_comments_array": [
"new comment 1"
]
},
"changed": false,
"failed": false
}
}
TASK [common/motd_scratch/v1 : debug] **************************************************************************************************************
ok: [ansible-poc-cos6] => {
"ansible_facts|selectattr(\"common_motd_qsc_comments_array\", \"defined\")|list": []
}
TASK [common/motd_scratch/v1 : Create an empty array if there is no array] *************************************************************************
ok: [ansible-poc-cos6]
TASK [common/motd_scratch/v1 : debug] **************************************************************************************************************
ok: [ansible-poc-cos6] => {
"t_array_exists[\"ansible_facts\"]|selectattr(\"common_motd_qsc_comments_array\", \"defined\")|list|length": "0"
}
TASK [common/motd_scratch/v1 : Add a new comment if it doesn't exist] ******************************************************************************
ok: [ansible-poc-cos6] => (item=new comment 22)
TASK [common/motd_scratch/v1 : Saving comments to snps.fact file] **********************************************************************************
changed: [ansible-poc-cos6]
TASK [common/motd_scratch/v1 : debug] **************************************************************************************************************
ok: [ansible-poc-cos6] => {
"common_motd_qsc_add_comment": [
"new comment 22"
]
}
snps.fact output:
Before:
[motd]
comment_array = ['new comment 1']
After passing new_comment as "new comment 2":
[motd]
comment_array = ['new comment 2']
Expected:
[motd]
comment_array = ['new comment 1', 'new comment 2']
You can do all of the above in one single easy step. Here is an MCVE playbook to illustrate.
Prior to running the playbook, I added the following file on my machine:
/etc/ansible/facts.d/snps.fact
{
"motd_example1": [
"One message",
"A message",
"Message to delete"
],
"motd_example2": [
"Some messages",
"Mandatory message",
"Other message"
]
}
The playbook
---
- name: Add/Remove list elements demo
hosts: localhost
gather_facts: true
vars:
# The (list of) custom message(s) I want to add if not present
my_custom_mandatory_messages:
- Mandatory message
# The (list of) custom message(s) I want to remove if present
my_custom_messages_to_delete:
- Message to delete
# The list of custom vars I'm going to loop over for demo
# In your case, you can simply use your single motd var
# directly in the below task without looping. I added this
# for convenience as the expression is exactly the same
# in all cases. Only the input data changes
my_motd_example_vars:
- motd_example1
- motd_example2
- motd_example3 # This one does not even exist in ansible_local.snps
tasks:
- name: Show result using local facts for each demo test var
debug:
msg: "{{ ansible_local.snps[item] | default([])
| union(my_custom_mandatory_messages)
| difference(my_custom_messages_to_delete) }}"
loop: "{{ my_motd_example_vars }}"
- name: Proove it works with a totaly undefined var
debug:
msg: "{{ totally.undefined.var | default([])
| union(my_custom_mandatory_messages)
| difference(my_custom_messages_to_delete) }}"
The result
TASK [Gathering Facts] **********************************************************************************************************************************************************************************************************************
ok: [localhost]
TASK [Show result using local facts for each demo test var] *********************************************************************************************************************************************************************************
ok: [localhost] => (item=motd_example1) => {
"msg": [
"One message",
"A message",
"Mandatory message"
]
}
ok: [localhost] => (item=motd_example2) => {
"msg": [
"Some messages",
"Mandatory message",
"Other message"
]
}
ok: [localhost] => (item=motd_example3) => {
"msg": [
"Mandatory message"
]
}
TASK [Simple proof that it works with a totaly undefined var] *******************************************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
"Mandatory message"
]
}
PLAY RECAP **********************************************************************************************************************************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Not sure I understood exactly what the problem is, but if you are trying to merge two dictionaries (with one overwriting the other if the key already exists), then you might want to take a look at:
https://docs.ansible.com/ansible/latest/user_guide/playbooks_advanced_syntax.html#yaml-anchors-and-aliases-sharing-variable-values
The very first example shows exactly that, with just YAML. If that doesn't do the trick, then you might want to try the combine filter:
https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html#combining-hashes-dictionaries
HTH
Alex
I was able to fix it with the help of my colleague. I just had to check if comment_array is not defined instead of trying to find its length. A simple alternate idea which works :)
- name: Create an empty array if there is no array
set_fact:
common_motd_qsc_comments_array: []
when:
- ansible_local['snps']['motd']['comment_array'] is not defined
Thank you all for your help :)
Would anyone be able to recommend a way to take the contents of two register variables and pass them into one command? While also lining up the results of the outputs from each variable in a 1:1 fashion. (ie. VS1:rule1, VS2:rule2, and so on from the output shown below)
Here is what's stored in stdout_lines for 'Virtual_Check' and 'Rule_Check':
"Virtual_Check.stdout_lines": [
[
"ltm virtual VS1 ",
"ltm virtual VS2 ",
"ltm virtual VS3 ",
"ltm virtual VS",
"Rule_Check.stdout_lines": [
[
"myrule1",
" ",
"",
" myrule2",
" ",
"",
" myrule3",
" ",
"",
" myrule4",
" ",
"",
Now, I would like to pass the contents of the variables into one command as shown below. When I run this playbook the 'Virtual_Check' portion under 'with_nested' loops as expected, but the issue I'm running into is it won't loop properly for the 'Rule_Check' portion (I've left in the two methods I tried below)
So far I've tried using with_nested to accomplish this and it seems to no be looping over the second variable correctly.
- name: Update VS iRule
bigip_command:
commands:
- "modify ltm virtual {{ item.0 }} rules { {{ item.1 }} myrule10 }"
provider:
server: "{{ inventory_hostname }}"
password: "{{ remote_passwd }}"
user: "{{ remote_username }}"
validate_certs: no
delegate_to: localhost
with_nested:
- [ "{{ Virtual_Check['stdout'][0] | replace('ltm virtual', '') | replace('\n', '') }}"]
- [ "{{ Rule_Check['stdout'][0] | replace('\n', '') }}" ]
- [ "{{ Rule_Check['stdout_lines'][0] }}" ]
I would expect that the 'modify ltm virtual {{ item.0 }} rules { {{ item.1 }} myrule10 }' line would be processed with the content within the Virtual_Check & Rule_Check lists
For example:
modify ltm virtual VS1 rules { myrule1 myrule10 }
modify ltm virtual VS2 rules { myrule2 myrule10 }
modify ltm virtual VS3 rules { myrule3 myrule10 }
modify ltm virtual VS4 rules { myrule4 myrule10 }
The nested lookup does not accomplish what your are expecting: it creates a loop on first element with a sub-loop on second element and a sub-sub-loop on third element, etc...
What you are looking for is the zip filter which will allow you to assemble several lists in a single one joining all items of same index together in a list.
Example below with your original sample data in you question. You just have to adapt to your real case:
---
- name: zip example
hosts: localhost
gather_facts: false
vars:
servers: [ 'VS1', 'VS2', 'VS3', 'VS4' ]
rules: [ myrule1, myrule2, myrule3, myrule4 ]
tasks:
- name: Show zipped data from servers and rules
debug:
msg: "Server {{ item.0 }} has rule: {{ item.1 }}"
loop: "{{ servers | zip(rules) | list }}"
which gives
PLAY [zip example] ********************************************************************************************************************************************************************************************************
TASK [Show zipped data from servers and rules] ****************************************************************************************************************************************************************************
ok: [localhost] => (item=['VS1', 'myrule1']) => {
"msg": "Server VS1 has rule: myrule1"
}
ok: [localhost] => (item=['VS2', 'myrule2']) => {
"msg": "Server VS2 has rule: myrule2"
}
ok: [localhost] => (item=['VS3', 'myrule3']) => {
"msg": "Server VS3 has rule: myrule3"
}
ok: [localhost] => (item=['VS4', 'myrule4']) => {
"msg": "Server VS4 has rule: myrule4"
}
PLAY RECAP ****************************************************************************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
I have the following json structure.
"results": [
{
"ltm_pools": [
{
"members": [
{
"full_path": "/Common/10.49.128.185:8080",
},
{
"full_path": "/Common/10.49.128.186:8080",
}
"name": "Staging-1-stresslab",
},
{
"members": [
{
"full_path": "/Common/10.49.128.187:0",
},
{
"full_path": "/Common/10.49.128.188:0",
}
],
"name": "Staging-2-lab",
},
I get an error when trying to do something like this
- debug:
msg: "{{item[0].host}} --- {{ item[1] }} --- {{ item[2] }}"
with_nested:
- "{{F5_hosts}}"
- "{{bigip_facts | json_query('[results[0].ltm_pools[*].name]') | flatten }}"
- "{{bigip_facts | json_query('[results[0].ltm_pools[?name.contains(#,'Staging'].members[::2].full_path]') | flatten }}"
I am unable to get the third array working.
I want to print the even members full_path variable from all objects where name contains staging.
I hope someone can help me I've been struggling with this for days.
From what I see/read/tried myself, you fell in this bug: https://github.com/ansible/ansible/issues/27299
This is the problem of "contains" JMESPath function as it is run by Ansible, to quote:
https://github.com/ansible/ansible/issues/27299#issuecomment-331068246
The problem is related to the fact that Ansible uses own types for strings: AnsibleUnicode and AnsibleUnsafeText.
And as long as jmespath library has very strict type-checking, it fails to accept this types as string literals.
There is also a suggested workaround, if you convert the variable to json and back, the strings in there have the correct type. Making long story short, this doesn't work:
"{{bigip_facts | json_query('results[0].ltm_pools[?name.contains(#,`Staging`)==`true`].members[::2].full_path') }}"
but this does:
"{{bigip_facts | to_json | from_json | json_query('results[0].ltm_pools[?name.contains(#,`Staging`)==`true`].members[::2].full_path') }}"
I've managed to run such a code:
- hosts: all
gather_facts: no
tasks:
- set_fact:
bigip_facts:
results:
- ltm_pools:
- members:
- full_path: "/Common/10.49.128.185:8080"
- full_path: "/Common/10.49.128.186:8080"
name: "Staging-1-stresslab"
- members:
- full_path: "/Common/10.49.128.187:0"
- full_path: "/Common/10.49.128.188:0"
name: "Staging-2-stresslab"
- name: "Debug ltm-pools"
debug:
msg: "{{ item }}"
with_items:
- "{{bigip_facts | to_json | from_json | json_query('results[0].ltm_pools[?name.contains(#,`Staging`)==`true`].members[::2].full_path') }}"
And it works as you wanted:
PLAY [all] *****************************************************************************************
TASK [set_fact] ************************************************************************************
ok: [localhost]
TASK [Debug ltm-pools] *****************************************************************************
ok: [localhost] => (item=[u'/Common/10.49.128.185:8080']) => {
"msg": [
"/Common/10.49.128.185:8080"
]
}
ok: [localhost] => (item=[u'/Common/10.49.128.187:0']) => {
"msg": [
"/Common/10.49.128.187:0"
]
}
PLAY RECAP *****************************************************************************************
localhost : ok=2 changed=0 unreachable=0 failed=0
I need to merge two JSON objects based on first object keys
object1 = {
"params" : {
"type": ["type1", "type2"],
"requeststate": []
}
}
object2 = {
"params" : {
"type": ["type2", "type3", "type4"],
"requeststate": ["Original", "Revised" ],
"responsestate": ["Approved" ]
}
}
I need to merge two object based on first object key and my output should look like below
mergedobject = {
"params" : {
"type": ["type1", "type2", "type3", "type4"],
"requeststate": ["Original", "Revised"]
}
}
i searched for my case and didnt find much details
Please let me know is it possible to do with ansible
I can able to merge array with
set_fact:
mergedrequeststate: "{{ object1.params.requeststate + object2.params.requeststate }}"
but my case involved with morethan 15 params object and I cant declare all the param object . Also it may grow in future and I need handle that if possible.
Please comment if you need more details.
Thanks for your support
Use the combine filter.
- set_fact:
mergedobject: "{{ object1.params | combine (object2.params) }}"
the requirement is well described, i would only add that you want to merge the keys and get the unique values from the 2 objects (if that's not the case, pay attention to the union filter in the PB below). Also, your example variables assume that the we want to merge the keys under objectX.params.
without further due, here is a PB that will get you going. there is 1 debug step to display all the keys your object1.params has, then a loop to merge the values of the 2 objects, then a final print.
PB:
---
- hosts: localhost
gather_facts: false
vars:
object1:
params:
type:
- type1
- type2
requeststate: []
object2:
params:
type:
- type2
- type3
- type4
requeststate:
- Original
- Revised
responsestate:
- Approved
tasks:
- name: print all the keys in the object1.params variable
debug:
msg: "{{ object1['params'].keys() | list }}"
- name: for each key, merge from the 2 variables
set_fact:
mergedobj: "{{ mergedobj|default({}) | combine({item: object1['params'][item] | union(object2['params'][item]) }) }}"
with_items:
- "{{ object1['params'].keys() | list }}"
- name: print final result
debug:
var: mergedobj
execution result:
[http_offline#greenhat-29 tests]$ ansible-playbook test.yml
PLAY [localhost] *******************************************************************************************************************************************************************************************************
TASK [print all the keys in the object1.params variable] ***************************************************************************************************************************************************************
ok: [localhost] => {
"msg": [
"type",
"requeststate"
]
}
TASK [for each key, merge from the 2 variables] ************************************************************************************************************************************************************************
ok: [localhost] => (item=type)
ok: [localhost] => (item=requeststate)
TASK [print final result] **********************************************************************************************************************************************************************************************
ok: [localhost] => {
"mergedobj": {
"requeststate": [
"Original",
"Revised"
],
"type": [
"type1",
"type2",
"type3",
"type4"
]
}
}
PLAY RECAP *************************************************************************************************************************************************************************************************************
localhost : ok=3 changed=0 unreachable=0 failed=0
[http_offline#greenhat-29 tests]$
hope it helps
you should check out the documentation, you can simply do:
- set_fact:
mergedobject: "{{ object1 | combine(object2, recursive=True) }}"