I'm using Ansible to get a list of user emails from an API and I want to loop over them.
This is the json response I get from the API:
"users": [
{
"email": "email1#email.com",
"id": 1,
"is_admin": true
},
{
"email": "email2#email.com",
"id": 2,
"is_admin": false
},
]
edit:
the task after that which I need the email for:
- name: Send emails
register: result
uri:
url: http://api
method: GET
body_format: json
return_content: yes
body:
email: "{{ item.email }}"
scope: SCOPE
loop: "{{ users.json['users'] }}"
- name: Print result
debug:
var: result
the error I get:
fatal: [localhost]: FAILED! => {"msg": "template error while templating string: unexpected char '#' at 16. String: {{ email#email.com }}"}
If I use email: item.email the json request body will be "body": {"email": "item.email"} instead of the email value
How Can I get the full email of each user?
You need to remove the {{ and }} markers from your var: directive. The value of var is implicitly evaluated in a Jinja template context so you don't need those markers:
- name: Print returned json dictionary
debug:
var: item.email
loop: "{{ users.json['users'] }}"
(Updated based on edits to your question)
The example you've shown doesn't generate the error you're asking about. Here's a complete reproducer with just the uri changed:
- hosts: localhost
gather_facts: false
vars:
users:
json:
users:
- email: email1#email.com
id: 1
is_admin: true
- email: email2#email.com
id: 2
is_admin: false
tasks:
- name: Send emails
register: result
uri:
url: https://eny1tdcqj6ghp.x.pipedream.net
method: GET
body_format: json
return_content: true
body:
email: "{{ item.email }}"
scope: SCOPE
loop: "{{ users.json['users'] }}"
This runs without errors (although note that I'm suspicious about the
use of a GET request with a JSON body; typically you would expect
that to be a POST request).
You can see the result of running this playbook here, which shows that the values being sent in the request are exactly what we expect from the data.
Related
I have a config_file with values in the format: key-value pair
e.g. a: "{\"type\": \"some_type\"}"
Everything is ok. Util I tried to pass this JSON value as value inside uri utility:
- name: 'Push config values'
uri:
url: '{{ uri }}/api/{{ item.key }}'
method: POST
body: '{"alias":"{{ item.key }}","key":"{{ item.key }}","value":"{{ item.value }}"}'
body_format: json
validate_certs: no
status_code: 204
headers:
X-Token: "{{ token }}"
with_dict: '{{ config_overrides_file }}'
register: response
I tried different way of escaping, also filter combination | from_json | to_json. But nothing is working. I got 'failed to parse JSON input: invalid character ''t'' after object key:value pair' in case of escaping
Probably, exist some standard approach to handle it?
I found just one way to achieve what I want: instead of a: "{\"type\": \"some_type\"}" this value in config file it could be a: {"type": "some_type"} and in uri it will be body: {"alias":"{{ item.key }}","key":"{{ item.key }}", "value": {{ item.value | to_json }}}. It's working approach but I would like not create additional task for json and non-json properties for such type of request.
P.S. When I hardcode this value everything is ok
You're getting bitten by ansible's "helpful" behavior of coercing any string it finds that starts with { back into a dict; the usual fix for that is to send it explicitly through |string to indicate to ansible you do not wish for that to happen. For sure any loop-ish behavior on a task (with_dict: being one of them) makes that problem much worse, because ansible gets really, really weird about coercion and recursive jinja2 templates in that circumstance
- name: test data for SO
set_fact:
config_values_file:
a: '{"type": "some_type"}'
- name: 'Push config values'
uri:
url: '{{ uri }}/api/{{ item.key }}'
method: POST
body: >-
{{ {"alias": item.key,
"key": item.key,
"value": item.value|string
} }}
body_format: json
validate_certs: no
status_code: 204
headers:
X-Token: "{{ token }}"
with_dict: '{{ config_overrides_file }}'
register: response
POSTs:
{"alias": "a", "key": "a", "value": "{\"type\": \"some_type\"}"} which I believe is what you were asking for
I want to push ios_facts to gitlab using ansibles uri module.
- name: get ios facts
ios_facts:
gather_subset: all
register: ios_facts
- name: commit to gitlab
delegate_to: localhost
uri:
url: http://gitlab/api/v4/projects/2/repository/commits
method: POST
body_format: json
status_code: 201
headers:
PRIVATE-TOKEN: "xxxxxxxxxxxxxx"
Content-Type: "application/json"
body: |
{
"branch": "master",
"commit_message": "{{ ansible_net_hostname }} update",
"actions": [
{
"action": "update",
"file_path": "conf/{{ ansible_net_hostname }}",
"content": "{{ ansible_net_config }}"
}
]
}
The Playbook works fine if I am using any other variable than ansible_net_config, or if I am pasting the raw content of ansible_net_config instead of using the jinja2 reference. The ansible_net_config is a large string using \n as new line and contains some special characters. I guess the problem occurs because I don't get valid json when the playbook parses.
I then get the HTTP Error 400: Bad Request
Is there any filter I can apply or any other thing I might be missing out?
I managed to solve the issue:
The API Call failed when the variable contains "\n". I could make it work by replacing "\n" with escaped "\\n":
...
"content": {{ ansible_net_config | replace('\n','\\n') }}
...
Here is the Ansible source code where with_items is linked to a dictionary type variable named "grafana_datasource_body". I always get an error message "ith_items expects a list or a set". Here is the source code and related output.
Ansible source code :
- name: Configure datasource creation http body
set_fact:
grafana_datasource_body:
name: "{{Ass_grafana_Datasource_Name}}"
type: "{{Ass_grafana_Datasource_Type}}"
isDefault: "{{Ass_grafana_Datasource_IsDefault}}"
access: "{{Ass_grafana_Datasource_Access}}"
basicAuth: "{{Ass_grafana_Datasource_BasicAuth}}"
url: "{{InfluxDB_Server}}:{{Ass_grafana_Datasource_Http_Port}}"
database: "{{Ass_grafana_Datasource_Db}}"
username: "{{Ass_grafana_Datasource_DbUser}}"
password: "{{Ass_grafana_Datasource_DbPassword}}"
when: Ass_grafana_Datasource_Name not in grafana_datasources_output.json|map(attribute='name')|list
- debug: msg="Datasource creation http body = {{grafana_datasource_body}}"
when: Ass_grafana_Datasource_Name not in grafana_datasources_output.json|map(attribute='name')|list
# Create non existing datasources
- name: datasource > Create datasource
register: grafana_datasources_create_output
failed_when: "'Datasource added' not in grafana_datasources_create_output|to_json"
uri:
url: "http://127.0.0.1:{{Ass_grafana_Listen_Http_Port}}/api/datasources"
method: POST
HEADER_Content-Type: application/json
body: '{{ item|to_json }}'
force_basic_auth: true
user: "{{Ass_grafana_Api_User}}"
password: "{{Ass_grafana_Api_Password}}"
status_code: 200
with_items: "{{grafana_datasource_body}}"
when: item.name not in grafana_datasources_output.json|map(attribute='name')|list
Output
TASK: [./grafana/grafana_Role | Configure datasource creation http body] ******
<10.0.20.7> ESTABLISH CONNECTION FOR USER: centos
ok: [10.0.20.7] => {"ansible_facts": {"grafana_datasource_body": {"access": "proxy", "basicAuth": "false", "database": "webcom-int", "isDefault": "true", "name": "MonTonton", "password": "webcom", "type": "influxdb", "url": "http://localhost:8086", "username": "webcom"}}}
TASK: [./grafana/grafana_Role | debug msg="Datasource creation http body = {{grafana_datasource_body}}"] ***
<10.0.20.7> ESTABLISH CONNECTION FOR USER: centos
ok: [10.0.20.7] => {
"msg": "Datasource creation http body = {'username': u'webcom', 'name': u'MonTonton', 'database': u'webcom-int', 'url': u'http://localhost:8086', 'basicAuth': u'false', 'access': u'proxy', 'password': u'webcom', 'type': u'influxdb', 'isDefault': u'true'}"
}
TASK: [./grafana/grafana_Role | datasource > Create datasource] ***************
fatal: [10.0.20.7] => with_items expects a list or a set
FATAL: all hosts have already failed -- aborting
Any real reason to use with_items loop here?
Replace
with_items: "{{grafana_datasource_body}}"
with
with_items: "{{ [grafana_datasource_body] }}"
this will work.
But you can use body: '{{ grafana_datasource_body|to_json }}' and don't use with_items.
There is a code which call the web services of an application.
- uri:
url: http://10.164.52.61:8080/ems/v74/ws/customer.ws?customerRefId=f4XXXb15d69c3
method: GET
content_as_json: true
password: admin
user: admin
validate_certs: no
return_content: yes
HEADER_Cookie: "{{login.set_cookie}}"
register: customerId
- debug:
var: customerId.content
The sample response is
"customerId.content": "<listResponse type=\"customer\" count=\"1\"><instance customerId=\"28\" name=\"abc\" customerRefId=\"xyz\" refId1=\"12\" type=\"org\" enabled=\"true\" phone=\"\" fax=\"\" billingZip=\"\" billingAddress=\"\" billingCity=\"\" billingCountry=\"\" billingState=\"\" vendor=\"1\" defaultEmail=\"test\" defaultContactName=\"test\"/></listResponse>"
I want to access the list response in my next block of code. Like i need just the value of "customerId". How this can be achieved using anisble
This is similar to sebaszw's answer, however doesn't require the xml response to be written to a file, instead storing it in a variable.
- uri:
url: http://10.164.52.61:8080/ems/v74/ws/customer.ws?customerRefId=f4XXXb15d69c3
method: GET
content_as_json: true
password: admin
user: admin
validate_certs: no
return_content: yes
HEADER_Cookie: "{{login.set_cookie}}"
register: customerId
- xml:
xmlstring: "{{customerId.content}}"
xpath: /listResponse/instance
content: attribute
register: instance_attributes
- debug:
var: instance_attributes.matches.0.instance.customerId
There is no XML support in Ansible out of the box.
You can use regex filters to make some simple searches in your XML data.
In your example:
- debug: msg="{{ customerId.content | regex_findall('customerId=\"(\d+)\"') }}"
Will search for customerId=\"<number>\" strings and return a list of numbers in quotes.
Update: fully working example:
- hosts: localhost
gather_facts: no
vars:
myvar: "<listResponse type=\"customer\" count=\"1\"><instance customerId=\"28\" name=\"abc\" customerRefId=\"xyz\" refId1=\"12\" type=\"org\" enabled=\"true\" phone=\"\" fax=\"\" billingZip=\"\" billingAddress=\"\" billingCity=\"\" billingCountry=\"\" billingState=\"\" vendor=\"1\" defaultEmail=\"test\" defaultContactName=\"test\"/></listResponse>"
tasks:
- debug: msg="{{ myvar | regex_findall('customerId=\"(\d+)\"') }}"
I use ansible 2.1.1.0.
From Ansible 2.4 you can do this:
- get_url:
url: http://10.164.52.61:8080/ems/v74/ws/customer.ws?customerRefId=f4XXXb15d69c3
dest: ./response.xml
url_password: admin
url_user: admin
validate_certs: no
headers:
Cookie: "{{login.set_cookie}}"
- xml:
path: ./response.xml
xpath: /listResponse/instance
content: attribute
register: instance_attributes
- debug:
var: instance_attributes.matches.0.instance.customerId
I just added XML parsing support in the uri module because I needed it too.
https://github.com/ansible/ansible/pull/53045
Just like the JSON support, it will return an 'xml' key with a dictionary consisting of the XML content for the convenience of accessing the data in the payload.
Your example would look like this:
- uri:
url: http://10.164.52.61:8080/ems/v74/ws/customer.ws?customerRefId=f4XXXb15d69c3
method: GET
password: admin
user: admin
validate_certs: no
HEADER_Cookie: "{{login.set_cookie}}"
register: customerId
- debug:
var: customerId.xml.listResponse.instance['#customerId']
The output in customerId.xml would be:
{
'listResponse': {
'#type': 'customer',
'#count', '1',
'instance': {
'#customerId': '28',
'#name': 'abc'
'#customerRefId': 'xyz',
'#refId1': '12',
'#type': 'org',
'#enabled': 'true',
'#phone': '',
'#fax': '',
'#billingZip': '',
'#billingAddress': '',
'#billingCity': '',
'#billingCountry': '',
'#billingState': '',
'#vendor': '1',
'#defaultEmail': 'test',
'#defaultContactName': 'test'
}
}
}
Its worth looking at ansible-xml for this. It's not part of Ansible but its a super useful module for parsing xml, it uses the lxml Python library under the hood.
As this was reasonably complex getting this to work I thought I'd post my Ansible - XML parameters for parsing a SOAP response:
<S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/">
<SOAP-ENV:Header/>
<S:Body xmlns:ns2="http://ws.coverity.com/v9">
<ns2:getLdapServerDomainsResponse>
<return>
<name>soapui_test</name>
</return>
</ns2:getLdapServerDomainsResponse>
</S:Body>
</S:Envelope>
The trick was getting namespaces and xpath right. Also as I wanted the actual name I needed content: text
- name: Extract the name from the SOAP
xml:
xmlstring: "{{ ldap_response.content }}"
xpath: "/S:Envelope/S:Body/ns2:getLdapServerDomainsResponse/return/name"
namespaces:
S: "http://schemas.xmlsoap.org/soap/envelope/"
SOAP-ENV: "http://schemas.xmlsoap.org/soap/envelope/"
ns2: "http://ws.coverity.com/v9"
content: text
register: ldap_name
I still don't understand why the xpath section at the end doesn't work with .../ns2:return/ns2:name as the Ansible - XML docs example suggests. The above fills ldap_name with (simplified):
"ldap_name": {
"matches": [
{
"name": "soapui_test"
}
],
}
I want to make a POST request to an API endpoint via Ansible where some of the items inside the post data are dynamic, here is what I try and fail:
My body_content.json:
{
apiKey: '{{ KEY_FROM_VARS }}',
data1: 'foo',
data2: 'bar'
}
And here is my Ansible task:
# Create an item via API
- uri: url="http://www.myapi.com/create"
method=POST return_content=yes HEADER_Content-Type="application/json"
body="{{ lookup('file','create_body.json') | to_json }}"
Sadly this doesn't work:
failed: [localhost] => {"failed": true}
msg: this module requires key=value arguments
....
FATAL: all hosts have already failed -- aborting
My ansible version is 1.9.1
You can't use newlines like this in yaml. Try this instead (the ">" indicates that the next lines are to be concatenated):
# Create an item via API
- uri: >
url="http://www.myapi.com/create"
method=POST return_content=yes HEADER_Content-Type="application/json"
body="{{ lookup('file','create_body.json') | to_json }}"
But I find this much better:
# Create an item via API
- uri:
url: "http://www.myapi.com/create"
method: POST
return_content: yes
HEADER_Content-Type: "application/json"
body: "{{ lookup('file','create_body.json') | to_json }}"
I'm posting below what I ended up using for my usecase (Ansible 2.0). This is useful if your json payload is stated inline (and not in a file).
this task expects 204 as its success return code.
And since the body_format is json, the header is inferred automatically
- name: add user to virtual host
uri:
url: http://0.0.0.0:15672/api/permissions/{{ rabbit_virtualhost }}/{{ rabbit_username }}
method: PUT
user: "{{ rabbit_username }}"
password: "{{ rabbit_password }}"
return_content: yes
body: {"configure":".*","write":".*","read":".*"}
body_format: json
status_code: 204
it is basically equivalent to:
curl -i -u user:pass -H "content-type:application/json" -XPUT http://0.0.0.0:15672/api/permissions/my_vhost/my_user -d '{"configure":".*","write":".*","read":".*"}'