I have a harness to build VMs using Packer that in turn calls Ansible (in local mode) to do the heavy lifting.
I'd like to be able to parameters to Packer (got that), which is passes to Ansible as extra vars.
I can pass an external variables files and also a simple variable such as the example below.
ansible-playbook -v -c local something.yml --extra-vars "deploy_loc=custom"
Thats okay, but I really need to pass more complex array of variables, such as the examples below.
I've tried a number of formatting such as the one below and usually get some kind of delimiter error.
ansible-playbook -v -c local something.yml --extra-vars 'deploy_loc=custom deploy_scen: [custom][ip=1.2.34]}'
Role variable file
# Which location
deploy_loc: 'external-dhcp'
# location defaults
deploy_scen:
custom:
ipv4: yes
net_type: dhcp
ip: '1.1.1.1'
prefix: '24'
gw: '1.1.1.1.254'
host: 'custom'
domain: 'domain.com'
dns1: '1.1.1.2'
standard-eng:
ipv4: yes
net_type: none
ip: '12.12.12.5'
prefix: '24'
external-dhcp:
ipv4: yes
net_type: dhcp
I think it's more robust and readable to generate a yaml file and use that with vars_files.
Alternatively you can generate a json file and read and parse it using a file lookup and the from_json filter. Something like this:
- name: Read objects
set_fact: deploy_scen={{lookup('file', 'deploy_scen.json') | from_json}}
However, if you really want --extra-var you can use the dict() function:
-e 'var={{dict(key1=dict(subkey1="value"),key2=dict(subkey1="value2"))}}'
You can pass a json structure in to ansible via the extra-vars parameter. You have to be a little careful to make sure it's set up properly though. You want to have the parameter look something like this:
--extra-vars {"param1":"foo","param2":"bar","file_list":[ "file1", "file2", "file3" ]}
When I do things like this I typically write a small bash wrapper script that will ensure extra-vars is set up properly and then invokes ansible.
Related
I have an ec2 tag 'Name' on my hosts in AWS. I need to append a suffix to the hostname in my inventory. I'm essentially trying to do this:
plugin: aws_ec2
strict: False
hostnames: '"tag:Name" + ".suffix.com"'
as soon as I do anything resembling Jinja in my inventory plugin yaml file, ansible-inventory stops parsing the file as YAML and I get errors.
previously, when using the aws_ec2.py script for inventory, I could to something like:
destination_format: "{0}.suffix.com"
destination_format_tags: "Name"
I'd much rather use the plugin because it has nice features that the old, crusty aws_ec2.py script does not.
A change in v3.3.0 of the Ansible Amazon AWS collection made it possible to use literal strings in the hostnames parameter -- previously the only accepted values were valid DescribeInstances filters.
This snippet will set inventory_hostname to the value of the instances' Name tag with .example.com appended:
hostnames:
- name: .example.com
separator: ''
prefix: tag:Name
I'm aware I can add host variables dynamically via an inventory script. I'm wondering if I can scripts in the host_vars directory which will be executed instead of simple read.
I have tried to create a simple script that outputs some variables. It seems only .json and .yml or no extension are read by the ansible-playbook. Since these are not executed the raw source will result in an error.
So, hence the question. Is this even possible and if not, would you be aware of a method to achieve the same results: Query a (local) dynamic source for variables of a particular host.
I'm pretty sure lookup("pipe") will do what you want, provided the script is available on the target host:
- set_fact:
my_vars: '{{ lookup("pipe", "./my_script.py") | from_json }}'
(substituting from_json with from_yaml or whatever to coerce the textual output from the script into a python datastructure; it's possible that ansible would coerce it automagically, but explicit is better than implicit)
If you want the script that is on the control machine to run, you'll likely have to do some hoopjumpery with delegate_to: and some hostvars ninjary to promote the set_fact: off of the control host over to all playbook hosts
The source of my host inventory is from an internal tool that outputs pairs of values, example, here are six observations, I currently have 160 observations:
servername1 processname1
servername1 processname2
servername1 processname3
servername2 processname1
servername3 processname1
servername4 processname1
So column 1 is my target host list (my inventory). Column2 are unique processname values, assigned specifically to the value of the servername. Often the same server will occur. Some servers have only one processname, others may have 2 to N. Meaning my target host may repeat for a unique list of processnames. I want to use both dynamic inventory from this output list of pairs, and I need both values on each observation to be associated and assigned to variables. I'm not absolutely required to use dynamic inventory, I just need a solution. I also need to pass to the target host and the value in {{ processname# }}, via the command: or shell: modules. (This is unique, there are no modules related to this need)
If required, I have a way to filter this data and output it in JSON format or YAML, making a separate YML file for each host. While I'd prefer to process these dynamically; pre-processing the list is acceptable.
Because ansible-playbook, requires some known host inventory list, I'm getting stuck understanding how I can create this list, from my dynamic output, at the time I start the play.
What I've done so far: I've tried reading up and trying to set these pairs as in the /etc/ansible/hosts/host_vars/servername#.yml files. This is extremely ugly, as I have to pre-process the output of the data, into YML format. But it does not give me a host list to reference in my playbook. So while it seems that hostvar is the logical choice, I cannot get my head around it.
What I need:
- The suggested format of the data? JSON? YAML? Other? (if I cannot read it in dynamically.
- Is putting this in host_vars correct?
- Last night I saw another answer using set_fact, would that help?
Thank you for any insight. I've now been using Ansible for 3.5 weeks! I've done pretty good using static and dynamic inventories, but this stumps me as the inventory list is not obvious, give the format of the matched pairs.
Note: MANY have suggested using host_vars, but that seems to me, to be reserved to hostnames, and related port and proxy values. I could be wrong.
===================================================================
UPDATE: Thanks for the help in the right direction.
I have updated our inventory script to output host list in JSON.
The first new option is to output the hosts in JSON.
Example:
{"my_host":["servername1","servername2",]}
Calling this as a dynamic inventory script, works great!
ansible all -m ping
servername1 | SUCCESS => {
"changed": false,
"ping": "pong"
}
servername2 | SUCCESS => {
"changed": false,
"ping": "pong"
}
Next: The second new option to the inventory script was to add a new switch to input a hostname. This part is still confusing me. Here is the output:
showInv --host=servername1
{"servername1":["processname1","processname2","processname3",]}
The final part that I am missing is how I call the inventory script with a specific "--host={{ my_host }} , from within my playbook.
It seems that I need to find the variable for the existing hostname and pass that back to the inventory script as the switch option "--host= "
You say that you are OK with dynamic inventories. Make your own.
Here is the docs.
You need to make a script that will do two things:
when executed with --list, processes your file and prints this JSON to stdout:
{ "myhosts": ["servername1", "servername2", "servername3"] }
when executed with --host servername1, prints this JSON to stdout:
{ "myprocesses": ["processname1", "processname2"] }
So with --list you should provide uniq list of hosts. In my example they belong to myhosts group.
And with --host <hostname> you should provide list dict of host vars for that host (<hostname>). In my example there is a list variable myprocesses that contains all processes for that host.
Then just call ansible-playbook -i my_inv_script myplaybook.yml.
Example playbook:
---
- hosts: myhosts
tasks:
- debug:
msg: "Process name is {{ item }}"
with_items: "{{ myprocesses }}"
This playbook will go trough all hosts in your dynamic inventory and print all processes for each host.
You will need to develop a dynamic inventory script, that takes the first column as the hostname, and the second column as variables for that host.
Please, find below the link to my dynamic inventory written in php
https://github.com/walden-it/ansible-ij/blob/master/inventory.php
take a look at the functions get_vars() and get_hosts() to see how is the array being populated.
And in case you need it, here is the dump for the database this script is looking at:
https://github.com/walden-it/ansible-ij/blob/master/ansible.sql
Then you just specify it with -i inventory in the ansible run, or add it as inventory_file to the ansible.cfg
Closing this out. With the help of Konstantin's suggestions, I now have a working play. What is not immediately apparent, is that Ansible is doing some "magic" behind the scenes. I had to modify my inventory script, that generates my dynamic inventory to accept the "--list" switch option, and the "--host hostname" option.
Once this was done, I could run the playbook with the -i listInv and Ansible internally calls this script as listInv --list, which produces my dynamic inventory list. Then it loops, through to the with_items, and internally calls the script as, listInv --host {{ items }} and outputs the matching processnames.
Additionally, the JSON output generated by my script, had to make the "group" (first) field, "myprocess". Initially, I had it as "my_process", and this failed. Removing the underscore, fixed, that error.
All working now. This is a great example for learning, but it's still magic.
The playbook looks like this:
- hosts: all
gather_facts: no
connection: local
tasks:
- debug:
msg: "Process name is {{ item }}"
with_items: "{{ myprocess }}"
I have the following version installed: ansible 2.3.0 (devel 2131eaba0c)
I want to specify my host variable as external variable and then use it in the playbook similar to this:
hosts: "{{integration}}"
In my group_vars/all file I have the following defined variable:
integration: "int60"
The host file looks like this:
[int60]
hostA
[int61]
hostB
Unfortunately this does not work. I also tried to define the host var in the following way:
[integration]
127.0.0.1 ansible_host="{{ integration_env }}"
and have the integration_env specified in my group_vars/all file. In this case it seemed like it ran the tasks locally and not in the desired environment.
Is it possible to do something like this? I'd be open to whole new ways of doing this. The main goal is simply to define the host variable in a var file.
This will work if you pass integration variable as extra variable:
ansible-playbook -e integration=int60 myplaybook.yml
Any variables used in play "header", should be defined before Ansible parses playbook.
In your example you define integration as host facts. Facts are only defined on task level, not play level.
Update: and you can use other ways of passing variables, not only extra vars.
For example:
- hosts: "{{ lookup('env','DYN_HOSTS') }}"
will also work.
How do I get the value of a variable from the inventory if the variable could be either in the inventory file or group_vars directory?
For example, region=place-a could be in the inventory file or in a file in the group_vars somewhere. I would like a command to be able to retrieve that value using ansible or something retrieve that value. like:
$ ansible -i /somewhere/production/web --get-value region
place-a
That would help me with deployments and knowing which region is being deployed to.
.
A longer explanation to clarify, my inventory structure looks like this:
/somewhere/production/web
/somewhere/production/group_vars/web
The contents with the variables of the inventory file, /somewhere/production/web looks like this:
[web:children]
web1 ansible_ssh_host=10.0.0.1
web2 ansible_ssh_host=10.0.0.2
[web:vars]
region=place-a
I could get the value from the inventory file by simply parsing the file. like so:
$ awk -F "=" '/^region/ {print $2}' /somewhere/production/web
place-a
But that variable could be in the group_vars file, too. For example:
$ cat /somewhere/production/group_vars/web
region: place-a
Or it could look like an array:
$ cat /somewhere/production/group_vars/web
region:
- place-a
I don't want to look for and parse all the possible files.
Does Ansible have a way to get the values? Kind of like how it does with --list-hosts?
$ ansible web -i /somewhere/production/web --list-hosts
web1
web2
Short Answer
NO there is no CLI option to get value of variables.
Long Answer
First of all, you must look into how variable precedence works in Ansible.
The exact order is defined in the documentation. Here is an summary excerpt:
Basically, anything that goes into “role defaults” (the defaults
folder inside the role) is the most malleable and easily overridden.
Anything in the vars directory of the role overrides previous versions
of that variable in namespace. The idea here to follow is that the
more explicit you get in scope, the more precedence it takes with
command line -e extra vars always winning. Host and/or inventory
variables can win over role defaults, but not explicit includes like
the vars directory or an include_vars task.
With that cleared out, the only way to see what value is a variable taking is to use the debug task as follows:
- name: debug a variable
debug:
var: region
This will print out the value of the variable region.
My suggestion would be to maintain a single value type, either a string or a list to prevent confusion across tasks in different playbooks.
You could use something like this:
- name: deploy to regions
<task_type>:
<task_options>: <task_option_value>
// use the region variable as you wish
region: {{ item }}
...
with_items: "{{ regions.split(',') }}"
In this case you could have a variable regions=us-west,us-east comma-separated. The with_items statement would split it over the , and repeat the task for all the regions.
The CLI version of this is important for people who are trying to connect ansible to other systems. Building on the usage of the copy module elsewhere, if you have a POSIX mktemp and a copy of jq locally, then here's a bash one-liner that does the trick from the CLI:
export TMP=`mktemp` && ansible localhost -c local -i inventory.yml -m copy -a "content={{hostvars['SOME_HOSTNAME']}} dest=${TMP}" >/dev/null && cat ${TMP}|jq -r .SOME_VAR_NAME && rm ${TMP}
Breaking it down
# create a tempfile
export TMP=`mktemp`
# quietly use Ansible in local only mode, loading inventory.yml
# digging up the already-merged global/host/group vars
# for SOME_HOSTNAME, copying them to our tempfile from before
ansible localhost --connection local \
--inventory inventory.yml --module-name copy \
--args "content={{hostvars['SOME_HOSTNAME']}} dest=${TMP}" \
> /dev/null
# CLI-embedded ansible is a bit cumbersome. After all the data
# is exported to JSON, we can use `jq` to get a lot more flexibility
# out of queries/filters on this data. Here we just want a single
# value though, so we parse out SOME_VAR_NAME from all the host variables
# we saved before
cat ${TMP}|jq -r .SOME_VAR_NAME
rm ${TMP}
Another more easy solution get a variable through cli from ansible could be this:
export tmp_file=/tmp/ansible.$RANDOM
ansible -i <inventory> localhost -m copy -a "content={{ $VARIABLE_NAME }} dest=$tmp_file"
export VARIBALE_VALUE=$(cat $tmp_file)
rm -f $tmp_file
Looks quite ugly but is really helpful.