I have what I think is a fairly common task of taking a local archive file, transferring it to a server, and extracting it there. I'm using the unarchive module for this but struggling with an elegant way to deal with the local filename always being different because the filename includes the software version. For example, here's a excerpt from a playbook:
- name: Upload code
unarchive: src={{payload_file}} dest={{install_dir}}
I can run this and use -e to set the variable:
ansible-playbook -e payload_file=/tmp/payload-4.2.1.zip -i production deploy.yml
My question is is this the best way to handle this situation? All the docs have hardcoded example file names and paths and I haven't seen any evidence in the docs that makes a variable that will be different each deployment straightforward to set.
While passing extra args on the command line will work, I generally don't like having to type to run the job -- this is to say that I don't think your solution is inelegant or bad in any way.
However, if you're willing to change your process a little, you can avoid having to add options when calling your playbook.
Option 1: Put the payload-version.zip file in a directory and target that directory with a glob. Any file in that directory will get unarchived (if there's only one file, this achieves the same behavior):
- name: Upload code
unarchive: src={{item}} dest={{install_dir}}
with_fileglob:
- /tmp/payloads/*
Option 2: Symlink the versioned file to a simple, general name of payload-latest.zip and target it with your play (using the modified date or setting a fact for metadata). I assume you have a build process or script to generate the zip, so add a step to symlink the build version to payload-latest.zip and target that in your play.
Related
I'm trying to write a playbook that will go and download the version of ombi I supply on the command line as a variable, then parse part of it so I can rename the file and keep a local copy of it. Then gunzip then untar then stop the service overwrite the existing app, then restart the service.
I've written several other playbooks but parsing this part out has me stumped.
So if say this was the URL
https://github.com/Ombi-app/Ombi/releases/download/v4.32.0/linux-x64.tar.gz
I want to extract the 4.32.0 out of that url. So my playbook run line might be something like:
ansible-playbook updateombi.yml --extra-vars "ombi_release=https://github.com/Ombi-app/Ombi/releases/download/v4.32.0/linux-x64.tar.gz"
I'm assuming I would declare a var like:
ombi_version: "{{ ombi_release | urlsplit('path') }}"
but the urlsplit is what's got me stumped. Anyone able to throw me a bone?
I'm trying to write a playbook that will go and download the version of Ombi I supply on the command line as a variable ...
To do so you could simply provide the version number only
ansible-playbook updateombi.yml --extra-vars "ombi_release=4.32.0"
and construct the URL and filename afterwards within your playbook
url: "https://github.com/Ombi-app/Ombi/releases/download/v{{ ombi_release }}/linux-x64.tar.gz"
dest: /tmp/linux-x64-v{{ ombi_release }}.tar.gz
since they don't have a variable part except the version number. By doing this there would be no need for
... then parse part of it so I can rename the file ...
I am using Ansible and have a directory structure like the example below:
configs
something
files
1.conf
2.conf
// and so on
Those files are templates and I am using Ansible to parse these templates and create them automatically in the destination server.
My problem is that with_fileglob is working only of first level directory and cannot seem to enable some recursive mode.
I have
- name: "Apply templates"
template:
src: "{{ item }}"
dest: "{{ item | replace('.j2', '') }}"
with_fileglob:
- "{{ user_configs_path }}/*"
by the way user_configs_path=configs exists and all good here.
The above does nothing.
If I add something under configs, example configs/blabla.j2 and re-run the playbook it is parsed and copied fine.
So seems somehow that the directories are not searched recursively.
I am not limited to only use the fileglob solution so feel free to suggest anything I can learn to reach my goal.
Basically I want to recursively iterate all directories for files only, and in a loop apply the template module to them and create them in remote server
Thank you
Regarding
My problem is that with_fileglob is working only of first level directory and cannot seem to enable some recursive mode.
and according the documentation fileglob
Matches all files in a single directory, non-recursively, that match a pattern. It calls Python’s “glob” library.
for
How to make Ansible with_fileglob work recursively for all subdirectories
one would need to enhance the module code.
Regarding
Basically I want to recursively iterate all directories for files only, and in a loop apply the template module to them and create them in remote server
as solution and depending on your requirements and what you try to achieve, you could use just
find module – Return a list of files based on specific criteria, in example .conf files. It works with parameter recursive: true and will you provide with a list of full path(s) over which you can loop after.
shell module – Execute shell commands on targets like find and register: result
Similar Q&A
Ansible playbook to find out specific files in sub directories
Ansible: How to find latest files from a directory recursively?
How to search for files containing a particular text with Ansible?
I have to perform the below steps.
Create a Playbook test.yml.
This playbook should copy the file (somefile.j2) to the Host machine's folder1, only if
somefile.j2 does not exist in host01.
By using vi editor you can add the tasks to test.yml.
[Hint: Use the stat and template module].
somefile.j2 is present at /root.
An inventory file named "myhosts" is present at /root
$ cat myhosts
host01 ansible_ssh_user=root
What should be the contents of test.yml?
Homework questions without "the work done so far to solve the problem and a description of the difficulty" is off-topic. But the conflict in the assignment, which might be considered "a practical, answerable problem that is unique to software development", deserves the answer.
Copy the file to the host only if somefile.j2 does not exist
Use the stat and template module
The assignment requires to use the module stat to find out whether the file exists or not. If it does not exist use the module template to create it.
It is not necessary to use the module stat. The module template "will only transfer the file if the destination does not exist" when "force: no" (default yes). Such "idempotent" behavior of Ansible modules is essential, should be expected, and searched for.
Simply take a look at the examples to see "What should be the contents of test.yml?"
I have a playbook with the following task that must copy the 2 Gb file from local to remote servers and extract files:
- name: Copy archived file to target server and extract
unarchive:
src: /path_to_source_dir/file.tar.gz
dest: /path_to_dest_dir
This task fails because ansible copies file to /home mount point on the target server and there's not enough space there:
sftp> put /path_to_source_dir/file.tar.gz /home/my_user_name/.ansible/tmp/ansible-tmp-1551129648.53-14181330218552/source
scp: /home/my_user_name/.ansible/tmp/ansible-tmp-1551129648.53-14181330218552/source: No space left on device
The reason for that is because ansible.cfg has a default parameter:
remote_tmp = ~/.ansible/tmp
How to overwrite this parameter from the playbook (if possible) and make ansible to copy file to the same destination directory specified in the task? So it would be like this:
remote_tmp = /path_to_dest_dir/.ansible/tmp
And the destination path is going to be different each time for a different target server!
Cleaning /home is not an option for me.
The answer here unfortunately is not very clear to me.
There are a few different ways to achieve what you are looking to do. Which one is a matter of preference and your use case.
You found the first way, setting an environment variable before running the playbook. Great for a quick on-the-fly job. Remembering to do that every time you run a certain playbook is indeed annoying. A slight variation of that is to use the environment keyword to set that variable for the play. You can also set environment variable in a role, block or a single task. https://docs.ansible.com/ansible/devel/reference_appendices/playbooks_keywords.html?highlight=environment%20directive. Here is an example of it in use: https://docs.ansible.com/ansible/devel/reference_appendices/faq.html?highlight=environment.
Using the environment keyword in a play et al works well for a specific application of automation, but what if you want Ansible to always use a different remote tmp path for specific servers? Like all variables, the remote_tmp can be sourced from inventory host and group variables not just the config file or environment variables. You need to mind you variable precedence if it is being set in different places. With this you could set remote_tmp in your inventory for that host or a group of hosts. Ansible will always use that path for that host or hosts in that group without having to define it in every play or roles. If you need to change that path, you change it in your inventory and it changes the behavior for all playbook runs without any additional edits. Here is an example of it being used as a host variable in static inventory: https://docs.ansible.com/ansible/devel/reference_appendices/faq.html?highlight=remote_tmp#running-on-solaris
So while the specific issue of "dynamically" setting the remote tmp directory on a host is not a best practice topic per se, it does become an example of the best practice of making the most of variables in Ansible.
For reference, remote temporary directories are handled by the shell plugins. While many parameters are shared, there are others that are specific to the shell Ansible using. Ansible uses sh by default. https://docs.ansible.com/ansible/latest/plugins/shell/sh.html
Hope that helps. Happy automating.
We are using vagrant and ansible to create standard development environments.
The ansible playbooks, vagrant files, etc. are in a git repository.
I've using variable file separation to refer to variable files in the developer's home directory for some senstitive and/or user-specific information (e.g. email address).
We use the variables by doing a vars_file: as part of the playbook, but have to do it for every play.
I don't want to put it in the group_vars/all file because it would then be in the repository and not specific to the user.
I would rather not have a file in the repository that is ignored because people still manage to include it and it screw everybody else up.
Is there a way of doing an equivalent of groups/all which can contain tasks and/or variable definitions that will automatically run whenever a playbook is run?
We use the variables by doing a vars_file: as part of the playbook, but have to do it for every play.
Nope, you can do it on playbook level. (But this might be a new thing, could have been impossible back then, I did not check.)
Is there a way of doing an equivalent of groups/all which can contain tasks and/or variable definitions that will automatically run whenever a playbook is run?
Automatically run/included when?! I don't think this is possible as there would be a lot of open questions like:
Should this be specified on the target machine or the ansible server?
How do you specify for which user should this happen on which host?
If there are tasks: do you want this to be executed on each playbook
when it is run using the given user? What about tasks which specifies
that they run as root (become)? What about tasks that specify a
given user to be executed as? What about tasks that are run as root
but creates a file with the owner matching the given user?
As there are no user scopes with variables and we don't really have a "user context" outlined (see the last questions) we are currently stuck with inclusion of variable files explicitly. Hence the below options:
You can keep using vars_file and specify a first found list.
vars_file:
- - ~/ansible_config/vars.yml
- <default vars file somewhere on the machine>
This way the ansible executor user can redefine values...
You can use the --extra-vars #<filepath> syntax to include all variables from a file, and you can have more than one of these.
A similar thing I do is that I include every variable from every yml file within my GLOBAL_INPUT_DIR (which is an environment variable that can be defined before running the bash script executing ansible-playbook or in a your bash profile or something).
EXTRA_ARGS=`{
{
find "${GLOBAL_INPUT_DIR}" -iname "*.yml";
}\
| while read line; do echo "--extra-vars #${line} "; done \
| tr -d "\n"
}`
ansible-playbook $# ${EXTRA_ARGS}
I usually include something like this in my doings to provide an easy way of redifining variables...
BUT: be aware that this will redefine ALL occurances of a variable name within the playbook (but this was also true with vars_file).