On Ubuntu 18 server in directory /home/adminuser/keys are 5 files that contain key parts:
/home/adminuser/key/
|- unseal_key_0
|- unseal_key_1
|- unseal_key_2
|- unseal_key_3
|- unseal_key_4
File contents:
1bbeaafab5037a287bde3e5203c8b2cd205f4cc55b4fcffe7931658dc20d8cdcdf
bdf7a6ee4c493aca5b9cc2105077ec67738a0e8bf21936abfc5d1ff8080b628fcb
545c087d3d59d02556bdbf8690c8cc9faafec0e9766bb42de3a7884159356e91b8
053207b0683a8a2886129f7a1988601629a9e7e0d8ddbca02333ce08f1cc7b3887
2320f6275804341ebe5d39a623dd309f233e454b4453c692233ca86212a3d40b5f
Part of Ansible playbook (task):
- name: Reading file contents
command: cat {{item}}
register: unseal_keys
with_fileglob: "/home/adminuser/keys/*"
The error that I get:
"[WARNING]: Unable to find '/home/adminuser/keys' in expected paths (use -vvvvv to see paths)"
I have tried to:
change user that creates directory and files
change path to /home/adminuser/keys/ and /home/adminuser/keys
I expect all of the file contents (that is parts of a single key) to be merged into one string:
1bbeaafab5037a287bde3e5203c8b2cd205f4cc55b4fcffe7931658dc20d8cdcdfbdf7a6ee4c493aca5b9cc2105077ec67738a0e8bf21936abfc5d1ff8080b628fcb545c087d3d59d02556bdbf8690c8cc9faafec0e9766bb42de3a7884159356e91b8 053207b0683a8a2886129f7a1988601629a9e7e0d8ddbca02333ce08f1cc7b38872320f6275804341ebe5d39a623dd309f233e454b4453c692233ca86212a3d40b5f
Given the files below for testing
shell> tree /tmp/admin/
/tmp/admin/
└── key
├── key_0
├── key_1
└── key_2
1 directory, 3 files
shell> cat /tmp/admin/key/key_0
abc
shell> cat /tmp/admin/key/key_1
def
shell> cat /tmp/admin/key/key_2
ghi
Use the module assemble to: "assemble a configuration file from fragments."
Declare the path
key_all_path: /tmp/admin/key_all
and assemble the fragments
- assemble:
src: /tmp/admin/key
dest: "{{ key_all_path }}"
This will create the file /tmp/admin/key_all
shell> cat /tmp/admin/key_all
abc
def
ghi
Read the file and join the lines. Declare the variable
key_all: "{{ lookup('file', key_all_path).splitlines()|join('') }}"
gives
key_all: abcdefghi
Example of a complete playbook for testing
- hosts: localhost
vars:
key_all_path: /tmp/admin/key_all
key_all: "{{ lookup('file', key_all_path).splitlines()|join('') }}"
tasks:
- assemble:
src: /tmp/admin/key
dest: "{{ key_all_path }}"
- debug:
var: key_all
Thanks !
Problem was in paths and hosts where task had to be executed.
Problem is solved by locating and reading files localy and executing this task:
- name: Reading file contents
command: cat "{{item}}"
register: keys ----> all file contents to variable "keys"
with_fileglob: "~/keys/*" ----> this is path to directory all files are storedon my local machine
delegate_to: localhost ----> here I specify that this task will be executed on local machine
become: false ----> remove sudo so that password is not requested
Related
---
- name: Find the latest file
find:
paths: /tmp/data
file_type: file
age: -2m
age_stamp: mtime
register: files
- debug:
msg: '{{ (files.files | sort(attribute="mtime", reverse=true) | first).path if//files.files|count > 0 else "cannot find any file" }}'
- name: Copy file, if found
fetch:
src: '{{ (files.files | sort(attribute="mtime", reverse=true) | first).path }}'
dest: '/tmp'
when: files.files|count > 0
In the above code when the fetch module is running it's just copying the full path. I just need the file. Can someone assist with what src can be set so that it just copies the latest file which is found and not the entire path?
Q: "Copy the latest file which is found and not the entire path."
A: Set flat: true and end the dest path by the slash /. See the notes below. Quoting from flat:
If dest ends with '/', it will use the basename of the source file, similar to the copy module. This can be useful if working with a single host, or if retrieving files that are uniquely named per host.
For example, given the files
shell> tree data1
data1
├── file1
├── file2
└── file3
The playbook below will copy the latest file (but not older than 2 minutes) from the directory data1 to the directory /tmp/localhost/data1/
shell> cat playbook.yml
- hosts: localhost
tasks:
- name: find the latest file
find:
paths: data1
file_type: file
age: -2m
age_stamp: mtime
register: files
- name: Copy file if found
fetch:
src: "{{ (files.files|sort(attribute='mtime', reverse=true)|first).path }}"
dest: /tmp
when: files.files|count > 0
The result will be the file stored in /tmp/localhost/data1/file3
shell> ll /tmp/localhost/data1/file3
-rw-rw-r-- 1 admin admin 0 Mar 1 15:18 /tmp/localhost/data1/file3
If you want to copy the file only change the attributes dest and flat, e.g.
- name: Copy file if found
fetch:
src: "{{ (files.files|sort(attribute='mtime', reverse=true)|first).path }}"
dest: /tmp/
flat: true
when: files.files|count > 0
The result will be the file stored in /tmp/file3
shell> ll /tmp/file3
-rw-rw-r-- 1 admin admin 0 Mar 1 15:18 /tmp/file3
Notes
The module fetch is used for fetching files from remote machines and storing them locally.
By default, the fetched files are stored in a 'host-based directory'. For example, if the dest directory is /backup an src file named /etc/profile on host host.example.com, would be saved into /backup/host.example.com/etc/profile. The hostname is based on the inventory name. See dest
By default, the attribute flat is False
I have a problem when I want to add a variable into a path which is used by with_fileglob - it seems that the variable is always expended to "[]".
I ran the playbook with parameter --extra-vars environment="dev" and got from debug output extra_vars: ('environment=dev',).
Unfortunatelly copy task with with_fileglob failed:
- name: Copy all files from environment subdirectory
copy:
src: "{{item}}"
dest: /etc/
with_fileglob: directory/{{ environment }}/*
TASK [Copy all files from environment subdirectory] ************************************************************************
task path: /home/ansible/playbook/playbook.yml:511
looking for "files/directory/[]" at "/home/ansible/playbook/files/files/directory/[]"
looking for "files/directory/[]" at "/home/ansible/playbook/files/directory/[]"
looking for "files/directory/[]" at "/home/ansible/playbook/files/files/directory/[]"
looking for "files/directory/[]" at "/home/ansible/playbook/files/directory/[]"
[WARNING]: Unable to find 'files/directory/[]' in expected paths (use -vvvvv to see paths)
I am using ansible 2.9.3.
May I ask you what I did wrong?
Thanks a lot for your hints in advance.
environment is a reserved keyword and can't be used as the name of a variable. See Creating valid variable names. The fixed variable in the playbook below works as expected
shell> cat pb.yml
- hosts: localhost
tasks:
- debug:
var: item
with_fileglob: "directory/{{ env }}/*"
Given the tree
shell> tree directory
directory
└── dev
├── file1
├── file2
└── file3
1 directory, 3 files
the abridged result is
shell> ansible-playbook pb.yml -e "env=dev" | grep item:
item: /scratch/tmp/directory/dev/file2
item: /scratch/tmp/directory/dev/file1
item: /scratch/tmp/directory/dev/file3
In one of my ansible roles, I have created a variable via the "defaults/main.yml" file. I am able to reference this variable just fine in "tasks/main.yml" file. However, the variable does not appear to work in "files/some_file.txt"
Is this expected ?
Yes that's expected. tasks/main.yml is being parsed by ansible and will replace variables as you've seen.
Generally files/some_file.txt should contain static files or scripts that should be used with the copy module. As you've discovered it will not be parsed beyond that.
If you want to use variables in a file you should use the template module. Create a templates directory and copy your files there e.g template/some_file.txt. Note that it's common to rename the file with a .j2 extension to indicate that it is a jinja template e.g some_file.j2 but this is not required.
- name: Create fact
set_fact:
my_variable: 123456
- name: Create file from template
template:
src: some_file.j2
dest: "/tmp/some_file.txt"
mode: 0755
some_file.j2 might contain:
This file contains this sentence and the number {{ my_variable }}
After the template task run /tmp/some_file.txt will looks like:
This file contains this sentence and the number 123456
Use template. For example, the role
shell> cat roles/role-17/defaults/main.yml
var1: value1
shell> cat roles/role-17/files/some_file.txt
[{{ var1 }}]
shell> cat roles/role-17/tasks/main.yml
- debug:
msg: task/main.yml [{{ var1 }}]
- debug:
msg: files/some_file.txt {{ lookup('template', 'files/some_file.txt') }}
and the playbook
shell> cat test-17.yml
- hosts: localhost
roles:
- role-17
give
"msg": "task/main.yml [value1]"
"msg": "files/some_file.txt [value1]\n"
I have a following directory structure:
parent_dir/
├── subdir_1
├── subdir_2
└── subdir_3
The subdirs don't have a fixed name and there can be an arbitrary number of them.
How to make ansible run a task for each sub directory?
(any task will do, eventually every dir will be a python package to install, but that isn't important for the context of this question)
This is the solution I managed to come up with, perhaps there is a cleaner way with lookups to achieve this in a single task.
Copy pasting the following code will create a directory structure with a minimal ansible playbook that does the required. (tested on Ubuntu/dash)
mkdir action_per_dir
cd action_per_dir
mkdir -p parent_dir/subdir_1 parent_dir/subdir_2 parent_dir/subdir_3
cat > action_per_dir.yml << "EOF"
---
# Gets all the directories and stores all the return values of `find`
# into the results_of_find
# The return value will consist of:
# https://docs.ansible.com/ansible/latest/modules/find_module.html#return-values
- hosts: localhost
tasks:
- name: Get all dirs
find:
paths: parent_dir
file_type: directory
register: result_of_find
# We're interested only in the `files` part of results of find.
# In pseudo code what's happening here is:
# for each item in result_of_find.files:
# print item.path
#
# The output will be very verbose but for debugging purposes it can be filtered
# ansible-playbook action_per_dir.yml | grep msg
- name: Print all the dirs
debug:
msg: "{{ item.path }}"
with_items: "{{ result_of_find.files }}"
EOF
After that it just needs to be run:
ansible-playbook action_per_dir.yml | grep msg
How to create a directory using ansible where directory names should be taken from a different file
I have tried using with_file command which didnt help much
-bash-4.2$ cat main.yml
---
- name: It is a test yml
file:
dest: "/tmp/destination/{{ item }}"
state: directory
with_file:
- "/tmp/stuff.yml"
-bash-4.2$ cat /tmp/stuff.yml
test
hello
world
The expected output is - 3 Folders to be created as below
/tmp/destination/test
/tmp/destination/hello
/tmp/destination/world
But the output what I received is - Only 1 Folder created as below
/tmp/destination/test?hello?world
Use with_lines
- name: It is a test yml
file:
dest: "/tmp/destination/{{ item }}"
state: directory
with_lines: cat /tmp/stuff.yml