Ansible find module giving error "does not seem to be a valid directory or it cannot be accessed" absolute path - ansible

Ansible find module isn't working as expected.
So i have three instances
One is test node , second controller node and third is from where i am running my ansible playbook
I am trying to generate ssh-keys on test_nodes and then fetching the public keys from those nodes. This is working fine.
Then I am trying to appending these public keys in the authorized_keys file of a different host(controller_node). For this, I am using the find module to get list of files and then loop over these files in authorized_key module.
I was using :
- name: Set authorized key file taken from file
authorized_key:
user: absrivastava
key: "{{ lookup('file','item') }}"
state: present
#with_file:
- "/home/absrivastava/ANSIBLE/ssh-keys/*/home/ribbon/.ssh/id_rsa.pub" This didnt work
#with_filetree:
- "/home/absrivastava/ANSIBLE/ssh-keys/*/home/ribbon/.ssh/id_rsa.pub" This was not appending data
But it didnt seem to work. So i am using find to get list of files and then iterate over them.
- name: Generate ssh keys
hosts: media_nodes
gather_facts: false
tasks:
- name: key generation
openssh_keypair:
path: ~/.ssh/id_ssh_rsa
force: True
register: public_key
- debug:
var: public_key.public_key
- name: fetch public key from all nodes
fetch:
src: ~/.ssh/id_ssh_rsa.pub
dest: ssh-keys/
- name: Controller play
hosts: controller
gather_facts: false
tasks:
- name: Find list of public key files
find:
paths: /home/abhilasha/ANSIBLE/ssh-keys/
file_type: file
recurse: yes
patterns: ".*pub"
use_regex: yes
register: files_matched
- name: debug files matched
debug:
var: files_matched.files
- name: Debug files_matched loop
debug:
var: item.path
loop: "{{ files_matched.files|flatten(levels=1) }}"
loop_control:
label: "{{ item.path }}"
- name: Set authorized key file taken from file
authorized_key:
key: "{{ lookup('file','item') }}"
state: present
with_file:
- "{{ files_matched.files }}"
- name: Find list of public key files
This play is not working giving error
TASK [Find list of public keys] *****************************************************************************************************************************************************************************************************************
ok: [test_controller] => {"changed": false, "examined": 0, "files": [], "matched": 0, "msg": "/home/abhilasha/ANSIBLE/ssh-keys/ was skipped as it does not seem to be a valid directory or it cannot be accessed\n"}

Okay so i got the issue , i was using hosts: controller for this play but the files are on my test VM instance .
But I am not sure how to still solve my problem. I want to use publoc keys on my local and then append it to controller server
- name: Fetch public key files from localhost
gather_facts: false
hosts: 127.0.0.1
connection: local
tasks:
- name: Find list of public keys
find:
paths: ssh-keys/
file_type: file
recurse: yes
patterns: "pub"
use_regex: yes
hidden: yes
register: files_matched
- name: Debug files_matched loop
debug:
var: item.path
loop: "{{ files_matched.files|flatten(levels=1) }}"
loop_control:
label: "{{ item.path }}"
- name: Add Public keys to controller authorized keys
hosts: controller
gather_facts: false
tasks:
- name: Set authorized key file taken from file
authorized_key:
key: "{{ lookup('file','item') }}"
state: present
with_file:
- "{{ files_matched.files }}"
I am unable to use files_matched variable outside the scope of that play. How can i make this work. Thanks in advance

Q: "msg": "/home/abhilasha/ANSIBLE/ssh-keys/ was skipped as it does not seem to be a valid directory or it cannot be accessed\n"
A: Take a look at the directory ssh-keys/ at controller and check the content. Instead of
paths: /home/abhilasha/ANSIBLE/ssh-keys/
find it in the same path
path: ssh-keys/
it has been fetch to
dest: ssh-keys/

Can you change the paths as below and try
- name: fetch public key from all nodes
fetch:
src: ~/.ssh/id_ssh_rsa.pub
dest: /tmp/ssh-keys/
- name: Find list of public key files
find:
paths: /tmp/ssh-keys/
file_type: file
recurse: yes
patterns: ".*pub"
use_regex: yes
register: files_matched

If you are trying to copy/find files from your local machine to the remote, by default the find module will run on the remote, not your local machine. As a result, the error will be thrown if those directories don't exist on your remote.
So you just tell it to "find" on your local machine by specifying delegate_to: localhost and it should work.
tasks:
- find:
paths:
- local_dir1
- local_dir2
file_type: file
patterns: '*.tgz'
register: files_output
# Execute task on this host instead of the target (inventory_hostname).
delegate_to: localhost
- block:
- set_fact:
files: "{{ files_output.files | map(attribute='path') }}"
- set_fact:
files: '{{ files + more }}'
vars:
more:
- '{{playbook_dir}}/remote.sh'
- debug:
msg: '{{ item }}'
loop: '{{ files }}'

Related

Iterate over dict object using ansible

I have a local.yml file shown below where I am defining the variable JDK_VERSION as list. calling the role from different repo
- role: jdk_install
vars:
JDK_VERSION: [ 'jdk1.8', 'jdk11', 'jdk17' ]
I have defined list varibles in local.yml as key in vars.yml
JDK_VERSIONS:
"jdk1.8": ["1.8.0_352", "1.8.0_322"]
"jdk11": ["11.0.17_8", "11.0.18_8"
"jdk17": "17.0.5_8"
1.8.0_352:
"package_name": "sincro-jdk-1.8.0_352"
"jdk_dirname": "jdk1.8.0_352"
"sym_link": "/opt/jdk1.8"
"installer": "rpm"
1.8.0_322:
"package_name": "sincro-jdk-1.8.0_322"
"jdk_dirname": "jdk1.8.0_322"
"sym_link": "/opt/jdk1.8"
"installer": "rpm"
11.0.17_8:
"download_url": https://artifactory.sincrod.com/artifactory/github-releases/adoptium/temurin11-binaries/releases/download/jdk-11.0.17+8/OpenJDK11U-jdk_x64_linux_hotspot_11.0.17_8.tar.gz
"package_name": "OpenJDK11U-jdk_x64_linux_hotspot_11.0.17_8.tar.gz"
"jdk_dirname": "jdk-11.0.17+8"
"sym_link": "/opt/jdk11"
"installer": "tar"
17.0.5_8:
"download_url": https://artifactory.sincrod.com/artifactory/github-releases/adoptium/temurin17-binaries/releases/download/jdk-17.0.5+8/OpenJDK17U-jdk_x64_linux_hotspot_17.0.5_8.tar.gz
"package_name": "OpenJDK17U-jdk_x64_linux_hotspot_17.0.5_8.tar.gz"
"jdk_dirname": "jdk-17.0.5+8"
"sym_link": "/opt/jdk17"
"installer": "tar"
The main.yml file looks like below
---
- name: Install Java
include: install_jdk.yml
vars:
"install_jdk" : "{{ item }}"
with_items:
- "{{ JDK_VERSION }}"
- name: Setting default jdk
include: default_jdk.yml
vars:
"default_jdk" : "{{ JDK_VERSION.0 }}"
And install.yml file looks like below
---
- name: JDK version
debug:
msg: "{{ install_jdk }}"
- name: Process JDK details
set_fact:
jdk_details: "{{ lookup('vars', JDK_VERSIONS[install_jdk], default='1.8.0_352') }}"
- name: Print JDK version
debug:
msg: "{{ jdk_details }}"
- name: Installing JDK from rpm
yum:
name: "{{ item }}"
update_cache: true
state: installed
when: jdk_details.installer == "rpm"
- name: Installing JDK from source
block:
- name: download jdk tar
get_url:
url: "{{ jdk_details.download_url }}"
dest: /tmp
mode: 0755
group: root
owner: root
- name: Create jdk installation directory path
file:
path: "/opt/data/services/jdks/"
state: directory
- name: Untar JDK installtion files
unarchive:
src: /tmp/{{ jdk_details.package_name }}
dest: /opt/data/services/jdks
remote_src: True
when: jdk_details.installer == "tar"
- name: create symlink for JDK version
file:
src: "/opt/data/services/jdks/{{ jdk_details.jdk_dirname }}"
dest: "{{ jdk_details.sym_link }}"
state: link
force: yes
follow: False
I want to iterate over the list mentioned in local.yml.
Also local.yml list item act as key for vars.yml. Now vars.yml or vars/main.yml has keys with multiple values.
So, basically I want to iterate similar to nested list or list[dict{[list]}] kind of structure.
In the above mentioned example the key "jdk1.8": ["1.8.0_352", "1.8.0_322"] has two item, so it should install two packages

How to use the lookup plugin to get the directory path and file in Ansible

I have a two playbooks where one creates SSH Keys and the other one creates a new user and deploys the public ssh key for the new user created.
My issue is I created a task that create a new directory with a timestamp to store the relevant data, I was able to get the path to a variable where I added it as a dummy host so that I can be able to call that path with all my plays but it seems like I am unable to use the same variable in lookup so that I can be able to deploy the ssh key. Kindly assist, below are the relevant tasks.
# Create the directory with timestamp
- name: Create Directory with timestamp to store data that was run multiple times that day
when: inventory_hostname in groups['local']
file:
path: "{{store_files_path}}/{{ansible_date_time.date}}/{{ansible_date_time.time}}"
state: directory
mode: "0755"
register: dir_path
# Add the directory path to dummy host called save so that I can call it from other plays
- name: Add dir path:"{{dir_path.path}}" as a 'save' host
when: inventory_hostname in groups['local']
add_host:
name: "save"
dir: "{{dir_path.path}}"
# Deploying SSH Key I tried this -->
- name: Deploy Public Key to the server
when: inventory_hostname in groups['Servers']
authorized_key:
user: "{{hostvars['new-user']['user']}}"
state: present
key: "{{dir_path.path}}/SSH-Key.pub"
# ...this -->
- name: Deploy Public Key to the server
when: inventory_hostname in groups['Servers']
authorized_key:
user: "{{hostvars['new-user']['user']}}"
state: present
key: "{{ lookup('file','{{dir_path.path}}/SSH-Key.pub') }}"
# .... and this -->
- name: Deploy Public Key to the server
when: inventory_hostname in groups['Servers']
authorized_key:
user: "{{hostvars['new-user']['user']}}"
state: present
key: "{{ lookup('file','{{hostvars['save']['dir']}}/SSH-Key.pub') }}"
None of them worked, what am I doing wrong?
If you put a Jinja expression into a string in a Jinja expression, then you indeed end up with a your variable not being interpreted.
A basic example of this is:
- hosts: all
gather_facts: no
tasks:
- debug:
msg: "{{ '{{ foo }}' }}"
vars:
foo: bar
Which gives:
ok: [localhost] => {
"msg": "{{ foo }}"
}
When
- hosts: all
gather_facts: no
tasks:
- debug:
msg: "{{ foo }}"
vars:
foo: bar
Gives thes expected:
ok: [localhost] => {
"msg": "bar"
}
So in order to achieve what you want here, you should use the concatenation operator of Jinja: ~, in order to let Jinja interpret your variable and concatenate it with the rest of your "hardcoded" string.
Effectively ending with the instruction:
key: "{{ lookup('file', hostvars['save']['dir'] ~ '/SSH-Key.pub') }}"

In Ansible loop, test existence of files from registered results

I have several files that I need to backup in different directories. I have tried the code below and not working for me.
vars:
file_vars:
- {name: /file1}
- {name: /etc/file2}
- {name: /etc/file/file3}
tasks:
- name: "Checking if config files exists"
stat:
path: "{{ item.name }}"
with_items: "{{ file_vars }}"
register: stat_result
- name: Backup Files
copy: src={{ item.name }} dest={{ item.name }}{{ ansible_date_time.date }}.bak
with_items: "{{ file_vars }}"
remote_src: yes
when: stat_result.stat.exists == True
The problem is the condition
when: stat_result.stat.exists == True
There is no attribute stat_result.stat. Instead, the attribute stat_result.results is a list of the results from the loop. It's possible to create a dictionary of files and their statuses. For example
- set_fact:
files_stats: "{{ dict(my_files|zip(my_stats)) }}"
vars:
my_files: "{{ stat_result.results|json_query('[].item.name') }}"
my_stats: "{{ stat_result.results|json_query('[].stat.exists') }}"
Then simply use this dictionary in the condition
when: files_stats[item.name]
Below is a shorter version which creates the dictionary more efficiently
- set_fact:
files_stats: "{{ dict(stat_result.results|
json_query('[].[item.name, stat.exists]')) }}"
Please try using below worked for me:
---
- name: Copy files
hosts: localhost
become: yes
become_user: root
vars_files:
- files.yml
tasks:
- name: "Checking if config files exists"
stat:
path: "{{ item }}"
with_items: "{{ files }}"
register: stat_result
- name: Ansible
debug:
msg: "{{ stat_result }}"
- name: Backup Files
copy:
src: "{{ item }}"
dest: "{{ item.bak }}"
with_items: "{{ files }}"
when: stat_result == "True"
and files.yml will look like:
---
files:
- /tmp/file1
- /tmp/file2
you can check you playbook syntax using below command:
ansible-playbook copy.yml --syntax-check
Also you do dry run your playbook before actual execution.
ansible-playbook -i localhost copy.yml --check

win_find FAILED! => {"msg": "'dict object' has no attribute 'files'"}

I am trying to run win_find with variables for "paths" to find temp files (e.g. c:\users\public\appdata\local\temp) but when I pass it to debug I get the error message
"FAILED! => {"msg": "'dict object' has no attribute 'files'"}"
I have tried swapping the "register" and "with_items" lines but that seems to make no difference at all.
- hosts: windows-02
vars:
dir: \AppData\Local\Temp\
tasks:
- name: find user directories
win_find:
paths: c:\users
recurce: yes
file_type: directory
get_checksum: no
register: user_dir
- name: debug 1
debug:
msg: "{{ item.path + dir }}"
loop: "{{ user_dir.files }}"
- name: find temp files
win_find:
paths: "{{ item.path + dir }}"
recurce: yes
hidden: yes
get_checksum: no
register: files_to_delete
with_items: "{{ user_dir.files }}"
- name: debug
debug:
msg: "{{ item }}"
loop: "{{ files_to_delete.files }}"
- name: remove
win_file:
path: "{{ item.path }}"
state: absent
with_items: "{{ files_to_delete.files }}"
I expect to get a list of files to be deleted which will be passed to the "win_files" module. Instead I'm getting the error message
"FAILED! => {"msg": "'dict object' has no attribute 'files'"}"
There is a typo in win_find module, it is recurse: yes
Please find the below code whcih worked for me
---
- name: Find files
win_find:
paths: "{{paths}}"
age: "{{duration}}"
register: log_files
- name: Delete the files
win_file:
path: "{{item.path}}"
state: absent
with_items: "{{log_files.files}}"
Also files_to_delete will not have files_to_delete.files as it is in loop. It will have files_to_delete.results
So, it appears that win_find, at least in my case, does not return files when using "when_items". I got around this by creating a list of paths and passing it into the "paths" parameter of the module.
Here is my code:
- hosts: "{{ host }}"
vars:
dir: \AppData\Local\Temp\
temp_paths: []
line_break: \n
tasks:
- name: find user directories
win_find:
paths: c:\users
recurse: no
file_type: directory
get_checksum: no
register: user_dir
- name: debug 1
debug:
msg: "{{ item.path + dir }}"
loop: "{{ user_dir.files }}"
- name: set temp path
set_fact: temp_paths="{{temp_paths + [ item.path + dir ] }}"
when: item.path != 'C:\\users\\Public'
with_items: "{{ user_dir.files }}"
- name: find temp files
win_find:
paths: "{{ temp_paths }}"
recurse: True
patterns: '*'
hidden: False
get_checksum: False
register: files_to_delete
# ignore_errors: yes
- name: debug
debug:
msg: "{{ item.path }}"
loop: "{{ files_to_delete.files }}"
- name: remove
win_file:
path: "{{ item.path }}"
state: absent
with_items: "{{ files_to_delete.files }}"

Ansible access same variables from multiple Json files

I have multiple .json files on local host where I place my playbook:
json-file-path/{{ testName }}.json
{{ testName }}.json are: testA.json, testB.json, testC.json ... etc.
All .json files have same keys with different values like this:
json-file-path/testA.json:
{
“a_key”: “a_value1”
“b_key”: “b_value1”
}
json-file-path/testB.json:
{
“a_key”: “a_value2”
“b_key”: “b_value2”
}
json-file-path/testC.json:
{
“a_key”: “a_value3”
“b_key”: “b_value3”
}
.....
I need to access the key-value variables from all .json files and if the values meet some condition, I will perform some task in target host. For example, I have:
a_value1=3
a_value2=4
a_value3=1
I go through my .json file one by one, if a_key[value]>3, I will copy this .json file to target host, otherwise skip the task. In this case, I will only copy testC.json to target host.
How would I achieve this? I was thinking of re-constructing my .json files using {{ testName }} as dynamic key of dict like this:
{
“testName”: “testA”
{
“a_key”: “a_value1”
“b_key”: “b_value1”
}
So I can access my variable as {{ testName}}.a_key. So far I haven’t been able to achieve this.
I have tried the following in my playbook:
—-
- host: localhost
tasks:
- name: construct json files
vars:
my_vars:
a_key: “{{ a_value }}”
b_key: “{{ b_value }}”
with_dict: “{{ testName }}”
copy:
content: “{{ my_vars | to_nice_json }}”
dest: /json-file-path/{{ testName }}.json
My updated playbook are:
/mypath/tmp/include.yaml:
—-
- hosts: remote_hostName
tasks:
- name: load json files
set_fact:
json_data: “{{ lookup(‘file’, item) | from_json }}”
- name: copy json file if condition meets
copy:
src: “{{ item }}”
dest: “{{ /remote_host_path/tmp}}/{{item | basename }}”
delegate_to: “{{ remote_hostName }}”
when: json_data.a_key|int>5
/mypath/test.yml:
—-
- hosts: localhost
vars:
local_src_ dir: /mypath/tmp
remote_host: remote_hostName
remote_dest_dir: /remote_host_path/tmp
tasks:
- name: looping
include: include.yaml
with_fileglob:
- “{{ local_src_dir }}/*json”
All json files on localhost under /mypath/tmp/.
Latest version of playbook. It is working now:
/mypath/tmp/include.yaml:
—-
- name: loafing json flies
include_vars:
file: “{{ item }}”
name: json_data
- name: copy json file to remote if condition meets
copy:
src: “{{ item }}”
dest: ‘/remote_host_path/tmp/{{item | basename}}’
delegate_to: “{{ remote_host }}”
when: json_data.a_key > 5
/mypath/test.yml:
—-
- hosts: localhost
vars:
local_src_dir: /mypath/tmp
remote_host: remote_hostName
remote_dest_dir: /remote_host_path/tmp
tasks:
- name: looping json files
include: include.yaml
with_fileglob:
- “{{ local_src_dir }}”/*json”
I am hoping that I have understood your requirements correctly, and that this helps move you forward.
Fundamentally, you can load each of the JSON files so you can query the values as native Ansible variables. Therefore you can loop through all the files, read each one, compare the value you are interested in and then conditionally copy to your remote host via a delegated task. Therefore, give this a try:
Create an include file include.yaml:
---
# 'item' contains a path to a local JSON file on each pass of the loop
- name: Load the json file
set_fact:
json_data: "{{ lookup('file', item) | from_json }}"
- name: Delegate a copy task to the remote host conditionally
copy:
src: "{{ item }}"
dest: "{{ remote_dest_dir }}/{{ item | basename }}"
delegate_to: "{{ remote_host }}"
when: json_data.a_key > value_threshold
then in your playbook:
---
- hosts: localhost
connection: local
# Set some example vars, tho these could be placed in a variety of places
vars:
local_src_dir: /some/local/path
remote_host: <some_inventory_hostname>
remote_dest_dir: /some/remote/path
value_threshold: 3
tasks:
- name: Loop through all *json files, passing matches to include.yaml
include: include.yaml
loop: "{{ lookup('fileglob', local_src_dir + '/*json').split(',') }}"
Note: As you are running an old version of Ansible, you may need older alternate syntax for all of this to work:
In your include file:
- name: Load the json file
set_fact:
include_vars: "{{ item }}"
- name: Delegate a copy task to the remote host conditionally
copy:
src: "{{ item }}"
dest: "{{ remote_dest_dir }}/{{ item | basename }}"
delegate_to: "{{ remote_host }}"
when: a_key > value_threshold
and in your playbook:
- name: Loop through all *json files, passing matches to include.yaml
include: include.yaml
with_fileglob:
- "{{ local_src_dir }}/*json"

Resources