I have a VPS sever where I deploy frequently releases and the dir structure is that I have a current dir, what is a symlink to an actual release under a releases dir.
How can I achieve, that only X ( in my case 3) releases stay in the releases dir, the rest can be deleted, to spare free HDD and because I don't need them any more.
This setup is what capifony uses.
Something along this lines should work, where bin would be your symlinked directory :
- name: Set timestamp
set_fact: release_timestamp="{{ansible_date_time.epoch}}"
- name: Deploy code from repository
action: git repo={{repo_url}} dest={{app_dir}}/releases/{{release_timestamp}} remote={{repo_remote}} version={{branch}}
- name: Create symlink to bin folder
file: src={{app_dir}}/releases/{{release_timestamp}} dest={{app_dir}}/bin state=link
- name: List old releases and clean them up
shell: "ls -t {{app_dir}}/releases | tail -n +{{releases_to_keep + 1}}"
register: ls_output
- file: name={{app_dir}}/releases/{{ item }} state=absent
with_items: ls_output.stdout_lines
If you want to do rollback:
- name: Find the previous release
shell: "ls -t {{app_dir}}/releases | head -2 | tail -1"
register: ls_output
- name: Create symlink to previous release folder
file: src={{app_dir}}/releases/{{item}} dest={{app_dir}}/bin state=link
with_items: ls_output.stdout_lines
This is an old topic but I think this answer will be helpful.
- name: Find all directories in releases folder
find:
paths="{{ app_dir }}/{{ releases }}"
file_type=directory
register: dirs
- name: Only last three releases
file:
path="{{ item.path }}"
state=absent
with_items:
- "{{ (dirs.files | sort(attribute='ctime'))[:-3] }}"
Perhaps a bit off-topic, but it will give you a place to start:
There is an Ansible role in the galaxy specifically pointed at replacing Capistrano/Capifony. With an example for a Symfony2 project even :-)
https://galaxy.ansible.com/list#/roles/732
Example usage:
https://github.com/SweetLakePHP/SweetLakePHP/blob/master/ansible/deploy.yml
To answer your question more specifically:
The loop required to clean the releases folder would be either a shell script or a composition of Ansible tasks (saving ls output, sorting and removing). To make this a smoother process, we decided to place this logic in an Ansible module. This module is used in the project_deploy role linked above. But if you prefer to wrote your own role, the module was extracted and placed in ot's own galaxy role:
https://galaxy.ansible.com/list#/roles/2266
Finally, if you're interested in a deeper look at why we came to the current construction:
http://future500.nl/articles/2014/07/thoughts-on-deploying-with-ansible/
Related
How to delete the oldest directory with ansible.
suppose I have the following tree structure
Parent Directory
-Dir2020-05-20
-Dir2020-05-21
-Dir2020-05-22
-Dir2020-05-23
now every time an ansible playbook is run, it should delete the oldest directory, For e.g it should delete Dir2020-05-20 in its first run if we consider its creation date to be 2020-05-20.
age attribute of file module does not seen helpful as i have to run this playbook very randomly and i want to keep limited no. of these directories.
Just assign dirpath to the path of your "Parent Directory" where all these directories are present
---
- hosts: localhost
vars:
dir_path: "/home/harshit/ansible/test/" ##parent directory path, make sure it ends with a slash
tasks:
- name: find oldest directory
shell:
cmd: "ls `ls -tdr | head -n 1 `"
chdir: "{{dir_path}}"
register: dir_name_to_delete
- name: "delete oldest directory: {{dir_path}}{{dir_name_to_delete.stdout}}"
file:
state: absent
path: "{{dir_path}}{{dir_name_to_delete.stdout}}"
Considering a recommended practice is not to use shell or command modules wherever possible I suggest a pure ansible solution for this case:
- name: Get directory list
find:
paths: "{{ target_directory }}"
file_type: directory
register: found_dirs
- name: Get the oldest dir
set_fact:
oldest_dir: "{{ found_dirs.files | sort(attribute='mtime') | first }}"
- name: Delete oldest dir
file:
state: absent
path: "{{ oldest_dir.path }}"
when:
- found_dirs.files | count > 3
There are two ways to know how many files were found with find module - either using its return value matched like this when: found_dirs.matched > 3 or using count filter. I prefer the latter method because I just use this filter in a lot of other cases so this is just a habit.
For your reference, ansible has a whole bunch of useful filters (e.g. I used count and sort here, but there are dozens of them). One does not need to remember those filter names, of course, just keep in mind they exist and might be useful in many cases.
In my Ansible Playbook I'd like to have a task that removes old files and folders from the application's directory. The twist to this otherwise simple task is that a few files or folders need to remain. Imagine something like this:
/opt/application
- /config
- *.properties
- special.yml
- /logs
- /bin
- /var
- /data
- /templates
Let's assume I'd like to keep /logs completely, /var/data and from /config I want to keep special.yml.
(I cannot provide exact code at the moment because I left work frustrated by this and, after cooling down, I am now writing up this question at home)
My idea was to have two lists of exclusions, one holding the folders and one the file. Then I use the find module to first get the folders in the application's directory into a variable and the same for the remaining files into another variable. Afterwards I wanted to remove every folder and file that are not in the lists of exclusions using the file module.
(Pseudo-YML because I'm not yet fluent enough in Ansible that I can whip up a properly structured example; it should be close enough though)
file:
path: "{{ item.path }}"
state: absent
with_items: "{{ found_files_list.files }}"
when: well, that is the big question
What I can't figure out is how to properly construct the when clause. Is it even possible like this?
I don't believe there is a when clause with the file module.
But you can probably achieve what you need as follows:
- name: Find /opt/application all directories, exclude logs, data, and config
find:
paths: /opt/application
excludes: 'logs,data,config'
register: files_to_delete
- name: Ansible remove file glob
file:
path: "{{ item.path }}"
state: absent
with_items: "{{ files_to_delete.files }}"
I hope this is what you need.
First use the find module like you said to get a total list of all files and directories. Register to a variable like all_objects.
- name: Get list of all files recursively
find:
path: /opt/application/
recurse: yes
register: all_objects
Then manually make a list of things you want to keep.
vars:
keep_these:
- /logs
- /var/data
- /config/special.yml
Then this task should delete everything except things in your list:
- name: Delete all files and directories except exclusions
file:
path: "{{ item.path }}"
state: absent
recurse: true
with_items: "{{ all_objects.files }}"
when: item.path not in keep_these
I think this general strategy should work... only thing I'm not sure about is the exact nesting heiararchy of the registered variable from the find module. You might have to play around with the debug module to get it exactly right.
Is possible to edit find module command to find X latest folders inside remotebackup dir? Thanks
- find: paths="{{ remotebackupdir }}" age="**find_oldest_folders_except_X_newer**" recurse=no
register: result
- debug: var=result.files
- file: path: "{{ remotebackupdir }}" state: absent
with_items: result.files
Ive done something similar.
Try something like this: (to be executed in the directory where the files are that you want to delete)
- name: Get first 4 files
command: ls -trd -1 {{path}}/* | head -4
register: filenames
- name: actually delete files
file: {{path}}/{{ item }} state=absent
with_items: filenames.stdout_lines
Basically -tr lists things in reverse date order, and -1 ensures everything is on one line, and -d says directories only
Is there with_fileglob that works remotely in ansible?
Mainly I do want to use something similar with the with_fileglob but that will glob the files on the remote/target machine, not on the one that is running ansible.
Use find module to filter the files and then process the resulting list:
- name: Get files on remote machine
find:
paths: /path/on/remote
register: my_find
- debug:
var: item.path
with_items: "{{ my_find.files }}"
All of the with_* looping mechanisms are local lookups unfortunately so there's no really clean way to do this in Ansible. Remote operations by design must be enclosed in tasks as it would need to deal with connections and inventory etc.
What you can do is generate your fileglob by shelling out to the host and then registering the output and looping over the stdout_lines part of the output.
So a trivial example may be something like this:
- name : get files in /path/
shell : ls /path/*
register: path_files
- name: fetch these back to the local Ansible host for backup purposes
fetch:
src : /path/"{{item}}"
dest: /path/to/backups/
with_items: "{{ path_files.stdout_lines }}"
This would connect to the remote host (e.g., host.example.com), get all the file names under /path/ and then copy them back to the Ansible host to the path: /path/host.example.com/.
Using ls /path/* didn't work for me, so here's an example that uses find and some simple regex to delete all nginx managed virtual hosts:
- name: get all managed vhosts
shell: find /etc/nginx/sites-enabled/ -type f -name \*-managed.conf
register: nginx_managed_virtual_hosts
- name: delete all managed nginx virtual hosts
file:
path: "{{ item }}"
state: absent
with_items: "{{ nginx_managed_virtual_hosts.stdout_lines }}"
You could use it to find all files with a specific extension or any other mix. For instance to simply get all files in a directory: find /etc/nginx/sites-enabled/ -type f.
Here's a way to do it so that you can loop through all found. In my example, i had to look for all instances of pip to wipe out awscli in preparation to install awscli v2.0. I've done similar with lineinfile to strip out vars in /etc/skel dotfiles
- name: search for pip
find:
paths: [ /usr/local/bin, /usr/bin ]
file_type: any
pattern: pip*
register: foundpip
- name: Parse out pip paths (say that 3 times fast)
set_fact:
pips: "{{ foundpip | json_query('files[*].path') }}"
- name: List all the found versions of pip
debug:
msg: "{{ pips }}"
#upgrading pip often leaves broken symlinks or older wrappers behind which doesn't affect pip but breaks playbooks so ignore!
- name: remove awscli with found versions of pip
pip:
name: awscli
state: absent
executable: "{{ item }}"
loop: "{{ pips }}"
ignore_errors: yes
I'm fairly new to Ansible and I'm trying to create a role that copies a file to a remote server. The local file can have a different name every time I'm running the playbook, but it needs to be copied to the same name remotely, something like this:
- name: copy file
copy:
src=*.txt
dest=/path/to/fixedname.txt
Ansible doesn't allow wildcards, so when I wrote a simple playbook with the tasks in the main playbook I could do:
- name: find the filename
connection: local
shell: "ls -1 files/*.txt"
register: myfile
- name: copy file
copy:
src="files/{{ item }}"
dest=/path/to/fixedname.txt
with_items:
- myfile.stdout_lines
However, when I moved the tasks to a role, the first action didn't work anymore, because the relative path is relative to the role while the playbook executes in the root dir of the 'roles' directory. I could add the path to the role's files dir, but is there a more elegant way?
It looks like you need access to a task that looks up information locally, and then uses that information as input to the copy module.
There are two ways to get local information.
use local_action:. That's shorthand for running the task agains 127.0.0.1, more info found here. (this is what you've been using)
use a lookup. This is a plugin system specifically designed for getting information locally. More info here.
In your case, I would go for the second method, using lookup. You could set it up like this example:
vars:
local_file_name: "{{ lookup('pipe', 'ls -1 files/*.txt') }}"
tasks:
- name: copy file
copy: src="{{ local_file_name }}" dest=/path/to/fixedname.txt
Or, more directly:
tasks:
- name: copy file
copy: src="{{ lookup('pipe', 'ls -1 files/*.txt') }}" dest=/path/to/fixedname.txt
With regards to paths
the lookup plugin is run from the context of the task (playbook vs role). This means that it will behave differently depending on where it's used.
In the setup above, the tasks are run directly from a playbook, so the working dir will be:
/path/to/project -- this is the folder where your playbook is.
If you where to add the task to a role, the working dir would be:
/path/to/project/roles/role_name/tasks
In addition, the file and pipe plugins run from within the role/files folder if it exists:
/path/to/project/roles/role_name/files -- this means your command is ls -1 *.txt
caveat:
The plugin is called every time you access the variable. This means you cannot trust debugging the variable in your playbook, and then relying on the variable to have the same value when used later in a role!
I do wonder though, about the use-case for a file that resides inside a projects ansible folders, but who's name is not known in advance. Where does such a file come from? Isn't it possible to add a layer in between the generation of the file and using it in Ansible... or having a fixed local path as a variable? Just curious ;)
Just wanted to throw in an additional answer... I have the same problem as you, where I build an ansible bundle on the fly and copy artifacts (rpms) into a role's files folder, and my rpms have versions in the filename.
When I run the ansible play, I want it to install all rpms, regardless of filenames.
I solved this by using the with_fileglob mechanism in ansible:
- name: Copy RPMs
copy: src="{{ item }}" dest="{{ rpm_cache }}"
with_fileglob: "*.rpm"
register: rpm_files
- name: Install RPMs
yum: name={{ item }} state=present
with_items: "{{ rpm_files.results | map(attribute='dest') | list }}"
I find it a little bit cleaner than the lookup mechanism.