I want to make a rsync with an update of the distant tree. I'd like my command to recursively create missings leaf folders ex :
Before:
Source
A/A/C_file
A/B/C_file
A/C/C_file
B/A/C_file
B/B/C_file
B/C/C_file
distant
A/A/C_file
A/C/C_file
B/A/C_file
B/C/C_file
After the Rsync command "rsync -atvrz source/dir/ distant/dir " :
Distant :
A/A/C_file
A/B/C_file
A/C/C_file
B/A/C_file
B/B/C_file
B/C/C_file
The --relative solution doesn't work for me because it creates the new path inside the distant : "distant/dir/source/dir"
Seems to work when the user right is harmonized.
So a chown -r user:user /distant solved the issue.
Related
I'm wanting to create a script which can run k apply -Rf ./service-token-auth for each of the logical groups here. Mainly all of the graphql-* and data-service-* folders.
Is this something that would be quite easy to implement?
$ ls
README.md data-service-notifications orchestration-workflows-service
argo-cd data-service-reports postgresql-operator
argocd data-service-user prometheus
azure-identities diagnostic-tools pushgateway
azure-nginx-ingress gloo-gateway reloader
azure-private-dns graphql-gateway service-auth
azure-rbacs graphql-service-applications service-b2c-gateway
azure-secrets graphql-service-clients service-dast-auth
blackbox-exporter graphql-service-findings service-dast-ml
cadence graphql-service-logging service-mesh
data-service-application graphql-service-user service-token-auth
data-service-clients kube-state-metrics strimzi-kafka
data-service-findings kuberhealthy tartarus
data-service-logging kubernetes-reflector whs-opa
you can iterate over files in bash
first make sure that it only hits the folders that you want
for i in graphql-* data-service-*; do echo $i; done
then execute
for i in graphql-* data-service-*; do k apply -Rf ./$i; done
I have a directory containing a big number of sub-directories within.
I need to loop over all subdiretories and save it names (without a path!) as a distinct variable
for d in ${output}/*/
do
dir_name=${d%*/}
echo ${dir_name}
done
the problem of the current version that it gives me a full path of the directory instead. Here is the result of echo
/Users/gleb/Desktop/DOcking/clusterizator/sub_folders_to_analyse/7000_CNE_lig992
/Users/gleb/Desktop/DOcking/clusterizator/sub_folders_to_analyse/7000_CNE_lig993
/Users/gleb/Desktop/DOcking/clusterizator/sub_folders_to_analyse/7000_CNE_lig994
/Users/gleb/Desktop/DOcking/clusterizator/sub_folders_to_analyse/7000_CNE_lig995
/Users/gleb/Desktop/DOcking/clusterizator/sub_folders_to_analyse/7000_CNE_lig996
/Users/gleb/Desktop/DOcking/clusterizator/sub_folders_to_analyse/7000_CNE_lig997
/Users/gleb/Desktop/DOcking/clusterizator/sub_folders_to_analyse/7000_CNE_lig998
/Users/gleb/Desktop/DOcking/clusterizator/sub_folders_to_analyse/7000_CNE_lig999
With the dir_name=${d%*/}, you remove the trailing / only. You will want to remove everything upto the last / as well. Or try basename, which is perhapse a better option.
As in:
for d in /var/*/ ; do
dir_name=${d%/}
base=$(basename "$d")
echo "$d $dir_name ${dir_name##*/} $base"
done
which produces:
/var/adm/ /var/adm adm adm
/var/cache/ /var/cache cache cache
/var/db/ /var/db db db
/var/empty/ /var/empty empty empty
/var/games/ /var/games games games
/var/heimdal/ /var/heimdal heimdal heimdal
/var/kerberos/ /var/kerberos kerberos kerberos
/var/lib/ /var/lib lib lib
/var/lock/ /var/lock lock lock
/var/log/ /var/log log log
/var/mail/ /var/mail mail mail
/var/man/ /var/man man man
/var/named/ /var/named named named
/var/netatalk/ /var/netatalk netatalk netatalk
/var/run/ /var/run run run
/var/slapt-get/ /var/slapt-get slapt-get slapt-get
/var/spool/ /var/spool spool spool
/var/state/ /var/state state state
/var/tmp/ /var/tmp tmp tmp
/var/www/ /var/www www www
/var/yp/ /var/yp yp yp
(on my system).
Can you cd to that parent directory?
cd ${output}/
lst=( */ )
for d in "${lst[#]}"; do echo "${d*/}"; done
If that's not an option, then you can strip it each time.
lst=( ${output}/*/ )
for d in "${lst[#]}"; do dir="${d*/}"; echo "${dir##/}"; done
As a hybrid, you can sometimes use a trick of changing directory inside a subshell, as the cd is local to the subshell and "goes away" when it ends, but so do any assignments.
cd /tmp
( cd ${output}/; lst=( */ ); for d in "${lst[#]}"; do echo "${d*/}"; done )
# in /tmp here, lst array does not exist any more...
I have this following content in a file named "/rsync/include.txt"
+ /home/**
+ /opt**
- *
I then call the rsync command as follows:
rsync -avr --include-from="/rsync/include.txt" . ../backup
This produces the following output:
sending incremental file list
created directory ../archive
./
opt/
opt/some-file
opt/include-me/
opt/include-me/me-too
opt/include-me/and-me/
sent 299 bytes received 106 bytes 810.00 bytes/sec
total size is 0 speedup is 0.00
The /home directory exists, and contains files.
Why does the + /home/** pattern not work? I do not want to use the + /home** pattern, as that could match other folder names, e.g., /homeopathy.
Can anybody help me understand why this command doesn't work, and point me in the direction of the working command?
EDIT: While I'd still like an answer to this question, I sincerely suggest using rdiff-backup as it uses similar filtering files and patterns, but is substantially easier to use. I've spent a good deal of time today on this issue with rsync, which was resolved in a few minutes using rdiff-backup.
General debugging info
The easiest way to see if your filter rules do what you want, is to use -vv (increase verbosity twice) in combination with -n (dry-run).
With double verbosity you will see which pattern caused which file to be ignored.
You may grep the output to only see the relevant parts.
Example
% rsync -avv --include 'opt/1' --exclude 'opt/*' -n ./ /tmp \
| grep '\[sender\]'
[sender] showing directory opt/1 because of pattern opt/1
[sender] hiding directory opt/2 because of pattern opt/*
[sender] hiding directory opt/3 because of pattern opt/*
Specific answer
Your example fails because + /home/** is excluded by - *.
man rsync states:
Note that, when using the --recursive (-r) option (which is implied by -a), every subdir component of every path is visited left to right, with each directory having a chance for exclusion before its content.
So the pattern /home/** will be evaluated after /home is traversed, but this will never happen, because - * excludes /home.
To include /home/ you just have to insert it before /home/**, so your exclude file becomes:
+ /home/
+ /home/**
+ /opt/
+ /opt/**
- *
I just noticed that some unknown folder has been created in my /etc/ansible directory.
Here it is ,
monitoring-server#monitoring-server:/etc/ansible$ ls
[' ansible.cfg hosts java.retry java.yml nginx.retry nginx.yml roles test.retry test.yml
This [' is the folder.
monitoring-server#monitoring-server:/etc/ansible$ ls -l
total 56
drw-r--r-- 3 root root 4096 Jul 27 07:12 ['
I need to be root to open it.
root#monitoring-server:/etc/ansible/['# tree
.
└── opt
└── rsyslog']
2 directories, 0 files
Well, the /opt/rsyslog is one of the directories I made on some remote servers using ansible.
How is this folder created and Why is it created?
Directory tree is ['/opt/rsyslog'] – seems like you mistyped string path as list sometime ago and fed it into module with local host as target.
As long as /etc/ansible also seems to be your working directory (which is really weird practice), path [' / opt / rsyslog'] has been created.
You may safely remove it.
I have a private GitHub repo (which I can't share here) cloned locally. I want to split a subfolder in this repo into a new subtree repo. I'm following these instructions Using Git subtrees for repository separation (under Splitting code into its own repository).
My specific command is:
> git subtree split -P .\plugins\rg-feed-client -b rg-feed-client
however it fails with exactly 24 "assertion failed" error messages that look like this:
1/ 26 (0)2/ 26 (1)assertion failed: [ plugins/rg-feed-client = .\plugins\rg-fee
3/ 26 (2)assertion failed: [ plugins/rg-feed-client = .\plugins\rg-feed-client ]
...
26/ 26 (25)assertion failed: [ plugins/rg-feed-client = .\plugins\rg-feed-client ]
If I try any other subfolder, the exact same happens. I have no idea what may be wrong here... HELP!
My repo has 2 remotes: origin, and a remote for an existing subtree that I added to my repo.
This was probably due to the backslashes in --prefix (I was running Windows back then.)
Split -P can't gracefully handle directory path . Use following command instead -
git subtree split --prefix=plugins/rg-feed-client -b rg-feed-client
A few points to remember -
Avoid prefixing ./ with path i.e instead of ./plugins/rg-feed-client use plugins/rg-feed-client
Avoid any trailing / after the path , i.e NO plugins/rg-feed-client/