I have the following code, which removes old files in a directory based on their timestamps:
ls -tp | grep -v '/$' | tail -n +2 | xargs -I {} rm -- {}
I am trying to make an executable script out of this and I don't want to cd into the directory where the above command should be run, but rather simply pass the path e.g. /tmp/backups/ to it.
How would I do this? Attaching the path directly after each command ls, grep, tail, xargs and rm ?
Assuming that your path is the first parameter of your script, you could do a
cd "${1:-/tmp/backups}" # Use the parameter, supply default if none
# Do your cleanup here
If you have afterwards more stuff to do in the original working directory, just do a
cd - >/dev/null
# Do what ever you need to do in the original working directory
The sole purpose of the redirection of stdout into the bit bucket is, because cd - by default prints to stdout the directory it changes into.
Related
I'm trying to get subdirectories list in a folder
echo "ls -1 /path/to/folder/*/" | sftp -i /path/to/key user#host | grep -v 'sftp>'
If there is more than one subdirectory I get list of subdirectories:
/path/to/folder/subdirectory1/
/path/to/folder/subdirectory2/
If there is only one subdirectory I get nothing.
Thank you for your suggestions.
Note: using SSH is not allowed
If there is only one subdirectory I get nothing.
You should only get nothing if the only one subdirectory is empty, because ls if given a single directory argument lists its contents. With the normal ls we could solve this problem simply by means of the option -d, but unfortunately sftp's ls doesn't have that option. The only way coming to my mind is to filter the desired directories from a long listing:
echo "ls -l /path/to/folder" | sftp -i /path/to/key user#host | awk '/^d/{print "/path/to/folder/"$NF}'
I have a directory which has 70000 xml files in it. Each file has a tag which looks something like this, for the sake of simplicity:
<ns2:apple>, <ns2:orange>, <ns2:grapes>, <ns2:melon>. Each file has only one fruit tag, i.e. there cannot be both apple and orange in the same file.
I would like rename every file (add "1_" before the beginning of each filename) which has one of: <ns2:apple>, <ns2:orange>, <ns2:melon> inside of it.
I can find such files with egrep:
egrep -r '<ns2:apple>|<ns2:orange>|<ns2:melon>'
So how would it look as a bash script, which I can then user as a cron job?
P.S. Sorry I don't have any bash script draft, I have very little experience with it and the time is of the essence right now.
This may be done with this script:
#!/bin/sh
find /path/to/directory/with/xml -type f | while read f; do
grep -q -E '<ns2:apple>|<ns2:orange>|<ns2:melon>' "$f" && mv "$f" "1_${f}"
done
But it will rescan the directory each time it runs and append 1_ to each file containing one of your tags. This means a lot of excess IO and files with certain tags will be getting 1_ prefix each run, resulting in names like 1_1_1_1_file.xml.
Probably you should think more on design, e.g. move processed files to two directories based on whether file has certain tags or not:
#!/bin/sh
# create output dirs
mkdir -p /path/to/directory/with/xml/with_tags/ /path/to/directory/with/xml/without_tags/
find /path/to/directory/with/xml -maxdepth 1 -mindepth 1 -type f | while read f; do
if grep -q -E '<ns2:apple>|<ns2:orange>|<ns2:melon>'; then
mv "$f" /path/to/directory/with/xml/with_tags/
else
mv "$f" /path/to/directory/with/xml/without_tags/
fi
done
Run this command as a dry run, then remove --dry_run to actually rename the files:
grep -Pl '(<ns2:apple>|<ns2:orange>|<ns2:melon>)' *.xml | xargs rename --dry-run 's/^/1_/'
The command-line utility rename comes in many flavors. Most of them should work for this task. I used the rename version 1.601 by Aristotle Pagaltzis. To install rename, simply download its Perl script and place into $PATH. Or install rename using conda, like so:
conda install rename
Here, grep uses the following options:
-P : Use Perl regexes.
-l : Suppress normal output; instead print the name of each input file from which output would normally have been printed.
SEE ALSO:
grep manual
I'm trying to run a series of commands on a list of files in multiple directories located directly under the current branch.
An example hierarchy is as follows:
/tmp
|-1
| |-a.txt
| |-b.txt
| |-c.txt
|-2
| |-a.txt
| |-b.txt
| |-c.txt
From the /tmp directory I'm sitting at my prompt and I'm trying to run a command against the a.txt file by renaming it to d.txt.
How do I get it to go into each directory and rename the file? I've tried the following and it won't work:
for i in ./*; do
mv "$i" $"(echo $i | sed -e 's/a.txt/d.txt/')"
done
It just doesn't jump into each directory. I've also tried to get it to create files for me, or folders under each hierarchy from the current directory just 1 folder deep, but it won't work using this:
for x in ./; do
mkdir -p cats
done
OR
for x in ./; do
touch $x/cats.txt
done
Any ideas ?
Place the below script in your base directory
#!/bin/bash
# Move 'a.txt's to 'd.txt's recursively
mover()
{
CUR_DIR=$(dirname "$1")
mv "$1" "$CUR_DIR/d.txt"
}
export -f mover
find . -type f -name "a.txt" -exec bash -c 'mover "$0"' {} \;
and execute it.
Note:
If you wish be a bit more innovative and generalize the script, you could accept directory name to search for as a parameter to the script and pass the directory name to find
> for i in ./*; do
As per your own description, this will assign ./1 and then ./2 to i. Neither of those matches any of the actual files. You want
for i in ./*/*; do
As a further aside, the shell is perfectly capable of replacing simple strings using glob patterns. This also coincidentally fixes the problem with not quoting $i when you echo it.
mv "$i" "${i%/a.txt}/d.txt"
I am flattening a directory of nested folders/picture files down to a single folder. I want to move all of the nested files up to the root level.
There are 3,381 files (no directories included in the count). I calculate this number using these two commands and subtracting the directory count (the second command):
find ./ | wc -l
find ./ -type d | wc -l
To flatten, I use this command:
find ./ -mindepth 2 -exec mv -i -v '{}' . \;
Problem is that when I get a count after running the flatten command, my count is off by 46. After going through the list of files before and after (I have a backup), I found that the mv command is overwriting files sometimes even though I'm using -i.
Here's details from the log for one of these files being overwritten...
.//Vacation/CIMG1075.JPG -> ./CIMG1075.JPG
..more log
..more log
..more log
.//dog pics/CIMG1075.JPG -> ./CIMG1075.JPG
So I can see that it is overwriting. I thought -i was supposed to stop this. I also tried a -n and got the same number. Note, I do have about 150 duplicate filenames. Was going to manually rename after I flattened everything I could.
Is it a timing issue?
Is there a way to resolve?
NOTE: it is prompting me that some of the files are overwrites. On those prompts I just press Enter so as not to overwrite. In the case above, there is no prompt. It just overwrites.
Apparently the manual entry clearly states:
The -n and -v options are non-standard and their use in scripts is not recommended.
In other words, you should mimic the -n option yourself. To do that, just check if the file exists and act accordingly. In a shell script where the file is supplied as the first argument, this could be done as follows:
[ -f "${1##*/}" ]
The file, as first argument, contains directories which can be stripped using ##*/. Now simply execute the mv using ||, since we want to execute when the file doesn't exist.
[ -f "${1##*/}" ] || mv "$1" .
Using this, you can edit your find command as follows:
find ./ -mindepth 2 -exec bash -c '[ -f "${0##*/}" ] || mv "$0" .' '{}' \;
Note that we now use $0 because of the bash -c usage. It's first argument, $0, can't be the script name because we have no script. This means the argument order is shifted with respect to a usual shell script.
Why not check if file exists, prior move? Then you can leave the file where it is or you can rename it or do something else...
Test -f or, [] should do the trick?
I am on tablet and can not easyly include the source.
Duplicate
Unable to remove everything else in a folder except FileA
I guess that it is slightly similar to this:
delete [^Music]
However, it does not work.
Put the following command to your ~/.bashrc
shopt -s extglob
You can now delete everything else in the folder except the Music folder by
rm -r !(Music)
Please, be careful with the command.
It is powerful, but dangerous too.
I recommend to test it always with the command
echo rm -r !(Music)
The command
rm (ls | grep -v '^Music$')
should work. If some of your "files" are also subdirectories, then you want to recursively delete them, too:
rm -r (ls | grep -v '^Music$')
Warning: rm -r can be dangerous and you could accidentally delete a lot of files. If you would like to confirm what you will be deleting, try looking at the output of
ls | grep -v '^Music$'
Explanation:
The ls command lists directory contents; without an argument, it defaults to the current directory.
The pipe symbol | redirects output to another command; when the output of ls is redirected in this way, it prints filenames one-per-line, rather than in a column format as you would see if you type ls at an interactive terminal.
The grep command matches lines for patterns; the -v switch means to print lines that don't match the pattern.
The pattern ^Music$ means to match a line starting and ending with Music -- that is, only the string Music; the effect of the ^ (beginning of line) and $ (end of line) characters can also be achieved with the -x switch, as in grep -vx Music.
The syntax command (subcommand) is fish's way of taking the output of one command and passing it over as command-line arguments to another.
The rm command removes files. By default, it does not remove directories, but the -r ("recursive") option changes that.
You can learn about these commands and more by typing man command, where command is what you want to learn about.
So I was looking all over for a way to remove all files in a directory except for some directories, and files, I wanted to keep around. After much searching I devised a way to do it using find.
find -E . -regex './(dir1|dir2|dir3)' -and -type d -prune -o -print -exec rm -rf {} \;
Essentially it uses regex to select the directories to exclude from the results then removes the remaining files. Just wanted to put it out here in case someone else needed it.