I have a rename command as follows:
rename ".csv" "-en.csv" Daily_vills.csv
rename "^" "07302019" Daily*
when i run this, i get an error "rename: invalid option -- 'e'". I already tried "\-en.csv", but this results in "Daily_vills\-en.csv"
my question is how to make bash understand that -en is a replacement value and not a parameter.
original:
Daily_vills.csv
Target;
07302019Daily_vills-en.csv
Any help on this is greatly appreciated
Use -- to notify rename of end of options.
rename -- ".csv" "-en.csv" Daily_vills.csv
From posix utility conventions:
Guideline 10:
The first -- argument that is not an option-argument should be accepted as a delimiter indicating the end of options. Any following arguments should be treated as operands, even if they begin with the '-' character.
It is common in many *nix tools to notify with -- the end of options. Examples:
# touch a file named -a
touch -- -a
# cat a file named -a
cat -- -a
# printf the string -b
printf -- -b
# find a string '-c' in files in current directory
grep -- -c *
The rename utility also follows the guideline from posix utility conventions. The -- may be used to notify rename of the end of options.
Related
In my directory I have thousands of PDF files. I want to write a shell script where goes through all the files and trims the last 16 characters and and saves back to the directory without keeping the old filename.
Now:
KUD_1234_Abc_DEF_9055_01.pdf
New:
KUD_1234.pdf
How can I solve that.
Thank you all
To the importance of analyzing and describing a problem properly to find a proper solution.
Here I implement exactly what you ask for:
#!/usr/bin/env sh
for oldname
do
# Capture old file name extension for re-use, by trimming-out the leading
# characters up-to including the dot
extension=${oldname##*.}
# Capture the old name without extension
extensionless=${oldname%.*}
# Compose new name by printing the old file name
# up to its length minus 16 characters
# and re-adding the extension
newname=$(
printf '%.*s.%s\n' $((${#extensionless}-16)) "$extensionless" "$extension"
)
# Demonstrate rename as a dummy
echo mv -- "$oldname" "$newname"
done
Works for your sample case:
mv -- KUD_1234_Abc_DEF_9055_01.pdf KUD_1234.pdf
Will collide not rename this:
mv -- KUD_1234_ooh_fail_666_02.pdf KUD_1234.pdf
Will not work with names shorter than 16 characters:
mv -- notwork.pdf notwork.pdf
Will probably not do what you expect if name has no dot extension:
mv -- foobar foobar.foobar
This should work for you (please backup data before trying):
find -type f | sed -E 's|^(.+)(.{16})(\.pdf)$|\1\2\3\ \1\3|g' | xargs -I f -- bash -c "mv f"
However, it's much easier to do it with python:
import os
os.chdir("/home/tkhalymon/dev/tmp/empty")
for f in os.listdir("."):
name, ext = os.path.splitext(f)
os.rename(f, f"{name[:-16]}{ext}")
There are few files in a directory. I would like to find all the files based on wildcard criteria and then rename it by appending a date or timestamp field to it using single line command
Example :
foo1.txt
foo1.log
foo2.txt
foo2.log
I want to find all .log files and rename them by appending date field to it
Expected output :
foo1.txt
foo1_20210609.log
foo2.txt
foo2_20210609.log
I would use a "for" loop over the wildcard list of matches and then use parameter expansion and command substitution to splice out the rest:
for file in *.log
do
echo mv -- "$file" "${file%.log}_$(date +%Y%m%d).log"
done
The pieces in the middle break down as:
mv -- -- invoke "mv" and explicitly tell it that there are no more options; this insulates us from filenames that might start with -i, for example
"${file%.log} -- expands the "$file" variable and removes the ".log" from the end
_ -- just adds the underscore where we want it
$(date +%Y%m%d) -- calls out to the date command and inserts the resulting output
.log -- just adds the ".log" part back at the end
Remove the "echo" if you like the resulting commands.
If you want a static timestamp, then just use that text in place of the whole $(date ...) command substitution.
On your sample input, but with today's date, the output is:
mv -- foo1.log foo1_20210610.log
mv -- foo2.log foo2_20210610.log
ls | perl -lne '$date=`date '+%Y%m%d'`; chomp($date); `mv $_ $_$date`;'
ls the file
pipe it into a perl script
e option executes in line
n option splits the input into lines
l option adds new line to each
back ticks execute unix commands
My bash script create an associative array with files as keys.
declare -A fileInFolder
for f in "${zipFolder}/"*".zip"; do
read -r fileInFolder[$f] _ < <(md5sum "$f")
done
... code for removing some entry in fileInFolder ...
unzip -qqc "${!fileInFolder[#]}"
And unzip prevent me caution: filename not matched: for everyone except first file.
The following command work without any problem:
unzip -qqc "${zipFolder}/"\*".zip"
I try using 7z but I did't find the way to give more than one zip file as input (using -ai option this need a list of file separated by new line if my understanding is correct...)
Ignoring the reason for the Associative array storing MD5 of zip files.
As #Enrico Maria De Angelis pointed-out, unzip only accepts one zip file argument per invocation. So you can not expand the associative array file names indexes into arguments for a single call to unzip.
I propose this solution:
#!/usr/bin/env bash
# You don't want to unzip the pattern name if none match
shopt -s nullglob
declare -A fileInFolder
for f in "${zipFolder}/"*".zip"; do
# Store MD5 of zip file into assoc array fileInFolder
# key: zip file name
# value: md5sum of zip file
read -r fileInFolder["$f"] < <(md5sum "$f")
# Unzip file content to stdout
unzip -qqc "$f"
done | {
# Stream the for loop's stdout to the awk script
awk -f script.awk
}
Alternative implementation calling md5sum only once for all zip files
shopt -s nullglob
# Iterate the null delimited entries output from md5sum
# Reading first IFS=' ' space delimited field as sum
# and remaining of entry until null as zipname
while IFS=' ' read -r -d '' sum zipname; do
# In case md5sum file patterns has no match
# It will return the md5sum of stdin with file name -
# If so, break out of the while
[ "$zipname" = '-' ] && break
fileInFolder["$zipname"]="$sum"
# Unzip file to stdout
unzip -qqc -- "$zipname"
done < <(md5sum --zero -- "$zipFolder/"*'.zip' </dev/null) | awk -f script.awk
Foreword
This answer, which basically boils down to you can't do this with one unzip command, assumes you are aware you can put unzip -qqc "$f" in the for loop you wrote in your question, and that you don't want to do it for some reason.
My answer
You are not getting an error for all files; rather, you are getting an error for all files from the second one on.
Simply try the following
unzip -qqc file1.zip file2.zip
and you will get the error
caution: filename not matched: file2.zip
which is just the one you are getting.
From the unzip's man page
SYNOPSIS
unzip [-Z] [-cflptTuvz[abjnoqsCDKLMUVWX$/:^]] file[.zip] [file(s) ...] [-x xfile(s) ...] [-d exdir]```
it looks like you are only allowed to provide one zip file on the command line.
Well, actually not quite, in that you can specify more zip files on the command line, but to do so, you have to rely on unzip's own way of interpreting its own command line; this partly mimics the shell, but all it can do is listed in the man page:
ARGUMENTS
file[.zip]
Path of the ZIP archive(s). If the file specification is a wildcard, each matching file is processed in an order determined by the
operating system (or file system). Only the filename can be a wildcard; the path itself cannot. Wildcard expressions are similar to
those supported in commonly used Unix shells (sh, ksh, csh) and may contain:
* matches a sequence of 0 or more characters
? matches exactly 1 character
[...] matches any single character found inside the brackets; ranges are specified by a beginning character, a hyphen, and an ending
character. If an exclamation point or a caret (`!' or `^') follows the left bracket, then the range of characters within the
brackets is complemented (that is, anything except the characters inside the brackets is considered a match). To specify a
verbatim left bracket, the three-character sequence ``[[]'' has to be used.
(Be sure to quote any character that might otherwise be interpreted or modified by the operating system, particularly under Unix and
VMS.) If no matches are found, the specification is assumed to be a literal filename; and if that also fails, the suffix .zip is ap‐
pended. Note that self-extracting ZIP files are supported, as with any other ZIP archive; just specify the .exe suffix (if any) ex‐
plicitly. ```
So you are technically facing the same issue you've found with 7z.
I am new to programming and just starting in bash.
I'm trying to print a list of directories and files to a txt file, and remove some of the path that gets printed to make it cleaner.
It works with this:
TODAY=$(date +"%Y-%m-%d")
cd
cd Downloads
ls -R ~/Music/iTunes/iTunes\ Media/Music | sed 's/\/Users\/BilPaLo\/Music\/iTunes\/iTunes\ Media\/Music\///g' > music-list-$TODAY.txt
But to clean it up I want to use variables like so,
# Creates a string of the date, format YYYY-MM-DD
TODAY="$(date +"%Y-%m-%d")"
# Where my music folders are
MUSIC="$HOME/Music/iTunes/iTunes\ Media/Music/"
# Where I want it to go
DESTINATION="$HOME/Downloads/music-list-"$TODAY".txt"
# Path name to be removed from text file
REMOVED="\/Users\/BilPaLo\/Music\/iTunes\/iTunes\ Media\/Music\/"
ls -R "$MUSIC" > "$DESTINATION"
sed "s/$REMOVED//g" > "$DESTINATION"
but it gives me a 'no such file or directory' error that I can't seem to get around.
I'm sure there are many other problems with this code but this one I don't understand.
Thank you everyone! I followed the much needed formatting advice and #amo-ej1's answer and now this works:
# Creates a string of the date format YYYY-MM-DD
today="$(date +"%Y-%m-%d")"
# Where my music folders are
music="$HOME/Music/iTunes/iTunes Media/Music/"
# Where I want it to go
destination="$HOME/Downloads/music-list-$today.txt"
# Temporary file
temp="$HOME/Downloads/temp.txt"
# Path name to be removed of text file to only leave artist name and album
remove="\\/Users\\/BilPaLo\\/Music\\/iTunes\\/iTunes\\ Media\\/Music\\/"
# lists all children of music and writes it in temp
ls -R "$music" > "$temp"
# substitutes remove by nothing and writes it in destination
sed "s/$remove//g" "$temp" > "$destination"
rm $temp #deletes temp
First when debugging bash it can be helpful to start bash with the -x flags (bash -x script.sh) or within the script enter set -x, that way bash will print out the commands it is executing (with the variable expansions) and you can more easily spot errors that way.
In this specific snippet our ls output is being redirected to a file called $DESTINATION and and sed will read from standard input and write also to $DESTINATION. So however you wanted to replace the pipe in your oneliner is wrong. As a result this will look as if your program is blocked but sed will simply wait for input arriving on standard input.
As for the 'no such file or directory', try executing with set -x and doublecheck the paths it is trying to access.
I am trying to write a custom command to copy files into a specific directory.
I am not sure the best way to do this. Right now, the script is this
#!/bin/sh
cp -rf $1 /support/save/
I called this command filesave, it works great for 1 file, but if you do *.sh or something similar it only copies the first file. This makes sense as that is the point of $1. Is there an input variable that will just collect all inputs not just the specific one?
#!/bin/sh
cp -rf -- "$#" /support/save
Use "$#" to expand to your entire argument list. It is essential that this be placed in double-quotes, or else it will behave identically to $* (which is to say, incorrectly).
The -- is a widely implemented extension which ensures that all following arguments are treated as literal arguments rather than parsed as options, thus making filenames starting with - safe.
To demonstrate the difference, name the following script quotdemo.
#!/bin/sh
printf '$#: '; printf '<%s>\n' "$#"
printf '$*: '; printf '[%s]\n' $*
...and try running:
touch foo.txt bar.txt "file with spaces.txt" # create some matching files
quotdemo *.txt # ...then test this...
quotdome "*.txt" # ...and this too!