I am making a script. i need to automate a portion that looks at all the files in the directory containing the string "HNAZXLCOM" in the name then take the newest file with that string in the name and put the filename into a variable.
Thanks
function latest {
if [[ $FUNCNAME == ${FUNCNAME[1]} ]]; then
unset -v x latest files
printf -v "$#"
elif (($# > 2)); then
printf '%s\n' "Usage: $FUNCNAME <glob> <varname>" 'Error: Takes at most 2 arguments. Glob defaults to *'
return 1
else
if ! shopt -q nullglob; then
typeset -f +t "$FUNCNAME"
trap 'shopt -u nullglob; trap - RETURN' RETURN
shopt -s nullglob
fi
IFS= typeset -a 'files=(${1:-*})'
typeset latest x
for x in "${files[#]}"; do
[[ -d $x || $x -ot $latest ]] || latest=$x
done
${2:+"$FUNCNAME"} "${2:-printf}" -- %s "$latest"
fi
}
latest '*HNAZXLCOM*' myVar
If I read you right
var="$(ls -t1 *HNAZXLCOM*|head -n 1)"
From man ls, the -t switch is relevant. Also see the -1 switch.
-t sort by modification time, newest first
-1 list one file per line
The -1 switch is not relevant, because ls automatically writes only one line per name when outputting to a pipe.
At a command prompt, the following function will list the 13 most recent files; eg, xt *HNAZXLCOM* would list the 13 most recent files with that string in their names, while xt by itself would list the most recent files (of any pattern) in the current working directory. Adjust the number to whatever you prefer. Some versions of head may require -n 13 instead of -13. In your application,
H=$(ls -t *HNAZXLCOM* | head -n 1)
is appropriate.
xt ()
{
date;
ls --color -Glt --color --time-style="+%Y-%m-%d %T" $* | grep -v "/" | head -13
}
Related
I have made two programms and I'm trying to call the one from the other but this is appearing on my screen:
cp: cannot stat ‘PerShip/.csv’: No such file or directory
cp: target ‘tmpship.csv’ is not a directory
I don't know what to do. Here are the programms. Could somebody help me please?
#!/bin/bash
shipname=$1
imo=$(grep "$shipname" shipsNAME-IMO.txt | cut -d "," -f 2)
cp PerShip/$imo'.csv' tmpship.csv
dist=$(octave -q ShipDistance.m 2>/dev/null)
grep "$shipname" shipsNAME-IMO.txt | cut -d "," -f 2 > IMO.txt
idnumber=$(cut -b 4-10 IMO.txt)
echo $idnumber,$dist
#!/bin/bash
rm -f shipsdist.csv
for ship in $(cat shipsNAME-IMO.txt | cut -d "," -f 1)
do
./FindShipDistance "$ship" >> shipsdist.csv
done
cat shipsdist.csv | sort | head -n 1
The code and error messages presented suggest that the second script is calling the first with an empty command-line argument. That would certainly happen if input file shipsNAME-IMO.txt contained any empty lines or otherwise any lines with an empty first field. An empty line at the beginning or end would do it.
I suggest
using the read command to read the data, and manipulating IFS to parse out comma-delimited fields
validating your inputs and other data early and often
making your scripts behave more pleasantly in the event of predictable failures
More generally, using internal Bash features instead of external programs where the former are reasonably natural.
For example:
#!/bin/bash
# Validate one command-line argument
[[ -n "$1" ]] || { echo empty ship name 1>&2; exit 1; }
# Read and validate an IMO corresponding to the argument
IFS=, read -r dummy imo tail < <(grep -F -- "$1" shipsNAME-IMO.txt)
[[ -f PerShip/"${imo}.csv" ]] || { echo no data for "'$imo'" 1>&2; exit 1; }
# Perform the distance calculation and output the result
cp PerShip/"${imo}.csv" tmpship.csv
dist=$(octave -q ShipDistance.m 2>/dev/null) ||
{ echo "failed to compute ship distance for '${imo}'" 2>&1; exit 1; }
echo "${imo:3:7},${dist}"
and
#!/bin/bash
# Note: the original shipsdist.csv will be clobbered
while IFS=, read -r ship tail; do
# Ignore any empty ship name, however it might arise
[[ -n "$ship" ]] && ./FindShipDistance "$ship"
done < shipsNAME-IMO.txt |
tee shipsdist.csv |
sort |
head -n 1
Note that making the while loop in the second script part of a pipeline will cause it to run in a subshell. That is sometimes a gotcha, but it won't cause any problem in this case.
I am i need to find & remove empty files. The definition of empty files in my use case is a file which has zero lines.
I did try testing the file to see if it's empty However, this behaves strangely as in even though the file is empty it doesn't detect it so.
Hence, the best thing I could write up is the below script which i way too slow given it has to test several hundred thousand files
#!/bin/bash
LOOKUP_DIR="/path/to/source/directory"
cd ${LOOKUP_DIR} || { echo "cd failed"; exit 0; }
for fname in $(realpath */*)
do
if [[ $(wc -l "${fname}" | awk '{print $1}') -eq 0 ]]
then
echo "${fname}" is empty
rm -f "${fname}"
fi
done
Is there a better way to do what I'm after or alternatively, can the above logic be re-written in a way that brings better performance please?
Your script is slow beacuse wc reads every file to the end, which is not needed for your purpose. This might be what you're looking for:
#!/bin/bash
lookup_dir='/path/to/source/directory'
cd "$lookup_dir" || exit
for file in *; do
if [[ -f "$file" && -r "$file" && ! -L "$file" ]]; then
read < "$file" || echo rm -f -- "$file"
fi
done
Drop the echo after making sure it works as intended.
Another version, calling the rm only once, could be:
#!/bin/bash
lookup_dir='/path/to/source/directory'
cd "$lookup_dir" || exit
for file in *; do
if [[ -f "$file" && -r "$file" && ! -L "$file" ]]; then
read < "$file" || files_to_be_deleted+=("$file")
fi
done
rm -f -- "${files_to_be_deleted[#]}"
Explanation:
The core logic is in the line
read < "$file" || rm -f -- "$file"
The read < "$file" command attempts to read a line from the $file. If it succeeds, that is, a line is read, then the rm command on the right-hand side of the || won't be executed (that's how the || works). If it fails then the rm command will be executed. In any case, at most one line will be read. This has great advantage over the wc command because wc would read the whole file.
if ! read < "$file"; then rm -f -- "$file"; fi
could be used instead. The two lines are equivalent.
To check a "$fname" is a file and is empty or not, use [ -s "$fname" ]:
#!/usr/bin/env sh
LOOKUP_DIR="/path/to/source/directory"
for fname in "$LOOKUP_DIR"*/*; do
if ! [ -s "$fname" ]; then
echo "${fname}" is empty
# remove echo when output is what you want
echo rm -f "${fname}"
fi
done
See: help test:
File operators:
...
-s FILE True if file exists and is not empty.
Yet another method
wc -l ~/tmp/* 2>/dev/null | awk '$1 == 0 {print $2}' | xargs echo rm
This will break if any of your files have whitespace in the name.
To work around that, with awk still
wc -l ~/tmp/* 2>/dev/null \
| awk 'sub(/^[[:blank:]]+0[[:blank:]]+/, "")' \
| xargs echo rm
This works because the sub function returns the number of substitutions made, which can be treated as a boolean zero/not-zero condition.
Remove the echo to actually delete the files.
I'm still new to the shell and need some help.
I have a file stapel_old.
Also I have in the same directory files like english_old_sync, math_old_sync and vocabulary_old_sync.
The content of stapel_old is:
english
math
vocabulary
The content of e.g. english is:
basic_grammar.md
spelling.md
orthography.md
I want to manipulate all files which are given in stapel_old like in this example:
take the first line of stapel_old 'english', (after that math, and so on)
convert in this case english to english_old_sync, (or after that what is given in second line, e.g. math to math_old_sync)
search in english_old_sync line by line for the pattern '.md'
And append to each line after .md :::#a1
The result should be e.g. of english_old_sync:
basic_grammar.md:::#a1
spelling.md:::#a1
orthography.md:::#a1
of math_old_sync:
geometry.md:::#a1
fractions.md:::#a1
and so on. stapel_old should stay unchanged.
How can I realize that?
I tried with sed -n, while loop (while read -r line), and I'm feeling it's somehow the right way - but I still get errors and not the expected result after 4 hours inspecting and reading.
Thank you!
EDIT
Here is the working code (The files are stored in folder 'olddata'):
clear
echo -e "$(tput setaf 1)$(tput setab 7)Learning directories:$(tput sgr 0)\n"
# put here directories which should not become flashcards, command: | grep -v 'name_of_directory_which_not_to_learn1' | grep -v 'directory2'
ls ../ | grep -v 00_gliederungsverweise | grep -v 0_weiter | grep -v bibliothek | grep -v notizen | grep -v Obsidian | grep -v z_nicht_uni | tee olddata/stapel_old
# count folders
echo -ne "\nHow much different folders: " && wc -l olddata/stapel_old | cut -d' ' -f1 | tee -a olddata/stapel_old
echo -e "Are this learning directories correct? [j ODER y]--> yes; [Other]-->no\n"
read lernvz_korrekt
if [ "$lernvz_korrekt" = j ] || [ "$lernvz_korrekt" = y ];
then
read -n 1 -s -r -p "Learning directories correct. Press any key to continue..."
else
read -n 1 -s -r -p "Learning directories not correct, please change in line 4. Press any key to continue..."
exit
fi
echo -e "\n_____________________________\n$(tput setaf 6)$(tput setab 5)Found cards:$(tput sgr 0)$(tput setaf 6)\n"
#GET && WRITE FOLDER NAMES into olddata/stapel_old
anzahl_zeilen=$(cat olddata/stapel_old |& tail -1)
#GET NAMES of .md files of every stapel and write All to 'stapelname'_old_sync
i=0
name="var_$i"
for (( num=1; num <= $anzahl_zeilen; num++ ))
do
i="$((i + 1))"
name="var_$i"
name=$(cat olddata/stapel_old | sed -n "$num"p)
find ../$name/ -name '*.md' | grep -v trash | grep -v Obsidian | rev | cut -d'/' -f1 | rev | tee olddata/$name"_old_sync"
done
(tput sgr 0)
I tried to add:
input="olddata/stapel_old"
while IFS= read -r line
do
sed -n "$line"p olddata/stapel_old
done < "$input"
The code to change only the english_old_sync is:
lines=$(wc -l olddata/english_old_sync | cut -d' ' -f1)
for ((num=1; num <= $lines; num++))
do
content=$(sed -n "$num"p olddata/english_old_sync)
sed -i "s/"$content"/""$content":::#a1/g"" olddata/english_old_sync
done
So now, this need to be a inner for-loop, of a outer for-loop which holds the variable for english, right?
stapel_old should stay unchanged.
You could try a while + read loop and embed sed inside the loop.
#!/usr/bin/env bash
while IFS= read -r files; do
echo cp -v "$files" "${files}_old_sync" &&
echo sed '/^.*\.md$/s/$/:::#a1/' "${files}_old_sync"
done < olddata/staple_old
convert in this case english to english_old_sync, (or after that what is given in second line, e.g. math to math_old_sync)
cp copies the file with a new name, if the goal is renaming the original file name from the content of the file staple_old then change cp to mv
The -n and -i flag from sed was ommited , include it, if needed.
The script also assumes that there are no empty/blank lines in the content of staple_old file. If in case there are/is add an addition test after the line where the do is.
[[ -n $files ]] || continue
It also assumes that the content of staple_old are existing files. Just in case add an additional test.
[[ -e $files ]] || { printf >&2 '%s no such file or directory.\n' "$files"; continue; }
Or an if statement.
if [[ ! -e $files ]]; then
printf >&2 '%s no such file or directory\n' "$files"
continue
fi
See also help test
See also help continue
Combining them all together should be something like:
#!/usr/bin/env bash
while IFS= read -r files; do
[[ -n $files ]] || continue
[[ -e $files ]] || {
printf >&2 '%s no such file or directory.\n' "$files"
continue
}
echo cp -v "$files" "${files}_old_sync" &&
echo sed '/^.*\.md$/s/$/:::#a1/' "${files}_old_sync"
done < olddata/staple_old
Remove the echo's If you're satisfied with the output so the script could copy/rename and edit the files.
I would like to get file (zip file) from path with this part of code file=$(cd '/path_to_zip_file' | ls -t | head -1). Instead that I got my .sh file in directory where I am running this file.
Why I can't file from /path_to_zip_file
Below is my code in .sh file
file=$(cd '/path_to_zip_file' | ls -t | head -1)
last_modified=`stat -c "%Y" $file`;
current=`date +%s`
echo $file
if [ $(($current-$last_modified)) -gt 86400 ]; then
echo 'Mail'
else
echo 'No Mail'
fi;
If you were going to use ls -t | head -1 (which you shouldn't), the cd would need to be corrected as a prior command (happening before ls takes place), not a pipeline component (running parallel with ls, with its stdout connected to ls's stdin):
set -o pipefail # otherwise, a failure of ls is ignored so long as head succeeds
file=$(cd '/path_to_zip_file' && ls -t | head -1)
A better-practice approach might look like:
newest_file() {
local result=$1; shift # first, treat our first arg as latest
while (( $# )); do # as long as we have more args...
[[ $1 -nt $result ]] && result=$1 # replace "result" if they're newer
shift # then take them off the argument list
done
[[ -e $result || -L $result ]] || return 1 # fail if no file found
printf '%s\n' "$result" # more reliable than echo
}
newest=$(newest_file /path/to/zip/file/*)
newest=${newest##*/} ## trim the path to get only the filename
printf 'Newest file is: %s\n' "$newest"
To understand the ${newest##*/} syntax, see the bash-hackers' wiki on parameter expansion.
For more on why using ls in scripts (except for output displayed to humans) is dangerous, see ParsingLs.
Bot BashFAQ #99, How do I get the latest (or oldest) file from a directory? -- and BashFAQ #3 (How can I sort or compare files based on some metadata attribute (newest / oldest modification time, size, etc)?) have useful discussion on the larger context in which this question was asked.
I have the following script which does a "which -a" on a command then a "ls -l" to let me know if it's a link or not .. ie "grep" since I have gnu commands installed (Mac with iTerm).
#!/usr/bin/env bash
which -a $1 | xargs -I{} ls -l "{}" \
| awk '{for (i = 1; i < 9; i++) $i = ""; sub(/^ */, ""); print}'
When I run this from the script "test grep" I receive no output, but when I run it via "bash -x test grep" I receive the following:
bash -x test grep
+ which -a grep
+ xargs '-I{}' ls -l '{}'
+ awk '{for (i = 1; i < 9; i++) $i = ""; sub(/^ */, ""); print}'
/usr/local/bin/grep -> ../Cellar/grep/3.1/bin/grep
/usr/bin/grep
The last 2 lines is what I'm looking to display. Thought this would be easier to do ;-) .. I also tried appending the pipe thinking printf would fix the issue:
| while read path
do
printf "%s\n" "$path"
done
Thanks and .. Is there a better way to get what I need?
The problem is that you named your script test.
If you want to run a command that's not in your PATH, you need to specify the directory it's in, e.g. ./test.
You're not getting an error for trying to run test because there is a built-in bash command called test that is used instead. For extra confusion, the standard test produces no output.
In conclusion:
Use ./ to run scripts in the current directory.
Never call your test programs test.
Thanks for the never naming a script "test" .. old habits are hard to break (I came from a non-unix background.
I ended with the following
for i in $(which -a $1)
do
stat $i | awk NR==1{'$1 = ""; sub(/^ */, ""); print}'
done
or simpler
for i in $(which -a $1)
do
stat -c %N "$i"
done
Consider the following shell function:
cmdsrc() {
local cmd_file cmd_file_realpath
case $(type -t -- "$1") in
file) cmd_file=$(type -P -- "$1")
if [[ -L "$cmd_file" ]]; then
echo "$cmd_file is a symlink" >&2
elif [[ -f "$cmd_file" ]]; then
echo "$cmd_file is a regular file" >&2
else
echo "$cmd_file is not a symlink or a regular file" >&2
fi
cmd_file_realpath=$(readlink -- "$cmd_file") || return
if [[ $cmd_file_realpath != "$cmd_file" ]]; then
echo "...the real location of the executable is $cmd_file_realpath" >&2
fi
;;
*) echo "$1 is not a file at all: $(type -- "$1")" >&2 ;;
esac
}
...used as such:
$ cmdsrc apt
/usr/bin/apt is a symlink
...the real location of the executable is /System/Library/Frameworks/JavaVM.framework/Versions/A/Commands/apt
$ cmdsrc ls
/bin/ls is a regular file
$ cmdsrc alias
alias is not a file at all: alias is a shell builtin
Took some suggestions and came up with the following:
The prt-underline is just a fancy printf function. I decided not to go with readline since the ultimate command resolution may be unfamiliar to me and I only deal with regular files .. so does't handle every situation but in the end gives me the output I was looking for. Thanks for all the help.
llt ()
{
case $(type -t -- "$1") in
function)
prt-underline "Function";
declare -f "$1"
;;
alias)
prt-underline "Alias";
alias "$1" | awk '{sub(/^alias /, ""); print}'
;;
keyword)
prt-underline "Reserved Keyword"
;;
builtin)
prt-underline "Builtin Command"
;;
*)
;;
esac;
which "$1" &> /dev/null;
if [[ $? = 0 ]]; then
prt-underline "File";
for i in $(which -a $1);
do
stat "$i" | awk 'NR==1{sub(/^ File: /, ""); print}';
done;
fi
}