How would one go about creating a script for creating 25 empty files in succession? (I.e 1-25, 26-51, 52-77)
I can create files 1-25 but I’m having trouble figuring out how to create a script that continues that process from where it left off, every time I run the script.
#!/bin/bash
higher=$( find files -type f -exec basename {} \; | sort -n | tail -1 )
if [[ "$higher" == "" ]]
then
start=1
end=25
else
(( start = higher + 1 ))
(( end = start + 25 ))
fi
echo "$start --> $end"
for i in $(seq $start 1 $end)
do
touch files/"$i"
done
I put my files in a directory called "files".
hence the find on directory "files".
for each file found, I run a basename on it. That will return only integer values, since the files all have a number filename.
sort -n puts them in order.
tail -1 extracts the highest number.
if there are no files, higher will be empty, so the indexes will be 1 and 25.
otherwise, they will be higher + 1, and higher + 26.
I used seq for the for loop to avoid problems with variables inside a range definition (you did {1..25})
#! /usr/bin/env bash
declare -r base="${1:-base-%d.txt}"
declare -r lot="${2:-25}"
declare -i idx=1
declare -i n=0
printf -v filename "${base}" ${idx}
while [[ -e "${filename}" ]]; do
idx+=1
printf -v filename "${base}" "${idx}"
done
while [[ $n -lt $lot ]]; do
printf -v filename "${base}" ${idx}
if [[ ! -e "${filename}" ]]; then
> "$filename"
n+=1
fi
idx+=1
done
This script accepts two optional parameters.
The first is the basename of your future files with a %d token automatically replaced by the file number. Default value is base-%d.txt;
The number of file to create. Default value is 25.
How script works:
Variable declarations
base: file basename (constant)
lot: number of file to create (constant)
idx: search index
n: counter for new files
Search files already created from 1
The loop stop at first hole in the numbering
Loop to create empty files
The condition in the loop allows to fill in the numbering holes
> filename create an empty file
Related
So I have this code, to puke 5 files per script. But, I need a counter
find cobacoba -type f | xargs -n 5 bash -c 'script.sh $counter ${0} ${1} ${2} ${3} ${4}' bash
Script.sh:
#!/usr/bin/env bash
echo "Group $0: $1 $2 $3 $4 $5"
##This is just a simple example. The actual script will use each variable, including the counter, for further processing.
##So, really need all the variables being passed by "find | xargs"
Hoping Result:
Group 1: cobacoba/1.3 cobacoba/1.6 cobacoba/1.q cobacoba/1.5
Group 2: cobacoba/1.1 cobacoba/1.q2 cobacoba/1.q23 cobacoba/1.4
Group 3: cobacoba/1.2
What strategy can I use to create $counter ?
There's no need to use find or xargs if all you want to do is recursively walk a directory:
i=0
shopt -s globstar
for f in cobacoba/**; do
[[ -f $f ]] || continue
(( i > 5 )) && wait
script.sh "$i" "${0}" "${1}" "${2}" "${3}" "${4}" "$f" &
(( i++ ))
done
Since the right side of a | pipe is a sub-shell, it can not persist or update a variable. It is not possible to use a counter in a variable like you do.
Fortunately Bash allow you to reverse the operation and feed the main shell (here a while loop) with the output of a command running in a sub-shell.
Since we update our variables within the main shell, it works like that:
#!/usr/bin/env bash
group=1 # Group counter
files_per_group=4 # How many files per group
filescount=0 # Files counter
newline='' # Newline code used between groups
# loop reading all null delimited files names returned by find -print0
while read -d '' -r file; do
# If count of file is a multiple of files per group, then start a new group
if [ $((filescount % files_per_group)) -eq 0 ]; then
printf '%sGroup %d:' "$newline" "$group"
# Increment group counter for upcoming group
group=$((group + 1))
newline=$'\n' # To separate next group in another line
fi
# Print space delimited file-name
printf ' %s' "$file"
# Increment the files counter
filescount=$((filescount + 1))
done < <(
# feed the whole while loop with the output of find
find cobacoba -type f -print0
)
echo
Solution with building parameters to call a bash script:
#!/usr/bin/env bash
# Dummy bashscript as a function to test call with parameters
bashscript() {
printf 'Called bashscript.sh\n'
printf 'group %s\n' "$1"
shift
printf '%d files:\n' "$#"
printf ' %s' "$#"
printf $'\n\n'
}
group=1 # Group counter
files_per_group=5 # How many files per group
filescount=0 # Files counter
bashscriptparams=() # Arguments for the bash scripts
# loop reading all null delimited files names returned by find -print0
while read -d '' -r file; do
# If count of file is a multiple of files per group, then start a new group
if [ $((filescount % files_per_group)) -eq 0 ]; then
#printf '%sGroup %d:' "$newline" "$group"
# Set the group number as first param of the bash script
bashscriptparams=("$group")
# Increment group counter for upcoming group
((group++))
fi
# Add the file as next parameter of the bash script
bashscriptparams+=("$file")
# Print space delimited file-name
#printf ' %s' "$file"
# If last file of group, then group is complete
if [ $((filescount % files_per_group)) -eq $((files_per_group - 1)) ]; then
# Launch the bash script with its arguments (group file_1 .. file_n)
bashscript "${bashscriptparams[#]}"
fi
# Increment the files counter
((filescount++))
done < <(
# feed the whole while loop with the output of find
find cobacoba -type f -print0
)
# If we reach here with an incomplete files group
if [ $((filescount % files_per_group)) -le $((files_per_group - 1)) ]; then
# Launch the bash script with its incomplete files arguments (group file_1 .. file_n)
bashscript "${bashscriptparams[#]}"
fi
Output:
Called bashscript.sh
group 1
5 files:
cobacoba/1.q2 cobacoba/1.3 cobacoba/1.1 cobacoba/1.5 cobacoba/1.6
Called bashscript.sh
group 2
4 files:
cobacoba/1.2 cobacoba/1.4 cobacoba/1.q cobacoba/1.q23
I have a list of files:
file_name_FOO31101.txt
file_name_FOO31102.txt
file_name_FOO31103.txt
file_name_FOO31104.txt
And I want to use pairs of files for input into a downstream program such as:
program_call file_name_01.txt file_name_02.txt
program_call file_name_03.txt file_name_04.txt
...
I do not want:
program_call file_name_02.txt file_name_03.txt
I need to do this in a loop as follows:
#!/bin/bash
FILES=path/to/files
for file in $FILES/*.txt;
do
stem=$( basename "${file}" ) # stem : file_name_FOO31104_info.txt
output_base=$( echo $stem | cut -d'_' -f 1,2,3 ) # output_base : FOO31104_info.txt
id=$( echo $stem | cut -d'_' -f 3 ) # get the first field : FOO31104
number=$( echo -n $id | tail -c 2 ) # get the last two digits : 04
echo $id $((id+1))
done
But this does not produce what I want.
In each loop I want to call a program once, with two files as input (last 2 digits of first file always odd 01, last 2 digits of second file always even 02)
I actually wouldn't use a for loop at all. A while loop that shifts files off is a perfectly reasonable way to do this.
# here, we're overriding the argument list with the list of files
# ...you can do this in a function if you want to keep the global argument list intact
set -- "$FILES"/*.txt ## without these quotes paths with spaces break
# handle the case where no files were found matching our glob
[[ -e $1 || -L $1 ]] || { echo "No .txt found in $FILES" >&2; exit 1; }
# here, we're doing our own loop over those arguments
while (( "$#" > 1 )); do ## continue in the loop only w/ 2-or-more remaining
echo "Processing files $1 and $2" ## ...substitute your own logic here...
shift 2 || break ## break even if test doesn't handle this case
done
# ...and add your own handling for the case where there's an odd number of files.
(( "$#" )) && echo "Left over file $1 still exists"
Note that the $#s are quoted inside (( )) here for StackOverflow's syntax highlighting, not because they otherwise need to be. :)
By the way -- consider using bash's native string manipulation.
stem=${file##*/}
IFS=_ read -r p1 p2 id p_rest <<<"$stem"
number=${id:$(( ${#id} - 2 ))}
output_base="${p1}${p2}${id}"
echo "$id $((10#number + 1))" # 10# ensures interpretation as decimal, not octal
I am looping through a folder and depending on the length of files do certain condition. I seem not to come right with that. I evaluate and output the length of a string in the terminal.
echo $file|wc -c gives me the answer of all files in the terminal.
But incorporating this into a loop is impossible
for file in `*.zip`; do
if [[ echo $file|wc -c ==9]]; then
some commands
where I want to operate on files that have a length of nine characters
Try this one:
for file in *.zip ; do
wcout=$(wc -c "$file")
if [[ ${wcout%% *} -eq 9 ]] ; then
# some commands
fi
done
The %% operator in variable expansion deletes everything that match the pattern after it. This is glob pattern, not regular expression.
Opposite to natural good sense of typical programmers the == operator in BASH compares strings, not numbers.
Alternatively (following the comment) you can:
for file in *.zip ; do
wcout=$(wc -c < "$file")
if [[ ${wcout} -eq 9 ]] ; then
# some commands
fi
done
Additional observation is that if BASH cannot expand *.zip as there is no ZIP files in the current directory it will pass "*.zip" into $file and let single iteration of the loop. That leads to the error reported by wc command. So it would be recommended to add:
if [[ -e ${file} ]] ; then ...
as a prevention mechanism.
Comments leads to another form of this solution (plus I added my safety mechanism):
for file in *.zip ; do
if [[ -e "$file" && (( $(wc -c < "$file") == 9 )) ]] ; then
# some commands
fi
done
using filter outside the loop
ls -1 *.zip \
| grep -E '^.{9}$' \
| while read FileName
do
# Your action
done
using filter inside loop
ls -1 *.zip \
| while read FileName
do
if [ ${#FileName} -eq 9 ]
then
# Your action
fi
done
alternative to ls -1 that is always a bit dangereous, find . -name '*.zip' -print [ but you neet to add 2 char length or filter the name form headin ./ and maybe limit to current folder depth ]
I am trying to match first few letters of a file.
for entry in `ls`; do
echo $entry
done
With the above code I get the name of all the files.
I have a few files with similar name at the start:
Beaglebone-v1
Beaglebone-v3
Beaglebone-v2
How can I compare $entry with Beaglebone* and then extract the latest version file name?
If you want to loop over all Beaglebone-* files:
for entry in Beaglebone-* ; do
echo $entry
done
if you just need the file with the latest version, you can depend on the fact that ls sorts your names alphabetically, so you could just do:
LATEST_FILE_NAME=$(ls Beaglebone-* | tail -n 1)
which will just take the last one alphabetically.
To deal with larger numbers, you could use numeric comparison like this:
stem="Beaglebone-v"
for file in $stem*; do
ver=${file#"$stem"} # cut away stem to get version number
(( ver > max )) && max=$ver # conditionally assign `ver` to `max`
done
echo "$stem$max"
Testing it out:
bash-4.3$ ls Beaglebone-v*
Beaglebone-v1 Beaglebone-v10 Beaglebone-v2 Beaglebone-v3
bash-4.3$ stem="Beaglebone-v" &&
for file in $stem*
do
ver=${file#"$stem"}
(( ver > max )) && max=$ver
done; echo "$stem$max"
Beaglebone-v10
You can store the filenames matching the pattern in an array and then pick the last element of the array.
shopt -s nullglob
arr=( Beaglebone-* )
if (( ${#arr[#]} > 0 ))
then
latest="${arr[ (( ${#arr[#]} - 1 )) ]}"
echo "$latest"
fi
You need to enable nullglob so that if there are no files matching the pattern, you will get an empty array rather than the pattern itself.
If version numbers can go beyond single digits,
function version_numbers {
typeset w; for w in $1-v*; do echo ${w#$1-v}; done
}
version_numbers "Beaglebone" | sort -n | tail -1
Or, adding function max:
# read a stream of numbers, from stdin (one per line)
# and return the largest value
function max
{
typeset _max n
read _max || return
while read n; do
((_max < n)) && _max=$n
done
echo $_max
}
We can now do the whole thing without external commands:
version_numbers Beaglebone | max
Note that max will fail horribly if any one line fails the numerical comparison.
Example
for FILE in $DIR/*
do
if(<is last File>)
doSomethingSpecial($FILE)
else
doSomethingRegular($FILE)
fi
done
What to call for <is last file> to check if the current file is the last one in the array ?
Is there an easy built-in check without checking the array's length by myself ?
What to call for to check if the current file is the last one in the array ?
For a start, you are not using an array. If you were then it would be easy:
declare -a files
files=($DIR/*)
pos=$(( ${#files[*]} - 1 ))
last=${files[$pos]}
for FILE in "${files[#]}"
do
if [[ $FILE == $last ]]
then
echo "$FILE is the last"
break
else
echo "$FILE"
fi
done
I know of no way to tell that you are processing the last element of a list in a for loop. However you could use an array, iterate over all but the last element, and then process the last element outside the loop:
files=($DIR/*)
for file in "${files[#]::${#files[#]}-1}" ; do
doSomethingRegular "$file"
done
doSomethingSpecial "${files[#]: -1:1}"
The expansion ${files[#]:offset:length} evaluates to all the elements starting at offset (or the beginning if empty) for length elements. ${#files[#]}-1 is the number of elements in the array minus 1.
${files[#]: -1:1} evaluates to the last element - -1 from the end, length 1. The space is necessary as :- is treated differently to : -.
Try this
LAST_FILE=""
for f in *
do
if [ ! -z $LAST_FILE ]
then
echo "Process file normally $LAST_FILE"
fi
LAST_FILE=$f
done
if [ ! -z $LAST_FILE ]
then
echo "Process file as last file $LAST_FILE"
fi
Produces
bash[1051]: ls
1 2 3 4
bash[1052]: sh ../last_file.sh
Process file normally 1
Process file normally 2
Process file normally 3
Process file as last file 4
You can use find to find the total number of files.
Then when you are in the loop count to the total number and carry out your task when the total equals the count i.e, the last file.
f=0
tot_files=`find . -iname '*.txt' | wc -l`
for FILE in $DIR/*
do
f=($f+1)
if [[ $f == $tot_files ]];then
carryout your task
fi
done
Building on the current highest-voted answer from #cdarke (https://stackoverflow.com/a/12298757/415523), if looking at a general array of values (rather than specifically files on disk), the loop code would be as follows:
declare -a array
declare -i length current
array=( a b c d e c )
length=${#array[#]}
current=0
for VALUE in "${array[#]}"; do
current=$((current + 1))
if [[ "$current" -eq "$length" ]]; then
echo "$VALUE is the last"
else
echo "$VALUE"
fi
done
This yields the output:
a
b
c
d
e
c is the last
This ensures that only the last item in the array triggers the alternative action and that, if any other item in the array duplicates the last value, the alternative action is not called for the earlier duplicates.
In the case of an array of paths to files in a specific directory, e.g.
array=( $DIR/* )
...it is probably less of a concern, since individual filenames within the same directory are almost-certainly unique (unless you have a really odd filesystem!)
You can abuse the positional parameters, since they act similarly to an array,
but are a little easier to manipulate. You should either save the old positional
parameters, or execute in a subshell.
# Method 1: use a subshell. Slightly cleaner, but you can't always
# do this (for example, you may need to affect variables in the current
# shell
files=( $DIR/* )
(
set -- "${files[#]}"
until (( $# == 1 )); do
doSomethingRegular "$1"
shift
done
doSomethingSpecial "$1"
)
# Method 2: save the positional parameters. A bit uglier, but
# executes everything in the same shell.
files=( $DIR/* )
oldPP=( "$#" )
set -- "${files[#]}"
until (( $# == 1 )); do
doSomethingRegular "$1"
shift
done
doSomethingSpecial "$1"
set -- "${oldPP[#]}"
What makes a file the last one? Is there something special about it? Is it the file with the greatest name when sorted by name?
Maybe you can take the file names backwards. Then, it's the first file you want to treat special and not the last. figuring out the first is a much easier task than doing the last:
for file in $(ls -r1 $dir)
do
if [ ! $processedLast ]
then
doSomethingSpecial($file)
processedLast=1
else
doSomethingRegular($file)
fi
done
No arrays needed. Actually, I like chepner's answer about using positional parameters.
It's old question - but building on answer from #GregReynolds please use this one-liner if commands differ only by parameters on last pass. Ugly, ugly code for one-liner lovers
( ff="" ; for f in * "" ; do [ -n "$ff" ] && echo $(${f:+false} && echo $ff alternate params here || echo normal params $ff ) ; ff=$f ; done )
normal params 1
normal params 2
normal params 3
4 alternate params here