How would one go about creating a script for creating 25 empty files in succession? (I.e 1-25, 26-51, 52-77)
I can create files 1-25 but I’m having trouble figuring out how to create a script that continues that process from where it left off, every time I run the script.
#!/bin/bash
higher=$( find files -type f -exec basename {} \; | sort -n | tail -1 )
if [[ "$higher" == "" ]]
then
start=1
end=25
else
(( start = higher + 1 ))
(( end = start + 25 ))
fi
echo "$start --> $end"
for i in $(seq $start 1 $end)
do
touch files/"$i"
done
I put my files in a directory called "files".
hence the find on directory "files".
for each file found, I run a basename on it. That will return only integer values, since the files all have a number filename.
sort -n puts them in order.
tail -1 extracts the highest number.
if there are no files, higher will be empty, so the indexes will be 1 and 25.
otherwise, they will be higher + 1, and higher + 26.
I used seq for the for loop to avoid problems with variables inside a range definition (you did {1..25})
#! /usr/bin/env bash
declare -r base="${1:-base-%d.txt}"
declare -r lot="${2:-25}"
declare -i idx=1
declare -i n=0
printf -v filename "${base}" ${idx}
while [[ -e "${filename}" ]]; do
idx+=1
printf -v filename "${base}" "${idx}"
done
while [[ $n -lt $lot ]]; do
printf -v filename "${base}" ${idx}
if [[ ! -e "${filename}" ]]; then
> "$filename"
n+=1
fi
idx+=1
done
This script accepts two optional parameters.
The first is the basename of your future files with a %d token automatically replaced by the file number. Default value is base-%d.txt;
The number of file to create. Default value is 25.
How script works:
Variable declarations
base: file basename (constant)
lot: number of file to create (constant)
idx: search index
n: counter for new files
Search files already created from 1
The loop stop at first hole in the numbering
Loop to create empty files
The condition in the loop allows to fill in the numbering holes
> filename create an empty file
I need split big *.csv file for several smaller. Currently there is 661497 rows, I need each file with max. 40000. I've tried solution that I found on Github but with no success:
FILENAME=/home/cnf/domains/cnf.com.pl/public_html/sklep/dropshipping-pliki/products-files/my_file.csv
HDR=$(head -1 ${FILENAME})
split -l 40000 ${FILENAME} xyz
n=1
for f in xyz*
do
if [[ ${n} -ne 1 ]]; then
echo ${HDR} > part-${n}-${FILENAME}.csv
fi
cat ${f} >> part-${n}-${FILENAME}.csv
rm ${f}
((n++))
done
The error I get:
/home/cnf/domains/cnf.com.pl/public_html/sklep/dropshipping-pliki/download.sh: line 23: part-1-/home/cnf/domains/cnf.com.pl/public_html/sklep/dropshipping-pliki/products-files/my_file.csv.csv: No such file or directory
thanks for help!
Keep in mind FILENAME contains both a directory and a file so later in the script when you build the new filename you get something like:
part-1-/home/cnf/domains/cnf.com.pl/public_html/sklep/dropshipping-pliki/products-files/tyre_8.csv.csv
One quick-n-easy fix would be split the directory and filename into 2 separate variables, eg:
srcdir='/home/cnf/domains/cnf.com.pl/public_html/sklep/dropshipping-pliki/products-files'
filename='tyre_8.csv'
hdr=$(head -1 ${srcdir}/${filename})
split -l 40000 "${srcdir}/${filename}" xyz
n=1
for f in xyz*
do
if [[ ${n} -ne 1 ]]; then
echo ${hdr} > "${srcdir}/part-${n}-${filename}"
fi
cat "${f}" >> "${srcdir}/part-${n}-${filename}"
rm "${f}"
((n++))
done
NOTES:
consider using lowercase variables (using uppercase variables raises the possibility of problems if there's an OS variable of the same name)
wrap variable references in double quotes in case string contains spaces
don't need to add a .csv extension on the new filename since it's already part of $filename
I have a file which contains varoius data (date,time,speed, distance from the front, distance from the back), the file looks like this, just with more rows:
2003.09.23.,05:05:21:64,134,177,101
2009.03.10.,17:46:17:81,57,102,57
2018.01.05.,00:30:37:04,354,145,156
2011.07.11.,23:21:53:43,310,125,47
2011.06.26.,07:42:10:30,383,180,171
I'm trying to write a simple Bash program, which tells the dates and times when the 'distance from the front' is less than the provided parameter ($1)
So far I wrote:
#!/bin/bash
if [ $# -eq 0 -o $# -gt 1 ]
then
echo "wrong number of parameters"
fi
i=0
fdistance=()
input='auto.txt'
while IFS= read -r line
do
year=${line::4}
month=${line:5:2}
day=${line:8:2}
hour=${line:12:2}
min=${line:15:2}
sec=${line:18:2}
hthsec=${line:21:2}
fdistance=$(cut -d, -f 4)
if [ "$fdistance[$i]" -lt "$1" ]
then
echo "$year[$i]:$month[$i]:$day[$i],$hour[$i]:$min[$i]:$sec[$i]:$hthsec[$i]"
fi
i=`expr $i + 1`
done < "$input"
but this gives the error "whole expression required" and doesn't work at all.
If you have the option of using awk, the entire process can be reduced to:
awk -F, -v dist=150 '$4<dist {split($1,d,"."); print d[1]":"d[2]":"d[3]","$2}' file
Where in the example above, any record with distance (field 4, $4) less than the dist variable value takes the date field (field 1, $1) and splits() the field into the array d on "." where the first 3 elements will be year, mo, day and then simply prints the output of those three elements separated by ":" (which eliminates the stray "." at the end of the field). The time (field 2, $2) is output unchanged.
Example Use/Output
With your sample data in file, you can do:
$ awk -F, -v dist=150 '$4<dist {split($1,d,"."); print d[1]":"d[2]":"d[3]","$2}' file
2009:03:10,17:46:17:81
2018:01:05,00:30:37:04
2011:07:11,23:21:53:43
Which provides the records in the requested format where the distance is less than 150. If you call awk from within your script you can pass the 150 in from the 1st argument to your script.
You can also accomplish this task by substituting a ':' for each '.' in the first field with gsub() and outputting a substring of the first field with substr() that drops the last character, e.g.
awk -F, -v dist=150 '$4<dist {gsub(/[.]/,":",$1); print substr($1,0,length($1)-1),$2}' file
(same output)
While parsing the data is a great exercise for leaning string handling in shell or bash, in practice awk will be Orders of Magnitude faster than a shell script. Processing a million line file -- the difference in runtime can be seconds with awk compared to minutes (or hours) with a shell script.
If this is an exercise to learn string handling in your shell, just put this in your hip pocket for later understanding that awk is the real Swiss Army-Knife for text processing. (well worth the effort to learn)
Would you try the following:
#/bin/bash
if (( $# != 1 )); then
echo "usage: $0 max_distance_from_the_front" >& 2 # output error message to the stderr
exit 1
fi
input="auto.txt"
while IFS=, read -r mydate mytime speed fdist bdist; do # split csv and assign variables
mydate=${mydate%.}; mydate=${mydate//./:} # reformat the date string
if (( fdist < $1 )); then # if the front disatce is less than $1
echo "$mydate,$mytime" # then print the date and time
fi
done < "$input"
Sample output with the same parameter as Keldorn:
$ ./test.sh 130
2009:03:10,17:46:17:81
2011:07:11,23:21:53:43
There are a few odd things in your script:
Why is fdistance an array. It is not necessary (and here done wrong) since the file is read line by line.
What is the cut of the line fdistance=$(cut -d, -f 4) supposed to cut, what's the input?
(Note: When invalid parameters, better end the script right away. Added in the example below.)
Here is a working version (apart from the parsing of the date, but that is not what your question was about so I skipped it):
#!/usr/bin/env bash
if [ $# -eq 0 -o $# -gt 1 ]
then
echo "wrong number of parameters"
exit 1
fi
input='auto.txt'
while IFS= read -r line
do
fdistance=$(echo "$line" | awk '{split($0,a,","); print a[4]}')
if [ "$fdistance" -lt "$1" ]
then
echo $line
fi
done < "$input"
Sample output:
$ ./test.sh 130
2009.03.10.,17:46:17:81,57,102,57
2011.07.11.,23:21:53:43,310,125,47
$
I have a script to do research in my photo archive.
Beacause I've some files named with a merceological name before.
Since I want to avoid the problem that misspellings cause results to be lost. so I thought about this system.
Eg names files:
(Travel and Tourism) – titlea.jpg
(Travel and Tourism) – titleb.jpg
(Drinks) – titlea.jpg
I did this idea,
two researches, one precise with the exact name and the other extremely inaccurate with a few letters (consonants) that it is not possible for them to be missing (unless one does not type drunk). And that obviously are not all present together in other names.
If the subtraction is zero it means that the two results are the same and therefore without errors:
myfindTravelandTourism=$(find /PATH/ -type f -iname '*(Travel and Tourism)*' | wc -l)
StringVerifyTravelandTourism=$(find /PATH/ | cut -f1 -d '–' | grep
-i -F 'T' | grep -i -F 'R' | grep -i -F 'V' | wc -l)
if [ "${myfindTravelandTourism-StringVerifyTravelandTourism}" ≠ "0" ] ; then
echo "WARNING THE SCRIPT HAS DETECTED AN ORTOGRAPHICAL ERROR IN THE VIAGGI E TURISMO SECTOR!"
if [ {StringVerifyTravelandTourism} -ne ≠ "$0" ]; then
exit
fi
fi
But the error code bash:
[: ≠: integer expression expected.
I'm a beginner. This is my first conditional statement in bash
This ≠ is not an operator as a result it throws the error.
Also, to check for strings, you should use != instead of -ne. For example,
n1 -ne n2 To check whether numbers are algebraically equal or not.
s1 != s2 To check whether strings are not equal.
You can try the following code snippet:
if [[ $(( myfindTravelandTourism - StringVerifyTravelandTourism)) != 0 ]]; then
echo "not equal 0"
fi
Part 1) My script allows the command line options -r, -c, and -t for rectangle, circle and trapezoid respectively. The rectangle option requires two arguments, the length and the width. The circle option requires one argument, the radius. The trapezoid option requires three arguments, the height and each of the parallel bases.
Part 2) The script allows the option, -f. When this option is used in conjunction with -r, -c, or –t, each option takes one argument, the name of the input file. The input values would be read from the input file one line at a time and the areas printed to stdout. The script should terminate when a value of –1 is read.
I have part 1 and part 2 both done except the script terminating when -1 is read. What I want to do on the second part is for the rectangle when i give it an input file is to check whether that file has 2 values in each line or not. If it does not have 2 values then echo "file has invalid set(s) of data". Same with circle, it should not accept anything that has more or less value than 1 in each line of th input file and for trapezoid 3.
filename=$OPTARG;;
while read -a line
do
if (( $choice == 1 ))
then
Area=`echo "scale=3; ${line[0]}*${line[1]}" | bc`;
echo "Area of the rectangle is $Area";
fi
if (( $choice == 2 ))
then
Pi=3.1416
Area=`echo "scale=3; ${line[0]*$line[0]}*$Pi" | bc`;
echo "Area of the circle is $Area";
fi
if (( $choice == 3 ))
then
Area=`echo "scale=3; (${line[0]}+${line[1]})/2*${line[2]}" | bc`;
echo "Area of the trapezoid is $Area";
fi
done < "$filename"
just grep for -1 in the file and echo $?. This will be zero if -1 is present and 1 if -1 is not present.
To check the no. of values passed do the following
SEPERATOR=" ";
#Assumed that seperator is space in your case
#remove all ending spaces
sed -i 's/ *$//g' $filename
#take the count of columns seperated by spaces
cat $filename | grep -o " " | wc -l
#this value +1 gives you the total values in the file
Hope this hellps