Hi I have a file that sorts some code and reformats it. I have over 200 files to apply this to with incremental names run001, run002 etc. Is there a quick way to write a shell script to execute this file over all the files? The executable creates a new file called run001an etc so just running over all files containing run doesnt work, how do i increment the file number?
Cheers
how about:
for i in ./run*; do
process_the_file $i
done
which is valid Bash/Ksh
To be more specific with run### files you can have
for file in dir/run[0-9][0-9][0-9]; do
do_something "$file"
done
dir could simply be just . or other directories. If they have spaces, quote them around "" but only the directory parts.
In bash, you can make use of extended patterns to generate all number matches not just 3 digits:
shopt -s extglob
for file in dir/run+([0-9]); do
do_something "$file"
done
Related
I have a list of files in a folder and I want to just work on a few of them. The folder contains files with file.qc, file.qc.gz file.qc.stat file.qc.count and so on.
I want to write a loop in bash that will open only the file.qc and file.qc.gz, while ignoring other file extensions (such as qc.stats or qc.count)
Just specify multiple globs in your loop:
#!/bin/bash
# Gracefully cases where there are no matches
shopt -s nullglob
for f in *.qc *.qc.gz
do
echo "Found: $f"
done
You can also write this shorter as *.qc{,.gz}, which expands to the same thing.
EDIT: In the course of working on and reediting this question, I was able to get this to work. However, I'm sure there's a better way to do it, so I'm leaving it up to hear from those more experienced.
Periodically I need to reproduce several dozen copies of a few files. For example, given:
company_a_results_30d.py
company_a_results_90d.py
company_a_results_120d.py
company_a_results_all_time.py
I need to make copies where company_a is replaced with company_b, company_c....etc. (The next step is to find and replace a number of terms within the files, but this I have managed to do with a perl script.)
I'm sure this should be possible with a bash script and mv, but I haven't quite got the hang of it. Something like:
#!/usr/bin/env bash
my_array=(company_b company_c company_d)
for i in "${my_array[#]}"
do
for file in *.py
do
cp "$file" "${file/company_a/$i}"
done
done
I'd prefer a solution compatible with zsh, which is what I use.
bash
Slightly modified from the OP's answer:
#!/usr/bin/env bash
set -x # So you can see what's happening - feel free to omit
company_a_files=(company_a*.py) # <== Save the list of files first
my_array=(company_b company_c company_d)
for i in "${my_array[#]}"
do
for file in "${company_a_files[#]}" # <== Use the saved list
do
cp "$file" "${file/company_a/$i}"
done
done
When the inner loop in the OP's answer runs for file in *.py, the glob will pick up whatever company_b &c. files have already been created. So you wind up with a lot of set -x output like:
+ cp company_b_1.py company_b_1.py
cp: 'company_b_1.py' and 'company_b_1.py' are the same file
Instead, save the glob of company_a files into a shell array first, and then
loop over that array.
perl
As a one-liner for Perl 5.14+:
perl -MFile::Copy=copy -E 'for my $file (#ARGV) { copy $file, $file =~ s/company_a/$_/r foreach qw(company_b company_c company_d) }' company_a*.py
The Perl version switches the loop order compared to the bash version. For each file given on the command line (the for ... #ARGV), it copies from that file to each name-modified file in turn (the foreach).
$file =~ s/company_a/$_/r is a non-destructive (/r) replace in $file (the filename) that changes company_a to $_ (the current value from foreach).
This was the solution I came up with:
#!/usr/bin/env bash
my_array=(company_b company_c company_d)
for i in "${my_array[#]}"
do
for file in *.py
do
cp "$file" "${file/company_a/$i}"
done
done
All,
I am running BASH in Solaris 10
I have the following shell script that loops in a directory depending on the presence of CSV files.
The problem is with this piece of code is that it still does one loop even if there is no CSV files in that directory and then calls SQL loader.
SQLLoader then produces a log file because there is no file to process and this is beginning to mess up my directory filling it with log files.
for file in *.csv ;
do
echo "SQLLoader is reading : " $file
sqlldr <User>/<Password>#<DBURL>:<PORT>/<SID> control=sqlloader.ctl log=$inbox/$file.log data=$inbox/$file
done
How do I stop it going into a loop if there is no CSV files in that directory of $inbox
Say:
shopt -s nullglob
before your for loop.
This is not the default, and saying for file in *.csv when you don't have any matching files expands it to *.csv.
Quoting from the documentation:
nullglob
If set, Bash allows filename patterns which match no files to expand to a null
string, rather than themselves.
Use find to search files
for file in `find -name "*.csv"` ;
First off, using nullglob is the correct answer if it is available. However, a POSIX-compliant option is available.
The pattern will be treated as literal text if there are no matches. You can catch this with a small hack:
for file in *.csv; do
[ -f "$file" ] || break
...
done
When there are no matches, file will be set to the literal string *.csv, which is not the name of a file, so -f "$file" will fail. Otherwise, file will be set in turn to the name of each file matching the pattern, and -f "$file" will succeed every time. Note this will work even if there is an file named *.csv. The drawback is that you have to make a redundant test for each existing file.
I am very new with linux usage maybe this is my first time so i hope some detailed help please.
I have more than 500 files in multiple directories on my server (Linux) I want to change their extensions to .xml using bash script
I used a lot of codes but none of them work some codes i used :
for file in *.txt
do
mv ${file} ${file/.txt}/.xml
done
or
for file in *.*
do
mv ${file} ${file/.*}/.xml
done
i do not know even if the second one is valid code or not i tried to change the txt extension beacuse the prompt said no such file '.txt'
I hope some good help for that thank you
Explanation
For recursivity you need Bash >=4 and to enable ** (i.e. globstar) ;
First, I use parameter expansion to remove the string .txt, which must be anchored at the end of the filename (%) :
the # anchors the pattern (plain word or glob) to the beginning,
and the % anchors it to the end.
Then I append the new extension .xml
Be extra cautious with filename, you should always quote parameters expansion.
Code
This should do it in Bash (note that I only echothe old/new filename, to actually rename the files, use mv instead of echo) :
shopt -s globstar # enable ** globstar/recursivity
for i in **/*.txt; do
[[ -d "$i" ]] && continue; # skip directories
echo "$i" "${i/%.txt}.xml";
done
If its a matter of a one or two sub-directories, you can use the rename command:
rename .txt .xml *.txt
This will rename all the .txt to .xml files in the directory from which the command is executed.
If all the files are in same directory, it can be done using a single command. For example you want to convert all jpg files to png, go to the related directory location and then use command
rename .jpg .png *
I wanted to rename "file.txt" to "file.jpg.txt", used rename easy peezy:
rename 's/.txt$/.jpg.txt/' *.txt
man rename will tell you everything you need to know.
Got to love Linux, there's a tool for everything :-)
passing command line argument for dir path
#!/bin/sh
cd $1
names_1=`ls`
for file in ${names_1}
do
mv ${file} ${file}.jpg
done
I have written a shell script which picks all the files recursively inside all the directories and prepared a report with the file last modified, size.
The problem that I am facing, there are few files with name as "User Interface"(space in between). How to use there files in the for loop of the shell script and fetch the files and directories inside this.
Thanks in advance
Just put the file name variable between double quotes "$FILENAME"
You're probably trying to use something like for file in $(command). Instead, use a while read loop or a for loop with globbing. Make sure you quote variables that contain filenamess.
#!/bin/sh
command | while read -r file
do
something_with "$file"
done
or, in shells that support process substitution:
#!/bin/bash
while read -r file
do
something_with "$file"
done < <(command)
If you're simply iterating over a list of files:
for file in "$dir"/*
do
something_with "$file"
done
Google Search led me to this page