I have an executable program that have input and output likes
./my_exe -i filename_input -o filename_output
I want to use the program to run all filename in my folder that has structure likes
root
|-folder_A
|-abc.txt
|-folder_B
|cdf.txt
So, we can use for to do it. But the problem is that I want to automatically make the filename_output from the filename_input by adding the extension '_processed' before '.txt' likes abc.txt is input file name
. Then the output will be abc_processed.txt
How to do it in shell script? This is my current for code
for sub_folder in "${root_folder[#]}"
do
filename_input=$sub_folder/*.txt
filename_output= filename_input/*.txt/processed.txt
echo filename_output
done
The output of my script is root/folder_A/*processed.txt. I do not know why abc is lost
The right solution depends on what you want to do with it.
You should loop over the files you want to rename, not the directories.
for f in */*/*txt; do
echo "With path ${f} ==> ${f//.txt/processed.txt}"
base_f=${f##*/}
echo "Basenames: ${base_f} ==> ${base_f//.txt/processed.txt}"
done
You might want to use find ... | xargs for this when you want to call my_exe with these files.
Make sure your *processed.txt are not converted again!
$ tree root
root
|-- directory-A
| `-- abc.txt
`-- directory-B
`-- def.txt
2 directories, 2 files
$ find root -type f -exec sh -c 'echo ${1%.txt}_processed.txt' _ {} \;
root/directory-B/def_processed.txt
root/directory-A/abc_processed.txt
or:
$ for dir in root/*; do ( cd $dir; for file in *.txt;
do echo "$file --> ${file%.txt}_processed.txt"; done ) done
abc.txt --> abc_processed.txt
def.txt --> def_processed.txt
Related
Given a root path, I am trying to loop through the sub-directories to loop through the files in each subdirectory and print the names of the files.
The directory structure is like this:
Root directory
dir2,
file{1..10}
dir3,
file{1..10}
dir4
file{1..10}
I want to loop through dir2 and print all the filenames in it. Then loop through dir3 and print all the file names...and so on
Here is what I have so far:
#!/bin/bash
#!/bin/sh
cd /the/root/directory
for dir in */
do
for FILE in dir
do
echo "$FILE"
done > /the/root/directory/filenames.txt
done
This is the output I get in filenames.txt:
dir
My expected output is supposed to be:
file{1..10}
file{1..10}
file{1..10}
I am a beginner to bash scripting...well scripting in general.
Any help is greatly appreciated!
You didn't mention what your end goal is, so I'll speculate here.
If your end goal is to only see the files recursively in a list, you can run just a simple find command:
find . -type f
Or if you want to see the details:
find . -type f -ls
A nice way to view them with colors and nice-looking ansi bars is to install the tree command. Example:
https://www.tecmint.com/linux-tree-command-examples/
If your needs are simple, for example, you want to do an action such as a tail -n1 on each file, you can pipe the command to xargs like this:
find . -type f | xargs tail -n1
But if your end goal is to use bash to process them in some way, then you can continue down the bash looping method as mentioned by #tjm3772.
You mentioned you were just looking for the filenames so you can just run:
find . -type f | sed 's/.*\///'
If you want to write that to a file, just redirect the output to a filename of your choice:
find . -type f | sed 's/.*\///' > filename.txt
You can use the find command, and this will loop through the directories without needing the for loop
my_bash_script.sh:
find * -type d > filenames.txt
Put this script in the same level of the directories, or you point it to the location by changing the * to the path
note: if it says permission denied in the terminal run this: chmod u+x the_script_name.sh
You forgot to expand $dir in your inner loop, so the loop is executing one time with FILE set to the literal string 'dir' instead of the directory name.
After that, you need a globbing pattern to expand to the filenames inside the directory.
Fixed example:
#!/bin/bash
cd /the/root/directory
for dir in */
do
for FILE in "$dir/"*
do
echo "$FILE"
done > /the/root/directory/filenames.txt
done
The bash's way to do it is with the globstar expansion which expands recursively in directories
#!/usr/bin/env bash
shopt -s globstar # This enables recursively expanding files in directories
# This prints all the files in all the directories starting from /the/root/directory
printf '%s\n' /the/root/directory/**
I'm trying to run a series of commands on a list of files in multiple directories located directly under the current branch.
An example hierarchy is as follows:
/tmp
|-1
| |-a.txt
| |-b.txt
| |-c.txt
|-2
| |-a.txt
| |-b.txt
| |-c.txt
From the /tmp directory I'm sitting at my prompt and I'm trying to run a command against the a.txt file by renaming it to d.txt.
How do I get it to go into each directory and rename the file? I've tried the following and it won't work:
for i in ./*; do
mv "$i" $"(echo $i | sed -e 's/a.txt/d.txt/')"
done
It just doesn't jump into each directory. I've also tried to get it to create files for me, or folders under each hierarchy from the current directory just 1 folder deep, but it won't work using this:
for x in ./; do
mkdir -p cats
done
OR
for x in ./; do
touch $x/cats.txt
done
Any ideas ?
Place the below script in your base directory
#!/bin/bash
# Move 'a.txt's to 'd.txt's recursively
mover()
{
CUR_DIR=$(dirname "$1")
mv "$1" "$CUR_DIR/d.txt"
}
export -f mover
find . -type f -name "a.txt" -exec bash -c 'mover "$0"' {} \;
and execute it.
Note:
If you wish be a bit more innovative and generalize the script, you could accept directory name to search for as a parameter to the script and pass the directory name to find
> for i in ./*; do
As per your own description, this will assign ./1 and then ./2 to i. Neither of those matches any of the actual files. You want
for i in ./*/*; do
As a further aside, the shell is perfectly capable of replacing simple strings using glob patterns. This also coincidentally fixes the problem with not quoting $i when you echo it.
mv "$i" "${i%/a.txt}/d.txt"
I need a bash script that can count directories that are inside other directories on a FreeBSD.
The case is like this:
The path is home/myuser/direct than inside this directory there are like 20 directories named only by one letter like A B C D E F and so on. Inside every A directory, B directory there are many other directories with different names, such as mydirectory1, mydirectory2 and so on. Inside mydirectory1 there are differnet files and directories, and I need to count only the directories that are under the mydirectory1, and not the files. I came up with this, but using this, I will have to do that manually for each directory:
home/myuser/direct# ls -l A/* | grep ^d | wc -l
than for the B directory I will have to:
home/myuser/direct# ls -l B/* | grep ^d | wc -l
and so on. Is there a way, to automatically do this, I mean change the letter A to B and so on?
P.S, sorry about the confusion as English is not my first language :(
This solution assumes you want the number of subfolders for each folder in the current directory. If you want to sum them all up into one value, that is a different question... It is not incredibly robust to variations in folder names, but should work for most cases when there is not strange punctuation:
for D in */; do echo "$D": $(ls -d "$D"*/| wc -l); done
Example output:
DATA/: 14
LOGS/: 2
PLOTS/: 3
SCRIPTS/: 2
ls: libraries//*/: No such file or directory
libraries/: 0
Here is a version which suppresses the error for empty folders:
for D in */; do echo "$D": $(ls -d "$D"*/ 2>/dev/null |wc -l); done
Try
find * -type d -print | wc -l
I need to do a bash command that will look through every home directory on a system, and copy the contents of the .forward file to a single file, along with copying the name of the directory it just copied from. So for example the final file would be something like forwards.txt and listeings would be
/home/user1
user1#email.com
/home/user2
user2#email.com
I've used this to list them to screen.
find /home -name '*' | cat /home/*/.forward
and it will print out the forward in each file but I'm not getting it to prefix it with which home directory it came from. Would I need to use a loop to do this? I had this test loop,
#!/bin/bash
for i in /home/*
do
if [ -d $i ]
then
pwd >> /tmp/forwards.txt
cat /home/*/.forward >> /tmp/forwards.txt
fi
done
But it went through the four home directories on my test setup and the forwards.txt file had the following listed four times.
/tmp
user1#email.com
user2#email.com
user3#email.com
user3#email.com
Thanks.
There is corrected version of your script:
#!/bin/bash
for i in /home/*
do
if [ -f "$i/.forward" ]
then
echo "$i" >> /tmp/forwards.txt
cat "$i/.forward" >> /tmp/forwards.txt
fi
done
Some points:
we checks for presents of .forward file inside home directory instead of existence of home directory itself
on each iteration $i contains name of home directory (like /home/user1). So we use its value instead of output of pwd command which always returns current directory (it doesn't change in our case)
instead of /home/*/.forward we use "/home/$i/.forward" because * after substitution gives to us all directories, while we need only current
Another, shortest version of this script may looks like this:
find /home -type f -name '.forward' | while read F; do
dirname "$F" >>/tmp/forwards.txt
cat "$F" >>/tmp/forwards.txt
done
I would write
for fwd in /home/*/.forward; do
dirname "$fwd"
cat "$fwd"
done > forwards.txt
A one liner (corrected):
find /home -maxdepth 2 -name ".forward" -exec echo "{}" >> /tmp/forwards.txt \; -exec cat "{}" >> /tmp/forwards.txt \;
This will output:
/home/user1/.forward
a#a.a
b#b.b
/home/user2/.forward
a#b.c
My directory structure is as follows
Directory1\file1.jpg
\file2.jpg
\file3.jpg
Directory2\anotherfile1.jpg
\anotherfile2.jpg
\anotherfile3.jpg
Directory3\yetanotherfile1.jpg
\yetanotherfile2.jpg
\yetanotherfile3.jpg
I'm trying to use the command line in a bash shell on ubuntu to take the first file from each directory and rename it to the directory name and move it up one level so it sits alongside the directory.
In the above example:
file1.jpg would be renamed to Directory1.jpg and placed alongside the folder Directory1
anotherfile1.jpg would be renamed to Directory2.jpg and placed alongside the folder Directory2
yetanotherfile1.jpg would be renamed to Directory3.jpg and placed alongside the folder Directory3
I've tried using:
find . -name "*.jpg"
but it does not list the files in sequential order (I need the first file).
This line:
find . -name "*.jpg" -type f -exec ls "{}" +;
lists the files in the correct order but how do I pick just the first file in each directory and move it up one level?
Any help would be appreciated!
Edit: When I refer to the first file what I mean is each jpg is numbered from 0 to however many files in that folder - for example: file1, file2...... file34, file35 etc... Another thing to mention is the format of the files is random, so the numbering might start at 0 or 1a or 1b etc...
You can go inside each dir and run:
$ mv `ls | head -n 1` ..
If first means whatever the shell glob finds first (lexical, but probably affected by LC_COLLATE), then this should work:
for dir in */; do
for file in "$dir"*.jpg; do
echo mv "$file" "${file%/*}.jpg" # If it does what you want, remove the echo
break 1
done
done
Proof of concept:
$ mkdir dir{1,2,3} && touch dir{1,2,3}/file{1,2,3}.jpg
$ for dir in */; do for file in "$dir"*.jpg; do echo mv "$file" "${file%/*}.jpg"; break 1; done; done
mv dir1/file1.jpg dir1.jpg
mv dir2/file1.jpg dir2.jpg
mv dir3/file1.jpg dir3.jpg
Look for all first level directories, identify first file in this directory and then move it one level up
find . -type d \! -name . -prune | while read d; do
f=$(ls $d | head -1)
mv $d/$f .
done
Building on the top answer, here is a general use bash function that simply returns the first path that resolves to a file within the given directory:
getFirstFile() {
for dir in "$1"; do
for file in "$dir"*; do
if [ -f "$file" ]; then
echo "$file"
break 1
fi
done
done
}
Usage:
# don't forget the trailing slash
getFirstFile ~/documents/
NOTE: it will silently return nothing if you pass it an invalid path.