String of a filename used within a bash script - bash

I have a list of files:
name1andlast1.address
name2andlast2.address
name2andlast2.address
...
I want to create a bash script that use each *.address file + use files which names are strings of the *.address filename. The delimiter will be "and" (without the file extension), so I can use name#.list and last#.list
For example, within a bash script that run for name1andlast1.address, I would like to use a set of files (in another directory) called name1.list and last1.list:
grep "something" name1.list > output1
grep "something" last1.list > output2
grep "something" name1andlast1.address > output3
The order of using grep is not important. What is more important is how to use the filename (i.e name1andlast1.address) to input in my bash script name1.list and last1.list. I need to find a way to extract those bnames separate by “and”. I need to find a way to do it iterative over multiple *.address files

to find a way to extract those bnames separate by “and”. I need to
find a way to do it iterative over multiple *.address files
bash solution:
for f in *.address; do
name_l=${f%and*}.list
last_l=${f#*and}
last_l=${last_l//.address/.list}
# ... do futher search/processing with "$name_l" and "$last_l"
done

Related

Modify text file based on file's name, repeat for all files in folder

I have a folder with several files named : something_1001.txt; something_1002.txt; something_1003.txt; etc.
Inside the files there is some text. Of course each file has a different text but the structure is always the same: some lines identified with the string ">TEXT", which are the ones I am interested in.
So my goal is :
for each file in the folder, read the file's name and extract the number between "_" and ".txt"
modify all the lines in this particular file that contain the string ">TEXT" in order to make it ">{NUMBER}_TEXT"
For example : file "something_1001.txt"; change all the lines containing ">TEXT" by ">1001_TEXT"; move on to file "something_1002.txt" change all the lines containing ">TEXT" by ">1002_TEXT"; etc.
Here is the code I wrote so far :
for i in /folder/*.txt
NAME=`echo $i | grep -oP '(?<=something_/).*(?=\.txt)'`
do
sed -i -e 's/>TEXT/>${NAME}_TEXT/g' /folder/something_${NAME}.txt
done
I created a small bash script to run the code but it's not working. There seems to be syntax errors and a loop error, but I can't figure out where.
Any help would be most welcome !
There are two problems here. One is that your loop syntax is wrong; the other is that you are using single quotes around the sed script, which prevents the shell from interpolating your variable.
The grep can be avoided, anyway; the shell has good built-in facilities for extracting the base name of a file.
for i in /folder/*.txt
do
base=${i#/folder/something_}
sed -i -e "s/>TEXT/>${base%.txt}_TEXT/" "$i"
done
The shell's ${var#prefix} and ${var%suffix} variable manipulation facility produces the value of $var with the prefix and suffix trimmed off, respectively.
As an aside, avoid uppercase variable names, because those are reserved for system use, and take care to double-quote any variable whose contents may include shell metacharacters.

bash cat all files that contains a certain string in file name

In bash, how would you cat all files in a directory that contains a certain string in its filename. For example I have files named:
test001.csv
test002.csv
test003.csv
result001.csv
result002.csv
I want to cat all .csv that contains the string test in the file name together, and all .csv that contains the string result in the file name together.
Just:
cat *test*.csv
cat *result*.csv
For all files with test (or in case of the second one result) in their name.
The shell itself can easily find all files matching a simple wildcard.
cat *test*.csv >testresult
You want to take care so that the output file's name does not match the wildcard. (It's technically harmless, but good practice.)
The shell will expand the wildcard in alphabetical order. Most shells will obey your locale, so the definition of "alphabetical order" may depend on current locale settings.
Here's very simple way
cat `find . -name "*test*.csv"`

Extend parsing command to multiple files

There are a number of files I use which can all be parsed into three files (per input file) with:
third_file_1_1.txt
first_file_1_2.txt....
in an automated process?
You can use the FILENAME builtin variable to know what file you are currently processing. This allows you to do:
awk '/^[01]+$/ {print > "third_"FILENAME; next}/ [...]' $(<list.txt)

Call script on all file names starting with string in folder bash

I have a set of files I want to perform an action on in a folder that i'm hoping to write a scipt for. Each file starts with mazeFilex where x can vary from any number , is there a quick and easy way to perform an action on each file? e.g. I will be doing
cat mazeFile0.txt | ./maze_ppm 5 | convert - maze0.jpg
how can I select each file knowing the file will always start with mazeFile?
for fname in mazeFile*
do
base=${fname%.txt}
base=${base#mazeFile}
./maze_ppm 5 <"$fname" | convert - "maze${base}.jpg"
done
Notes
for fname in mazeFile*; do
This codes starts the loop. Written this way, it is safe for all filenames, whether they have spaces, tabs or whatever in their names.
base=${fname%.txt}; base=${base#mazeFile}
This removes the mazeFile prefix and .txt suffix to just leave the base name that we will use for the output file.
./maze_ppm 5 <"$fname" | convert - "maze${base}.jpg"
The output filename is constructed using base. Note also that cat was unnecessary and has been removed here.
for i in mazeFile*.txt ; do ./maze_ppm 5 <$i | convert - `basename maze${i:8} .txt`.jpg ; done
You can use a for loop to run through all the filenames.
#!/bin/bash
for fn in mazeFile*; do
echo "the next file is $fn"
# do something with file $fn
done
See answer here as well: Bash foreach loop
I see you want a backreference to the number in the mazeFile. Thus I recommend John1024's answer.
Edit: removes the unnecessary ls command, per #guido 's comment.

Using bash wildcards with prefix

I am trying to write a bash script that takes a variable number of file names as arguments.
The script is processing those files and creating a temporary file for each of those files.
To access the arguments in a loop I am using
for filename in $*
do
...
generate t_$(filename)
done
After the loop is done, I want to do something like cat t_$* .
But it's not working. So, if the arguments are a b c, it is catting t_a, b and c.
I want to cat the files t_a, t_b and t_c.
Is there anyway to do this without having to save the list of names in another variable?
You can use the Parameter expansion:
cat "${#/#/t_}"
/ means substitute, # means at the beginning.

Resources