I have the template file 12A-r.inp . I want to prepare files from this file whose name will be 16A-r.inp, 20A-r.inp, 24A-r.inp. And I want to change some parameter in those files according to their names. For example, I want to replace the string "12A" in all places in file 12A-r.inp, with 16A in 16A-r.inp, and 20A in 20A-r.inp. I have written the code below for this:
for ((i=12;i<=24;i=i+4))
do
cat 12A-r.inp >> $i\A-r.inp
done
for ((i=12;i<=24;i=i+4))
do
sed -i "s/12A/${i}/g" $i\A-r.inp
done
But the problem is 12A gets replaced by ${i}, not with strings like 16A, 20A etc.
Observations:
In for ((i=12;i<=24;i=i+4)) counts 12,16,20,24. There's no need
to start at 12, since the template is already correct. Worse,
when i=12, this code cat 12A-r.inp >> $i\A-r.inp appends a
copy of the template file onto itself, doubling it, which causes every ensuing
created file to be twice as long as the original template.
The \ in $i\A-r.inp is unnecessary, since A is not a special character.
The cat is unnecessary, sed without -i can do it all.
In sed, s/12A/${i}/g would replace the string "12A", with whatever number $i is, without the "A", unless the variable includes that letter.
The for loop uses a bashism to enumerate i... in this instance there's a simpler equivalent bashism, (see below).
Suggested revision:
for i in {16..24..4}A
do
sed "s/12A/${i}/g" 12A-r.inp > ${i}-r.inp
done
How it works:
$i is set to 16A,20A, and 24A.
sed repeatedly reads in the template, replaces 12A with $i,
prints everything to STDOUT...
which is redirected to the appropriately named file.
Related
I have a script that I call with an application, I can't run it from command line. I derive the directory where the script is called and in the next variable go up 1 level where my files are stored. From there I have 3 variables with the full path and file names (with wildcard), which I will refer to as "masks".
I need to find and "do something with" (copy/write their names to a new file, whatever else) to each of these masks. The do something part isn't my obstacle as I've done this fine when I'm working with a single mask, but I would like to do it cleanly in a single loop instead of duplicating loop and just referencing each mask separately if possible.
Assume in my $FILESFOLDER directory below that I have 2 existing files, aaa0.csv & bbb0.csv, but no file matching the ccc*.csv mask.
#!/bin/bash
SCRIPTFOLDER=${0%/*}
FILESFOLDER="$(dirname "$SCRIPTFOLDER")"
ARCHIVEFOLDER="$FILESFOLDER"/archive
LOGFILE="$SCRIPTFOLDER"/log.txt
FILES1="$FILESFOLDER"/"aaa*.csv"
FILES2="$FILESFOLDER"/"bbb*.csv"
FILES3="$FILESFOLDER"/"ccc*.csv"
ALLFILES="$FILES1
$FILES2
$FILES3"
#here as an example I would like to do a loop through $ALLFILES and copy anything that matches to $ARCHIVEFOLDER.
for f in $ALLFILES; do
cp -v "$f" "$ARCHIVEFOLDER" > "$LOGFILE"
done
echo "$ALLFILES" >> "$LOGFILE"
The thing that really spins my head is when I run something like this (I haven't done it with the copy command in place) that log file at the end shows:
filesfolder/aaa0.csv filesfolder/bbb0.csv filesfolder/ccc*.csv
Where I would expect echoing $ALLFILES just to show me the masks
filesfolder/aaa*.csv filesfolder/bbb*.csv filesfolder/ccc*.csv
In my "do something" area, I need to be able to use whatever method to find the files by their full path/name with the wildcard if at all possible. Sometimes my network is down for maintenance and I don't want to risk failing a change directory. I rarely work in linux (primarily SQL background) so feel free to poke holes in everything I've done wrong. Thanks in advance!
Here's a light refactoring with significantly fewer distracting variables.
#!/bin/bash
script=${0%/*}
folder="$(dirname "$script")"
archive="$folder"/archive
log="$folder"/log.txt # you would certainly want this in the folder, not $script/log.txt
shopt -s nullglob
all=()
for prefix in aaa bbb ccc; do
cp -v "$folder/$prefix"*.csv "$archive" >>"$log" # append, don't overwrite
all+=("$folder/$prefix"*.csv)
done
echo "${all[#]}" >> "$log"
The change in the loop to append the output or cp -v instead of overwrite is a bug fix; otherwise the log would only contain the output from the last loop iteration.
I would probably prefer to have the files echoed from inside the loop as well, one per line, instead of collect them all on one humongous line. Then you can remove the array all and instead simply
printf '%s\n' "$folder/$prefix"*.csv >>"$log"
shopt -s nullglob is a Bash extension (so won't work with sh) which says to discard any wildcard which doesn't match any files (the default behavior is to leave globs unexpanded if they don't match anything). If you want a different solution, perhaps see Test whether a glob has any matches in Bash
You should use lower case for your private variables so I changed that, too. Notice also how the script variable doesn't actually contain a folder name (or "directory" as we adults prefer to call it); fixing that uncovered a bug in your attempt.
If your wildcards are more complex, you might want to create an array for each pattern.
tmpspaces=(/tmp/*\ *)
homequest=($HOME/*\?*)
for file in "${tmpspaces[#]}" "${homequest[#]}"; do
: stuff with "$file", with proper quoting
done
The only robust way to handle file names which could contain shell metacharacters is to use an array variable; using string variables for file names is notoriously brittle.
Perhaps see also https://mywiki.wooledge.org/BashFAQ/020
I need to update potentially hundreds of configuration files for a program by adding new lines to text files. I will be adding additional properties, such as background color, to these files and want to automate the process with bash. All of these properties are contained in ".m" files. Its essentially updating the properties of widgets on GUIs. Each object's properties in a gui is labeled with an object type followed by a name. One example of an object is called a Form.
The problem is, each name that follows the object is different so I need to add the line based off of the name of the other properties in each section of the .m file. For example, one file has two form objects. The section for one is called "*FO_mm" while the second object's section is named "*FO_test_area". After the name is an extension for what property is specified, such as "*FO_mm.class:". While the properties each object has tends to vary, I found that all objects share a property called ".parent" so I am using that as a search reference. I want to use the sed command to add a line after the .parent line with the new property, in this case background color. So the idea is to search for a string that starts with "*FO_" and ends with ".parent", with everything inbetween being something different for each section. I want to use a loop to capture the string preceding ".parent" as a variable and attach it to the beginning of the new property line so it matches the current section. Here is my current script:
//The top level directory
script_dir="/project/guis/"
//The extension to look for
file_ext="*.m"
fileList=$(find $script_dir -type f -name "$file_ext")
declare -a file_list
readarray -t file_list < <(printf '%s\n' "$fileList")
cd $script_dir
//Loop through each m file
for m_file in ${file_list[#]}; do
var1=($(grep '*FO_.*.parent:' $m_file))
declare -a var_list
readarray -t var_list < <(printf '%s\n' "$var1")
for i in ${var_list[#]}; do
echo $i
sed -i "/^*FO_.*.parent:.*/a\$i.background: #2b3856 " $m_file
done
done
When I run it, the script adds the line "$i.background: #2b3856" below the .parent line. And the "echo $i" line returns "*FO_mm.parent: FO_mm". So there are several problems.
The value of the variable is not being substituted into the sed statement.
As the echo states, only the first section "*FO_mm" is being saved as a variable, which means the second section "*FO_test_area" is not being implemented.
I only want the object name to be stored and placed into the new line. So the result should give me for the first section "*FO_mm.background: #2b3856" with everything from .background on being tacked on by the last part of the sed statement. Since I am still fairly new to bash and especially sed, I have no idea how to strip the variable down to just the object name.
Here is an example of what a single object section looks like prior to running the script:
*FO_test_area.class: Form
*FO_test_area.static: true
*FO_test_area.parent: FO_mm
*FO_test_area.resizePolicy: "resize_none"
And here is what this section looks like after running the WIP script:
*FO_test_area.class: Form
*FO_test_area.static: true
*FO_test_area.parent: FO_mm
$i.background: #33b342
*FO_test_area.resizePolicy: "resize_none"
Its a lot to describe, but I've hit a wall and I would really appreciate any help you can provide.
If ed is available/acceptable.
The script named script.ed (name it whatever you like).
g/^\*FO_.*\.parent:.*/t.\
s/^\(\*\).*parent: */\1/
s/$/.background: #33b342/
%p
Q
The g/^\*FO_.*\.parent:.*/ will match every line that starts with *FO_ and with .parent: somewhere after it. It will match either *FO_test_area.parent and *FO_mm.class.parent. You gonna have to be specific about the regex to match a specific *FO_.*\.parent: pattern to be able to do a specific search & replace/insert
Here is a specific script for the *FO_test_area.parent:
g/^\*FO_test_area\.parent:.*/t.\
s/^\(\*\).*parent: */\1/
s/$/.background: #33b342/
%p
Q
Modify the script above and add another pattern before the line where %p is at, do the rest of the substitution after that.
Your sample file/data.
*FO_test_area.class: Form
*FO_test_area.static: true
*FO_test_area.parent: FO_mm
*FO_test_area.resizePolicy: "resize_none"
Running the script against your data/file. (file ending with a .m)
ed -s file.m < script.ed
Output
*FO_test_area.class: Form
*FO_test_area.static: true
*FO_test_area.parent: FO_mm
*FO_mm.background: #33b342
*FO_test_area.resizePolicy: "resize_none"
If you're satisfied with the output, next thing is to do the loop.
Doing some adjustment to your script. Instead of a nested for loop, the script is using a while + read loop and Process Substitution. See How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?
#!/usr/bin/env bash
script_dir="/project/guis/"
file_ext="*.m"
while IFS= read -r m_file; do
if grep -q '^\*FO_.*.parent:' "$m_file"; then
ed -s "$m_file" < script.ed
fi
done < <(find "$script_dir" -type f -name "$file_ext")
The script.ed is inside the current directory where your script is at. It can be anywhere just need to give it the correct absolute path, e.g.
/path/to/script.ed
If in-place editing is needed change the Q to w inside the ed script.
Remove the line where %p is at if the output is not needed to stdout.
See:
GNU ed
POSIX ed
ed
ed in pdf
MirBSD ed
bash hackers wiki ed
Also your local man pages.
man 1p ed
I have a variable which contains several special characters. Now I want to change these into space
If I do it one by one, the change goes fine
e.g. txt=$(printf "$txt" | sed 's/\xE2/ /g')
Now to change them all at once I inserted all special characters into a file like this :
\xE1
\xE2
\xC3
...
If I want to change this by doing this nothing happens :
while IFS=: read -r special
do
txt=$(printf "txt" | sed 's/$special/ /g')
done <"/home/u555/specialchar.txt"
What is wrong with this loop ?
sed (or any other external utility or a loop) is not needed for this job. You can use builtin bash parameter expansions:
var=${var//[$'\xe1\xe2\xc3']/ }
will do the job at once.
Update after the comment "But what if you need to change a lot of special characters (approx 50) I want to keep them in a file, so if I add to add one I don't need to change the program, only the file"
One method is to define a variable, say, spchars, as spchars=$'\xe1\xe2\xc3' within a file named, say, special_characters and source that file into your script:
. special_characters
var=${var//[$spchars]/ }
then you won't need to modify your script, only the parameter spchars in the file special_characters.
I have a folder with several files named : something_1001.txt; something_1002.txt; something_1003.txt; etc.
Inside the files there is some text. Of course each file has a different text but the structure is always the same: some lines identified with the string ">TEXT", which are the ones I am interested in.
So my goal is :
for each file in the folder, read the file's name and extract the number between "_" and ".txt"
modify all the lines in this particular file that contain the string ">TEXT" in order to make it ">{NUMBER}_TEXT"
For example : file "something_1001.txt"; change all the lines containing ">TEXT" by ">1001_TEXT"; move on to file "something_1002.txt" change all the lines containing ">TEXT" by ">1002_TEXT"; etc.
Here is the code I wrote so far :
for i in /folder/*.txt
NAME=`echo $i | grep -oP '(?<=something_/).*(?=\.txt)'`
do
sed -i -e 's/>TEXT/>${NAME}_TEXT/g' /folder/something_${NAME}.txt
done
I created a small bash script to run the code but it's not working. There seems to be syntax errors and a loop error, but I can't figure out where.
Any help would be most welcome !
There are two problems here. One is that your loop syntax is wrong; the other is that you are using single quotes around the sed script, which prevents the shell from interpolating your variable.
The grep can be avoided, anyway; the shell has good built-in facilities for extracting the base name of a file.
for i in /folder/*.txt
do
base=${i#/folder/something_}
sed -i -e "s/>TEXT/>${base%.txt}_TEXT/" "$i"
done
The shell's ${var#prefix} and ${var%suffix} variable manipulation facility produces the value of $var with the prefix and suffix trimmed off, respectively.
As an aside, avoid uppercase variable names, because those are reserved for system use, and take care to double-quote any variable whose contents may include shell metacharacters.
I have a 'testfiles' files that has list of files
Ex-
Tc1
Tc2
calling above file in script
test=`cat testfiles`
for ts in $test
do
feed.sh $ts >>results
done
This script runs fine when there only 1 test file in 'testfiles',but when there are multiple files ,it fails with 'file not found'
Let me know if this is correct approach
you ll have to read files one by one since you are taking testfiles='Tc1 Tc2' cat is searching for file named 'Tc1 Tc2' which does not exist so use cut command with " " as the delimiter and rad files one by one in a loop.or u can use sed command also to seperate file names
Your approach should work if the filenames have no spaces or other tricky characters. An approach that handles spaces in file names successfully is:
while IFS= read -r ts
do
feed.sh "$ts" >>results
done <testfiles
If your file names have newline characters in them, then the above won't work and you would need to create testfiles with the names separated by a null character in place of a newline.
Let's consider the original code. When bash substitutes for $test in the for statement, all the file names appear on the same line and bash will perform word splitting which will make a mess of any file names containing white space. The same happens on the line feed.sh $ts. Since $ts is not quoted, it will also undergo word splitting.