I'm a newbie here. I've really tried to google it but failed.
I've got a simple script that reads from file line by line and then prints it. "b.txt" has the first line of "02083192846".
#!/bin/bash
while IFS= read -r line; do
printf 'downloading %s .html\n' $line
done < "$1"
however, the output is screwed up:
User#User-pk ~/test
$ ./test.sh b.txt
.htmlading 02083192846
Once this is fixed, my next question is how to use this line as part of the file name that some command will store into. i.e. the filename should be "02083192846.html". I've tried setting using it as ${line}.html but it doesn't work. For example grep "foo" -f ${line}.html doesn't work. But curl http://foo -o $line.html does work still!
What I would do :
#!/bin/bash
while IFS= read -r line; do
printf 'downloading %s .html\n' "${line//$'\r'/}"
# ________
done < "$1"
# ^
# remove \r with bash parameter expansion
So I was just starting learning bash scripting. I encountered a question in a book.
An example testfile contains following content.
$ cat testfile
This is the first line.
This is the second line.
This is the third line.
And the script file is like:
#!/bin/bash
# testing input/output file descriptor
exec 3<> testfile
read line <&3
echo "Read: $line"
echo "This is a test line" >&3
After running the script, the testfile became:
$ cat testfile
This is the first line.
This is a test line
ine.
This is the third line.
I understand why that script changes the testfile. My question is why
"ine." starts from a new line? Does echo command automatically add a newline character to the end of the string?
echo -n is what you seek: the option -n
instructs echo to "do not output the trailing newline".
FWIW: man echo on your platform will instruct what options the /bin/echo command understands. But since you mention bash as shell: bash has an internal implementation of echo (a so-called "builtin")
I'm trying to write a shell script that takes in a file(ex. file_1_2.txt) and replaces any "_" with "."(ex. file.1.2.txt). This is what I have but its giving me a blank output when I run it.
read $var
x= `echo $var | sed 's/\./_/g'`
echo $x
I'm trying to store the changed filename in the variable "x" and then output x to the console.
I am calling this script by writing
./script2.sh < file_1_2.txt
There is two problems. First, your code has some bugs:
read var
x=`echo $var | sed 's/_/\./g'`
echo $x
will work. You had an extra $ in read var, a space too much (as mentioned before) and you mixed up the replacement pattern in sed (it was doing the reverse of what you wanted).
Also if you want to replace the _ by . in the filename you should do
echo "file_1_2.txt" | ./script2.sh
If you use < this will read the content of `file_1_2.txt" into your script.
Another solution, with bash only:
$ x=file_1_2.txt; echo "${x//_/.}"
file.1.2.txt
(See “Parameter expansion” section in bash manual page for details)
And you can also do this with rename:
$ touch file_1_2.txt
$ ls file*
file_1_2.txt
$ rename 'y/_/\./' file_1_2.txt
$ ls file*
file.1.2.txt
Threre is not need for sed as bash supports variable replacement:
$ cat ./script2
#!/bin/bash
ofile=$1
nfile=${ofile//_/./}
echo mv "$ofile" "$nfile"
$ ./script2 file_1_2.txt
mv "file_1_2.txt" "file.1.2.txt"
Then just remove echo if you are satisfied with the result.
I have a simple bash script, which receives a file with subject text, and processes line by line.
If the line begins with certain characters - then run block of compound commands.
I am trying to use grep to test the lines for patterns (but would accept other suggestions).
file: matcher.sh
#!/usr/bin/env bash
for i in "$#"
do
if grep "^M" $i # I want grep to "assume" $i was a file
# and test if the pattern "^M" is present
then
echo "This line started with an M: "$i
# command 1
# command 2
# etc
fi
done
Subject_text.txt
D bar
M shell_test.sh
M another_file
Then run script with
cat subject_text.txt | xargs --delimiter="\n" ./matcher.sh
How can I get grep to treat each iteration $i through the parameter list
as if $i were a file?
You can read the file Subject_text.txt in a loop and feed matcher.sh with the name of the file to check:
while IFS= read -r _ file_name
do
./matcher.sh "$file_name"
done < "Subject_text.txt"
However, now that I see, you are using matcher.sh over every line. Note that calling a script with every single line as parameter is a bit overkilling.
What about looping over the file normally and performing the grep?
#!/usr/bin/env bash
file=$1
while IFS= read -r line
do
if grep "^M" <<< "$line" # I want grep to "assume" $i was a file
# and test if the pattern "^M" is present
then
echo "This line started with an M: $i"
# command 1
# command 2
# etc
fi
done < "$file"
Consider a ASCII text file (lets say it contains code of a non-shell scripting language):
Text_File.msh:
spool on to '$LOG_FILE_PATH/logfile.log';
login 'username' 'password';
....
Now if this were a shell script I could run it as $ sh Text_File.msh and the shell would automatically expand the variables.
What I want to do is have shell expand these variables and then create a new file as Text_File_expanded.msh as follows:
Text_File_expanded.msh:
spool on to '/expanded/path/of/the/log/file/../logfile.log';
login 'username' 'password';
....
Consider:
$ a=123
$ echo "$a"
123
So technically this should do the trick:
$ echo "`cat Text_File.msh`" > Text_File_expanded.msh
...but it doesn't work as expected and the output-file while is identical to the source.
So I am unsure how to achieve this.. My goal is make it easier to maintain the directory paths embedded within my non-shell scripts. These scripts cannot contain any UNIX code as it is not compiled by the UNIX shell.
This question has been asked in another thread, and this is the best answer IMO:
export LOG_FILE_PATH=/expanded/path/of/the/log/file/../logfile.log
cat Text_File.msh | envsubst > Text_File_expanded.msh
if on Mac, install gettext first: brew install gettext
see:
Forcing bash to expand variables in a string loaded from a file
This solution is not elegant, but it works. Create a script call shell_expansion.sh:
echo 'cat <<END_OF_TEXT' > temp.sh
cat "$1" >> temp.sh
echo 'END_OF_TEXT' >> temp.sh
bash temp.sh >> "$2"
rm temp.sh
You can then invoke this script as followed:
bash shell_expansion.sh Text_File.msh Text_File_expanded.msh
If you want it in one line (I'm not a bash expert so there may be caveats to this but it works everywhere I've tried it):
when test.txt contains
${line1}
${line2}
then:
>line1=fark
>line2=fork
>value=$(eval "echo \"$(cat test.txt)\"")
>echo "$value"
line1 says fark
line2 says fork
Obviously if you just want to print it you can take out the extra value=$() and echo "$value".
If a Perl solution is ok for you:
Sample file:
$ cat file.sh
spool on to '$HOME/logfile.log';
login 'username' 'password';
Solution:
$ perl -pe 's/\$(\w+)/$ENV{$1}/g' file.sh
spool on to '/home/user/logfile.log';
login 'username' 'password';
One limitation of the above answers is that they both require the variables to be exported to the environment. Here's what i came up with that would allow the variables to be local to the current shell script:
#!/bin/sh
FOO=bar;
FILE=`mktemp`; # Let the shell create a temporary file
trap 'rm -f $FILE' 0 1 2 3 15; # Clean up the temporary file
(
echo 'cat <<END_OF_TEXT'
cat "$#"
echo 'END_OF_TEXT'
) > $FILE
. $FILE
The above example allows the variable $FOO to be substituted in the files named on the command line. I'm sure it can be improved, but this works for me so far.
Thanks to both previous answers for their ideas!
If the variables you want to translate are known and limited in number, you can always do the translation yourself:
sed "s/\$LOG_FILE_PATH/$LOG_FILE_PATH/g" input > output
And also assuming the variable itself is already known
This solution allows you to keep the same formatting in the ouput file
Copy and paste the following lines in your script
cat $1 | while read line
do
eval $line
echo $line
eval echo $line
done | uniq | grep -v '\$'
this will read the file passed as argument line by line, and then process to try and print each line twice:
- once without substitution
- once with substitution of the variables.
then remove the duplicate lines
then remove the lines containing visible variables ($)
Yes eval should be used carefully, but it provided me this simple oneliner for my problem. Below is an example using your filename:
eval "echo \"$(<Text_File.msh)\""
I use printf instead of echo for my own purposes, but that should do the trick. Thank you abyss.7 providing the link that solve my problem. Hope it helps.
Create an ascii file test.txt with the following content:
Try to replace this ${myTestVariable1}
bla bla
....
Now create a file “sub.sed” containing variable names, eg
's,${myTestVariable1},'"${myTestVariable1}"',g;
s,${myTestVariable2},'"${myTestVariable2}"',g;
s,${myTestVariable3},'"${myTestVariable3}"',g;
s,${myTestVariable4},'"${myTestVariable4}"',g'
Open a terminal move to the folder containing test.txt and sub.sed.
Define the value of the varible to be replaced
myTestVariable1=SomeNewText
Now call sed to replace that variable
sed "$(eval echo $(cat sub.sed))" test.txt > test2.txt
The output will be
$cat test2.txt
Try to replace this SomeNewText
bla bla
....
#logfiles.list:
$EAMSROOT/var/log/LinuxOSAgent.log
$EAMSROOT/var/log/PanacesServer.log
$EAMSROOT/var/log/PanacesStrutsGUI.log
#My Program:
cat logfiles.list | while read line
do
eval Eline=$line
echo $Eline
done