I'm looking at creating a bash script to print out a text file with some extra variables, see below
I want to have a text file with something like this in it
bot1
bot2
bot3
And then have a bash script print it out like so
--exclude-agent="bot1" --exclude-agent="bot2" --exclude-agent="bot3"
Is this possible? So that if I add another line to that first file it'll just print another --exclude-agent="whatever I put in the file"
At the moment I've got the below which is close, but not quite what I want
#!/bin/bash
while read line
do
echo "--exclude-agent="$line" \\"
done < bots.txt
Any help would be great!
It depends on what did you mean by
not quite what I want
First of all, you have a problem with quoting. Correct way is
echo "--exclude-agent=$line \\"
If you want to print it on the same line try
echo -n "--exclude-agent=$line"
If you want to hold it in a variable try this code
params=''
while read -r line
do
params+="--exclude-agent=$line "
done < bots.txt
printf '%s\n' "$params"
Also it seems like you're trying to store parameters in a variable. Which is the worst idea ever
echo -n "--exclude-agent="$line". -n does not print the last \n so that the all options come in the same line
to print out the expected output:
echo "--exclude-agent="\"$line"\" "
Related
I need to add new lines with specific information to one or multiple files at the same time.
I tried to automate this task using the following script:
for i in /apps/data/FILE*
do
echo "nice weather 20190830 friday" >> $i
done
It does the job yet I wish I can automate it more and let the script ask me for to provide the file name and the line I want to add.
I expect the output to be like
enter file name : file01
enter line to add : IWISHIKNOW HOWTODOTHAT
Thank you everyone.
In order to read user input you can use
read user_input_file
read user_input_text
read user_input_line
You can print before the question as you like with echo -n:
echo -n "enter file name : "
read user_input_file
echo -n "enter line to add : "
read user_input_text
echo -n "enter line position : "
read user_input_line
In order to add line at the desired position you can "play" with head and tail
head -n $[$user_input_line - 1] $user_input_file > $new_file
echo $user_input_text >> $new_file
tail -n +$user_input_line $user_input_file >> $new_file
Requiring interactive input is horrible for automation. Make a command which accepts a message and a list of files to append to as command-line arguments instead.
#!/bin/sh
msg="$1"
shift
echo "$msg" | tee -a "$#"
Usage:
scriptname "today is a nice day" file1 file2 file3
The benefits for interactive use are obvious -- you get to use your shell's history mechanism and filename completion (usually bound to tab) but also it's much easier to build more complicated scripts on top of this one further on.
The design to put the message in the first command-line argument is baffling to newcomers, but allows for a very simple overall design where "the other arguments" (zero or more) are the files you want to manipulate. See how grep has this design, and sed, and many many other standard Unix commands.
You can use read statement to prompt for input,
read does make your script generic, but if you wish to automate it then you have to have an accompanying expect script to provide inputs to the read statement.
Instead you can take in arguments to the script which helps you in automation.. No prompting...
#!/usr/bin/env bash
[[ $# -ne 2 ]] && echo "print usage here" && exit 1
file=$1 && shift
con=$1
for i in `ls $file`
do
echo $con >> $i
done
To use:
./script.sh "<filename>" "<content>"
The quotes are important for the content so that the spaces in the content are considered to be part of it. For filenames use quotes so that the shell does not expand them before calling the script.
Example: ./script.sh "file*" "samdhaskdnf asdfjhasdf"
I am currently writing some code for a shell script that needs a blank line between two parts of the script, as thus they can be separated when output is displayed to the user.
My question is, I am not sure what the preferred practice is in a shell script for a blank line.
Is it preferred practice to just write echo and nothing else or to write echo " " as in echo with quotes and blank between the quotes?
echo is preferred. echo " " outputs an unnecessary space character. echo "" would be better, but it's unnecessary.
but you can use
echo -e "Hi \n"
and print a blank line after Hi, with -e interprets the \n character.
In its first implementation, echo had no option and outputs optional arguments ending with a new line, so it perfectly suit your needs.
For formatted outputs ending with a new line, printf is a better choice, for example : printf "%s\n\n" "output".
All of these commands can be used to echo a blank line:
echo, echo '', echo ""
We cant use echo "\n" or echo '\n' as they will give output as \n in both cases.
printf
More portable and succinct. This prints a hundred lines.
shell
printf '\n%.0s' `seq 1 100`
bash
printf '\n%.0s' {1,100}
As John suggested use echo. However if you want to print a blank line followed by text and anther blank line - as in running a test then use echo -e suggested by wyanzes.
echo -e "\n Now we are going to load data \n"
puts a blank line before and after.
in shell scripts I usually append a string to variable with "${variable} end". However, I have a file "file.txt" in which I want all lines to be appended by "end". So command line I do, for instance, for i in `cat file.txt`; do echo "${i} end"; done. But the word "end" (pluse the space) will not be appended but appended. The same thing happends when I use a while loop. Could anybody tell me what is going on right there? I am using GNU bash version 4.2.37 on LinuxMint13 64bit (both Cinammon and Mate).
Thank you for any help!
You should use a while loop instead of a for loop, as explained here.
while IFS= read -r line
do
echo "$line end"
done < "file.txt"
It may just be your syntax - don't forget do. That is:
for i in `cat file.txt`; do echo "${i} end"; done
If you're asking how to make a new file with "end" appended to each line, try this:
for i in `cat file.txt`; do echo "${i} end" >> some_new_file; done
Is using a loop the only option? If all you want to do is append something to the end of every line, it's probably easier to use sed:
sed -ie 's/.*/& end/' file.txt
I need a shell script to find and replace text that would go like:
For each line in a file
find equation mark "="
remove everything up to the equation mark on that line and replace it with the string cuts[Counter] where Counter counts how many times such substitutions have been made.
Could anybody help me out with getting started with a script like that?
Assuming you mean "up to the first equation mark..." and you want to keep the =, this should do it:
awk '{c += sub(/[^=]+=/,"cuts["c+0"]=") }1' file
In pure bash:
counts=0
while IFS= read -r line; do
if [[ "$line" =~ "=" ]]; then
echo "${counts}${line#*=}"
counts=$((counts+1))
fi
done <infile
Note that this will exclude the '='. You can reinclude it in the ehco statement if necessary.
Here is the perl one liner for that:
perl -plne 'if($.==1){$count=1}if(/=/){$_=~s/[^\=]*[=]/cut[$count]/g;$count++}' temp
Given a text file with multiple lines, I would like to iterate over each line in a Bash script. I had attempted to use cut, but cut does not accept \n (newline) as a delimiter.
This is an example of the file I am working with:
one
two
three
four
Does anyone know how I can loop through each line of this text file in Bash?
I found myself in the same problem, this works for me:
cat file.cut | cut -d$'\n' -f1
Or:
cut -d$'\n' -f1 file.cut
Use cat for concatenating or displaying. No need for it here.
file="/path/to/file"
while read line; do
echo "${line}"
done < "${file}"
Simply use:
echo -n `cut ...`
This suppresses the \n at the end
cat FILE|while read line; do # 'line' is the variable name
echo "$line" # do something here
done
or (see comment):
while read line; do # 'line' is the variable name
echo "$line" # do something here
done < FILE
So, some really good (possibly better) answers have been provided already. But looking at the phrasing of the original question, in wanting to use a BASH for-loop, it amazed me that nobody mentioned a solution with change of Field Separator IFS. It's a pure bash solution, just like the accepted read line
old_IFS=$IFS
IFS='\n'
for field in $(<filename)
do your_thing;
done
IFS=$old_IFS
If you are sure that the output will always be newline-delimited, use head -n 1 in lieu of cut -f1 (note that you mentioned a for loop in a script and your question was ultimately not script-related).
Many of the other answers, including the accepted one, have multiple lines unnecessarily. No need to do this over multiple lines or changing the default delimiter on the system.
Also, the solution provided by Ivan with -d$'\n' did not work for me either on Mac OSX or CentOS 7. Since his answer is four years old, I assume something must have changed on the logic of the $ character for this situation.
While loop with input redirection and read command.
You should not be using cut to perform a sequential iteration of each line in a file as cut was not designed to do this.
Print selected parts of lines from each FILE to standard output.
— man cut
TL;DR
You should use a while loop with the read -r command and redirect standard input to your file inside a function scope where IFS is set to \n and use -E when using echo.
processFile() { # Function scope to prevent overwriting IFS globally
file="$1" # Any file that exists
local IFS="\n" # Allows spaces and tabs
while read -r line; do # Read exits with 1 when done; -r allows \
echo -E "$line" # -E allows printing of \ instead of gibberish
done < $file # Input redirection allows us to read file from stdin
}
processFile /path/to/file
Iteration
In order to iterate over each line of a file, we can use a while loop. This will let us iterate as many times as we need to.
while <condition>; do
<body>
done
Getting our file ready to read
We can use the read command to store a single line from standard input in a variable. Before we can use that to read a line from our file, we need to redirect standard input to point to our file. We can do this with input redirection. According to the man pages for bash, the syntax for redirection is [fd]<file where fd defaults to standard input (a.k.a file descriptor 0). We can place this before or after our while loop.
while <condition>; do
<body>
done < /path/to/file
# or the non-traditional way
</path/to/file while <condition>; do
<body>
done
Reading the file and ending the loop
Now that our file can be read from standard input, we can use read. The syntax for read in our context is read [-r] var... where -r preserves the \ (backslash) character, instead of using it as an escape sequence character, and var is the name of the variable to store the input in. You can have multiple variables to store pieces of the input in but we only need one to read an entire line. Along with this, to preserve any backslashes in any output from echo you will likely need to use the -E flag to disable the interpretation of backslash escapes. If you have any indentation (spaces or tabs), you will need to temporarily change the IFS (Input Field Separators) variable to only "\n"; normally it is set to " \t\n".
main() {
local IFS="\n"
read -r line
echo -E "$line"
}
main
How do we use read to end our while loop?
There is really only one reliable way, that I know of, to determine when you've finished reading a file with read: check the exit value of read. If the exit value of read is 0 then we successfully read a line, if it is 1 or higher then we reached EOF (end of file). With that in mind, we can place the call to read in our while loop's condition section.
processFile() {
# Could be any file you want hardcoded or dynamic
file="$1"
local IFS="\n"
while read -r line; do
# Process line here
echo -E "$line"
done < $file
}
processFile /path/to/file1
processFile /path/to/file2
A visual breakdown of the above code via Explain Shell.
If I am executing a command and want to cut the output but it has multiple lines I found it helpful to do
echo $([command]) | cut [....]
This puts all the output of [command] on a single line that can be easier to process.
My opinion is that "cut" uses '\n' as its default delimiter.
If you want to use cut, I have two ways:
cut -d^M -f1 file_cut
I make ^M By click Enter After Ctrl+V. Another way is
cut -c 1- file_cut
Does that help?