How to prevent bash script from putting all output into one line? - bash

I have the following bash script top_script.sh
#!/bin/bash
# "Usage: $0 JOBNAME LOGFILE"
JOBNAME=$1
LOGFILE=$2
JOB_OUTPUT=$($1 2>&1)
echo ${JOB_OUTPUT} >> "${LOGFILE}"
that is supposed to be invoked like this
top_script.sh script_to_run.sh log.txt
If script_to_run.sh has multiple echo statements, e.g.
echo Line 1 from $0
echo Line 2 from $0
then what I get in log.txt is
Line 1 from script_to_run.sh Line 2 from script_to_run.sh
i.e. all this output gets concatenated into a single line. I suspect that the reason is line #5 in the first code block above. How can I modify it to ensure separate echos print into separate lines in log.txt?
Not that it really matters, but in case you are wondering, top_script.sh gets generated automatically form some a config file.

echo "${JOB_OUTPUT}" >> "${LOGFILE}"

Related

Is there a way to extract the first line from stderr and store it to a variable in the shell script executing the function?

Say I have a command I'm running in my script whose first line in stderr is something I need. I'm using stderr because stdout is already being used for transferring some other data. I still need the rest of stderr for user feedback, so I still want to display everything after the first line.
cmd() {
ssh usr#remote.machine.com '
printf "stderr data line 1 (important)\n" 1>&2
printf "stdout data line 1\n"
printf "stderr data line 2\n" 1>&2
printf "stdout data line 2\n"
printf "stdout data line 3\n"
printf "stderr data line 3\n" 1>&2'
}
# What sort of shell magic would I need to extract
# only the 1st line of stderr?
cmd > store_stdout_to_this_file ??? read -a first_line_of_stderr
echo "$first_line_of_stderr"
I can't use a pipe, as pipes only pipe stdout, and even if I were to rearrange them, then the other end of the pipe is in a different process space.
I see several methods to do this, all with different limitations and oddities. Probably the simplest is to redirect stderr to a background subshell (via process substitution) that runs read once, then cat:
cmd >outputfile 2> >(read firstline; echo "First line is: '$firstline'"; cat -u)
But that puts the first-line processing in a subshell, so any variables it sets will be lost when that shell exits. I suppose you could have it write the line to another file, and then read that from the main script afterward...
Another possibility is to put the command you're trying to capture in the process substitution, and read and cat in the main shell:
{ read firstline; cat -u; } < <(cmd 2>&1 >outputfile)
echo "First line is: '$firstline'"
Note that the output redirection from cmd must be done inside the process-substituted part (since its stdout is being sent to the read-cat part), and the 2>&1 must be first so stderr doesn't go to the same place.
You can do like this:
line=
while read -r; do
[[ -z $line ]] && line="$REPLY" || echo "$REPLY"
done < <(cmd 2>&1 >out.log)
stderr data line 2
stderr data line 3
# check $line
echo "$line"
stderr data line 1 (important)
i.e. redirect stderr to stdout first and then redirect stdout to out.log and finally pipe output using head -n 1.

Store multiline eval output in variable bash

Can someone help me to store multiline output in variable in bash. I have the following code:
# FILES[1] contents:
# age people.csv
# ...data...
FILE="${FILES[1]}"
cmd=$(head -n 1 "$FILE")
cmd="./corona $cmd"
echo "Command to run: ${cmd[*]}"
output=$(eval "$cmd")
echo "$output"
I'm trying to store the output of corona script in output variable. But it doesn't seem to work. The output stucks at
Command to run: ./corona age people.csv
And on the second line I can see only the blinking cursor. But when I press Ctrl+D it suddenly prints all the output from corona script and stops. So, probably, the echo command works just after pressing the shortcut.
Also, I'd like to mention, that variable FILES is an array of filenames. So the FILE variable is a name of the file. It has command arguments to run on the first line and other data starting from the second line.
Here is a sample script I developed to read the output of ls into a variable. You could use the same technique.
#!/bin/bash
my_array=()
while IFS= read -r line; do
my_array+=( "$line" )
done < <( ls )
echo ${#my_array[#]}
printf '%s\n' "${my_array[#]}"

Does echo command automatically add a new line character to the end of the input string?

So I was just starting learning bash scripting. I encountered a question in a book.
An example testfile contains following content.
$ cat testfile
This is the first line.
This is the second line.
This is the third line.
And the script file is like:
#!/bin/bash
# testing input/output file descriptor
exec 3<> testfile
read line <&3
echo "Read: $line"
echo "This is a test line" >&3
After running the script, the testfile became:
$ cat testfile
This is the first line.
This is a test line
ine.
This is the third line.
I understand why that script changes the testfile. My question is why
"ine." starts from a new line? Does echo command automatically add a newline character to the end of the string?
echo -n is what you seek: the option -n
instructs echo to "do not output the trailing newline".
FWIW: man echo on your platform will instruct what options the /bin/echo command understands. But since you mention bash as shell: bash has an internal implementation of echo (a so-called "builtin")

How can I write new line - Echo command

I have a log file and simple bash script:
echo -e "$date $totalsize $dupsize $per" > log.txt
But when I execute this script, it write first line of log.txt and show me just last data.
I tried \n but does not working.
How can I write to new line when I execute this script?
I assume what you mean is that you want to append to the log.txt? In that case, use:
echo -e "$date $totalsize $dupsize $per" >> log.txt
Note the >> which means append.
Using > will create a new file (overwrite it) each time it's run, containing whatever you echo to it.

Creating files with some content with shell script

I need to configure a server with a few files and I want to do it programmatically.
I need to create files say /home/a.config, /var/spool/b.config, /etc/c.config
Files above have some contents (multi lines).
I want to create ONE shell script which can create all three file with multiple lines (around 10).
I would like to know the how can I use CAT command to do that. (inside shell script).
I am looking something like this
echo " going to create /home/a.config"
cat "HOW CAN I HAVE MULTIPLE LINES HERE?? " > /home/a.config
thanks
You can use a here document:
cat <<EOF >filename
first line
second line
third line
EOF
You can place several of these in the same script.
file="/tmp/test.txt"
echo "Adding first line" > $file
echo "Adding first line replaced" > $file
echo "Appending second line " >> $file
echo "Appending third line" >> $file
cat $file
> to add/replace the content ( here actual content got replaced by the 2nd line)
>> to append
Result
Adding first line replaced
Appending second line
Appending third line
Like so:
#!/bin/bash
var="your text"
echo "simply put,
just so: $var" > a.config
For further info, see Input/Output part of abs.
Hope, this helps.
If you've got variables like $1 or $HOMEDIR in your text then these normally get evaluated and substituted with actual values. If you want to prevent these from getting substituted then you need to quote the opening limit string (EOF in example below) with single quote 'EOF', double quote "EOF" or precede it with backslash \EOF
Closing limit string stays as is. EOF
This is especially useful if you are writing shell scripts to a file.
cat << 'EOF' >/etc/rc.d/init.d/startup
case $1 in
start)
start
;;
stop)
stop
;;
restart)
stop
start
;;
status)
pid=$(tomcat_pid)
if [ -n "$pid" ]
then
echo "Tomcat is running with pid: $pid"
else
echo "Tomcat is not running"
fi
;;
esac
EOF
Refer Example 19.7 Parameter Substitution Turned off in Here Documents
>\#!/bin/bash
>
>var="your text" <br>
>echo "simply put, <br>
>just so: $var" > a.config
Note that you also need to escape out certain characters to avoid them interfering with what you're trying to do, for example $ ` and " will all break such a statement unless preceded with a backslash, i.e. \` \$ or \"
so if we define the following:
var="100"
the following would not behave as expected:
echo "simply put,
just "lend" me US$ $var" > a.config
but the following would work correctly:
echo "simply put,
just \"lend\" me US\$ $var" > a.config

Resources