I need a shell script to find and replace text that would go like:
For each line in a file
find equation mark "="
remove everything up to the equation mark on that line and replace it with the string cuts[Counter] where Counter counts how many times such substitutions have been made.
Could anybody help me out with getting started with a script like that?
Assuming you mean "up to the first equation mark..." and you want to keep the =, this should do it:
awk '{c += sub(/[^=]+=/,"cuts["c+0"]=") }1' file
In pure bash:
counts=0
while IFS= read -r line; do
if [[ "$line" =~ "=" ]]; then
echo "${counts}${line#*=}"
counts=$((counts+1))
fi
done <infile
Note that this will exclude the '='. You can reinclude it in the ehco statement if necessary.
Here is the perl one liner for that:
perl -plne 'if($.==1){$count=1}if(/=/){$_=~s/[^\=]*[=]/cut[$count]/g;$count++}' temp
Related
I'm having an issue in something that seems to be a rookie error, but I can't find a way to find a solution.
I have a bash script : log.sh
which is :
#!/bin/bash
echo $1 >> log_out.txt
And with a file made of filenames (taken from the output of "find" which names is filesnames.txt and contains 53 lines of absolute paths) I try :
./log.sh $(cat filenames.txt)
the only output I have in the log_out.txt is the first line.
I need each line to be processed separately as I need to put them in arguments in a pipeline with 2 softwares.
I checked for :
my lines being terminated with /n
using a simple echo without writing to a file
all the sorts of cat filenames.txt or (< filenames.txt) found on internet
I'm sure it's a very dumb thing, but I can't find why I can't iterate more than one line :(
Thanks
It is because your ./log.sh $(cat filenames.txt) is being treated as one argument.
while IFS= read -r line; do
echo "$line";
done < filenames.txt
Edit according to: https://mywiki.wooledge.org/DontReadLinesWithFor
Edit#2:
To preserve leading and trailing whitespace in the result, set IFS to the null string.
You could simplify more and skip using explicit variable and use the default $REPLY
Source: http://wiki.bash-hackers.org/commands/builtin/read
You need to quote the command substitution. Otherwise $1 will just be the first word in the file.
./log.sh "$(cat filenames.txt)"
You should also quote the variable in the script, otherwise all the newlines will be converted to spaces.
echo "$1" >> log_out.txt
If you want to process each word separately, you can leave out the quotes
./log.sh $(cat filenames.txt)
and then use a loop in the script:
#!/bin/bash
for word in "$#"
do
echo "$word"
done >> log_out.txt
Note that this solution only works correctly when the file has one word per line and there are no wildcards in the words. See mywiki.wooledge.org/DontReadLinesWithFor for why this doesn't generalize to more complex lines.
You can iterate with each line.
#!/bin/bash
for i in $*
do
echo $i >> log_out.txt
done
I'm new to UNIX and have this really simple problem:
I have a text-file (input.txt) containing a string in each line. It looks like this:
House
Monkey
Car
And inside my shell script I need to read this input file line by line to get to a variable like this:
things="House,Monkey,Car"
I know this sounds easy, but I just couldnt find any simple solution for this. My closest attempt so far:
#!/bin/sh
things=""
addToString() {
things="${things},$1"
}
while read line; do addToString $line ;done <input.txt
echo $things
But this won't work. Regarding to my google research I thought the while loop would create a new sub shell, but this I was wrong there (see the comment section). Nevertheless the variable "things" was still not available in the echo later on. (I cannot just write the echo inside the while loop, because I need to work with that string later on)
Could you please help me out here? Any help will be appreciated, thank you!
What you proposed works fine! I've only made two changes here: Adding missing quotes, and handling the empty-string case.
things=""
addToString() {
if [ -n "$things" ]; then
things="${things},$1"
else
things="$1"
fi
}
while read -r line; do addToString "$line"; done <input.txt
echo "$things"
If you were piping into while read, this would create a subshell, and that would eat your variables. You aren't piping -- you're doing a <input.txt redirection. No subshell, code works without changes.
That said, there are better ways to read lists of items into shell variables. On any version of bash after 3.0:
IFS=$'\n' read -r -d '' -a things <input.txt # read into an array
printf -v things_str '%s,' "${things[#]}" # write array to a comma-separated string
echo "${things_str%,}" # print that string w/o trailing comma
...on bash 4, that first line can be:
readarray -t things <input.txt # read into an array
This is not a shell solution, but the truth is that solutions in pure shell are often excessively long and verbose. So e.g. to do string processing it is better to use special tools that are part of the “default” Unix environment.
sed ':b;N;$!bb;s/\n/,/g' < input.txt
If you want to omit empty lines, then:
sed ':b;N;$!bb;s/\n\n*/,/g' < input.txt
Speaking about your solution, it should work, but you should really always use quotes where applicable. E.g. this works for me:
things=""
while read line; do things="$things,$line"; done < input.txt
echo "$things"
(Of course, there is an issue with this code, as it outputs a leading comma. If you want to skip empty lines, just add an if check.)
This might/might not work, depending on the shell you are using. On my Ubuntu 14.04/x64, it works with both bash and dash.
To make it more reliable and independent from the shell's behavior, you can try to put the whole block into a subshell explicitly, using the (). For example:
(
things=""
addToString() {
things="${things},$1"
}
while read line; do addToString $line ;done
echo $things
) < input.txt
P.S. You can use something like this to avoid the initial comma. Without bash extensions (using short-circuit logical operators instead of the if for shortness):
test -z "$things" && things="$1" || things="${things},${1}"
Or with bash extensions:
things="${things}${things:+,}${1}"
P.P.S. How I would have done it:
tr '\n' ',' < input.txt | sed 's!,$!\n!'
You can do this too:
#!/bin/bash
while read -r i
do
[[ $things == "" ]] && things="$i" || things="$things","$i"
done < <(grep . input.txt)
echo "$things"
Output:
House,Monkey,Car
N.B:
Used grep to tackle with empty lines and the probability of not having a new line at the end of file. (Normal while read will fail to read the last line if there is no newline at the end of file.)
Can someone please help with this because I can't seem to find a solution. I have the following script that works fine:
#!/bin/bash
#Checks the number of lines in the userdomains file
NUM=`awk 'END {print NR}' /etc/userdomains.hristian`;
echo $NUM
#Prints out a particular line from the file (should work with $NUM eventually)
USER=`sed -n 4p /etc/userdomains.hristian`
echo $USER
#Edits the output so that only the username is left
USER2=`echo $USER | awk '{print $NF}'`
echo $USER2
However, when I substitute the 4 on line 12 with the variable $NUM, like this, it doesn't work:
USER=`sed -n $NUMp /etc/userdomains.hristian`
I tried a number of different combinations of quotes and ${}, however none of them seem to work because I'm a BASH newbie. Help please :)
I'm not sure exactly what you've already tried but this works for me:
$ cat out
line 1
line 2
line 3
line 4
line 5
$ num=4
$ a=`sed -n ${num}p out`
$ echo "$a"
line 4
To be clear, the issue here is that you need to separate the expansion of $num from the p in the sed command. That's what the curly braces do.
Note that I'm using lowercase variable names. Uppercase ones should be be reserved for use by the shell. I would also recommend using the more modern $() syntax for command substitution:
a=$(sed -n "${num}p" out)
The double quotes around the sed command aren't necessary but they don't do any harm. In general, it's a good idea to use them around expansions.
Presumably the script in your question is a learning exercise, which is why you have done all of the steps separately. For the record, you could do the whole thing in one go like this:
awk 'END { print $NF }' /etc/userdomains.hristian
In the END block, the values from the last line in the file can still be accessed, so you can print the last field directly.
Your trying to evaluate the variable $NUMp rather than $NUM. Try this instead:
USER=`sed -n ${NUM}p /etc/userdomains.hristian`
I am trying to check the ends of code for semi-colons as they are causing me some issues for a server I have running. To do this I am using a bash script (as I am more familiar with bash) to read through the lines and return those that doesn't end with a semi-colon. My bash script is as follows
while read line
do
if[$line!=*;]
echo $line
fi
done < $1
When I run the script, it says there is an error by fi but I cannot figure it out. I also realize this will return statements like if and while but that will be fine for my needs.
Given the sample input
use CGI;
print "<html>"
print "<head>";
print "</head>";
print "<body><p> HELLO WORLD </p>";
print "</body>";
print "</html>"
this should be the output
print "<html>"
print "</html>"
I think the easiest way would be with grep. Given an input.txt file like this:
spam
foo;<Space><Space>
sausage
baked;<Tab>
beans
unladen;
You could do
grep -v ';\s*$' input.txt
and obtain
spam
sausage
beans
grep's -v flag means "return all lines not matching this regular expression", so it will skip all lines ending with semi-colons.
If your lines have also spaces after the semi-colons, the \s* means "any sequence of space characters" so grep will remove those lines aswell.
The reason you have a problem is that your if statement requires a then. You also need some more spaces and to quote your variables. It still won't work, though. Your comparison is wrong, too - that's not how [ works to compare strings. You can use bash's [[ instead:
while read line
do
if [[ "$line" != *\; ]]
then
echo "$line"
fi
done < $1
But even with all that, what you really should be doing is:
grep -v ';$' $1
To search lines with semicolon ; and without semicolon in a linux file
$ grep ';' file_name
$ grep -v ';\s*$' file_name
in shell scripts I usually append a string to variable with "${variable} end". However, I have a file "file.txt" in which I want all lines to be appended by "end". So command line I do, for instance, for i in `cat file.txt`; do echo "${i} end"; done. But the word "end" (pluse the space) will not be appended but appended. The same thing happends when I use a while loop. Could anybody tell me what is going on right there? I am using GNU bash version 4.2.37 on LinuxMint13 64bit (both Cinammon and Mate).
Thank you for any help!
You should use a while loop instead of a for loop, as explained here.
while IFS= read -r line
do
echo "$line end"
done < "file.txt"
It may just be your syntax - don't forget do. That is:
for i in `cat file.txt`; do echo "${i} end"; done
If you're asking how to make a new file with "end" appended to each line, try this:
for i in `cat file.txt`; do echo "${i} end" >> some_new_file; done
Is using a loop the only option? If all you want to do is append something to the end of every line, it's probably easier to use sed:
sed -ie 's/.*/& end/' file.txt