I want to use grep to count the number of W occurrences in the file.txt, save it as the variable WATER_NUMBER. Then append that number to the end of file.txt.
Following here,
I tried
#!/bin/bash -l
WATER_NUMBER="$(grep -c W file.txt)"
sed -i -e '$a\"${WATER_NUMBER}"' file.txt
but I got "${WATER_NUMBER}" printed out, instead of the number. Can I ask how to modify it?
The command
sed -i '$a\"${WATER_NUMBER}"' file.txt
will simply add the line "${WATER_NUMBER}" at the end of the file. You could try
sed -i "$ a\$WATER_NUMBER" file.txt
but this will still add the line $WATER_NUMBER. The problem is that the variable WATER_NUMBER is not expanded in the sed script. In order to pass its value to sed, place it outside the quoting, like this
sed -i '$ a\'$WATER_NUMBER file.txt
Edit: I actually wrote my answer yesterday without really thinking about the reason as to why the variable is not expanded. This morning I wondered why this is the case even though the variable is in double quotes as opposed to single quotes. The reason is actually just the coincidence that the \ from the append command is in front of the $ from the variable, thus escaping it. To prevent this, you need to escape the \. On the other hand, a backslash is actually not needed to separate the a from the line you want to add, hence
sed -i "$ a $WATER_NUMBER" file.txt
will do the job.
How to save a number to a variable?
WATER_NUMBER=42
How to append the variable content to a file?
echo $WATER_NUMBER >> file.txt
Related
I'm trying to use
however the lines starting with "#" are not ignored. I want to ignore the lines starting with "#" and make permanent changes to my file with "|" delimiter.
Any ideas how can i fix this or make this in one command?
You are passing the filename argument to the second sed and hence it would ignore the input coming in through the pipe. You can combine both statements with a ; inside sed expression:
sed -i "/^#/d;/$id/{s/[^|]*/$value/$column}" "$myfile"
As an aside, it is always better to enclose your variables in double quotes, "$myfile" in this case.
I am aware that the "deleting n rows" and "deleting column x" questions have both been answered individually before. My current problem is that I'm writing my first bash script, and am having trouble making that script work the way I want it to.
file0001.csv (there are several hundred files like these in one folder)
Data number of lines 540
No.,Profile,Unit
1,1027.84,µm
2,1027.92,µm
3,1028,µm
4,1028.81,µm
Desired output
1,1027.84
2,1027.92
3,1028
4,1028.81
I am able to use sed and cut individually but for some reason the following bash script doesn't take cut into account. It also gives me an error "sed: can't read ls: No such file or directory", yet sed is successful and the output is saved to the original files.
sem2csv.sh
for files in 'ls *.csv' #list of all .csv files
do
sed '1,2d' -i $files | cut -f '1-2' -d ','
done
Actual output:
1,1027.84,µm
2,1027.92,µm
3,1028,µm
4,1028.81,µm
I know there may be awk one-liners but I would really like to understand why this particular bash script isn't running as intended. What am I missing?
The -i option of sed modifies the file in place. Your pipeline to cut receives no input because sed -i produces no output. Without this option, sed would write the results to standard output, instead of back to the file, and then your pipeline would work; but then you would have to take care of writing the results back to the original file yourself.
Moreover, single quotes inhibit expansion -- you are "looping" over the single literal string ls *.csv. The fact that you are not quoting it properly then causes the string to be subject to wildcard expansion inside the loop. So after variable interpolation, your sed command expands to
sed -i 1,2d ls *.csv
and then the shell expands *.csv because it is not quoted. (You should have been receiving a warning that there is no file named ls in the current directory, too.) You probably attempted to copy an example which used backticks (ASCII 96) instead of single quotes (ASCII 39) -- the difference is quite significant.
Anyway, the ls is useless -- the proper idiom is
for files in *.csv; do
sed '1,2d' "$files" ... # the double quotes here are important
done
Mixing sed and cut is usually not a good idea because you can express anything cut can do in terms of a simple sed script. So your entire script could be
for f in *.csv; do
sed -i -e '1,2d' -e 's/,[^,]*$//' "$f"
done
which says to remove the last comma and everything after it. (If your sed does not like multiple -e options, try with a semicolon separator: sed -i '1,2d;s/,[^,]*$//' "$f")
You may use awk,
$ awk 'NR>2{sub(/,[^,]*$/,"",$0);print}' file
1,1027.84
2,1027.92
3,1028
4,1028.81
or
sed -i '1,2d;s/,[^,]*$//' file
1,2d; for deleting the first two lines.
s/,[^,]*$// removes the last comma part in remaining lines.
If I want to add content to the 10th line of a file, how do I do it?
This is what I came up with:
sed -ie '' "10s/^/new_content/g" file.txt
I keep getting the message "no such file or directory"
Also, if I want to replace 10 with N+1 and the new_content with a variable $VAR, would the format still be the same?
VAR= $(cat fileA.txt)
sed -ie '' "`expr $N +1`s/^/$VAR/g" fileB.txt
Thanks for your help!!!
Shell is fussy about spacing:
VAR= $(cat fileA.txt)
You do not want the space after the =. Actually, you don't want the assignment either, unless fileA.txt has only one line. You'd have to escape newlines with backslashes, etc, to get it to work, which is both hard and unnecessary.
Unless you're using an antique (non-POSIX) shell, use built-in arithmetic:
sed -i.bak -e "$(($N + 1))r fileA.txt" fileB.txt
The $(($N + 1)) has the shell evaluate the arithmetic expression. The r has sed read the file that is named.
[1addr]r file
Copy the contents of file to the standard output immediately before the next attempt to read a line of input. If file cannot be read for any reason, it is silently ignored and no error condition is set.
It turns out I needed to split -i and -e.
The final result that works was:
sed -i '' -e "10s/^/CONTENT/g" file.txt
Or to increase Ns by +1 each time it loops:
sed -i '' -e "$((N + 1))s/^/CONTENT/g" ~/dance/heatlist/prep.txt
I still have not figured out how to make CONTENT a variable $CONTENT
I have a bash script which checks for a string pattern in file and delete entire line i same file but somehow its not deleting the line and no throwing any error .same command from command prompt deletes from file .
#array has patterns
for k in "${patternarr[#]}
do
sed -i '/$k/d' file.txt
done
sed version is >4
when this loop completes i want all lines matching string pattern in array to be deleted from file.txt
when i run sed -i '/pataern/d file.txt from command prompt then it works fine but not inside bash
Thanks in advance
Here:
sed -i '/$k/d' file.txt
The sed script is singly-quoted, which prevents shell variable expansion. It will (probably) work with
sed -i "/$k/d" file.txt
I say "probably" because what it will do depends on the contents of $k, which is just substituted into the sed code and interpreted as such. If $k contains slashes, it will break. If it comes from an untrustworthy source, you open yourself up to code injection (particularly with GNU sed, which can be made to execute shell commands).
Consider k=^/ s/^/rm -Rf \//e; #.
It is generally a bad idea to substitute shell variables into sed code (or any other code). A better way would be with GNU awk:
awk -i inplace -v pattern="$k" '!($0 ~ pattern)' file.txt
Or to just use grep -v and a temporary file.
first of all, you got an unclosed double quote around ${patternarr[#]} in your for statement.
Then your problem is that you use single quotes in the sed argument, making your shell not evaluate the $k within the quotes:
% declare -a patternarr=(foo bar fu foobar)
% for k in ${patternarr[#]}; do echo sed -i '/$k/d' file.txt; done
sed -i /$k/d file.txt
sed -i /$k/d file.txt
sed -i /$k/d file.txt
sed -i /$k/d file.txt
if you replace them with double quotes, here it goes:
% for k in ${patternarr[#]}; do echo sed -i "/$k/d" file.txt; done
sed -i /foo/d file.txt
sed -i /bar/d file.txt
sed -i /fu/d file.txt
sed -i /foobar/d file.txt
Any time you write a loop in shell just to manipulate text you have the wrong approach. This is probably closer to what you really should be doing (no surrounding loop required):
awk -v ks="${patternarr[#]}" 'BEGIN{gsub(/ /,")|(",ks); ks="("ks")} $0 !~ ks' file.txt
but there may be even better approaches still (e.g. only checking 1 field instead of the whole line, or using word boundaries, or string comparison or....) if you show us some sample input and expected output.
You need to use double quotes to interpolate shell variables inside the sed command, like:
for k in ${patternarr[#]}; do
sed -i "/$k/d" file.txt
done
I am having trouble using sed to substitute values and write to a new file. It writes to a new file, but fails to change any values. Here is my code:
cd/mydirectory
echo "Enter file name:"
read file_input
file1= "$file_input"
file1= "$file1.b"
file2= "$file_input"
file2= "${file2}Ins.b"
sed "/\!cats!/s/\!cats!.*/cats!300!/g $file1>$file2
I simply want to substitute whatever text was after cats with the value 300. Whenever I run this script it doesn't overwrite the previous value with 300. Any suggestions?
Try changing
sed "/\!cats!/s/\!cats!.*/cats!300!/g $file1>$file2
to
sed "s/cats.*/cats300/g" $file1 > $file2
To replace text, you often have to use sed like sed "s/foo/bar/g" file_in > file_out, to change all occurrences of foo with bar in file_in, redirecting the output to file_out.
Edit
I noticed that you are redirecting the output to the same file - you can't do that. You have 2 options:
Redirect the results to another file, with a different filename. e.g.:
sed "s/cats.*/cats300/g" $file1 > $file2.tmp
Note the .tmp after $file2
Use the -i flag (if using GNU sed):
sed -i "s/cats.*/cats300/g" $file1
The i stands for inline replacement.
I think this modified version of your script should work:
echo "Enter file name:"
read file_input
file1="$file_input" # No space after '='
file1="$file1.b" # No space after '='
file2="$file_input" # No space after '='
file2="${file2}Ins.b" # No space after '='
sed 's/!cats!.*/!cats!300!/g' "$file1" > "$file2"
Note the single quotes around sed expression: with them, there's no need to escape the !s in your expression. Note also the double quotes around "$file1" and "$file2": if one of those variables contain spaces, this will prevent your command from breaking.
Some further remarks:
As pointed by jim, you may want to use the GNU sed -i option.
Your regex will currently replace everything after !cats! in matching lines. If they were several occurences of !cats! on your line, only one will remain. If instead you just want to replace the value between two ! delimiters, you may consider use following sed command instead:
sed 's/!cats![^!]*/!cats!300/g'