I'm trying to use
however the lines starting with "#" are not ignored. I want to ignore the lines starting with "#" and make permanent changes to my file with "|" delimiter.
Any ideas how can i fix this or make this in one command?
You are passing the filename argument to the second sed and hence it would ignore the input coming in through the pipe. You can combine both statements with a ; inside sed expression:
sed -i "/^#/d;/$id/{s/[^|]*/$value/$column}" "$myfile"
As an aside, it is always better to enclose your variables in double quotes, "$myfile" in this case.
Related
I want to use grep to count the number of W occurrences in the file.txt, save it as the variable WATER_NUMBER. Then append that number to the end of file.txt.
Following here,
I tried
#!/bin/bash -l
WATER_NUMBER="$(grep -c W file.txt)"
sed -i -e '$a\"${WATER_NUMBER}"' file.txt
but I got "${WATER_NUMBER}" printed out, instead of the number. Can I ask how to modify it?
The command
sed -i '$a\"${WATER_NUMBER}"' file.txt
will simply add the line "${WATER_NUMBER}" at the end of the file. You could try
sed -i "$ a\$WATER_NUMBER" file.txt
but this will still add the line $WATER_NUMBER. The problem is that the variable WATER_NUMBER is not expanded in the sed script. In order to pass its value to sed, place it outside the quoting, like this
sed -i '$ a\'$WATER_NUMBER file.txt
Edit: I actually wrote my answer yesterday without really thinking about the reason as to why the variable is not expanded. This morning I wondered why this is the case even though the variable is in double quotes as opposed to single quotes. The reason is actually just the coincidence that the \ from the append command is in front of the $ from the variable, thus escaping it. To prevent this, you need to escape the \. On the other hand, a backslash is actually not needed to separate the a from the line you want to add, hence
sed -i "$ a $WATER_NUMBER" file.txt
will do the job.
How to save a number to a variable?
WATER_NUMBER=42
How to append the variable content to a file?
echo $WATER_NUMBER >> file.txt
I am aware that the "deleting n rows" and "deleting column x" questions have both been answered individually before. My current problem is that I'm writing my first bash script, and am having trouble making that script work the way I want it to.
file0001.csv (there are several hundred files like these in one folder)
Data number of lines 540
No.,Profile,Unit
1,1027.84,µm
2,1027.92,µm
3,1028,µm
4,1028.81,µm
Desired output
1,1027.84
2,1027.92
3,1028
4,1028.81
I am able to use sed and cut individually but for some reason the following bash script doesn't take cut into account. It also gives me an error "sed: can't read ls: No such file or directory", yet sed is successful and the output is saved to the original files.
sem2csv.sh
for files in 'ls *.csv' #list of all .csv files
do
sed '1,2d' -i $files | cut -f '1-2' -d ','
done
Actual output:
1,1027.84,µm
2,1027.92,µm
3,1028,µm
4,1028.81,µm
I know there may be awk one-liners but I would really like to understand why this particular bash script isn't running as intended. What am I missing?
The -i option of sed modifies the file in place. Your pipeline to cut receives no input because sed -i produces no output. Without this option, sed would write the results to standard output, instead of back to the file, and then your pipeline would work; but then you would have to take care of writing the results back to the original file yourself.
Moreover, single quotes inhibit expansion -- you are "looping" over the single literal string ls *.csv. The fact that you are not quoting it properly then causes the string to be subject to wildcard expansion inside the loop. So after variable interpolation, your sed command expands to
sed -i 1,2d ls *.csv
and then the shell expands *.csv because it is not quoted. (You should have been receiving a warning that there is no file named ls in the current directory, too.) You probably attempted to copy an example which used backticks (ASCII 96) instead of single quotes (ASCII 39) -- the difference is quite significant.
Anyway, the ls is useless -- the proper idiom is
for files in *.csv; do
sed '1,2d' "$files" ... # the double quotes here are important
done
Mixing sed and cut is usually not a good idea because you can express anything cut can do in terms of a simple sed script. So your entire script could be
for f in *.csv; do
sed -i -e '1,2d' -e 's/,[^,]*$//' "$f"
done
which says to remove the last comma and everything after it. (If your sed does not like multiple -e options, try with a semicolon separator: sed -i '1,2d;s/,[^,]*$//' "$f")
You may use awk,
$ awk 'NR>2{sub(/,[^,]*$/,"",$0);print}' file
1,1027.84
2,1027.92
3,1028
4,1028.81
or
sed -i '1,2d;s/,[^,]*$//' file
1,2d; for deleting the first two lines.
s/,[^,]*$// removes the last comma part in remaining lines.
I am having trouble using sed to substitute values and write to a new file. It writes to a new file, but fails to change any values. Here is my code:
cd/mydirectory
echo "Enter file name:"
read file_input
file1= "$file_input"
file1= "$file1.b"
file2= "$file_input"
file2= "${file2}Ins.b"
sed "/\!cats!/s/\!cats!.*/cats!300!/g $file1>$file2
I simply want to substitute whatever text was after cats with the value 300. Whenever I run this script it doesn't overwrite the previous value with 300. Any suggestions?
Try changing
sed "/\!cats!/s/\!cats!.*/cats!300!/g $file1>$file2
to
sed "s/cats.*/cats300/g" $file1 > $file2
To replace text, you often have to use sed like sed "s/foo/bar/g" file_in > file_out, to change all occurrences of foo with bar in file_in, redirecting the output to file_out.
Edit
I noticed that you are redirecting the output to the same file - you can't do that. You have 2 options:
Redirect the results to another file, with a different filename. e.g.:
sed "s/cats.*/cats300/g" $file1 > $file2.tmp
Note the .tmp after $file2
Use the -i flag (if using GNU sed):
sed -i "s/cats.*/cats300/g" $file1
The i stands for inline replacement.
I think this modified version of your script should work:
echo "Enter file name:"
read file_input
file1="$file_input" # No space after '='
file1="$file1.b" # No space after '='
file2="$file_input" # No space after '='
file2="${file2}Ins.b" # No space after '='
sed 's/!cats!.*/!cats!300!/g' "$file1" > "$file2"
Note the single quotes around sed expression: with them, there's no need to escape the !s in your expression. Note also the double quotes around "$file1" and "$file2": if one of those variables contain spaces, this will prevent your command from breaking.
Some further remarks:
As pointed by jim, you may want to use the GNU sed -i option.
Your regex will currently replace everything after !cats! in matching lines. If they were several occurences of !cats! on your line, only one will remain. If instead you just want to replace the value between two ! delimiters, you may consider use following sed command instead:
sed 's/!cats![^!]*/!cats!300/g'
I am trying to find a way to find all string between two patterns. This is easy:
cat file | sed -n "/pattern_start/,/pattern_end/p"
However, in this case I want to use a variable inside the sed script, which also is fine:
cat file | sed -n "/$var1/,/pattern_end/p"
However, if the variable contains special escape characters such as, '/' this does not work. Then I read that one could replace the escape character to anything, such as # or |
For example, lets say:
var1=/some/funny/path
cat file | sed -n "#$var1#,#pattern_end#p"
But this does not work for me. What am I doing wrong? I have tried to find the answer on Google etc but without success and I cant really find any other question here on Stackoverflow which deals with this exact problem.
You could do this:
sed -n "/$(sed 's#/#\\/#g' <<< "$var1")/,/end/p" file
You'll have to protect the contents of your variable from whatever delimiters you choose.
To address hek2mgl's concern, we could escape all potentially troublesome characters:
re_escape() {
sed 's#[^[:alnum:]_]#\\&#g' <<< "$*"
}
sed -n "/$(re_escape "$var1")/,/end/p" file
To change the delimiter, you need to escape the first one, i.e. something like, \!FIND!. Also, it is best to use single quotes, leave them when needed and use double quotes for variable interpolation. Then your command looks like this:
sed -n '\!'"$var1"'!,\!'"$var2"'!p' inpu
I want to pass this command in my script:
sed -n -e "/Next</a></p>/,/Next</a></p>/ p" file.txt
This command (should) extract all text between the two matched patterns, which are both Next</a></p> in my case. However when I run my script I keep getting errors. I've tried:
sed -n -e "/Next\<\/a\>\<\/p\>/,/Next<\/a\>\<\/p>/ p" file.txt with no luck.
I believe the generic pattern for this command is this:
sed -n -e "/pattern1/,/pattern2/ p" file.txt
I can't get it working for Next</a></p> though and I'm guessing it has something to do with the special characters I am encasing. Is there any way to pass Next</a></p> in the sed command? Thanks in advance guys! This community is awesome!
You don't need to use / as a regular expression delimiter. Using a different character will make quoting issues slightly easier. The syntax is
\cregexc
where c can be any character (other than \) that you don't use in the regex. In this case, : might be a good choice:
sed -n -e '\:Next</a></p>:,\:Next</a></p>: p' file.txt
Note that I changed " to ' because inside double quotes, \ will be interpreted by bash as an escape character, whereas inside single quotes \ is just treated as a regular character. Consequently, you could have written the version with escaped slashes like this:
sed -n -e '/Next<\/a><\/p>/,/Next<\/a><\/p>/ p' file.txt
but I think the version with colons is (slightly) easier to read.
You need to escape the forward slashes inside the regular expressions with a \, since the forward slashes serve as delimiters for the regexes
sed -n -e '/Next<\/a><\/p>/,/Next<\/a><\/p>/p' file.txt