I've been looking around for a while on this and can't seem to find a solution on how to use sed to do this. I have a file that is named:
FILE_772829345_D594_242_25kd_kljd.mov
that I want to be renamed
D594_242_25kd_kljd.mov
I currently have been trying to get sed to work for this but have only been able to remove the first second of the file:
echo 'FILE_772829345_D594_242_25kd_kljd.mov' | sed 's/[^_]*//'
_772829345_D594_242_25kd_kljd.mov
How would I get sed to do the same instruction again, up to the second underscore?
If the filename is in a shell variable, you don't even need to use sed, just use a shell expansion with # to trim through the second underscore:
filename="FILE_772829345_D594_242_25kd_kljd.mov"
echo "${filename#*_*_}" # prints "D594_242_25kd_kljd.mov"
BTW, if you're going to use mv to rename the file, use its -i option to avoid file getting overwritten if there are any name conflicts:
mv -i "$filename" "${filename#*_*_}"
If all your files are named similarly, you can use cut which would be a lot simpler than sed with a regex:
cut -f3- -d_ <<< "FILE_772829345_D594_242_25kd_kljd.mov"
Output:
D594_242_25kd_kljd.mov
Related
I am aware that the "deleting n rows" and "deleting column x" questions have both been answered individually before. My current problem is that I'm writing my first bash script, and am having trouble making that script work the way I want it to.
file0001.csv (there are several hundred files like these in one folder)
Data number of lines 540
No.,Profile,Unit
1,1027.84,µm
2,1027.92,µm
3,1028,µm
4,1028.81,µm
Desired output
1,1027.84
2,1027.92
3,1028
4,1028.81
I am able to use sed and cut individually but for some reason the following bash script doesn't take cut into account. It also gives me an error "sed: can't read ls: No such file or directory", yet sed is successful and the output is saved to the original files.
sem2csv.sh
for files in 'ls *.csv' #list of all .csv files
do
sed '1,2d' -i $files | cut -f '1-2' -d ','
done
Actual output:
1,1027.84,µm
2,1027.92,µm
3,1028,µm
4,1028.81,µm
I know there may be awk one-liners but I would really like to understand why this particular bash script isn't running as intended. What am I missing?
The -i option of sed modifies the file in place. Your pipeline to cut receives no input because sed -i produces no output. Without this option, sed would write the results to standard output, instead of back to the file, and then your pipeline would work; but then you would have to take care of writing the results back to the original file yourself.
Moreover, single quotes inhibit expansion -- you are "looping" over the single literal string ls *.csv. The fact that you are not quoting it properly then causes the string to be subject to wildcard expansion inside the loop. So after variable interpolation, your sed command expands to
sed -i 1,2d ls *.csv
and then the shell expands *.csv because it is not quoted. (You should have been receiving a warning that there is no file named ls in the current directory, too.) You probably attempted to copy an example which used backticks (ASCII 96) instead of single quotes (ASCII 39) -- the difference is quite significant.
Anyway, the ls is useless -- the proper idiom is
for files in *.csv; do
sed '1,2d' "$files" ... # the double quotes here are important
done
Mixing sed and cut is usually not a good idea because you can express anything cut can do in terms of a simple sed script. So your entire script could be
for f in *.csv; do
sed -i -e '1,2d' -e 's/,[^,]*$//' "$f"
done
which says to remove the last comma and everything after it. (If your sed does not like multiple -e options, try with a semicolon separator: sed -i '1,2d;s/,[^,]*$//' "$f")
You may use awk,
$ awk 'NR>2{sub(/,[^,]*$/,"",$0);print}' file
1,1027.84
2,1027.92
3,1028
4,1028.81
or
sed -i '1,2d;s/,[^,]*$//' file
1,2d; for deleting the first two lines.
s/,[^,]*$// removes the last comma part in remaining lines.
My script is as follow (variables are defined above by user input):
grep -RlI $OLD $PATH > $LIST
while read line
do
FILE=echo $line
sed -i '' -e 's|$OLD|$NEW|g' $FILE
done < $LIST
It seems to work except that sed fails because
"sed: -i may not be used with stdin"
What am I doing wrong? Maybe that's the wrong approach for what I am trying to do?
(which, by the way, is to replace occurrences of a string in many files, AND to create a file that lists all files that contain a match.)
Many thanks,
C
Try replacing
FILE=echo $line
with
FILE="$line"
sed is complaining because the $FILE variable doesn't contain anything, or just contains whitespace. Examine the contents of the file referenced by $LIST; make sure there are no empty lines or lines with just whitespace.
sed -i -r 's/\$[[:alnum:]]{32}-[[:digit:]]{8}\$[[:alnum:]+\.\_\-]{2,3}#[[:alnum:]+\.\_\-]*/****/' *.log
my variant to replace data like $1BC29B36F623BA82AAF6724FD3B16718-17082022$2sy#domain4.name with *****
I'm trying to find a way using variables and sed to do a specific text substitution using a changing input file, but only if there is a value given to replace the existing string with. No value= do nothing (rather than remove the existing string).
Example
Substitute.csv contains 5 lines=
this-has-text
this-has-text
this-has-text
this-has-text
and file.text has one sentence=
"When trying this I want to be sure that text-this-has is left alone."
If I run the following command in a shell script
Text='text-this-has'
Change=`sed -n '3p' substitute.csv`
grep -rl $Text /home/username/file.txt | xargs sed -i "s|$Text|$Change|"
I end up with
"When trying this I want to be sure that is left alone."
But I'd like it to remain as
"When trying this I want to be sure that text-this-has is left alone."
Any way to tell sed "If I give you nothing new, do nothing"?
I apologize for the overthinking, bad habit. Essentially what I'd like to accomplish is if line 3 of the csv file has a value - replace $Text with $Change inline. If the line is empty, leave $Text as $Text.
Text='text-this-has'
Change=$(sed -n '3p' substitute.csv)
if [[ -n $Change ]]; then
grep -rl $Text /home/username/file.txt | xargs sed -i "s|$Text|$Change|"
fi
Just keep it simple and use awk:
awk -v t="$Text" -v c="$Change" 'c!=""{sub(t,c)} {print}' file
If you need inplace editing just use GNU awk with -i inplace.
Given your clarified requirement, this is probably what you actually want:
awk -v t="$Text" 'NR==FNR{if (NR==3) c=$0; next} c!=""{sub(t,c)} {print}' Substitute.csv file.txt
Testing whether $Change has a value before launching into the grep and sed is undoubtedly the most efficient bash solution, although I'm a bit skeptical about the duplication of grep and sed; it saves a temporary file in the case of files which don't contain the target string, but at the cost of an extra scan up to the match in the case of files which do contain it.
If you're looking for typing efficiency, though, the following might be interesting:
find . -name '*.txt' -exec sed -i "s|$Text|${Change:-&}|" {} \;
Which will recursively find all files whose names end with the extension .txt and execute the sed command on each one. ${Change:-&} means "the value of $Change if it exists and is non-empty, and otherwise an &"; & in the replacement of a sed s command means "the matched text", so s|foo|&| replaces every occurrence of foo with itself. That's an expensive no-op but if your time matters more than your cpu time, it might have been worth it.
I am having trouble using sed to substitute values and write to a new file. It writes to a new file, but fails to change any values. Here is my code:
cd/mydirectory
echo "Enter file name:"
read file_input
file1= "$file_input"
file1= "$file1.b"
file2= "$file_input"
file2= "${file2}Ins.b"
sed "/\!cats!/s/\!cats!.*/cats!300!/g $file1>$file2
I simply want to substitute whatever text was after cats with the value 300. Whenever I run this script it doesn't overwrite the previous value with 300. Any suggestions?
Try changing
sed "/\!cats!/s/\!cats!.*/cats!300!/g $file1>$file2
to
sed "s/cats.*/cats300/g" $file1 > $file2
To replace text, you often have to use sed like sed "s/foo/bar/g" file_in > file_out, to change all occurrences of foo with bar in file_in, redirecting the output to file_out.
Edit
I noticed that you are redirecting the output to the same file - you can't do that. You have 2 options:
Redirect the results to another file, with a different filename. e.g.:
sed "s/cats.*/cats300/g" $file1 > $file2.tmp
Note the .tmp after $file2
Use the -i flag (if using GNU sed):
sed -i "s/cats.*/cats300/g" $file1
The i stands for inline replacement.
I think this modified version of your script should work:
echo "Enter file name:"
read file_input
file1="$file_input" # No space after '='
file1="$file1.b" # No space after '='
file2="$file_input" # No space after '='
file2="${file2}Ins.b" # No space after '='
sed 's/!cats!.*/!cats!300!/g' "$file1" > "$file2"
Note the single quotes around sed expression: with them, there's no need to escape the !s in your expression. Note also the double quotes around "$file1" and "$file2": if one of those variables contain spaces, this will prevent your command from breaking.
Some further remarks:
As pointed by jim, you may want to use the GNU sed -i option.
Your regex will currently replace everything after !cats! in matching lines. If they were several occurences of !cats! on your line, only one will remain. If instead you just want to replace the value between two ! delimiters, you may consider use following sed command instead:
sed 's/!cats![^!]*/!cats!300/g'
Hello I am using bash to create CSV file by extracting data from an html file using grep. The problem is after getting the data then using sed to take out , in it and put a word like My_com it gose a crazy on me. here is my code.
time=$(grep -oP 'data-context-item-time=.*.data-context-item-views' index.html \
| cut -d'"' -f2)
title=$(grep -oP 'data-context-item-title=.*.data-context-item-id' index.html |\
cut -d'"' -f2)
sed "s/,/\My_commoms/g" $title
echo "$user,$views,$time,$title" >> test
I keep getting this error
sed: can't read Flipping: No such file or directory
sed: can't read the: No such file or directory
and so on
any advice on what wrong with my code
You can't use sed on text directly on the command line like that; sed expects a file, so it is reading your text as a file name. Try this for your second to last line:
echo $title | sed 's/,/My_com/g'
that way sed sees the text on a file (stdin in this case). Also note that I've used single quotes in the argument to sed; in this case I don't think it will make any difference, but in general it is good practice to make sure bash doesn't mess with the command at all.
If you don't want to use the echo | sed chain, you might also be able to rewrite it like this:
sed 's/,/My_com/g' <<< "$title"
I think that only works in bash, not dash etc. This is called a 'here-string', and bash passes the stuff on the right of the <<< to the command on its stdin, so you get the same effect.