I am in use of bash shell.
I have a file filelist:
cat filelist
../1.txt
../2.txt
...
../100.txt
I want to remove "../" and I tried $ cat filelist | sed s/..\///
, but it gives an error message. How can i remove the slash?
You need quotes:
sed 's/..\///'
. means any character so you have to escape it too:
sed 's/\.\.\///'
and for readability you can use another character for the separator:
sed 's|\.\./||'
You need quotes around the sed argument and also need to include the g global flag. It is also not necessary to cat the file first. You should also escape the periods.
Use:
sed 's/\.\.\///g' filelist
Gives:
1.txt
2.txt
...
100.txt
remove starting ../ (i guess it's the purpose but not specified)
sed 's#^\.\./##' filelist
escaping the dot for avoiding regex meaning
changing default separator /by # to allow a readible / in path
adding ^ for limiting to starting path and not changing something like bad/../folder. If not the wanted behavior, just remove this caret.
direct use of file from sed (no need of cat with sed in this case)
Use an alternate separator.
sed 's|^\.\./||g' filelist
You can also use the -i flag to edit the file in-place.
sed -i 's|^\.\./||g' filelist
Related
What I have:
test
more text
#user653434 text and so
test
more text
#user9659333 text and so
I'd like to filter this text and finally get the following list as .txt file:
user653434
user9659333
It's important to get the names without "#" sign.
Thx for help ;)
Using grep -P (requires GNU grep):
$ grep -oP '(?<=#)\w+' File
user653434
user9659333
-o tells grep to print only the match.
-P tells grep to use Perl-style regular expressions.
(?<=#) tells sed that # must precede the match but the # is not included in the match.
\w+ matches one or more word characters. This is what grep will print.
To change the file in place with grep:
grep -oP '(?<=#)\w+' File >tmp && mv tmp File
Using sed
$ sed -En 's/^#([[:alnum:]]+).*/\1/p' File
user653434
user9659333
And, to change the file in place:
sed -En -i.bak 's/^#([[:alnum:]]+).*/\1/p' File
-E tells sed to use the extended form of regular expressions. This reduces the need to use escapes.
-n tells sed not to print anything unless we explicitly ask it to.
-i.bak tells sed to change the file in place while leaving a backup file with the extension .bak.
The leading s in s/^#([[:alnum:]]+).*/\1/p tells sed that we are using a substitute command. The command has the typical form s/old/new/ where old is a regular expression and sed replaces old with new. The trailing p is an option to the substitute command: the p tells sed to print the resulting line.
In our case, the old part is ^#([[:alnum:]]+).*. Starting from the beginning of the line, ^, this matches # followed by one or more alphanumeric characters, ([[:alnum:]]+), followed by anything at all, .*. Because the alphanumeric characters are placed in parens, this is saved as a group, denoted \1.
The new part of the substitute command is just \1, the alphanumeric characters from above which comprise the user name.
Here, the s indicates that we are using a sed substitute command. The usual form
With GNU grep:
grep -Po '^#\K[^ ]*' file
Output:
user653434
user9659333
See: The Stack Overflow Regular Expressions FAQ
I want to remove a line in a file containing a path. The path which should be removed is stored in a variable in a bash script.
Somewhere I read that filenames are allowed to contain any characters except "/" and "\0" on *nix systems.
Since I can't use "/" for this purpose (I have paths) I wanted to use the nul character.
What I tried:
#!/bin/bash
var_that_contains_path="/path/to/file.ext"
sed "\\\0$var_that_contains_path"\\0d file.txt > file1.txt #not working
sed "\\0$var_that_contains_path"\0d file.txt > file1.txt #not working
How can I make this work? Thanks in advance!
I think you may be using the wrong tool for the job here. Just use grep:
$ cat file
blah /path/to/file.ext more
some other text
$ var='/path/to/file.ext'
$ grep -vF "$var" file
some other text
As you can see, the line containing the path in the variable is not present in the output.
The -v switch means that grep does an inverse match, so that only lines that don't match the pattern are printed. The -F switch means that grep searches for fixed strings, rather than regular expressions.
Since the filename can contain at least a dozen different characters which have special meaning for sed (., ^, [, just to name a few), the right way to do this is to escape them all in the search string:
Escape a string for a sed replace pattern
So for the search pattern (in this case: the path), you need the following expression:
the_path=$(sed -e 's/[]\/$*.^|[]/\\&/g' <<< "$the_path")
Unix command to remove dot at end of each line in file.
Sample rec in file
11234567 0.
23456789 5569.
34567810 1.
10162056 0.
Just use sed:
sed 's/\.$//' yourfile
Escape the special character . using \.
Put an achor $ to only remove it from the end.
To make infile changes use -i option of sed.
sed -e 's/\.$//'
done. (padding to make answer long enough. grumble)
sed -i 's/\.$//' /path/to/file
This will match a literal period that is anchored to the end of the line, and replace it with nothing. The -i tells sed to make the change inline.
Suppose my file a.conf is as following
Include /1
Include /2
Include /3
I want to replace "Include /2" with a new line, I write the code in .sh file :
line="Include /2"
rep=""
sed -e "s/${line}/${rep}/g" /root/new_scripts/a.conf
But after running the sh file, It give me the following error
sed: -e expression #1, char 14: unknown option to `s'
If you are using a newer version of sed you can use -i to read from and write to the same file. Using -i you can specify a file extension so a backup will be made, incase something went wrong. Also you don't need to use the -e flag unless you are using multiple commands
sed -i.bak "s/${line}/${rep}/g" /root/new_scripts/a.conf
I have just noticed that as the variables you are using are quoted strings you may want to use single quotes around your sed expression. Also your string contains a forward slash, to avoid any errors you can use a different delimiter in your sed command (the delimiter doesn't need to be a slash):
sed -i.bak 's|${line}|${rep}|g' /root/new_scripts/a.conf
You have to write the changes to a new file and then, move the new file over the old one. Like this:
line="Include 2"
rep=""
sed -e "s/${line}/${rep}/g" /root/new_scripts/a.conf > /root/new_scripts/a.conf-new
mv /root/new_scripts/a.conf-new /root/new_scripts/a.conf
The redirection (> /root/new_scripts/a.conf) wipes the contents of the file before sed can see it.
You need to pass the -i option to sed to edit the file in-place:
sed -i "s/${line}/${rep}/g" /root/new_scripts/a.conf
You can also ask sed to create a backup of the original file:
sed -i.bak "s/${line}/${rep}/g" /root/new_scripts/a.conf
So, if you have to replace a substring in a file, you can use sed command like this, say we have a file as file.txt, so replacing a substring in it can be done like this
searchString="abc";
replaceString="def";
sed -i '' "s|$searchString|$replaceString|g" file.txt
This will all the occurrences of "abc" with "def" in file.txt. Also, this keeps a check for any / character present in the variables used, and with no backup file made.
I have a large number of words in a text file to replace.
This script is working up until the sed command where I get:
sed: 1: "*.js": invalid command code *
PS... Bash isn't one of my strong points - this doesn't need to be pretty or efficient
cd '/Users/xxxxxx/Sites/xxxxxx'
echo `pwd`;
for line in `cat myFile.txt`
do
export IFS=":"
i=0
list=()
for word in $line; do
list[$i]=$word
i=$[i+1]
done
echo ${list[0]}
echo ${list[1]}
sed -i "s/{$list[0]}/{$list[1]}/g" *.js
done
You're running BSD sed (under OS X), therefore the -i flag requires an argument specifying what you want the suffix to be.
Also, no files match the glob *.js.
This looks like a simple typo:
sed -i "s/{$list[0]}/{$list[1]}/g" *.js
Should be:
sed -i "s/${list[0]}/${list[1]}/g" *.js
(just like the echo lines above)
So myFile.txt contains a list of from:to substitutions, and you are looping over each of those. Why don't you create a sed script from this file instead?
cd '/Users/xxxxxx/Sites/xxxxxx'
sed -e 's/^/s:/' -e 's/$/:/' myFile.txt |
# Output from first sed script is a sed script!
# It contains substitutions like this:
# s:from:to:
# s:other:substitute:
sed -f - -i~ *.js
Your sed might not like the -f - which means sed should read its script from standard input. If that is the case, perhaps you can create a temporary script like this instead;
sed -e 's/^/s:/' -e 's/$/:/' myFile.txt >script.sed
sed -f script.sed -i~ *.js
Another approach, if you don't feel very confident with sed and think you are going to forget in a week what the meaning of that voodoo symbols is, could be using IFS in a more efficient way:
IFS=":"
cat myFile.txt | while read PATTERN REPLACEMENT # You feed the while loop with stdout lines and read fields separated by ":"
do
sed -i "s/${PATTERN}/${REPLACEMENT}/g"
done
The only pitfall I can see (it may be more) is that if whether PATTERN or REPLACEMENT contain a slash (/) they are going to destroy your sed expression.
You can change the sed separator with a non-printable character and you should be safe.
Anyway, if you know whats on your myFile.txt you can just use any.