I am new to shell scripting and i am trying to remove new line character from each line using SED. this is what i have done so far :
printf "{new\nto\nlinux}" | sed ':a;N;s/\n/ /g'
removes only Ist new line character.
I somewhere found this command :
printf "{new\nto\nlinux}" | sed ':a;N;$!ba;s/\n/ /g'
but it gives :"ba: Event not found."
if i do:
printf "{new\nto\nlinux}" | sed ':a;N;s/\n/ /g' | sed ':a;N;s/\n/ /g'
then it gives correct output but i am looking for something better as i am not sure how many new character i will get when i run the script.
incoming stream is from echo or printf or some variable in script.
To remove newlines, use tr:
tr -d '\n'
If you want to replace each newline with a single space:
tr '\n' ' '
The error ba: Event not found is coming from csh, and is due to csh trying to match !ba in your history list. You can escape the ! and write the command:
sed ':a;N;$\!ba;s/\n/ /g' # Suitable for csh only!!
but sed is the wrong tool for this, and you would be better off using a shell that handles quoted strings more reasonably. That is, stop using csh and start using bash.
This might work for you:
printf "{new\nto\nlinux}" | paste -sd' '
{new to linux}
or:
printf "{new\nto\nlinux}" | tr '\n' ' '
{new to linux}
or:
printf "{new\nto\nlinux}" |sed -e ':a' -e '$!{' -e 'N' -e 'ba' -e '}' -e 's/\n/ /g'
{new to linux}
Use perl instead of sed. perl is similar to sed:
ubuntu#ubuntu:/$ printf "{new\nto\nlinux}" | sed 's/\n/ /g'; echo ''
{new
to
linux}
ubuntu#ubuntu:/$ printf "{new\nto\nlinux}" | perl -pe 's/\n/ /g'; echo ''
{new to linux}
ubuntu#ubuntu:/$ echo -e "new\nto\nlinux\ntest\n1\n2 3" | perl -pe 's/\n/_ _/g'; echo ''
new_ _to_ _linux_ _test_ _1_ _2 3_ _
ubuntu#ubuntu:/$
Related
I have a command line string:
ls | sed -e 's/^/"/g' -e 's/$/"/g' | tr '\n' ' '
which outputs file names in the current folder, in a single line, surrounded by quotations.
I attempted to add this to my .bash_profile as an alias, however I think the single quotes are causing an issue and I can't get it to work.
I tried this with no luck:
alias='ls | sed -e 's/^/"/g' -e 's/$/"/g' | tr '\n' ' ''
How would one go about creating an alias for the above?
Thanks in advance to anyone who can help with this noob question and I appreciate your time :)
Cheers,
Stephen.
Enquote the whole command in double quotes and escape the double quotes inside the command with a backslash:
alias a="ls | sed -e 's/^/\"/g' -e 's/$/\"/g' | tr '\n' ' '"
or use a function
a() {
ls | sed -e 's/^/"/g' -e 's/$/"/g' | tr '\n' ' '
}
By the way: Parsing ls is a bad practice. It would be safer and easier to use globs and printf:
printf '"%s" ' *
or, if you want to properly quote for using the arguments inside eval or something similar
printf '%q ' *
work here with using \"
alias X="ls | sed -e 's/^/\"/g' -e 's/$/\"/g' | tr '\n' ' '"
The alias should enclose the command line into ' AND each already existing ' should be escaped with '\''
Give a try to this:
alias lsquoted='ls | sed -e '\''s/^/"/g'\'' -e '\''s/$/"/g'\'' | tr '\''\n'\'' '\'' '\'''
I'm trying to format output in a way that inserts newline characters after each 'line', with lines denoted by double quotes (""). The quotes themselves are temporary and to be stripped in a later step.
Input:
"a",1,"aa""b",2,"bb"
Output:
a,1,aa
b,2,bb
I've tried:
sed 's/""/\n/'
sed 's/""/\/g'
tr '""' '\n'
But tr seems to replace every quote character and sed seems to insert \n as text instead of a newline. What can I do to make this work?
echo '"a",1,"aa""b",2,"bb"' |awk -v RS='""' '{$1=$1} {gsub(/"/,"")}1'
a,1,aa
b,2,bb
or using sed:
echo '"a",1,"aa""b",2,"bb"' |sed -e 's/""/\n/' -e 's/"//g' # OR sed -e 's/""/\n/;s/"//g'
a,1,aa
b,2,bb
awk solution: Here the default record separator is changed from new line to "". So awk will consider the EOL when it hits "".
sed solution: Here first "" are converted into new line and second replacement is to remove " from each line.
neech#nicolaw.uk:~ $ cat file.txt
"a",1,"aa""b",2,"bb"
neech#nicolaw.uk:~ $ sed 's/""/\n/' file.txt | tr -d '"'
a,1,aa
b,2,bb
You seem to be dealing with POSIX sed, which does not have support for the \n notation. Insert an actual new-line into the pattern, either:
sed 's/""/\
/'
Or:
sed 's/""/\'$'\n''/'
E.g.:
sed 's/""/\
/' | tr -d \"
Output:
a,1,aa
b,2,bb
As suggested by George Vasiliou if you have perl you could use:
> echo '"a",1,"aa""b",2,"bb"' | perl -pe 's/""/"\n"/g;s/"//g'
This avoids the non portable sed problem.
Or for a crappy hack version.
Replace the "" with another character and then use tr (since tr should work with \n) to replace it with \n instead then remove the single " after.
So you can get the "" replaced with newline like this:
sed 's/""/#/g' | tr '#' '\n'
Then the rest follows:
> echo '"a",1,"aa""b",2,"bb"'| sed 's/""/#/g' | tr '#' '\n' | sed 's/\"//g'
trying to replace the sapces and underscores in this is just a (test)
I do the following:
echo "this is just a (test)" | sed -e 's/ /_/g' | sed -e 's/(//g' | sed -e 's/)//g'
And this gives me:
this_is_just_a_test
Is there a better way? shorter way of writing it in sed?
You can achieve the same thing using tr:
echo "this is just a (test)" | tr \ _ | tr -d \(\)
The first tr replaces spaces with underscores and the second one deletes all parenthesis.
I have paths.txt like:
pathO1/:pathD1/
pathO2/:pathD2/
...
pathON/:pathDN/
How can I 'sed' insert ' * ' after each pathOX/ ?
The script is:
while read line
do
cp $(echo $line | tr ':' ' ')
done < "paths.txt"
substituted by:
while read line
do
cp $(echo $line | sed 's/:/* /1')
done < "paths.txt"
This looks to be a similar question to which you asked earlier: Shell Script: Read line in file
Just apply the trick of removing additional '*' before appliying tr like:
cp $(echo $line | sed 's/\*//1' | tr ':' '* ')
while read line
do
path=`echo "$line" | sed 's/:/ /g'`
cmd="cp $path"
echo $cmd
eval $cmd
done < "./paths.txt"
quick and dirty awk one-liner without loop to do the job:
awk -F: '$1="cp "$1' paths.txt
this will output:
cp /home/Documents/shellscripts/Origen/* /home/Documents/shellscripts/Destino/
cp /home/Documents/shellscripts/Origen2/* /home/Documents/shellscripts/Destino2/
...
if you want the cmds to get executed:
awk -F: '$1="cp "$1' paths.txt|sh
I said it quick & dirty, because:
the format must be path1:path2
your path cannot contain special letters (like space) or :
Using pure shell
while IFS=: read -r p1 p2
do
cp $p1 "$p2"
done < file
I'm using
sed -e "s/\*DIVIDER\*/$DIVIDER/g" to replace *DIVIDER* with a user-specified string, which is stored in $DIVIDER. The problem is that I want them to be able to specify escape characters as their divider, like \n or \t. When I try this, I just end up with the letter n or t, or so on.
Does anyone have any ideas on how to do this? It will be greatly appreciated!
EDIT: Here's the meat of the script, I must be missing something.
curl --silent "$URL" > tweets.txt
if [[ `cat tweets.txt` == *\<error\>* ]]; then
grep -E '(error>)' tweets.txt | \
sed -e 's/<error>//' -e 's/<\/error>//' |
sed -e 's/<[^>]*>//g' |
head $headarg | sed G | fmt
else
echo $REPLACE | awk '{gsub(".", "\\\\&");print}'
grep -E '(description>)' tweets.txt | \
sed -n '2,$p' | \
sed -e 's/<description>//' -e 's/<\/description>//' |
sed -e 's/<[^>]*>//g' |
sed -e 's/\&\;/\&/g' |
sed -e 's/\<\;/\</g' |
sed -e 's/\>\;/\>/g' |
sed -e 's/\"\;/\"/g' |
sed -e 's/\&....\;/\?/g' |
sed -e 's/\&.....\;/\?/g' |
sed -e 's/^ *//g' |
sed -e :a -e '$!N;s/\n/\*DIVIDER\*/;ta' | # Replace newlines with *divider*.
sed -e "s/\*DIVIDER\*/${DIVIDER//\\/\\\\}/g" | # Replace *DIVIDER* with the actual divider.
head $headarg | sed G
fi
The long list of sed lines are replacing characters from an XML source, and the last two are the ones that are supposed to replace the newlines with the specified character. I know it seems redundant to replace a newline with another newline, but it was the easiest way I could come up with to let them pick their own divider. The divider replacement works great with normal characters.
You can use bash to escape the backslash like this:
sed -e "s/\*DIVIDER\*/${DIVIDER//\\/\\\\}/g"
The syntax is ${name/pattern/string}. If pattern begins with /, every occurence of pattern in name is replaced by string. Otherwise only the first occurence is replaced.
Maybe:
case "$DIVIDER" in
(*\\*) DIVIDER=$(echo "$DIVIDER" | sed 's/\\/\\\\/g');;
esac
I played with this script:
for DIVIDER in 'xx\n' 'xxx\\ddd' "xxx"
do
echo "In: <<$DIVIDER>>"
case "$DIVIDER" in (*\\*) DIVIDER=$(echo "$DIVIDER" | sed 's/\\/\\\\/g');;
esac
echo "Out: <<$DIVIDER>>"
done
Run with 'ksh' or 'bash' (but not 'sh') on MacOS X:
In: <<xx\n>>
Out: <<xx\\n>>
In: <<xxx\\ddd>>
Out: <<xxx\\\\ddd>>
In: <<xxx>>
Out: <<xxx>>
It seems to be a simple substitution:
$ d='\n'
$ echo "a*DIVIDER*b" | sed "s/\*DIVIDER\*/$d/"
a
b
Maybe I don't understand what you're trying to accomplish.
Then maybe this step could take the place of the last two of yours:
sed -n ":a;$ {s/\n/$DIVIDER/g;p;b};N;ba"
Note the space after the dollar sign. It prevents the shell from interpreting "${s..." as a variable name.
And as ghostdog74 suggested, you have way too many calls to sed. You may be able to change a lot of the pipe characters to backslashes (line continuation) and delete "sed" from all but the first one (leave the "-e" everywhere). (untested)
You just need to escape the escape char.
\n will match \n
\ will match \
\\ will match \
Using FreeBSD sed (e.g. on Mac OS X) you have to preprocess the $DIVIDER user input:
d='\n'
d='\t'
NL=$'\\\n'
TAB=$'\\\t'
d="${d/\\n/${NL}}"
d="${d/\\t/${TAB}}"
echo "a*DIVIDER*b" | sed -E -e "s/\*DIVIDER\*/${d}/"