I am trying to run this command on MacOSX terminal , which was initially intended to run on Linux
sed '1 i VISPATH=/mnt/local/gdrive/public/3DVis' init.txt >> ~/.bash_profile
but it gives me the error:
command i expects \ followed by text.
is there any way I could modify the above command to work on MacOSX terminal
Shelter is right but there's another way to do it. You can use the bash $'...' quoting to interpret the escapes before passing the string to sed.
So:
sed -iold '1i\'$'\n''text to prepend'$'\n' file.txt
^^^^^^^^ ^
/ |\|||/ \ |__ No need to reopen
| | \|/ | string to sed
Tells sed to | | | |
escape the next _/ | | +-----------------------------+
char | +-------------+ |
| | |
Close string The special bash Reopen string to
to sed newline char to send to sed
send to sed
This answer on unix.stackexchange.com led me to this solution.
Had the same problem and solved it with brew:
brew install gnu-sed
gsed YOUR_USUAL_SED_COMMAND
If you want to use the sed command, then you can set an alias:
alias sed=gsed
The OSX seds are based on older versions, you need to be much more literal in your directions to sed, AND you're lucky, in this case, sed is telling you exactly what to do. Untested as I don't have OSX, but try
sed '1 i\
VISPATH=/mnt/local/gdrive/public/3DVis
' init.txt >> ~/.bash_profile
Input via the i cmd is terminated by a blank line. Other sed instructions can follow after that. Note, NO chars after the \ char!
Also, #StephenNiedzielski is right. Use the single quote chars to wrap your sed statements. (if you need variable expansion inside your sed and can escape other uses of $, then you can also use dbl-quotes, but it's not recommended as a normal practices.
edit
As I understand now that you're doing this from the command-line, and not in a script or other editor, I have tested the above, and.... all I can say is that famous line from tech support ... "It works for me". If you're getting an error message
sed: -e expression #1, char 8: extra characters after command
then you almost certainly have added some character after the \. I just tested that, and I got the above error message. (I'm using a linux version of sed, so the error messages are exactly the same). You should edit your question to include an exact cut-paste of your command line and the new error message. Using curly-single-quotes will not work.
IHTH
Here's how I worked it out on OS X. In my case, I needed to prepend text to a file. Apparently, modern sed works like this:
sed -i '1i text to prepend' file.txt
But on OS X I had to do the following:
sed -i '' '1i\
text to prepend
' file.txt
It looks like you copied rich text. The single quotes should be straight not curly:
sed '1 i VISPATH=/mnt/local/gdrive/public/3DVis'
Related
I'm trying to format cut, paste output but sed not working.
file.txt
Apple
Banana
Apple
Banana
Orange
Apple
Orange
code.sh
cut -f2 file.txt | sort | uniq | sed 's/^\|$/#/g'| paste -sd,\& -
expected output / output on ubuntu
#Apple#,#Banana#&#Orange#
getting output / output on macos
Apple,Banana&Orange
Note: The code works on Ubuntu, but on MacOS it doesn't.
This can be done in a single gnu-awk:
awk '!seen[$1]++{} END {
PROCINFO["sorted_in"]="#ind_str_asc"
for (i in seen)
s = s (s == "" ? "" : (++j==1?",":"&")) "#" i "#"
print s
}' file
#Apple#,#Banana#&#Orange#
On OSX I have gnu awk installed via home brew.
As mentioned elsewhere, BSD sed doesn't support \|. Instead of replacing ^ and $, you can substitute # around the whole line.
sort -u file.txt | sed 's/.*/#&#/' | paste -sd,'&' -
As far as I know, BSD/Mac sed doesn't support \|. See sed not giving me correct substitute operation for newline with Mac - differences between GNU sed and BSD / OSX sed for details.
As an alternate, you can use ERE instead of BRE. I checked it on Linux, apparently this still doesn't seem to work on Mac (See also: MacOS sed: match either beginning or end).
$ echo 'Apple' | sed -E 's/^|$/#/g'
#Apple#
# workaround for Mac
$ echo 'Apple' | sed -e 's/^/#/' -e 's/$/#/'
#Apple#
Instead of sort+uniq+sed, you can also use awk (but note that awk solution shown here removes duplicates while preserving original order, doesn't sort the input):
$ awk '!seen[$0]++{print "#" $0 "#"}' ip.txt
#Apple#
#Banana#
#Orange#
Change $0 to $2 if you want only the second field, based on your use of cut
A simple way to do it using the sed command:
sed -E 's/[[:alnum:]]+/#&#/'
the -E option for enabling the POSIX ERE (extended regular
expression)
[[:alnum:]]+ The alphanumeric characters; in ASCII, equivalent to [A-Za-z0-9] with the plus (+) to refer to one or more.
the & symbol, does bring or refer to the content of the pattern we found. (on which we surrounded it with #)
$ ls | sed -n "/c/{/s/r/ssss/;p}"
When running this line in terminal, I got:
sed: -e expression #1, char 0: unmatched `{'
What's wrong?
I think there's one too many / characters in there.
You probably need:
ls | sed -n "/c/{ s/r/ssss/;p;}"
This looks for lines from ls containing a c, and replaces the first r with ssss and prints the result.
It works with the Mac (BSD) version of sed and the GNU version too. The GNU version is OK without the semicolon after the p but the Mac version is not.
However, there could be other ways to interpret what you're trying to do, and that might lead to a different command line. If you're trying to find a line containing a c and an s and replace the whole line with ssss (and maybe some other characters — it isn't clear what you've aiming for), then you'd have to do more work, in both GNU and Mac/BSD versions of sed.
I am running a script and this is part of it:
cat ../my_file.txt | sed -e $'s/\t#.*$/found_pattern/g'
This is quite working. So far, so good. Now I want to use this part as a variable $'s/\t#.*$/found_pattern/g'
When I am trying to run the following it wont work:
THISVAR="$'s/\t#.*$/found_pattern/g'"
cat ../my_file.txt | sed -e ${THISVAR}
I think the dollar sign won't get interpreted.
Can you guys help me out?
Thanks and have a great day.
When you put $ in quotes it became part of the variable's value instead of being used as a meta-character to interpret escape sequences. Try this:
THISVAR=$'s/\t#.*$/found_pattern/g'
cat ../my_file.txt | sed -e "$THISVAR"
Double-quote THISVAR to interpret the variable but prevent shell from tokenizing its value before passing to sed.
Also, you don't need the cat - just pass the file name to sed directly.
sed -e "$THISVAR" ../my_file.txt
Using bash, how would one replace all unquoted characters from a file?
I have a system that I can't modify that spits out CSV files such as:
code;prop1;prop2;prop3;prop4;prop5;prop6
0,1000,89,"a1,a2,a3",33,,
1,,,"a55,a10",1,1 L,87
2,25,1001,a4,,"1,5 L",
I need this to become, for a new system being added
code;prop1;prop2;prop3;prop4;prop5;prop6
0;1000;89;a1,a2,a3;33;;
1;;;a55,a10;1;1 L;87
2;25;1001;a4;1,5 L;
If the quotes can be removed after this substitution happens in one command it would be nice :) But I prefer clarity to complicated one-liners for future maintenance.
Thank you
With sed:
sed -e 's/,/;/g' -e ':loop; s/\("\)\([^;]*\);\([^"]*"\)/\1\2,\3/; t loop'
Test:
$ sed -e 's/,/;/g' -e ':loop; s/\("\)\([^;]*\);\([^"]*"\)/\1\2,\3/; t loop' yourfile
code;prop1;prop2;prop3;prop4;prop5;prop6
0;1000;89;"a1,a2,a3";33;;
1;;;"a55,a10";1;1 L;87
2;25;1001;a4;;"1,5 L";
You want to use a csv parser. Parsing csv with shell tools is hard (you will encounter regular expressions soon, and they rarely get all cases).
There is one in almost every language. I recommend python.
You can also do this using excel/openoffice variants by opening the file and then saving with ; as the separator.
You can used sed:
echo '0,1000,89,"a1,a2,a3",33,,' | sed -e "s|\"||g"
This will replace " with the empty string (deletes it), and you can pipe another sed to replace the , with ;:
sed -e "s|,|;|g"
$ echo '0,1000,89,"a1,a2,a3",33,,' | sed -e "s|\"||g" | sed -e "s|,|;|g"
>> 0;1000;89;a1;a2;a3;33;;
Note that you can use any separator you want instead of | inside the sed command. For example, you can rewrite the first sed as:
sed -e "s-\"--g"
I have a file of strings that are comma separated. I'm trying to replace the commas with a new line. I've tried:
sed 's/,/\n/g' file
but it is not working. What am I missing?
Use tr instead:
tr , '\n' < file
Use an ANSI-C quoted string $'string'
You need a backslash-escaped literal newline to get to sed.
In bash at least, $'' strings will replace \n with a real newline, but then you have to double the backslash that sed will see to escape the newline, e.g.
echo "a,b" | sed -e $'s/,/\\\n/g'
Note this will not work on all shells, but will work on the most common ones.
sed 's/,/\
/g'
works on Mac OS X.
If your sed usage tends to be entirely substitution expressions (as mine tends to be), you can also use perl -pe instead
$ echo 'foo,bar,baz' | perl -pe 's/,/,\n/g'
foo,
bar,
baz
MacOS is different, there is two way to solve this problem with sed in mac
first ,use \'$'\n'' replace \n, it can work in MacOS:
sed 's/,/\'$'\n''/g' file
the second, just use an empty line:
sed 's/,/\
/g' file
Ps. Pay attention the range separated by '
the third, use gnu-sed replace the mac-sed
Apparently \r is the key!
$ sed 's/, /\r/g' file3.txt > file4.txt
Transformed this:
ABFS, AIRM, AMED, BOSC, CALI, ECPG, FRGI, GERN, GTIV, HSON, IQNT, JRCC, LTRE,
MACK, MIDD, NKTR, NPSP, PME, PTIX, REFR, RSOL, UBNT, UPI, YONG, ZEUS
To this:
ABFS
AIRM
AMED
BOSC
CALI
ECPG
FRGI
GERN
GTIV
HSON
IQNT
JRCC
LTRE
MACK
MIDD
NKTR
NPSP
PME
PTIX
REFR
RSOL
UBNT
UPI
YONG
ZEUS
This works on MacOS Mountain Lion (10.8), Solaris 10 (SunOS 5.10) and RHE Linux (Red Hat Enterprise Linux Server release 5.3, Tikanga)...
$ sed 's/{pattern}/\^J/g' foo.txt > foo2.txt
... where the ^J is done by doing ctrl+v+j. Do mind the \ before the ^J.
PS, I know the sed in RHEL is GNU, the MacOS sed is FreeBSD based, and although I'm not sure about the Solaris sed, I believe this will work pretty much with any sed. YMMV tho'...
To make it complete, this also works:
echo "a,b" | sed "s/,/\\$(echo -e '\n\r')/"
Though I am late to this post, just updating my findings. This answer is only for Mac OS X.
$ sed 's/new/
> /g' m1.json > m2.json
sed: 1: "s/new/
/g": unescaped newline inside substitute pattern
In the above command I tried with Shift+Enter to add new line which didn't work. So this time I tried with "escaping" the "unescaped newline" as told by the error.
$ sed 's/new/\
> /g' m1.json > m2.json
Worked! (in Mac OS X 10.9.3)
$ echo $PATH | sed -e $'s/:/\\\n/g'
/usr/local/sbin
/Library/Oracle/instantclient_11_2/sdk
/usr/local/bin
...
Works for me on Mojave
Just to clearify: man-page of sed on OSX (10.8; Darwin Kernel Version 12.4.0) says:
[...]
Sed Regular Expressions
The regular expressions used in sed, by default, are basic regular expressions (BREs, see re_format(7) for more information), but extended
(modern) regular expressions can be used instead if the -E flag is given. In addition, sed has the following two additions to regular
expressions:
1. In a context address, any character other than a backslash (``\'') or newline character may be used to delimit the regular expression.
Also, putting a backslash character before the delimiting character causes the character to be treated literally. For example, in the
context address \xabc\xdefx, the RE delimiter is an ``x'' and the second ``x'' stands for itself, so that the regular expression is
``abcxdef''.
2. The escape sequence \n matches a newline character embedded in the pattern space. You cannot, however, use a literal newline charac-
ter in an address or in the substitute command.
[...]
so I guess one have to use tr - as mentioned above - or the nifty
sed "s/,/^M
/g"
note: you have to type <ctrl>-v,<return> to get '^M' in vi editor
The sed on macOS Mojave was released in 2005, so one solution is to install the gnu-sed,
brew install gnu-sed
then use gsed will do as you wish,
gsed 's/,/\n/g' file
If you prefer sed, just sudo sh -c 'echo /usr/local/opt/gnu-sed/libexec/gnubin > /etc/paths.d/brew', which is suggested by brew info gnu-sed. Restart your term, then your sed in command line is gsed.
FWIW, the following line works in windows and replaces semicolons in my path variables with a newline. I'm using the tools installed under my git bin directory.
echo %path% | sed -e $'s/;/\\n/g' | less
I have found another command that is working also.
find your_filename.txt -type f -exec sed -i 's/,/\n/g' {} \;