I've successfully used the following sed command to search/replace text in Linux:
sed -i 's/old_link/new_link/g' *
However, when I try it on my Mac OS X, I get:
"command c expects \ followed by text"
I thought my Mac runs a normal BASH shell. What's up?
EDIT:
According to #High Performance, this is due to Mac sed being of a different (BSD) flavor, so my question would therefore be how do I replicate this command in BSD sed?
EDIT:
Here is an actual example that causes this:
sed -i 's/hello/gbye/g' *
If you use the -i option you need to provide an extension for your backups.
If you have:
File1.txt
File2.cfg
The command (note the lack of space between -i and '' and the -e to make it work on new versions of Mac and on GNU):
sed -i'.original' -e 's/old_link/new_link/g' *
Create 2 backup files like:
File1.txt.original
File2.cfg.original
There is no portable way to avoid making backup files because it is impossible to find a mix of sed commands that works on all cases:
sed -i -e ... - does not work on OS X as it creates -e backups
sed -i'' -e ... - does not work on OS X 10.6 but works on 10.9+
sed -i '' -e ... - not working on GNU
Note Given that there isn't a sed command working on all platforms, you can try to use another command to achieve the same result.
E.g., perl -i -pe's/old_link/new_link/g' *
I believe on OS X when you use -i an extension for the backup files is required. Try:
sed -i .bak 's/hello/gbye/g' *
Using GNU sed the extension is optional.
This works with both GNU and BSD versions of sed:
sed -i'' -e 's/old_link/new_link/g' *
or with backup:
sed -i'.bak' -e 's/old_link/new_link/g' *
Note missing space after -i option! (Necessary for GNU sed)
Had the same problem in Mac and solved it with brew:
brew install gnu-sed
and use as
gsed SED_COMMAND
you can set as well set sed as alias to gsed (if you want):
alias sed=gsed
Or, you can install the GNU version of sed in your Mac, called gsed, and use it using the standard Linux syntax.
For that, install gsed using ports (if you don't have it, get it at http://www.macports.org/) by running sudo port install gsed. Then, you can run sed -i 's/old_link/new_link/g' *
Your Mac does indeed run a BASH shell, but this is more a question of which implementation of sed you are dealing with. On a Mac sed comes from BSD and is subtly different from the sed you might find on a typical Linux box. I suggest you man sed.
Insead of calling sed with sed, I do ./bin/sed
And this is the wrapper script in my ~/project/bin/sed
#!/bin/bash
if [[ "$OSTYPE" == "darwin"* ]]; then
exec "gsed" "$#"
else
exec "sed" "$#"
fi
Don't forget to chmod 755 the wrapper script.
Sinetris' answer is right, but I use this with find command to be more specific about what files I want to change. In general this should work (tested on osx /bin/bash):
find . -name "*.smth" -exec sed -i '' 's/text1/text2/g' {} \;
In general when using sed without find in complex projects is less efficient.
I've created a function to handle sed difference between MacOS (tested on MacOS 10.12) and other OS:
OS=`uname`
# $(replace_in_file pattern file)
function replace_in_file() {
if [ "$OS" = 'Darwin' ]; then
# for MacOS
sed -i '' -e "$1" "$2"
else
# for Linux and Windows
sed -i'' -e "$1" "$2"
fi
}
Usage:
$(replace_in_file 's,MASTER_HOST.*,MASTER_HOST='"$MASTER_IP"',' "./mysql/.env")
Where:
, is a delimeter
's,MASTER_HOST.*,MASTER_HOST='"$MASTER_IP"',' is pattern
"./mysql/.env" is path to file
As the other answers indicate, there is not a way to use sed portably across OS X and Linux without making backup files. So, I instead used this Ruby one-liner to do so:
ruby -pi -e "sub(/ $/, '')" ./config/locales/*.yml
In my case, I needed to call it from a rake task (i.e., inside a Ruby script), so I used this additional level of quoting:
sh %q{ruby -pi -e "sub(/ $/, '')" ./config/locales/*.yml}
Here's how to apply environment variables to template file (no backup need).
1. Create template with {{FOO}} for later replace.
echo "Hello {{FOO}}" > foo.conf.tmpl
2. Replace {{FOO}} with FOO variable and output to new foo.conf file
FOO="world" && sed -e "s/{{FOO}}/$FOO/g" foo.conf.tmpl > foo.conf
Working both macOS 10.12.4 and Ubuntu 14.04.5
Here is an option in bash scripts:
#!/bin/bash
GO_OS=${GO_OS:-"linux"}
function detect_os {
# Detect the OS name
case "$(uname -s)" in
Darwin)
host_os=darwin
;;
Linux)
host_os=linux
;;
*)
echo "Unsupported host OS. Must be Linux or Mac OS X." >&2
exit 1
;;
esac
GO_OS="${host_os}"
}
detect_os
if [ "${GO_OS}" == "darwin" ]; then
sed -i '' -e ...
else
sed -i -e ...
fi
sed -ie 's/old_link/new_link/g' *
Works on both BSD & Linux with gnu sed
Related
I'm trying to delete a line from a file and update the file to reflect that. I'm running a bash command inside a python program. The line delete works on the terminal, but the file isn't updated.
subprocess.call("sed -e $d {}".format(self._path).split())
How can I update the file to not have this line anymore.
Should add the -i flag to sed command to edit the file in-place.
On BSD's sed:
subprocess.call("sed -i '' -e $d {}".format(self._path).split())
On GNU's sed:
subprocess.call("sed -i -e $d {}".format(self._path).split())
I have a variable f in a bash script
f=/path/to/a/file.jpg
I'm using the variable as an input argument to a program that requires and input and an output path.
For example the program's usage would look like this
./myprogram -i inputFilePath -o outputFilePath
using my variable, I'm trying to maintain the same basename, change the extension, and put the output file into a sub directory. For example
./myprogram -i /path/to/a/file.jpg -o /path/to/a/new/file.tiff
I'm trying to do that by doing this
./myprogram -i "$f" -o "${f%.jpg}.tiff"
of course this keeps the basename, changes the extension, but doesn't put the file into the new subdirectory.
How can I modify f to to change /path/to/a/file.jpg into /path/to/a/new/file.tiff?
Actually you can do this in several ways:
Using sed as pointed out by #anubhava
Using dirname and basename:
./myprogram -i "$f" -o "$(dirname -- "$f")/new/$(basename -- "$f" .jpg).tiff"
Using only Bash:
./myprogram -i "$f" -o "${f%/*}/new/$(b=${f##*/}; echo -n ${b%.jpg}.tiff)"
Note that unlike the second solution (using dirname/basename) that is more robust, the third solution (in pure Bash) won't work if "$f" does not contain any slash:
$ dirname "file.jpg"
.
$ f="file.jpg"; echo "${f%/*}"
file.jpg
You may use this sed:
s='/path/to/a/file.jpg'
sed -E 's~(.*/)([^.]+)\.jpg$~\1new/\2.tiff~' <<< "$s"
/path/to/a/new/file.tiff
If you're on a system that supports the basename and dirnamecommands you could use a simple wrapper function eg:
$ type newSubDir
newSubDir is a function
newSubDir ()
{
oldPath=$(dirname "${1}");
fileName=$(basename "${1}");
newPath="${oldPath}/${2}/${fileName}";
echo "${newPath}"
}
$ newSubDir /path/to/a/file.jpg new
/path/to/a/new/file.jpg
If your system doesn't have those, you can accomplish the same thing using string manipulation:
$ file="/path/to/a/file.jpg"
$ echo "${file%/*}"
/path/to/a
$ echo "${file##*/}"
file.jpg
I am trying to store the start of a sed command inside a variable like this:
sedcmd="sed -i '' "
Later I then execute a command like so:
$sedcmd s/$orig_pkg/$package_name/g $f
And it doesn't work. Running the script with bash -x, I can see that it is being expanded like:
sed -i ''\'''\'''
What is the correct way to express this?
Define a shell function:
mysed () {
sed -i "" "$#"
}
and call it like this:
$ mysed s/$orig_pkg/$package_name/g $f
It works when the command is only one word long:
$ LS=ls
$ $LS
But in your case, the shell is trying the execute the program sed -i '', which does not exist.
The workaround is to use $SHELL -c:
$ $SHELL -c "$LS"
total 0
(Instead of $SHELL, you could also say bash, but that's not entirely reliable when there are multiple Bash installations, the shell isn't actually Bash, etc.)
However, in most cases, I'd actually use a shell function:
sedcmd () {
sed -i '' "$#"
}
Why not use an alias or a function? You can do alias as
alias sedcmd="sed -i '' "
Not exactly sure what you're trying to do, but my suggestion is:
sedcmd="sed -i "
$sedcmd s/$orig_pkg/$package_name/g $f
You must set variables orig_pkg package_name and f in your shell first.
If you're replacing variable names in a file, try:
$sedcmd s/\$orig_pkg/\$package_name/g $f
Still f must be set to the file name you're working on.
This is the right way for do that
alias sedcmd="sed -i ''"
Obviously remember that when you close your bash, this alias will be gone.
If you want to make it "permanent", you have to add it to your .bashrc home file (if you want to make this only for a single user) or .bashrc global file, if you want to make it available for all users
I am writing a bash script for an automator service that will take a Windows directory location and change it to Mac and open a finder window. It's working except for when it hits folders with spaces. I have put in to remove them but it won't work on anything with spaces still. I must have made some sort of syntax mistake.
sed -e 's:\\\\fmg_cifs1\\Dept_Shares:/Volumes/Dept_Shares:' -e 's: :\ :g' -e 's:\\:/:g' | pbcopy
TAG=$(pbpaste)
cd $TAG; open .
This is almost certainly all you have to change:
cd "$TAG"
Quoting fixes everything!
Do you need to use pbcopy and pbpaste and a variable?
cd "$(sed -e 's:\\\\fmg_cifs1\\Dept_Shares:/Volumes/Dept_Shares:' -e 's: :\ :g' -e 's:\\:/:g')"
As Jonathan pointed out, some of the sed command is unnecessary. Of course, something needs to be fed to sed. This may be all you need:
cd "$(echo "$dir" | sed -e 's:\\\\fmg_cifs1\\Dept_Shares:/Volumes/Dept_Shares:')"
The GNU version of rm has a cool -I flag. From the manpage:
-I prompt once before removing more than three files, or when removing recursively. Less
intrusive than -i, while still giving protection against most mistakes
Macs don't:
$ rm -I scratch
rm: illegal option -- I
usage: rm [-f | -i] [-dPRrvW] file ...
unlink file
Sometimes people have coreutils (the GNU version) installed on Macs and sometimes they don't. Is there a way to detect this command line flag before proceeding? I'd like to have something like this in my bash_profile:
if [ has_gnu_rm_version ]; then
alias rm="rm -I"
fi
strings /bin/rm | grep -q 'GNU coreutils'
if $? is 0, it is coreutils
I would recommend not starting down this road at all. Target your scripts to be as portable as possible, and only rely on flags/options/behaviors you can count on. Shell scripting is hard enough - why add more room for error?
To get a sense of the kind of thing I have in mind, check out Ryan Tomayko's Shell Haters talk. He also has a very well-organized page with links to POSIX descriptions of shell features and utilities. Here's rm, for example.
I'd say test the output of rm -I on a temp file, if it passes then use the alias
touch /tmp/my_core_util_check
if rm -I /tmp/my_core_util_check > /dev/null 2>&1 ; then
alias rm="rm -I"
else
rm /tmp/my_core_util_check;
fi
You could always ask rm its version with --version and check to see if it says gnu or coreutils like this:
rm --version 2>&1 | grep -i gnu &> /dev/null
[ $? -eq 0 ] && alias rm="rm -I"
how about something like this?
#!/bin/bash
rm -I &> /dev/null
if [ "$?" == "0" ]; then
echo coreutils detected
else
echo bsd version detected
fi