I have an audacity.cfg file in which I want to script the substitution of two plugin paths. The paths were previously different, so I need to inset the updated ones. I will provide one below.
First, I want to locate this text, which begins the line in question:
FFmpegLibPath
Next, I want to replace that entire line with:
FFmpegLibPath=/Library/Application Support/audacity/libs/libavformat.55.dylib
That's it. It should not be so difficult, but it is. I have done lots of experimenting using sed and awk, but have not been able to get anything to work. While there are LOTS of examples of this online and in this forum, none of them have worked. They all produce errors relating to escape characters, as well as some random other things. I have spent hours experimenting and researching, but have not made any headway.
I realize that the slashes and spaces are likely causing issues, and I have spent considerable time attempting to solve this. I've tried all sorts of things, but as I've said, nothing works.
Does anyone have any ideas about this?
Thanks in advance for your help.
Edit:
I am running MacOS 10.10.5, and one of the things I saw in my research was using GNU sed, because some arguments do not work without it. While I am sure that would produce a better result, I cannot use it because my users would not have it. I think this is part of the reason why this is so difficult, because many of the solutions I have seen are utilizing arguments that I cannot use.
If everything other fails, you always can use the old-school ed solution. :) :)
#!/bin/bash
{
printf 'H\n'
printf '/^FFmpegLibPath[ \t=]/\n'
printf '%s\n' c 'FFmpegLibPath=/Library/Application Support/audacity/libs/libavformat.55.dylib' . w q
} | ed -s "/path/to/audacity.cfg" >/dev/null
The quotes, spaces are mandatory.
The above searching for the line starting with FFmpegLibPath and followed by space or tab or =. So it tries avoid collisions with similar prefixes like: FFmpegLibPath2.
If such collisions are not possible, the above could be simply written as:
ed -s "/path/to/audacity.cfg" >/dev/null <<'EOF'
H
/^FFmpegLibPath/
c
FFmpegLibPath=/Library/Application Support/audacity/libs/libavformat.55.dylib
.
w
q
EOF
or
printf '%s\n' H '/^FFmpegLibPath/' c 'FFmpegLibPath=/Library/Application Support/audacity/libs/libavformat.55.dylib' . w q |
ed -s "/path/to/audacity.cfg" >/dev/null
You can escape the special character (forward slash) and assign it to a variable:
REPL=$(sed 's/[\/]/\\&/g' <<< "/Library/Application Support/audacity/libs/libavformat.55.dylib")
& is sed's meta-character to represent the pattern that was matched.
sed -E "s/(FFmpegLibPath=).+/\1$REPL/" audacity.cfg
Option -E is used to support extended regular expressions
output:
etc
FFmpegLibPath=/Library/Application Support/audacity/libs/libavformat.55.dylib
etc
etc
If you preferred to maintain the updates in a separate text file:
cfg_update.txt
key_name1=value
key_name2=value
key_name3=value
# define delimiter
IFS=\=
cat cfg_update.txt | while read KEY VALUE; do
sed -i -E "s/($KEY=).+/\1$VALUE/" audacity.cfg
done
option -i is used to edit file in place
Finally, be sure to make a backup before your tests, good luck!
Related
I have set up a sed expression that finds an old number in a file and replaces it with a new number. I have no problem with this.
I have many files. For each file, i, the new number needs to come from row i of a column of data in another file (let's called it "newNumbers"), like the example below.
1.2345
10.6789
100.101112
...
I can do this by doing (inside a for loop over file (i)):
i = 1
while read line
do
var[$i]="$line"
find ... # My sed expression for finding and replacing a word in file (i).
i=$((i+1))
done < newNumbers
However, this is not a good solution. "newNumbers" is so long that it will take days. I know the line I need from "newNumbers." It is line i, which corresponds to file i. So I want to read in the value from "newNumbers" at line i. I cannot get the syntax right to do this though (I am not experienced with bash). I'm using GNU sed on a Mac and have checked out several questions that seem related here. Examples that have resulted in "char1: missing command" or '"iq;d": command expects \ followed by text'-type errors are:
gsed -n -e "${i}" newNumbers
gsed 'iq;d' newNumbers
gsed "${i}q;d" newNumbers
I don't know if this is a Mac problem (GNU sed) or some syntax problem. Thank you for any help.
This is what worked but I have no idea why! I will post it, as there are probably other people who need to deal with this when working with input and output files.
Inside my for loop over file i, which starts at 0,
j=$((i+1))
newNumber_i=$(gsed "${j}q;d" newNumbers)
echo ${i%}
I needed the "echo" statement. I got it from ghostdog74's response here:
How to read output of sed into a variable
I have some text files $f resembling the following
function
%blah
%blah
%blah
code here
I want to append the following text before the first empty line:
%
%This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike
%3.0 Unported License. See notes at the end of this file for more information.
I tried the following:
top=$(cat ./PATH/text.txt)
top="${top//$'\n'/\\n}"
sed -i.bak 's#^$#'"$top"'\\n#' $f
where the second line (I think) preserves the new line in the text and the third line (I think) substitutes the first empty line with the text plus a new empty line.
Two problems:
1- My code appends the following text:
%n%This work is licensed under the Creative Commons
Attribution-NonCommercial-ShareAlike n%3.0 Unported License. See notes
at the end of this file for more information.\n
2- It appends it at end of the file.
Can someone please help me understand the problems with my code?
If you are using GNU sed, following would work.
Use ^$ to find the empty line and then use sed to replace/put the text that you want.
# Define your replacement text in a variable
a="%\n%This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike\n%3.0 Unported License. See notes at the end of this file for more information."
Note, $a should include those \n that will be directly interpreted by sed as newlines.
$ sed "0,/^$/s//$a/" inputfile.txt
In the above syntax, 0 represents the first occurrence.
Output:
function
%blah
%blah
%
%This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike
%3.0 Unported License. See notes at the end of this file for more information.
%blah
code here
test
You've included bash and sed tags in your question. Since I can't seem to come up with a way of doing this in sed, here's a bash-only solution. It's likely to perform the worst of all working solutions you might find.
The following works with your sample input:
$ while read -r x; do [[ -z "$x" ]] && cat boilerplate; printf '%s\n' "$x"; done < src
This will however insert the boilerplate before EVERY blank line, which is probably not what you're after. Instead, we should probably make this more than a one-liner:
#!/usr/bin/env bash
y=true
while read -r x; do
if [[ -z "$x" ]] && $y; then
cat boilerplate
y=false
fi
printf '%s\n' "$x"
done < src
Note that unlike the code in your question, this doesn't store your boilerplate in a variable, it just cats it "at the right time".
Note that this sends the combined output to stdout. If your goal is to modify the original file, you'll need to wrap this in something that moves around temporary files. (Note that sed's -i option also doesn't really edit files in place, it only hides the moving-around-temp-files from you.)
The following alternatives are probably a better idea.
A similar solution to the bash one might be achieved with better performance using awk:
awk 'NR==FNR{b=b $0 ORS;next} /^$/&&!y{printf "%s",b;y++} 1' boilerplate src
This awk solution obviously reads your boilerplate into a variable, though it's not a shell variable.
Notwithstanding non-standard platform-specific extensions, awk does not have any facility for editing files "in place" either. A portable solution using awk would still need to push temp files around.
And of course, the following old standard of ed is great to keep in your back pocket:
printf 'H\n/^$/\n-\n.r boilerplate\nw\nq\n' | ed src
In bash, of course, you could always use heretext, which might be clearer:
$ ed src <<< $'H\n/^$/\n-\n.r boilerplate\nw\nq\n'
The ed command is non-stream version of sed. Or rather, sed is the stream version of ed, which has been around since before the dinosaurs and is still going strong.
The commands we're using are separated by newlines and fed to ed's standard input. You can discard stdout if you feel the urge. The commands shown here are:
H - instruct ed to print more useful errors, if it gets any.
/^$/ - search for the first occurrence of a newline.
- - GO BACK ONE LINE. Awesome, right?
.r boilerplate - Read your boilerplate at the current line,
w - and write the file.
q - Quit.
Note that this does not keep a .bak file. You'll need to do that yourself if you really want one.
And if, as you suggested in comments, the filename you're reading is to be constructed from a variable, note that variable expansion does not happen inside format quoting ($' .. '). You can either switch quoting mechanisms mid-script:
ed "$file" <<< $'H\n/^$/\n-\n.r ./TATTOO_'"$currn"$'/top.txt\nw\nq\n'
Or you could put ed script in a variable constructed by printf
printf -v scr 'H\n/^$/\n-\n.r ./TATTOO_%s/top.txt\nw\nq\n' "$currn"
ed "$file" <<< "$scr"`
Adding the text to a variable so you can interpolate the variable is wasteful and an unnecessary complication. sed can easily read the contents of a file by itself.
sed -i.bak '1r./PATH/text.txt' "$f"
Unfortunately, this part of sed is poorly standardized, so you may have to experiment a little bit. Some dialects require a newline (perhaps, or perhaps not, preceded by a backslash) before the filename.
sed -i.bak '1r\
./PATH/text.txt' "$f"
(Notice also the double quotes around the file name. You generally always want double quotes around variables which contain file names. More here.)
Adapting the recipe from here we can extend this to apply to the first empty line instead of the first line.
sed -i.bak -e '/^$/!b' -e 'r./PATH/text.txt' -e :a -e '$!{' -e n -e ba -e } "$f"
This adds the boilerplate after the first empty line but perhaps that's acceptable. Refactoring it to replace it or add an empty line after should not be too challenging anyway. (Maybe use sed -n and instead explicitly print everything except the empty line.)
In brief terms, this skips to the end (simply prints) up until we find the first empty line. Then, we read and print the file, and go into a loop which prints the remainder of the file without returning to the beginning of the script.
sed that I think works. Uses files for the extra bit to be inserted.
b='##\n## comment piece\n##'
sed --posix -ne '
1,/^$/ {
/^$/ {
x;
/^true$/ !{
x
s/^$/true/
i\
'"$b"'
};
x;
s/^.*$//
}
}
p
' file1
with the examples using ranges of 1,/^$/, an empty first line would result in the disclaimer being printed twice. To avoid this, I've set it up to put a flag in the hold space ( x; s/^$/true/ ) that I can swap to the pattern space to check whether its the first blank. Once theres a match for blank line, i\ inserts the comment ($b) in front of the pattern space.
Thanks to ghoti for the initial plan.
I am relatively new to bash scripting and have no experience with LaTeX. I've been asked to develop a script which will replace convenience shortcuts in LaTeX documents with their more cumbersome long-form equivalents.
My approach thus far has been to isolate both the shortcut and the long-form in separate variables and then try to replace them in the text by using sed. I've attached short example files below.
As it is currently the script takes 2 arguments, a file expr from which it retrieves the shortcuts and long-form terminology and an infile to which is is supposed to make the appropriate changes. I know that the script is properly isolating both the shortcuts and long-forms and can return them, but it can't seem to execute the sed command.
I have tried searching this online and found multiple similar question where the suggestion was that sed has difficultly recognizing variable and that various type of quotation combinations might solve the problem. I have tried many permutations and none appear to work. The long-form terminologies in many cases contain special characters such as '$' and '{}', so I suspect that this might be the issue but I'm not sure. I am also very much open to other ideas about how to solve the problem. Please find below samples of both the script and the 2 argument files, expr and infile.
expr file containing shortcuts and long-forms
% a
\newcommand{\ao}{$^{18}$O}
\newcommand{\aodso}{$^{18}$O/$^{16}$O}
% b
\newcommand{\bea}{\begin{equation}}
\newcommand{\beaa}{\begin{eqnarray}}
% c
\newcommand{\cthree}{C$_3$}
\newcommand{\cfour}{C$_4$}
\newcommand{\coz}{CO$_2$}
infile containing shortcuts to be replaced by long-forms
This is my test {\ao}
{\aodso} my test is this
Does it work {\bea}
{\beaa} test test test
work work work {\cthree}
{\cfour} This is my test
my test is this {\coz}
Relevant subsection of script called with expr and infile as arguments
while read line; do
if [[ $line == \newcommand* ]]; then
temp=${line#*\{}
sc=${temp%%\}*}
templf=${temp#*\{}
lf=${templf%\}}
#echo $sc, $lf
sed -i -e 's/${sc}/${lf}/g' ${infile}
fi
done < ${expr}
UPDATE:
For clarification, this is what the desired result would be, the shortcuts present in infile would be substituted with the appropriate long-form
This is my test {$^{18}$O}
{$^{18}$O/$^{16}$O} my test is this
Does it work {\begin{equation}}
{\begin{eqnarray}} test test test
work work work {C$_3$}
{C$_4$} This is my test
my test is this {CO$_2$}
Code for GNU sed:
sed -r '/^%/d;s#.*\b(\{\\\w+\})(\{.*\})#\1 \2#;s#\\#\\\\#g;s#(\S+)\s(\S+)#\\|\1|s|\1|\2|g#' file1|sed -f - file2
$ cat file1
% a
\newcommand{\ao}{$^{18}$O}
\newcommand{\aodso}{$^{18}$O/$^{16}$O}
% b
\newcommand{\bea}{\begin{equation}}
\newcommand{\beaa}{\begin{eqnarray}}
% c
\newcommand{\cthree}{C$_3$}
\newcommand{\cfour}{C$_4$}
\newcommand{\coz}{CO$_2$}
$ cat file2
This is my test {\ao}
{\aodso} my test is this
Does it work {\bea}
{\beaa} test test test
work work work {\cthree}
{\cfour} This is my test
my test is this {\coz}
$ sed -r "/^%/d;s#.*\b(\{\\\w+\})(\{.*\})#\1 \2#;s#\\#\\\\#g;s#(\S+)\s(\S+)#\\|\1|s|\1|\2|g#" file1|sed -f - file2
This is my test {$^{18}$O}
{$^{18}$O/$^{16}$O} my test is this
Does it work {\begin{equation}}
{\begin{eqnarray}} test test test
work work work {C$_3$}
{C$_4$} This is my test
my test is this {CO$_2$}
Explanation:
There are two calls for sed, the first one makes from the file with the search/replace patterns a sed script:
sed -r '/^%/d;s#.*\b(\{\\\w+\})(\{.*\})#\1 \2#;s#\\#\\\\#g;s#(\S+)\s(\S+)#\\|\1|s|\1|\2|g#' file1
\|{\\ao}|s|{\\ao}|{$^{18}$O}|g
\|{\\aodso}|s|{\\aodso}|{$^{18}$O/$^{16}$O}|g
\|{\\bea}|s|{\\bea}|{\\begin{equation}}|g
\|{\\beaa}|s|{\\beaa}|{\\begin{eqnarray}}|g
\|{\\cthree}|s|{\\cthree}|{C$_3$}|g
\|{\\cfour}|s|{\\cfour}|{C$_4$}|g
\|{\\coz}|s|{\\coz}|{CO$_2$}|g
In the second call sed processes this script with the text file to make the replacements.
sed -f - file2
There's a lot of discussion of this issue on this question at tex.SE. But I'll take the opportunity to note that the best answer there (IMO) is to use the de-macro program, which is a python script that comes with TeXLive. It's quite capable, and can handle arguments as well as simple replacements.
To use it, you move the macros that you want expanded into a <something>-private.sty file, and include it into your document with \usepackage{<something>-private}, then run de-macro <mydocument>. It spits out <mydocument>-private.tex, which is the same as your original, but with your private macros replaced by their more basic things.
I know that this question has been marked as answered since quite a while and that you explicitly mention bash and sed as your desired tool.
However, in the interest of others and if you don't insist on bash and sed there exist other options for your problem, e.g. the perl script TME (as suggested here on SO). Usage:
tme [ -c ] [ -D | -Dn ] [ macros.tex ... ] <input.tex >output.tex
I went through lots of similar posts but none could be applied to mine.
I would like to search and replace using sed in some particular lines in a way that in only matches the first occurrence; lets say I have this part of the script:
processor <- read.table("../mall_all/adpcm/FULL_DB-constprop", header=TRUE, colClasses=c("reassociate"="factor", "scalarrepl"="factor", "inline"="factor", "sccp"="factor", "loop_reduce"="factor"))
processor<-processor[-c(20:40)]
processor$intensity <- processor$int_high - processor$int_low
processor$performance<- processor$perf_high - processor$perf_low
processor<-processor[-c(1:4)]
processor<-processor[,!names(processor) %in% c("constprop")]
I want to keep changing the $constprop variable in
"../mall_all/adpcm/FULL_DB-constprop"
AND
[,!names(processor) %in% c("constprop")]
in a loop that I wrote, the problem is; I want the colClasses parameteres AND the rest of the scripts remains the same while entering the loop (the loop has the compiler options like: reassociate, inline, constprop, etc)
I was wondering why my search and replace didn't work :
set -x
compilerOptionList="constprop dce inline instcombine licm loop_reduce loop_rotate loop_unroll loop_unswitch loop_unswitch mem2reg memcpyopt reassociate scalarrepl sccp simplifycfg "
stringToBeReplaced=constprop
for compilerOption in $compilerOptionList
do
echo "Using compiler option: $compilerOption"
//here you could see the sed scripts
sed -i "1,15 /FULL_DB/,/header/ s/$stringToBeReplaced/$compilerOption/" r.scr
stringToBeReplaced=$compilerOption
make
mv Rplots.pdf Rplots_adpcm_$compilerOption.pdf
echo "DONE! $compilerOption"
done
Thanks all for your time and help ;)
Amir
I'm not sure having rightly understood your need, but maybe someting like
sed -e "
1,15ba;
/FULL_DB/,/header/ba;
bb;
:a;
s/stringToBeReplaced/$compilerOption/;
:b;
" -i r.scr
could do the job.
This line is problematic
sed -i "1,15 /FULL_DB/,/header/ s/$stringToBeReplaced/$compilerOption/" r.scr
It's not a valid sed command syntax. You'll need to enclose part of it in braces like this
sed -i "1,15 { /FULL_DB/,/header/ s/$stringToBeReplaced/$compilerOption/ }" r.scr
But I think a tidier way is to use separate files for input and output of sed, i.e. change that line to
sed "1,15 s/constprop/$compilerOption/" r.scr_tmp >r.scr
You don't need the stringToBeReplaced variable. This way you always substitute "constprop", and don't have to worry that the string to be replaced appears elsewhere in the code.
r.scr_tmp would contain the same code as r.scr except that the constprop part of r.scr_tmpremains unchanged.
I'm thinking of using find or grep to collect the files, and maybe sed to make the change, but what to do with the output? Or would it be better to use "argdo" in vim?
Note: this question is asking for command line solutions, not IDE's. Answers and comments suggesting IDE's will be calmly, politely and serenely flagged. :-)
I am huge fan of the following
export MYLIST=`find . -type f -name *.java`
for a in $MYLIST; do
mv $a $a.orig
echo "import.stuff" >> $a
cat $a.orig >> $a
chmod 755 $a
done;
mv is evil and eventually this will get you. But I use this same construct for a lot of things and it is my utility knife of choice.
Update: This method also backs up the files which you should do using any method. In addition it does not use anything but the shell's features. You don't have to jog your memory about tools you don't use often. It is simple enough to teach a monkey (and believe me I have) to do. And you are generally wise enough to just throw it away because it took four seconds to write.
you can use sed to insert a line before the first line of the file:
sed -ie "1i import package.name.*;" YourClass.java
use a for loop to iterate through all your files and run this expression on them. but be careful if you have packages, because the import statements must be after the package declaration. you can use a more complex sed expression, if that's the case.
I'd suggest sed -i to obviate the need to worry about the output. Since you don't specify your platform, check your man pages; the semantics of sed -i vary from Linux to BSD.
I would use sed if there was a decent way to so "do this for the first line only" but I don't know of one off of the top of my head. Why not use perl instead. Something like:
find . -name '*.java' -exec perl -p -i.bak -e '
BEGIN {
print "import package.name.*;\n"
}' {} \;
should do the job. Check perlrun(1) for more details.
for i in `ls *java`
do
sed -i '.old' '1 i\
Your include statement here.
' $i
done
Should do it. -i does an in place replacement and .old saves the old file just in case something goes wrong. Replace the iterator *java as necessary (maybe 'find . | grep java' or something instead.)
You may also use the ed command to do in-file search and replace:
# delete all lines matching foobar
ed -s test.txt <<< $'g/foobar/d\nw'
see: http://bash-hackers.org/wiki/doku.php?id=howto:edit-ed
I've actually starting to do it using "argdo" in vim. First of all, set the args:
:args **/*.java
The "**" traverses all the subdir, and the "args" sets them to be the arg list (as if you started vim with all those files in as arguments to vim, eg: vim package1/One.java package1/Two.java package2/One.java)
Then fiddle with whatever commands I need to make the transform I want, eg:
:/^package.*$/s/$/\rimport package.name.*;/
The "/^package.*$/" acts as an address for the ordinary "s///" substitution that follows it; the "/$/" matches the end of the package's line; the "\r" is to get a newline.
Now I can automate this over all files, with argdo. I hit ":", then uparrow to get the above line, then insert "argdo " so it becomes:
:argdo /^package.*$/s/$/\rimport package.name.*;/
This "argdo" applies that transform to each file in the argument list.
What is really nice about this solution is that it isn't dangerous: it hasn't actually changed the files yet, but I can look at them to confirm it did what I wanted. I can undo on specific files, or I can exit if I don't like what it's done (BTW: I've mapped ^n and ^p to :n and :N so I can scoot quickly through the files). Now, I commit them with ":wa" - "write all" files.
:wa
At this point, I can still undo specific files, or finesse them as needed.
This same approach can be used for other refactorings (e.g. change a method signature and calls to it, in many files).
BTW: This is clumsy: "s/$/\rtext/"... There must be a better way to append text from vim's commandline...