I have this configuration file that has
# some other configuration settings
.....
wrapper.java.classpath.1=/opt/project/services/wrapper.jar
wrapper.java.classpath.2=/opt/project/RealTimeServer/RTEServer.jar
wrapper.java.classpath.3=/opt/project/mysql-connector-java-5.1.39-bin.jar
.....
# some other configuration settings
and I want it to look like this
# some other configuration settings
.....
wrapper.java.classpath.1=/opt/project/services/wrapper.jar
wrapper.java.classpath.2=/opt/project/RealTimeServer/RTEServer.jar
wrapper.java.classpath.3=/opt/project/mysql-connector-java-5.1.39-bin.jar
wrapper.java.classpath.4=/opt/project/RealTimeServer/some_other.jar
.....
# some other configuration settings
So I wrote this bash shell
#!/bin/bash
CONF_FILE=$1
JAR_FILE=$2
DIR=$3
# Get the last wrapper.java.classpath.N=/some_path line
CLASSPATH=`awk '/classpath/ {aline=$0} END{print aline}' $CONF_FILE`
echo $CLASSPATH
# Get the left side of the equation
IFS='=' read -ra LS <<< "$CLASSPATH"
# Get the value of N
NUM=${LS##*\.}
# Increment by 1
NUM=$((NUM+1))
echo $NUM
NEW_LINE="wrapper.java.classpath.$NUM=$DIR/$JAR_FILE"
echo $NEW_LINE
# Append classpath line to conf file
sed "/$CLASSPATH/a \\${NEW_LINE}" $CONF_FILE
I call it this way
./append_classpath.sh some_file.conf some_other.jar /opt/project/RealTimeServer
But I get
sed: -e expression #1, char 28: unknown command: `o'
I just saw your shell script. A shell is an environment from which to call tools, it is NOT a tool to manipulate text. The standard, general purpose UNIX tool to manipulate text is awk. Your entire shell script can be reduced to:
$ dir="/opt/project/RealTimeServer"
$ jar_file="some_other.jar"
$ awk -v new="$dir/$jar_file" 'p~/classpath/ && !/classpath/{match(p,/([^=]+\.)([0-9]+)=/,a); print a[1] (++a[2]) "=" new} {print; p=$0}' file
# some other configuration settings
.....
wrapper.java.classpath.1=/opt/project/services/wrapper.jar
wrapper.java.classpath.2=/opt/project/RealTimeServer/RTEServer.jar
wrapper.java.classpath.3=/opt/project/mysql-connector-java-5.1.39-bin.jar
wrapper.java.classpath.4=/opt/project/RealTimeServer/some_other.jar
.....
# some other configuration settings
The above uses GNU awk for the 3rd arg to match(). Read the book Effective Awk Programming, 4th Edition, by Arnold Robbins if you will ever have to manipulate text in a UNIX environment.
Now back to your question:
This is the syntax for what you are TRYING to do:
sed '/'"$some_string"'/a '"$some_line" "$some_file"
BUT DON'T DO IT or you'll be condemning yourself to cryptic, non-portable, unmaintainable, peeling the onion, escaping-everything hell (see Is it possible to escape regex metacharacters reliably with sed)!
sed is for simple subsitutions on individual lines, that is all. For anything else, e.g. what you are attempting, you should be using awk:
awk -v regexp="$some_string" -v line="$some_line" '{print} $0~regexp{print line}' file
Note that although your shell variable is named "some_string" you were using it in a regexp context (all you can do with sed) so I used it in a regexp context in the awk command too and named the awk variable "regexp" rather than "string" for clarity (it's just a variable name, though, no hidden meaning).
If you really DID want it treated as a string rather than a regexp then that'd be:
awk -v string="$some_string" -v line="$some_line" '{print} index($0,string){print line}' file
The only caveat to the above is that backslashes in the shell variables will be expanded when the awk variables are initialized from them so \t, for example, would become a literal tab character. If that's undesirable let us know and we can provide an alternative syntax for initing the awk variables that does not expand backslashes, see http://cfajohnson.com/shell/cus-faq-2.html#Q24.
The sed command will have problems with the slashes in your variables.
Look for some unique delimiter such as a # and try something like
CLASSPATH="wrapper.java.classpath.4=/opt/project/RealTimeServer/some_other.jar"
NEW_LINE="wrapper.java.classpath.5=your/data.rar"
echo "# some other configuration settings
.....
wrapper.java.classpath.1=/opt/project/services/wrapper.jar
wrapper.java.classpath.2=/opt/project/RealTimeServer/RTEServer.jar
wrapper.java.classpath.3=/opt/project/mysql-connector-java-5.1.39-bin.jar
wrapper.java.classpath.4=/opt/project/RealTimeServer/some_other.jar
.....
# some other configuration settings
Some more config lines
" | sed "s#${CLASSPATH}#&\n${NEW_LINE}#"
This is a one-pass pure bash solution - it should be fine if the configuration file is not huge
pfx=wrapper.java.classpath.
while IFS= read -r line; do
if [[ $line == $pfx*=* ]]; then
lastclasspath=$line
elif [[ -n $lastclasspath ]]; then
newline=${lastclasspath#$pfx}
num=${newline%%=*}
newline="$pfx$((num+1))=$DIR/$JAR_FILE"
echo "$newline"
unset lastclasspath
fi
echo "$line"
done <$CONF_FILE
Related
I have a golang file named ver.go consisting of 1 variable.
const (
ver = "1.1.1"
)
I want to be able to output 1.1.1 using a bash command. I am able to do this without a problem if we get rid of the tab at the beginning and the spaces like so
const (
ver="1.1.1"
)
by using this command awk -F= '/^ver=/{print $2}' ver.go | sed -e 's/^"//' -e 's/"$//'
However, since it must be formatted properly with gofmt I can't seem to figure it out with the tab in there as well as the space after the equal sign
Any help is appreciated.
You may use this awk:
awk -F= '$1 ~ /^[[:blank:]]*ver/{gsub(/["[:blank:]]+/, ""); print $2}' file
1.1.1
A simple solution if you just want "1.1.1" from the file ver.go is to let sed do the work for you. You can use the normal substitution form with a single backreference to capture the "1.1.1" and reinsert it as the replacement for the entire line. You can use sed -n to suppress output of normal pattern-space and add a p after the substitution so that only the line matching your REGEX prints following successful substitution, e..g
sed -n 's/^[[:space:]]*ver[[:space:]]*=[[:space:]]*"\([^"][^"]*\).*$/\1/p' ver.go
This will work with or without the spaces (or tabs) before the "ver".
Example Use/Output
With your ver.go contents, you would get:
$ sed -n 's/^[[:space:]]*ver[[:space:]]*=[[:space:]]*"\([^"][^"]*\).*$/\1/p' ver.go
1.1.1
If I misunderstood what you are after, please let me know. If you have further questions, just drop a comment below.
With bash and Parameter Expansion to remove all ":
while read -r key x value; do [[ "$key" == "ver" ]] && echo "${value//\"/}"; done < ver.go
Output:
1.1.1
This is very similar to the question "remove duplicate lines with similar prefix" but it's the other way around:
Given an input of sorted strings (in this case, directories) like:
a/
a/b/c/
a/d/
bar/foo/
bar/foo2/
c/d/
c/d/e/
I want to remove the lines from the output, if the previous line is a prefix of the current line. In this case, the output would be:
a/
bar/foo/
bar/foo2/
c/d/
This would be pretty easy to code in Python etc, but in this case I am in shell environment (bash, sort, sed, awk...). (Re-sorting is fine.)
use awk:
awk '{if(k && match($0, k))next; k="^"$0}1' file
k="^"$0 to anchor the pattern to the beginning of the string.
Probably need NF>0 before the main block in case there are EMPTY lines.
Update: there could be issues if regex meta characters exist in the variable k, the below line without using regex should be better:
awk '{if(k && index($0, k)==1)next; k=$0}1' file
Update-2: thanks #Ed, I've adjusted the 2nd method to cover non-empty lines which evaluated to zeros (empty lines will be kept as-is though):
awk '{if(k!="" && index($0,k)==1)next;k=$0}1' file
Perl 1-liner. Loop over the input lines -n and then execute -e the following program, checking to see if the beginning of the current line matches the last line, printing the non-matches.
perl -ne 'print unless m|^$last|; chomp($last=$_);' file_list.txt
Bash itself (in fact POSIX shell) provides all you need through parameter expansion with substring removal. All you need to do is check whether the line you read matches itself with the prefix removed. If it doesn't, you have a prefixed line, otherwise, you have a non-prefixed line. Then it is a simple matter of outputting the non-prefixed line and setting the prefix to the current line -- and repeat, e.g.
#!/bin/bash
pfx= ## prefix
## read each line
while read -r line; do
## if no prefix or line matches line with prefix removed
if [ -z "$pfx" -o "$line" = "${line#$pfx}" ]
then
printf "%s\n" "$line" ## output lile
pfx="$line" ## set prefix to line
fi
done < "$1"
(note: if there is a chance that an input file that does not contain a POSIX end-of-file, e.g. a '\n' on the final line of the file, then you should check the contents of line as a condition of your while, e.g. while read -r line || [ -n "$line" ]; do ... )
Example Input File
$ cat string.txt
a/
a/b/c/
a/d/
bar/foo/
bar/foo2/
c/d/
c/d/e/
Example Use/Output
$ bash nonprefix.sh string.txt
a/
bar/foo/
bar/foo2/
c/d/
$ awk 'NR==1 || index($0,prev)!=1{prev=$0; print}' file
a/
bar/foo/
bar/foo2/
c/d/
I have a file name
"PHOTOS_TIMESTAMP_5373382"
I want to extract from this filename "PHOTOS_5373382" and add "ABC" i.e. finally want it to look like
"abc_PHOTOS_5373382" in shell script.
echo "PHOTOS_TIMESTAMP_5373382" | awk -F"_" '{print "ABC_"$1"_"$3}'
echo will provide input for awk command.
awk command does the data tokenization on character '_' of input using the option -F.
Individual token (starting from 1) can be accessed using $n, where n is the token number.
You will need the following sequence of commands directly on your shell, preferably bash shell (or) as a complete script which takes a single argument the file to be converted
#!/bin/bash
myFile="$1" # Input argument (file-name with extension)
filename=$(basename "$myFile") # Getting the absolute file-path
extension="${filename##*.}" # Extracting the file-name part without extension
filename="${filename%.*}" # Extracting the extension part
IFS="_" read -r string1 string2 string3 <<<"$filename" # Extracting the sub-string needed from the original file-name with '_' de-limiter
mv -v "$myFile" ABC_"$string1"_"$string3"."$extension" # Renaming the actual file
On running the script as
$ ./script.sh PHOTOS_TIMESTAMP_5373382.jpg
`PHOTOS_TIMESTAMP_5373382.jpg' -> `ABC_PHOTOS_5373382.jpg'
Although I like awk
Native shell solution
k="PHOTOS_TIMESTAMP_5373382"
IFS="_" read -a arr <<< "$k"
echo abc_${arr[0]}_${arr[2]}
Sed solution
echo "abc_$k" | sed -e 's/TIMESTAMP_//g'
abc_PHOTOS_5373382
I want to turn this input_variable = 1
into input_variable = 01
From previous posts here I tried this but didn't work:
sed -e "s/\0" <<< "$input_variable"
I get:
Syntax error: redirection unexpected
What do I do wrong?
Thanks!
EDIT
Thanks to Benjamin I found a workaround (I would still like to know why the sed didn't work):
new_variable="0$input_variable"
While it can be done with sed, simple assignment in your script can do exactly what you want done. For example, if you have input_variable=1 and want input_variable=01, you can simply add a leading 0 by assignment:
input_variable="0${input_variable}"
or for additional types of numeric formatting you can use the printf -v option and take advantage of the format-specifiers provided by the printf function. For example:
printf -v input_variable "%02d" $input_variable
will zero-pad input_variable to a length of 2 (or any width you specify with the field-width modifier). You can also just add the leading zero regardless of the width with:
printf -v input_variable "0%s" $input_variable
sed is an excellent tool, but it isn't really the correct tool for this job.
You don't close the substitution command. Each substitution command must contain 3 delimiters
sed -e 's/pattern/replacement/' <<< 'text' # 3 backslashes
What you want to do could be done with:
sed -e 's/.*/0&/' <<< $input_variable
EDIT:
You are probably using Ubuntu and stumbled upon dash also known as the Almquist shell, which does not have the <<< redirection operator. The following would be a POSIX-compliant alternative, which works with dash as well:
sed -e 's/.*/0&/' <<~
$input_variable
~
And also this:
echo $input_variable | sed -e 's/.*/0&/'
To have the variable take on the new value, do this:
input_variable=$(echo $input_variable | sed -e 's/.*/0&/')
That's however not how you would write the shell script. Shell scripts usually give out some textual output, rather than setting external variables:
So, the script, let's call it append_zero.sh:
#!/bin/sh
echo $1 | sed 's/.*/0&/'
and you would execute it like this:
$ input_variable=1
$ input_variable=$(append_zero.sh input_variable)
$ echo $input_variable
01
This way you have a working shell script that you can reuse with any Unix system that has a POSIX compliant /bin/sh
I wanted to print the name from the entire address by shell scripting. So user1#12.12.23.234 should give output "user1" and similarly 11234#12.123.12.23 should give output 11234
Reading from the terminal:
$ IFS=# read user host && echo "$user"
<user1#12.12.23.234>
user1
Reading from a variable:
$ address='user1#12.12.23.234'
$ cut -d# -f1 <<< "$address"
user1
$ sed 's/#.*//' <<< "$address"
user1
$ awk -F# '{print $1}' <<< "$address"
user1
Using bash in place editing:
EMAIL='user#server.com'
echo "${EMAIL%#*}
This is a Bash built-in, so it might not be very portable (it won't run with sh if it's not linked to /bin/bash for example), but it is probably faster since it doesn't fork a process to handle the editing.
Using sed:
echo "$EMAIL" | sed -e 's/#.*//'
This tells sed to replace the # character and as many characters that it can find after it up to the end of line with nothing, ie. removing everything after the #.
This option is probably better if you have multiple emails stored in a file, then you can do something like
sed -e 's/#.*//' emails.txt > users.txt
Hope this helps =)
I tend to use expr for this kind of thing:
address='user1#12.12.23.234'
expr "$address" : '\([^#]*\)'
This is a use of expr for its pattern matching and extraction abilities. Translated, the above says: Please print out the longest prefix of $address that doesn't contain an #.
The expr tool is covered by Posix, so this should be pretty portable.
As a note, some historical versions of expr will interpret an argument with a leading - as an option. If you care about guarding against that, you can add an extra letter to the beginning of the string, and just avoid matching it, like so:
expr "x$address" : 'x\([^#]*\)'