I want to turn this input_variable = 1
into input_variable = 01
From previous posts here I tried this but didn't work:
sed -e "s/\0" <<< "$input_variable"
I get:
Syntax error: redirection unexpected
What do I do wrong?
Thanks!
EDIT
Thanks to Benjamin I found a workaround (I would still like to know why the sed didn't work):
new_variable="0$input_variable"
While it can be done with sed, simple assignment in your script can do exactly what you want done. For example, if you have input_variable=1 and want input_variable=01, you can simply add a leading 0 by assignment:
input_variable="0${input_variable}"
or for additional types of numeric formatting you can use the printf -v option and take advantage of the format-specifiers provided by the printf function. For example:
printf -v input_variable "%02d" $input_variable
will zero-pad input_variable to a length of 2 (or any width you specify with the field-width modifier). You can also just add the leading zero regardless of the width with:
printf -v input_variable "0%s" $input_variable
sed is an excellent tool, but it isn't really the correct tool for this job.
You don't close the substitution command. Each substitution command must contain 3 delimiters
sed -e 's/pattern/replacement/' <<< 'text' # 3 backslashes
What you want to do could be done with:
sed -e 's/.*/0&/' <<< $input_variable
EDIT:
You are probably using Ubuntu and stumbled upon dash also known as the Almquist shell, which does not have the <<< redirection operator. The following would be a POSIX-compliant alternative, which works with dash as well:
sed -e 's/.*/0&/' <<~
$input_variable
~
And also this:
echo $input_variable | sed -e 's/.*/0&/'
To have the variable take on the new value, do this:
input_variable=$(echo $input_variable | sed -e 's/.*/0&/')
That's however not how you would write the shell script. Shell scripts usually give out some textual output, rather than setting external variables:
So, the script, let's call it append_zero.sh:
#!/bin/sh
echo $1 | sed 's/.*/0&/'
and you would execute it like this:
$ input_variable=1
$ input_variable=$(append_zero.sh input_variable)
$ echo $input_variable
01
This way you have a working shell script that you can reuse with any Unix system that has a POSIX compliant /bin/sh
Related
Cannot seem to find an answer to this one online...
I have a string variable (externally sourced) with new lines "\n" encoded as strings.
I want to replace those strings with actual new line carriage returns. The code below can achieve this...
echo $EXT_DESCR | sed 's/\\n/\n/g'
But when I try to store the result of this into it's own variable, it converts them back to strings
NEW_DESCR=`echo $EXT_DESCR | sed 's/\\n/\n/g'`
How can this be achieved, or what I'm I doing wrong?
Here's my code I've been testing to try get the right results
EXT_DESCR="This is a text\nWith a new line"
echo $EXT_DESCR | sed 's/\\n/\n/g'
NEW_DESCR=`echo $EXT_DESCR | sed 's/\\n/\n/g'`
echo ""
echo "$NEW_DESCR"
No need for sed, using parameter expansion:
$ foo='1\n2\n3'; echo "${foo//'\n'/$'\n'}"
1
2
3
With bash 4.4 or newer, you can use the E operator in ${parameter#operator}:
$ foo='1\n2\n3'; echo "${foo#E}"
1
2
3
Other answers contain alternative solutions. (I especially like the parameter expansion one.)
Here's what's wrong with your attempt:
In
echo $EXT_DESCR | sed 's/\\n/\n/g'
the sed command is in single quotes, so sed gets s/\\n/\n/g as is.
In
NEW_DESCR=`echo $EXT_DESCR | sed 's/\\n/\n/g'`
the whole command is in backticks, so a round of backslash processing is applied. That leads to sed getting the code s/\n/\n/g, which does nothing.
A possible fix for this code:
NEW_DESCR=`echo $EXT_DESCR | sed 's/\\\\n/\\n/g'`
By doubling up the backslashes, we end up with the right command in sed.
Or (easier):
NEW_DESCR=$(echo $EXT_DESCR | sed 's/\\n/\n/g')
Instead of backticks use $( ), which has less esoteric escaping rules.
Note: Don't use ALL_UPPERCASE for your shell variables. UPPERCASE is (informally) reserved for system variables such as HOME and special built-in variables such as IFS or RANDOM.
Depending on what exactly you need it for:
echo -e $EXT_DESCR
might be all you need.
From echo man page:
-e
enable interpretation of backslash escapes
This printf would do the job by interpreting all escaped constructs:
printf -v NEW_DESCR "%b" "$EXT_DESCR"
-v option will store output in a variable so no need to use command substitution here.
Problem with your approach is use of old back-ticks. You could do:
NEW_DESCR=$(echo "$EXT_DESCR" | sed 's/\\n/\n/g')
Assuming you're using gnu sed as BSD sed won't work with this approach.
I have this configuration file that has
# some other configuration settings
.....
wrapper.java.classpath.1=/opt/project/services/wrapper.jar
wrapper.java.classpath.2=/opt/project/RealTimeServer/RTEServer.jar
wrapper.java.classpath.3=/opt/project/mysql-connector-java-5.1.39-bin.jar
.....
# some other configuration settings
and I want it to look like this
# some other configuration settings
.....
wrapper.java.classpath.1=/opt/project/services/wrapper.jar
wrapper.java.classpath.2=/opt/project/RealTimeServer/RTEServer.jar
wrapper.java.classpath.3=/opt/project/mysql-connector-java-5.1.39-bin.jar
wrapper.java.classpath.4=/opt/project/RealTimeServer/some_other.jar
.....
# some other configuration settings
So I wrote this bash shell
#!/bin/bash
CONF_FILE=$1
JAR_FILE=$2
DIR=$3
# Get the last wrapper.java.classpath.N=/some_path line
CLASSPATH=`awk '/classpath/ {aline=$0} END{print aline}' $CONF_FILE`
echo $CLASSPATH
# Get the left side of the equation
IFS='=' read -ra LS <<< "$CLASSPATH"
# Get the value of N
NUM=${LS##*\.}
# Increment by 1
NUM=$((NUM+1))
echo $NUM
NEW_LINE="wrapper.java.classpath.$NUM=$DIR/$JAR_FILE"
echo $NEW_LINE
# Append classpath line to conf file
sed "/$CLASSPATH/a \\${NEW_LINE}" $CONF_FILE
I call it this way
./append_classpath.sh some_file.conf some_other.jar /opt/project/RealTimeServer
But I get
sed: -e expression #1, char 28: unknown command: `o'
I just saw your shell script. A shell is an environment from which to call tools, it is NOT a tool to manipulate text. The standard, general purpose UNIX tool to manipulate text is awk. Your entire shell script can be reduced to:
$ dir="/opt/project/RealTimeServer"
$ jar_file="some_other.jar"
$ awk -v new="$dir/$jar_file" 'p~/classpath/ && !/classpath/{match(p,/([^=]+\.)([0-9]+)=/,a); print a[1] (++a[2]) "=" new} {print; p=$0}' file
# some other configuration settings
.....
wrapper.java.classpath.1=/opt/project/services/wrapper.jar
wrapper.java.classpath.2=/opt/project/RealTimeServer/RTEServer.jar
wrapper.java.classpath.3=/opt/project/mysql-connector-java-5.1.39-bin.jar
wrapper.java.classpath.4=/opt/project/RealTimeServer/some_other.jar
.....
# some other configuration settings
The above uses GNU awk for the 3rd arg to match(). Read the book Effective Awk Programming, 4th Edition, by Arnold Robbins if you will ever have to manipulate text in a UNIX environment.
Now back to your question:
This is the syntax for what you are TRYING to do:
sed '/'"$some_string"'/a '"$some_line" "$some_file"
BUT DON'T DO IT or you'll be condemning yourself to cryptic, non-portable, unmaintainable, peeling the onion, escaping-everything hell (see Is it possible to escape regex metacharacters reliably with sed)!
sed is for simple subsitutions on individual lines, that is all. For anything else, e.g. what you are attempting, you should be using awk:
awk -v regexp="$some_string" -v line="$some_line" '{print} $0~regexp{print line}' file
Note that although your shell variable is named "some_string" you were using it in a regexp context (all you can do with sed) so I used it in a regexp context in the awk command too and named the awk variable "regexp" rather than "string" for clarity (it's just a variable name, though, no hidden meaning).
If you really DID want it treated as a string rather than a regexp then that'd be:
awk -v string="$some_string" -v line="$some_line" '{print} index($0,string){print line}' file
The only caveat to the above is that backslashes in the shell variables will be expanded when the awk variables are initialized from them so \t, for example, would become a literal tab character. If that's undesirable let us know and we can provide an alternative syntax for initing the awk variables that does not expand backslashes, see http://cfajohnson.com/shell/cus-faq-2.html#Q24.
The sed command will have problems with the slashes in your variables.
Look for some unique delimiter such as a # and try something like
CLASSPATH="wrapper.java.classpath.4=/opt/project/RealTimeServer/some_other.jar"
NEW_LINE="wrapper.java.classpath.5=your/data.rar"
echo "# some other configuration settings
.....
wrapper.java.classpath.1=/opt/project/services/wrapper.jar
wrapper.java.classpath.2=/opt/project/RealTimeServer/RTEServer.jar
wrapper.java.classpath.3=/opt/project/mysql-connector-java-5.1.39-bin.jar
wrapper.java.classpath.4=/opt/project/RealTimeServer/some_other.jar
.....
# some other configuration settings
Some more config lines
" | sed "s#${CLASSPATH}#&\n${NEW_LINE}#"
This is a one-pass pure bash solution - it should be fine if the configuration file is not huge
pfx=wrapper.java.classpath.
while IFS= read -r line; do
if [[ $line == $pfx*=* ]]; then
lastclasspath=$line
elif [[ -n $lastclasspath ]]; then
newline=${lastclasspath#$pfx}
num=${newline%%=*}
newline="$pfx$((num+1))=$DIR/$JAR_FILE"
echo "$newline"
unset lastclasspath
fi
echo "$line"
done <$CONF_FILE
Okay, I am a newbie to Unix scripting. I was given the task to find a temporary work around for this:
cat /directory/filename1.xml |sed -e "s/ABCXYZ/${c}/g" > /directory/filename2.xml
$c is a variable from a sqlplus count query. I totally understand how this sed command is working. But here is where I am stuck. I am storing the count associated with the variable in another file called filename3 as count[$c] where $c is replaced with a number. So my question is how can I update this sed command to substitute ABCXYZ with the count from file3?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>
UPDATE: In case anyone has a similar issue I got mine to work using:
rm /directory/folder/variablefilename.dat
echo $c >> /directory/folder/variablefilename.dat
d=$(grep [0-9] /directory/folder/variablefilename.dat)
sed -3 "s/ABC123/${d}/g" /directory/folder/inputfile.xml >> /directory/folder/outputfile.xml
thank you to Kaz for pointing me in the right direction
Store the count in filename3 using the syntax c=number. Then you can source the file as a shell script:
. /filename3 # get c variable
sed -e "s/ABCXYZ/${c}/g" /directory/filename1.xml > /directory/filename2.xml
If you can't change the format of filename3, you can write a shell function which scrapes the number out of that file and sets the c variable. Or you can scrape the number out with an external program like grep, and then interpolate its output into a variable assignment using command substitution: $(command arg ...) syntax.
Suppose we can rely on file3 to contain exactly one line of the form count[42]. Then we can just extract the digits with grep -o:
c=$(grep -E -o '[0-9]+' filename3)
sed -e "s/ABCXYZ/$c/g" /directory/filename1.xml > /directory/filename2.xml
The c variable can be eliminated, of course; you can stick the $(grep ...) into the sed command line in place of $c.
A file which contains numerous instances of syntax like count[42] for various variables could be transformed into a set of shell variable assignments using sed, and then sourced into the current shell to make those assignments happen:
$ sed -n -e 's/^\([A-Za-z_][A-Za-z0-9_]\+\)\[\(.*\)\]/\1=\2/p' filename3 > vars.sh
$ . ./vars.sh
you can use sed like this
sed -r "s/ABCXYZ/$(sed -nr 's/.*count[[]([0-9])+[]].*/\1/p' path_to_file)/g" path_to_file
the expression is double quoted which allow the shell to execute below and find the number in count[$c] in the file and use it as a substitute
$(sed -nr 's/.*count[[]([0-9])+[]].*/\1/p' path_to_file)
I wanted to print the name from the entire address by shell scripting. So user1#12.12.23.234 should give output "user1" and similarly 11234#12.123.12.23 should give output 11234
Reading from the terminal:
$ IFS=# read user host && echo "$user"
<user1#12.12.23.234>
user1
Reading from a variable:
$ address='user1#12.12.23.234'
$ cut -d# -f1 <<< "$address"
user1
$ sed 's/#.*//' <<< "$address"
user1
$ awk -F# '{print $1}' <<< "$address"
user1
Using bash in place editing:
EMAIL='user#server.com'
echo "${EMAIL%#*}
This is a Bash built-in, so it might not be very portable (it won't run with sh if it's not linked to /bin/bash for example), but it is probably faster since it doesn't fork a process to handle the editing.
Using sed:
echo "$EMAIL" | sed -e 's/#.*//'
This tells sed to replace the # character and as many characters that it can find after it up to the end of line with nothing, ie. removing everything after the #.
This option is probably better if you have multiple emails stored in a file, then you can do something like
sed -e 's/#.*//' emails.txt > users.txt
Hope this helps =)
I tend to use expr for this kind of thing:
address='user1#12.12.23.234'
expr "$address" : '\([^#]*\)'
This is a use of expr for its pattern matching and extraction abilities. Translated, the above says: Please print out the longest prefix of $address that doesn't contain an #.
The expr tool is covered by Posix, so this should be pretty portable.
As a note, some historical versions of expr will interpret an argument with a leading - as an option. If you care about guarding against that, you can add an extra letter to the beginning of the string, and just avoid matching it, like so:
expr "x$address" : 'x\([^#]*\)'
I am writing shell script for embedded Linux in a small industrial box. I have a variable containing the text pid: 1234 and I want to strip first X characters from the line, so only 1234 stays. I have more variables I need to "clean", so I need to cut away X first characters and ${string:5} doesn't work for some reason in my system.
The only thing the box seems to have is sed.
I am trying to make the following to work:
result=$(echo "$pid" | sed 's/^.\{4\}//g')
Any ideas?
The following should work:
var="pid: 1234"
var=${var:5}
Are you sure bash is the shell executing your script?
Even the POSIX-compliant
var=${var#?????}
would be preferable to using an external process, although this requires you to hard-code the 5 in the form of a fixed-length pattern.
Here's a concise method to cut the first X characters using cut(1). This example removes the first 4 characters by cutting a substring starting with 5th character.
echo "$pid" | cut -c 5-
Use the -r option ("use extended regular expressions in the script") to sed in order to use the {n} syntax:
$ echo 'pid: 1234'| sed -r 's/^.{5}//'
1234
Cut first two characters from string:
$ string="1234567890"; echo "${string:2}"
34567890
pipe it through awk '{print substr($0,42)}' where 42 is one more than the number of characters to drop. For example:
$ echo abcde| awk '{print substr($0,2)}'
bcde
$
Chances are, you'll have cut as well. If so:
[me#home]$ echo "pid: 1234" | cut -d" " -f2
1234
Well, there have been solutions here with sed, awk, cut and using bash syntax. I just want to throw in another POSIX conform variant:
$ echo "pid: 1234" | tail -c +6
1234
-c tells tail at which byte offset to start, counting from the end of the input data, yet if the the number starts with a + sign, it is from the beginning of the input data to the end.
Another way, using cut instead of sed.
result=`echo $pid | cut -c 5-`
I found the answer in pure sed supplied by this question (admittedly, posted after this question was posted). This does exactly what you asked, solely in sed:
result=\`echo "$pid" | sed '/./ { s/pid:\ //g; }'\``
The dot in sed '/./) is whatever you want to match. Your question is exactly what I was attempting to, except in my case I wanted to match a specific line in a file and then uncomment it. In my case it was:
# Uncomment a line (edit the file in-place):
sed -i '/#\ COMMENTED_LINE_TO_MATCH/ { s/#\ //g; }' /path/to/target/file
The -i after sed is to edit the file in place (remove this switch if you want to test your matching expression prior to editing the file).
(I posted this because I wanted to do this entirely with sed as this question asked and none of the previous answered solved that problem.)
Rather than removing n characters from the start, perhaps you could just extract the digits directly. Like so...
$ echo "pid: 1234" | grep -Po "\d+"
This may be a more robust solution, and seems more intuitive.
This will do the job too:
echo "$pid"|awk '{print $2}'