I have a method that curls, formats the output and returns it as an array.
function hdfs_ls_curl() {
ls_url=$1
ls_result=$(curl -s --negotiate -u : ${ls_url})
#gets the foldernames from the json returned. works fine.
folder_names=$(echo $ls_result | grep -oP '(?<="pathSuffix":").+?(?=")')
#echo $folder_names <--- MAGIC ECHO
folder_array=( $folder_names )
return $folder_array
}
If i execute without the echo, nothing is returned. Adding the echo lets it behave properly.
Is the command substitution not happening without an actual execution statement?
As GordonDavisson pointed out, removing the return statement and just printing it as an array did the job.
The function will return the printed value and i can parse that in the main program.
> folder_names=$(echo $ls_result | grep -oP '(?<="pathSuffix":").+?(?=")')
> #echo $folder_names <--- MAGIC ECHO
folder_names="echo $ls_result | grep -oP \'\(?<=\"pathSuffix\":\"\).+?\(?=\"\)\'"
eval $folder_names
That might work, but watch those escapes. This saves the actual command itself in the variable, rather than its' output.
Related
I have two strings saved in a bash variable delimited by :. I want to get extract the second string, prepend that with THIS_VAR= and append it to a file named saved.txt
For example if myVar="abc:pqr", THIS_VAR=pqr should be appended to saved.txt.
This is what I have so far,
myVar="abc:pqr"
echo $myVar | cut -d ':' -f 2 >> saved.txt
How do I prepend THIS_VAR=?
printf 'THIS_VAR=%q\n' "${myVar#*:}"
See Shell Parameter Expansion and run help printf.
The more general solution in addition to #konsolebox's answer is piping into a compound statement, where you can perform arbitrary operations:
echo This is in the middle | {
echo This is first
cat
echo This is last
}
I have file abc.sh which contains below data -
a_a_1 was unsuccessful
a_a_5 was completed
a_a_2 was unsuccessful
a_a_4 was unsuccessful
a_a_9 was unsuccessful
now, I have a variable abc which contains value 2,1,9 ..i.e abc=2,1,9
now want to print only those lines from file which matches value 2,1,9 and above string.
output should be like-
a_a_2 was unsuccessful
a_a_1 was unsuccessful
a_a_9 was unsuccessful
How to achieve above output?
Since this is tagged tcl...
#!/usr/bin/env tclsh
proc main {abc} {
set abc [string map {, |} $abc]
set re [string cat {^a_a_(?:} $abc {)\M}]
while {[gets stdin line] >= 0} {
if {[regexp $re $line]} {
puts $line
}
}
}
main [lindex $argv 0]
Example usage:
$ ./findit.tcl 2,1,9 < abc.sh
Basically, it converts the CSV 2,1,9 into pipe delimited 2|1|9 and uses that as part of a bigger regular expression, and prints lines read from standard input that match it.
Since you also seem to be interested in a bash solution (you already got one for Tcl):
grep -E "_(${abc//,/|}) " abc.sh
The idea here is to translate the 2,1,9 into the regexp pattern 2|1|9. An alternative, similar in spirit, would be
grep "_[${abc//,/}] " abc.sh
which produces the pattern [219].
Your variable need to be:
abc="2\|1\|9" for grep $abc abc.sh
a=2 b=1 c=9 for grep "[$a]\|[$b]\|[$c]" abc.sh
Well as your question, if you want for abc=2,1,9 , you can just change the , as \| using sed when executing grep like this:
grep $(echo $abc | sed "s/,/\\\|/g") abc.sh
*ps: English is not my primary language so please excuse any grammar mistakes :)
I need to write bash script that converts a string of only integers "intString" to :id. intString always exists after /, may never contain any other types (create_step2 is not a valid intString), and may end at either a second / or end of line. intString may be any 1-8 characters. Script needs to be repeated for every line in a given file.
For example:
/sample/123456/url should be converted to /sample/:id/url
and /sample_url/9 should be converted to /sampleurl/:id however /sample_url_2/ should remain the same.
Any help would be appreciated!
It seems like the long way around the problem to go recursive but then I don't know what problem you are solving. It seems like a good sed command like
sed -E 's/\/[0-9]{1,}/\/:id/g'
could do it in one shot, but if you insist on being recursive then it might go something like this ...
#!/bin/bash
function restring()
{
s="$1"
s="$(echo $s | sed -E 's/\/[0-9]{1,}/\/:id/')"
if ( echo $s | grep -E '\/[0-9]{1,}' > /dev/null ) ; then
restring $s
else
echo $s
exit
fi
echo $s
}
restring "$1"
now run it
$ ./restring.sh "/foo/123/bar/456/baz/45435/andstuff"
/foo/:id/bar/:id/baz/:id/andstuff
I am using echo and sed to print a string between two strings, but it is giving me error "no such file of directory"
$SEARCH_1=12
$FO_FILE=myfile.txt
SEARCH=$(head -$SEARCH_1 $FO_FILE | tail -1 | grep BShare)
LOC=echo $SEARCH_2 | sed 's/\(.*\)BShare>\(.*\)<\/BShare\(.*\)/\2/g'
If I don't use LOC= and use only echo it prints the string correctly.
example
<test>mystring</test> will be printed as mystring
but if I assign the echo command to a variable it says "no such file or directory"
To capture the output of a command and assign it to a variable you need to use backticks or $( ... ), e.g.
LOC=`echo $SEARCH_2 | sed 's/\(.*\)BShare>\(.*\)<\/BShare\(.*\)/\2/g'`
or
LOC=$(echo $SEARCH_2 | sed 's/\(.*\)BShare>\(.*\)<\/BShare\(.*\)/\2/g')
The problem with the original command
LOC=echo $SEARCH_2 | sed 's/\(.*\)BShare>\(.*\)<\/BShare\(.*\)/\2/g'
is that the shell here tries to run what ever $SEARCH_2 evaluates to (with a environmental variable LOC set to value echo), and pipe the output of that to sed.
How to I concatenate stdin to a string, like this?
echo "input" | COMMAND "string"
and get
inputstring
A bit hacky, but this might be the shortest way to do what you asked in the question (use a pipe to accept stdout from echo "input" as stdin to another process / command:
echo "input" | awk '{print $1"string"}'
Output:
inputstring
What task are you exactly trying to accomplish? More context can get you more direction on a better solution.
Update - responding to comment:
#NoamRoss
The more idiomatic way of doing what you want is then:
echo 'http://dx.doi.org/'"$(pbpaste)"
The $(...) syntax is called command substitution. In short, it executes the commands enclosed in a new subshell, and substitutes the its stdout output to where the $(...) was invoked in the parent shell. So you would get, in effect:
echo 'http://dx.doi.org/'"rsif.2012.0125"
use cat - to read from stdin, and put it in $() to throw away the trailing newline
echo input | COMMAND "$(cat -)string"
However why don't you drop the pipe and grab the output of the left side in a command substitution:
COMMAND "$(echo input)string"
I'm often using pipes, so this tends to be an easy way to prefix and suffix stdin:
echo -n "my standard in" | cat <(echo -n "prefix... ") - <(echo " ...suffix")
prefix... my standard in ...suffix
There are some ways of accomplish this, i personally think the best is:
echo input | while read line; do echo $line string; done
Another can be by substituting "$" (end of line character) with "string" in a sed command:
echo input | sed "s/$/ string/g"
Why i prefer the former? Because it concatenates a string to stdin instantly, for example with the following command:
(echo input_one ;sleep 5; echo input_two ) | while read line; do echo $line string; done
you get immediatly the first output:
input_one string
and then after 5 seconds you get the other echo:
input_two string
On the other hand using "sed" first it performs all the content of the parenthesis and then it gives it to "sed", so the command
(echo input_one ;sleep 5; echo input_two ) | sed "s/$/ string/g"
will output both the lines
input_one string
input_two string
after 5 seconds.
This can be very useful in cases you are performing calls to functions which takes a long time to complete and want to be continuously updated about the output of the function.
You can do it with sed:
seq 5 | sed '$a\6'
seq 5 | sed '$ s/.*/\0 6/'
In your example:
echo input | sed 's/.*/\0string/'
I know this is a few years late, but you can accomplish this with the xargs -J option:
echo "input" | xargs -J "%" echo "%" "string"
And since it is xargs, you can do this on multiple lines of a file at once. If the file 'names' has three lines, like:
Adam
Bob
Charlie
You could do:
cat names | xargs -n 1 -J "%" echo "I like" "%" "because he is nice"
Also works:
seq -w 0 100 | xargs -I {} echo "string "{}
Will generate strings like:
string 000
string 001
string 002
string 003
string 004
...
The command you posted would take the string "input" use it as COMMAND's stdin stream, which would not produce the results you are looking for unless COMMAND first printed out the contents of its stdin and then printed out its command line arguments.
It seems like what you want to do is more close to command substitution.
http://www.gnu.org/software/bash/manual/html_node/Command-Substitution.html#Command-Substitution
With command substitution you can have a commandline like this:
echo input `COMMAND "string"`
This will first evaluate COMMAND with "string" as input, and then expand the results of that commands execution onto a line, replacing what's between the ‘`’ characters.
cat will be my choice: ls | cat - <(echo new line)
With perl
echo "input" | perl -ne 'print "prefix $_"'
Output:
prefix input
A solution using sd (basically a modern sed; much easier to use IMO):
# replace '$' (end of string marker) with 'Ipsum'
# the `e` flag disables multi-line matching (treats all lines as one)
$ echo "Lorem" | sd --flags e '$' 'Ipsum'
Lorem
Ipsum#no new line here
You might observe that Ipsum appears on a new line, and the output is missing a \n. The reason is echo's output ends in a \n, and you didn't tell sd to add a new \n. sd is technically correct because it's doing exactly what you are asking it to do and nothing else.
However this may not be what you want, so instead you can do this:
# replace '\n$' (new line, immediately followed by end of string) by 'Ipsum\n'
# don't forget to re-add the `\n` that you removed (if you want it)
$ echo "Lorem" | sd --flags e '\n$' 'Ipsum\n'
LoremIpsum
If you have a multi-line string, but you want to append to the end of each individual line:
$ ls
foo bar baz
$ ls | sd '\n' '/file\n'
bar/file
baz/file
foo/file
I want to prepend my sql script with "set" statement before running it.
So I echo the "set" instruction, then pipe it to cat. Command cat takes two parameters : STDIN marked as "-" and my sql file, cat joins both of them to one output. Next I pass the result to mysql command to run it as a script.
echo "set #ZERO_PRODUCTS_DISPLAY='$ZERO_PRODUCTS_DISPLAY';" | cat - sql/test_parameter.sql | mysql
p.s. mysql login and password stored in .my.cnf file