I'm trying to implement a decent random string generator in my buildfile using the following ExecTask
:
<exec command="cat /dev/urandom | tr -cd '[:alnum:]' | fold -w30 | head -n1" returnProperty="random" />
While this works fine when run on the command line, it causes my build script to just hang when I try and call it from Phing. I've tried various escape patterns to no avail. What am I doing wrong?
Try setting the escape to false.
Also, you might want to try to use passthru.
<exec escape="false" passthru="true" command="cat /dev/urandom | tr -cd '[:alnum:]' | fold -w30 | head -n1" returnProperty="random" />
Here is the phing documentation, with the available options you can pass to exec.
https://www.phing.info/docs/guide/trunk/ExecTask.html
And when should you use passthru.
PHP - exec() vs system() vs passthru()
Related
I'm trying to get a shell command with pipes working with Jenkins Pipelines. It's related to: Jenkins pipeline sh does not seem to respect pipe in shell command. I also found: Jenkins pipeline, bash, and pipes
However, my success with it has been unsuccessful. I can literally copy/paste the resolution of the first post in my pipeline and it does work (at least it tries execute the shell, fails due to lack of pom.xml).
When I attempt to modify the shell command to my needs, it falls flat:
def bgd_discovery = $/eval "cf apps | grep ${appHost}.${host} | cut -d ' ' -f 1 | sed 's/${args.appName}-bgd//'"/$
echo "${bgd_discovery}"
Jenkins throws a DSL method error:
java.lang.NoSuchMethodError: No such DSL method 'eval "cf apps | grep [...app host...] | cut -d ' ' -f 1 | sed 's' found among steps [...]
It seems to stop at first / of sed, but that's not the problem because this does work (from the first post):
def ver_script = $/eval "sed -n '/<version>/,/<version/p' pom.xml | head -1 | sed 's/[[:blank:]]*<\/*version>//g'"/$
Interestingly, here's another symptom to a problem: An added hyphen to the end of the sed regex
def bgd_discovery = $/eval "cf apps | grep ${appHost}.${host} | cut -d ' ' -f 1 | sed 's/${args.appName}-bgd-//'"/$
Produced this error:
WorkflowScript: 26: expecting '}', found '' # line 26, column 10.
echo "${bgd_discovery}"
I can't figure out what's wrong.
Okay, I'm going to answer my own question now. It would appear there's something flakey with the quotes. I removed them for sed and things are functioning, even without the whole eval route. This works just fine:
def current_deployment = sh (
returnStdout: true,
script: "cf apps | grep ${appName}.apps | cut -d ' ' -f 1 | sed s/${appName}-bgd-//"
);
I've a file that contains info that I'm retrieving such way
Command
cat 2018_02_15_09_01_08_result.tsv | grep -o [A-Z]\\*[0-9]*:[0-9]* | sort | uniq | sed -e 's/^/HLA-/' |tr '\n' ',' | sed '$ s/.$//'
Output
HLA-A*30:02,HLA-B*18:01,HLA-C*05:01
But I'm trying to save this in variable, the asterisk and a letter disappears, I've tried several ways, adding/removing commas etc and I'm yet not able to print it properly.
hla=`cat 2018_02_15_09_01_08_result.tsv | grep -o [A-Z]\\*[0-9]*:[0-9]* | sort | uniq | sed -e 's/^/HLA-/' |tr '\n' ',' | sed '$ s/.$//'`
echo $hla
HLA-05:01,HLA-18:01,HLA-30:02
echo "$hla"
HLA-05:01,HLA-18:01,HLA-30:02
There are multiple errors here, most of which will be aptly diagnosed by http://shellcheck.net/ without any human intervention.
You really should single-quote your regular expressions unless you specifically require the shell to perform wildcard expansion and whitespace tokenization on the regex before executing the command.
The obsolescent `command` in backticks introduces some unfortunate additional shell handling on the string inside the backticks. The solution since the 1990s is to prefer the $(command) syntax for command substitution, which does not exhibit this problem.
The cat is useless; grep knows full well how to read a file.
Try this refactored code:
hla=$(grep -o '[A-Z]*[0-9]*:[0-9]*' 2018_02_15_09_01_08_result.tsv |
sort -u | sed -e 's/^/HLA-/' |tr '\n' ',' | sed '$ s/.$//')
echo "$hla"
The double quotes around the variable interpolation in the echo are necessary and useful; notice also the line wraps for legibility and the use of sort -u in preference over sort | uniq (and generally try to reduce the number of processes -- once I understand what the sed | tr | sed does I can probably propose a simplification for that, too). Perhaps the simplest fix would be to refactor all of this into a single Awk script, but without access to the input, it's hard to tell you in more detail what that might look like.
(Also, are you really sure you need to capture the value to a variable? Often variable=value; echo "$variable" is just an obscure and inefficient way to say echo "value". And variable=$(command); echo "$variable" is better written simply command and capturing the command's standard output just so you can print it to standard output is a pure waste of cycles, unless you are planning to do something more with that variable's value.)
I've solved it by saving the output of the command with a redirection:
cat 2018_02_15_09_01_08_result.tsv |
grep -o [A-Z]\\*[0-9]*:[0-9]* |
sort | uniq |
sed -e 's/^/HLA-/' |tr '\n' ',' | sed '$ s/.$//' > out_file
hla=`cat out_file`
echo $hla
which gets me the expected HLA-A*30:02,HLA-B*18:01,HLA-C*05:01. Not the ideal solution, but it works.
I am trying to convert the following bash command into a format that can be run by python's sh package.
cat "Some_File.txt" | tr [:space:] '\n' | grep -v "^\s*$" | sort | uniq -c | sort -bnr
I have very little experience with sh and some experience with python. I understand how to use sh with simple bash commands but I am having trouble figuring out how to convert this complex command into sh format due in part to the piping. Any ideas?
I think you'd be better off using the shell.
from sh import sort, grep, tr, uniq
sort("-bnr",
uniq("-c",
sort(grep("-v",
"^\s*$",
tr("[:space:]",
"\n",
"Some_File.txt")))))
When I run this from the terminal:
$ grep -rnw 'PageObjects::CompanySettings::InfoPage' spec/**/*_spec.rb | cut -d: -f1 | uniq
It returns a single result:
spec/features/admins/change_payment_method_spec.rb
When I run this command (notice InfoPage is now TeamPage:
$ grep -rnw 'PageObjects::CompanySettings::TeamPage' spec/**/*_spec.rb | cut -d: -f1 | uniq
I get one result:
spec/features/team_page_spec.rb
Now, when I go into irb, and use the backticks to call a shell command, I get output for the second command (TeamPage), but no output for the first command (InfoPage).
2.1.6 :001 > `grep -rnw 'PageObjects::CompanySettings::InfoPage' spec/**/*_spec.rb | cut -d: -f1 | uniq`
""
2.1.6 :002 > # No result ^^
2.1.6 :003 > `grep -rnw 'PageObjects::CompanySettings::TeamPage' spec/**/*_spec.rb | cut -d: -f1 | uniq`
"spec/features/team_page_spec.rb\n"
2.1.6 :004 > # One result, as expected!
Can anyone help me figure out why this is?
I notice that in your first example, the file is nested several directories deep, and that you are using the non-standard ** wildcard.
My best guess is that your interactive shell supports that non-standard wildcard, but your batch shell does not.
One possible solution would be to use find instead of relying on shell wildcards. Another possibility would be to use Ruby, since you are already using Ruby anyway. (Note: grep is extremely optimized, so if you have a huge number of very large files, Ruby might be a bit slower.)
# Ruby's Dir class does support **
Dir['spec/**/*_spec.rb'].select {|filename|
File.open(filename) {|file|
file.each_line.any? {|line|
line.inlcude?('PageObjects::CompanySettings::InfoPage')
}
}
}
[UNTESTED]
I am trying to write a script that should take values from a xml file.
Here is the xml file :-
`<manifestFile>
<productInformation>
<publicationInfo>
<pubID pcsi-selector="P.S.">PACODE</pubID>
<pubNumber/>
</publicationInfo>
</productInformation>
</manifestFile>`
and i my code is
:-
#!/bin/sh
Manifest=""
Manifest= `/bin/grep 'pcsi-selector="' /LDCManifest.xml | cut -f 2 -d '"'`
echo $Manifest
I expect my result to be P.S. , but it keeps throwing error as :-
./abc.sh: P.S.: not found
I am new to shell and i am not able to figure out whats the error here ?
You can't have a space after the =.
When you run this command:
Manifest= `/bin/grep 'pcsi-selector="' /LDCManifest.xml | cut -f 2 -d '"'`
It's the same as this:
Manifest='' `/bin/grep 'pcsi-selector="' /LDCManifest.xml | cut -f 2 -d '"'`
That tells the shell to
Run the grep command.
Take its output
Run that output as a command, with the environment variable Manifest set to the empty string for the duration of the command.
Get rid of the space after the = and you'll get the result you want.
However, you should also avoid using backticks for command substitution, because they interfere with quoting. Use $(...) instead:
Manifest=$(grep 'pcsi-selector="' /LDCManifest.xml | cut -f2 -d'"')
Also, using text/regex-based tools like grep and cut to manipulate XML is clunky and error-prone. You'd be better off installing something like XMLStarlet:
Manifest=$(xmlstarlet sel -t \
-v '/manifestFile/productInformation/publicationInfo/pubID/#pcsiselector' -n \
/LDCManifest.xml)
Or simpler:
grep -oP 'pcsi-selector="\K[^"]+' /LDCManifest.xml
would print
P.S.
assign
Manifest=$(grep -oP 'pcsi-selector="\K[^"]+' /LDCManifest.xml)