I have used the below command, and It will get some substring in attribute value.
skipped=$(echo "$value" | jq -f '.[].output | scan("totalSkipped+: [[:digit:]]+")' | sed 's/"//g' )
I ran this script in shell through Jenkins job. and observed below error message:
/tmp/jenkins7615126817764256878.sh: command substitution: line 30: syntax error near unexpected token `"totalSkipped+: [[:digit:]]+"'
/tmp/jenkins7615126817764256878.sh: command substitution: line 30: `echo "$value" | jq .[].output | scan("totalSkipped+: [[:digit:]]+") | sed 's/"//g' )'
I have the entire json file which is stored in $value variable and echo "$value" returned the content of json file but not sure why its not working in jenkins.
I used the same command in jq online tool but It works as expected.
https://jqplay.org/s/7lBj_kDoB3
I'm using jq-1.6 version.
Can someone help me to resolve this?
skipped=$(jq -r '.[].output | scan("totalSkipped: [[:digit:]]+")' <<<"$value")
The pipeline is jq syntax, so it needs to be inside single quotes so the shell doesn't try to find a separate shell command named scan.
No reason for sed here -- using the -r argument to jq makes it emit raw strings as output, so they don't have syntactic quotes.
Related
So I have a bash command like
- name: Get waiting pull_request_id
id: get_pr_id
run: |
prId='${{ steps.get_in_progress_workflow_run.outputs.data }}' | jq '.workflow_runs' | jq '.[] | first(select(.status=="in_progress"))' | jq '.pull_requests' | jq '. | first' | jq '.number' \
echo "$prId" \
echo "prId=$prId" >> $GITHUB_OUTPUT
I am using this to lookup a workflow that is in progress and extract the pull request Id. I want to output the pull requestId to use in the next step
The error is
jq: error: Could not open file echo: No such file or directory
jq: error: Could not open file : No such file or directory
jq: error: Could not open file echo: No such file or directory
jq: error: Could not open file prId=: No such file or directory
So it seems that even with the \ I am still in the context of the previous line of script. I have tried semicolons to no avail. How can I make this work?
If I remove the backslashes, then the first line never seems to query anything or even execute as I get no output at all
I have verified that the first cmd line does return the prId if I echo it instead of assigning to a variable. I have tried to echo into the github output directly from that 1st line, but it requires quotation marks and I can't figure out how to do a jq select without using quotes.
prId='${{ steps.get_in_progress_workflow_run.outputs.data }}' | jq ...
Does not provide any input to jq. It sets a variable, then pipes nothing to jq. Setting a variable does not produce any output.
If you want to feed the variable to jq, you have to output it explicitly:
prId='${{ steps.get_in_progress_workflow_run.outputs.data }}';
printf '%s\n' "$prId" | jq ...
If that does not work, you could try the following:
prId='${{ steps.get_in_progress_workflow_run.outputs.data }}';
echo '${{ steps.get_in_progress_workflow_run.outputs.data }}' | jq ...
... | jq '.number' \
echo "$prId" \
echo "prId=$prId" >> $GITHUB_OUTPUT
Backslashes at the end of a line escape the newline character and continues the line to the next line. So the 3 lines above are parsed as a single line by your shell interpreter.
Also note that there's no need to start that many separate jq processes. Your shell pipeline can easily be merged into a single jq invocation:
... | jq '.workflow_runs[] | first(select(.status=="in_progress")) | .pull_requests[0] | .number'
After several comments and trying to understand the problem from a different angle, here's what are probably really trying to do:
prId="$(echo '${{ steps.get_in_progress_workflow_run.outputs.data }}' | jq '.workflow_runs[] | first(select(.status=="in_progress")) | .pull_requests[0] | .number')";
echo "$prId";
echo "prId=$prId" >> "$GITHUB_OUTPUT"
The magic word you are looking for here is command substitution
I want to edit a file and write it to anoter location without .template at the end. To do so, I use two commands in a row which should give me the new file name, but bash doesn't accept it. I heard you should do this with backquotes but I can't get it to work (the first sed is correct don't bother):
sed -E '/^\s*(#.*)?$/d; s/^\s*(\S+)\s+("([^"]*)"|(\S+)).*/s\/\1\/\3\4\/g/' "$file" > `$(basename $file) | sed -E 's/(.*).template$/.translated\1/g')`
I get such errors:
translate: command substitution: line 25: syntax error near unexpected token `)'
translate: command substitution: line 25: `$(basename $file) | sed -E 's/(.*).template$/.translated\1/g')'
translate: line 25: `$(basename $file) | sed -E 's/(.*).template$/.translated\1/g')`: ambiguous redirect
What should the redirect-to-filename part look like?
Part 1: Why It Happens
Let's look at why you get a "ambiguous redirect" error.
When you run:
foo > `$(basename $file) | sed -E 's/(.*).template$/.translated\1/g')`
...that's telling the shell to first run a command:
$(basename $file) | sed -E 's/(.*).template$/.translated\1/g'
...and use its output as the redirection target.
However, $(basename $file) | ... is not the same as basename $file | ...! Let's say your file is named /path/to/foo.template. After that first command substitution happens, it's translated to:
foo.template | sed -E 's/(.*).template$/.translated\1/g'
...which is running a command named foo.template and feeding its output to sed.
Do you have a command named foo.template? If you don't, then that just generates an error on stderr, and nothing at all on stdout. Since stdout is what's fed into sed, that means sed receives no input at all, so it writes no output at all, so you have an empty filename.
And trying to redirect to an empty filename... is ambiguous! There's your bug.
Part 2: Doing It Right
I'm assuming both your sed commands do what you want them to do (meaning that you want foo.template to create a file named .footranslated -- if you don't want to create this hidden file, your second sed operation is very wrong).
file_base=${file##*/}
sed -E '/^\s*(#.*)?$/d; s/^\s*(\S+)\s+("([^"]*)"|(\S+)).*/s\/\1\/\3\4\/g/' "$file" \
>".translated${file_base%.template}"
...or, if your actual intent is to replace the extension .template with an extension .translated, that would instead be:
file_base=${file##*/}
sed -E '/^\s*(#.*)?$/d; s/^\s*(\S+)\s+("([^"]*)"|(\S+)).*/s\/\1\/\3\4\/g/' "$file" \
>"${file_base%.template}.translated"
I'm trying to inline a jq construct that itself requires pipes. I suspect I'm running into issues because bash is treating them as bash-level pipes, rather than part of the jq.
Testing at jqplay.org, this .[1] | [.timeEnded, .lifecycleState] | flatten gets me the result I need.
Trying to embed that in bash, I am trying to do something like:
status=$(curl -X GET <URL> | jq -r -c '.[1] | [.timeEnded, .lifecycleState] | flatten' | awk -F, '{print $2}' | sed 's/"//g')
With no escaping the pipes within the jq, I get
[.timeEnded,: command not found
I tried to escape those pipes as jq -r -c '.[1] \| [.timeEnded, .lifecycleState] \| flatten' but that gets me a jq syntax error:
jq: error: syntax error, unexpected INVALID_CHARACTER, expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
.[1] \| [.timeEnded, .lifecycleState] \| flatten
jq: 1 compile error
Wrapping the entire jq command in double quotes (as well as the escape chars) gave me the same syntax error. I'm sure there's probably an easy answer here, but jq is new to me.
Any help would be appreciated.
I clearly suspect that you have an unbreakable space in this part:
jq -r -c '.[1] | [...
So, edit the line manually, and replace all spaces with real spaces (taking care to not type unbreakable spaces again with AltGr+space)
Embedding jq in bash - what needs to be escaped?
Using bash and bash-like shells, jq programs can often be specified quite simply on the command line using single-quoted strings, e.g.
$ jq -n '"abc"'
"abc"
However, using this technique, single quotes are a headache since bash
does not allow single quotes within ordinary single-quoted strings. The workaround is quite horrible:
$ jq -n '"a'"'"'b"'
"a'b"
So if the jq program does have embedded single-quotes, then
it's probably time to use the -f option, but if that is not
an option, then using the form $'STRING' should be considered.
In this case, though, there are two characters that can occur in jq programs and
that will require attention: single-quotes and backslashes
For example:
$ jq -n $'"a\\tb" | "\'\\(.)\'"'
"'a\tb'"
If I'm not mistaken, the required escaping can be done using:
sed -e $'s/\'/\\\'/g' -e $'s/\\\\/\\\\\\\\/g'
Im trying to create a variable using jq, let's say, for example:
firstName=($(curl -s https://www.easports.com/fifa/ultimate-team/api/fut/item | jq -r '.items[].firstName'))
The result I expected is "C. Ronaldo" but it gave me only "C." How can I fix it?
What about using .items[0] and command substitution instead of ($(...)) which is an array and is a subject to word-splitting, hence just the C.:
$ var=$(curl -s 'https://www.easports.com/fifa/ultimate-team/api/fut/item' | jq -r '.items[0].firstName')
$ echo "$var"
C. Ronaldo
I want to use the os.system command in order to get some information during a python script execution, but I get a syntax error, I'm literally using the same command on the terminal and the syntax it's correct.
the command is:
os.system(['df -h / | grep -E "\/$" | awk '{printf( $4)}''])
but the syntax error appears on the {
thanks in advance.
You're inconsistent with your single vs double quotes. Stack overflow's syntax coloring should make this clear.
You can fix your syntax error by escaping quotes as done by l'L'l, though I prefer wrapping the string in triple quotes. This avoids the need for escaped quotes (e.g. \').
os.system(["""df -h / | grep -E "\/$" | awk '{printf( $4)}'"""])
Then you'll just have a TypeError because the function doesn't expect a list. Fix it like this:
os.system("""df -h / | grep -E "\/$" | awk '{printf( $4)}'""")