Github Action - After using jq to extract a json value the following bash commands still attempt to execute inside jq - bash

So I have a bash command like
- name: Get waiting pull_request_id
id: get_pr_id
run: |
prId='${{ steps.get_in_progress_workflow_run.outputs.data }}' | jq '.workflow_runs' | jq '.[] | first(select(.status=="in_progress"))' | jq '.pull_requests' | jq '. | first' | jq '.number' \
echo "$prId" \
echo "prId=$prId" >> $GITHUB_OUTPUT
I am using this to lookup a workflow that is in progress and extract the pull request Id. I want to output the pull requestId to use in the next step
The error is
jq: error: Could not open file echo: No such file or directory
jq: error: Could not open file : No such file or directory
jq: error: Could not open file echo: No such file or directory
jq: error: Could not open file prId=: No such file or directory
So it seems that even with the \ I am still in the context of the previous line of script. I have tried semicolons to no avail. How can I make this work?
If I remove the backslashes, then the first line never seems to query anything or even execute as I get no output at all
I have verified that the first cmd line does return the prId if I echo it instead of assigning to a variable. I have tried to echo into the github output directly from that 1st line, but it requires quotation marks and I can't figure out how to do a jq select without using quotes.

prId='${{ steps.get_in_progress_workflow_run.outputs.data }}' | jq ...
Does not provide any input to jq. It sets a variable, then pipes nothing to jq. Setting a variable does not produce any output.
If you want to feed the variable to jq, you have to output it explicitly:
prId='${{ steps.get_in_progress_workflow_run.outputs.data }}';
printf '%s\n' "$prId" | jq ...
If that does not work, you could try the following:
prId='${{ steps.get_in_progress_workflow_run.outputs.data }}';
echo '${{ steps.get_in_progress_workflow_run.outputs.data }}' | jq ...
... | jq '.number' \
echo "$prId" \
echo "prId=$prId" >> $GITHUB_OUTPUT
Backslashes at the end of a line escape the newline character and continues the line to the next line. So the 3 lines above are parsed as a single line by your shell interpreter.
Also note that there's no need to start that many separate jq processes. Your shell pipeline can easily be merged into a single jq invocation:
... | jq '.workflow_runs[] | first(select(.status=="in_progress")) | .pull_requests[0] | .number'
After several comments and trying to understand the problem from a different angle, here's what are probably really trying to do:
prId="$(echo '${{ steps.get_in_progress_workflow_run.outputs.data }}' | jq '.workflow_runs[] | first(select(.status=="in_progress")) | .pull_requests[0] | .number')";
echo "$prId";
echo "prId=$prId" >> "$GITHUB_OUTPUT"
The magic word you are looking for here is command substitution

Related

Embedding jq in bash - what needs to be escaped?

I'm trying to inline a jq construct that itself requires pipes. I suspect I'm running into issues because bash is treating them as bash-level pipes, rather than part of the jq.
Testing at jqplay.org, this .[1] | [.timeEnded, .lifecycleState] | flatten gets me the result I need.
Trying to embed that in bash, I am trying to do something like:
status=$(curl -X GET <URL> | jq -r -c '.[1] | [.timeEnded, .lifecycleState] | flatten' | awk -F, '{print $2}' | sed 's/"//g')
With no escaping the pipes within the jq, I get
[.timeEnded,: command not found
I tried to escape those pipes as jq -r -c '.[1] \| [.timeEnded, .lifecycleState] \| flatten' but that gets me a jq syntax error:
jq: error: syntax error, unexpected INVALID_CHARACTER, expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
.[1] \| [.timeEnded, .lifecycleState] \| flatten
jq: 1 compile error
Wrapping the entire jq command in double quotes (as well as the escape chars) gave me the same syntax error. I'm sure there's probably an easy answer here, but jq is new to me.
Any help would be appreciated.
I clearly suspect that you have an unbreakable space in this part:
jq -r -c '.[1] | [...
So, edit the line manually, and replace all spaces with real spaces (taking care to not type unbreakable spaces again with AltGr+space)
Embedding jq in bash - what needs to be escaped?
Using bash and bash-like shells, jq programs can often be specified quite simply on the command line using single-quoted strings, e.g.
$ jq -n '"abc"'
"abc"
However, using this technique, single quotes are a headache since bash
does not allow single quotes within ordinary single-quoted strings. The workaround is quite horrible:
$ jq -n '"a'"'"'b"'
"a'b"
So if the jq program does have embedded single-quotes, then
it's probably time to use the -f option, but if that is not
an option, then using the form $'STRING' should be considered.
In this case, though, there are two characters that can occur in jq programs and
that will require attention: single-quotes and backslashes
For example:
$ jq -n $'"a\\tb" | "\'\\(.)\'"'
"'a\tb'"
If I'm not mistaken, the required escaping can be done using:
sed -e $'s/\'/\\\'/g' -e $'s/\\\\/\\\\\\\\/g'

(Bash) Pass the variable with jq that includes spaces

Im trying to create a variable using jq, let's say, for example:
firstName=($(curl -s https://www.easports.com/fifa/ultimate-team/api/fut/item | jq -r '.items[].firstName'))
The result I expected is "C. Ronaldo" but it gave me only "C." How can I fix it?
What about using .items[0] and command substitution instead of ($(...)) which is an array and is a subject to word-splitting, hence just the C.:
$ var=$(curl -s 'https://www.easports.com/fifa/ultimate-team/api/fut/item' | jq -r '.items[0].firstName')
$ echo "$var"
C. Ronaldo

Unexpected token error observed while using jq library in shell

I have used the below command, and It will get some substring in attribute value.
skipped=$(echo "$value" | jq -f '.[].output | scan("totalSkipped+: [[:digit:]]+")' | sed 's/"//g' )
I ran this script in shell through Jenkins job. and observed below error message:
/tmp/jenkins7615126817764256878.sh: command substitution: line 30: syntax error near unexpected token `"totalSkipped+: [[:digit:]]+"'
/tmp/jenkins7615126817764256878.sh: command substitution: line 30: `echo "$value" | jq .[].output | scan("totalSkipped+: [[:digit:]]+") | sed 's/"//g' )'
I have the entire json file which is stored in $value variable and echo "$value" returned the content of json file but not sure why its not working in jenkins.
I used the same command in jq online tool but It works as expected.
https://jqplay.org/s/7lBj_kDoB3
I'm using jq-1.6 version.
Can someone help me to resolve this?
skipped=$(jq -r '.[].output | scan("totalSkipped: [[:digit:]]+")' <<<"$value")
The pipeline is jq syntax, so it needs to be inside single quotes so the shell doesn't try to find a separate shell command named scan.
No reason for sed here -- using the -r argument to jq makes it emit raw strings as output, so they don't have syntactic quotes.

How to use an if statement after grep via pipe

I am trying to find all instances of "type":"FollowEvent", and then within those instances, if the string "actor": is not followed by {, then capture the string enclosed in " that comes immediately after "actor":. Else, capture the string enclosed in " that comes immediately after "login:".
What I have so far:
zgrep -e '"type":"FollowEvent"' /path/to/dir/* | zgrep -o '"actor":(?!{)*' | cut -f2- -d: | cut -d',' -f1 > results_file.txt
What this does:
For all files in /path/to/dir, for all lines that contain "type":"FollowEvent", find "actor:" not followed by {. Then take everything after the :, and before the next ,. Put the results in results_file.txt.
A single line in the files that are being grep'd could look like this:
{"repo":{"url":"https://url","name":"/"},"type":"FollowEvent","public":true,"created_at":"2011-05-29","payload":{"target":{"gravatar_id":"73","id":64,"repos":35,"followers":58,"login":"username3"}},"actor":{"gravatar_id":"06","id":439,"url":"https://url","avatar_url":"https://.png","login":"username4"},"id":"14"}
or like this:
{"repo":{"url":"https://url/","name":"/"},"type":"FollowEvent","public":true,"created_at":"2011-04-01","payload":{"target":{"gravatar_id":"40","repos":2,"followers":1,"login":"username2"},"actor":"username1","actor_gravatar":"de4"},"actor":{"gravatar_id":"de4","id":716,"url":"https://url","avatar_url":"https://.png","login":"username2"},"id":"12"}
What I want:
a file containing only the usernames of actors. Here, I want, in results_file.txt:
username4
username1
Let's say:
JSON='{"repo":{"url":"https://url","name":"/"},"type":"FollowEvent","public":true,"created_at":"2011-05-29","payload":{"target":{"gravatar_id":"73","id":64,"repos":35,"followers":58,"login":"username3"}},"actor":{"gravatar_id":"06","id":439,"url":"https://url","avatar_url":"https://.png","login":"username4"},"id":"14"}'
For a simple answer, I do suggest you to use jq: https://stedolan.github.io/jq/
$ echo "$JSON" | jq -r '. | select(.type=="FollowEvent") | .actor.login'
username4
You can install it in most of distros with the default package manager.
Anyway if you need to do it with GNU tools.
$ echo "$JSON" | grep '"type":"FollowEvent"' | sed 's/.*"login":"\([^"]*\).*/\1/g'
username4

Printing range of lines determined by variables with sed returns invalid command code

I'm trying to print a range of lines determined by two variables set outside the script. For some reason I keep getting the invalid command code error. If I set the variables inside the script (using FT_LINE1=7 for example) it works just fine, but if the variables are set outside it doesn't work anymore.
Here is my full script:
cat /etc/passwd | grep -v "#" | sed '1d; n; d' | sed 's/:\*:.*//' | rev | sort -r |
sed -n "$FT_LINE1,${FT_LINE2}p"
The error I receive is:
sed: 1: ",p": invalid command code ,
I am using MacOS.

Resources