I want to write a shell script to login and get bugs for a project. I want the dashboard values like bugs, Vulnerabilities, code smells and coverage.
The url of dashboard is: http://www.example.com/dashboard?id=example_project_name.
Here is what I tried:
curl GET -u username:password http://www.example.com/api/issues/search?project=example_project_name&types=BUG.
So, this prints all the data. I just need the value show in the below image:
Basically What I want to achieve is that I’m using a Sonarqube plugin in Jenkins, so I use extended email plugin to send email for job execution and in that email I want to give details like number of bugs in the repository after the build.
Is there any other way?
Finally after reading the documentation carefully, I got the values. Here is the script that I created.
#!/bin/bash
vul=$(curl -sX GET -u username:password 'http://www.example.com/api/issues/search?projectKeys=example_project_name&types=VULNERABILITY');
bug=$(curl -sX GET -u username:password 'http://www.example.com/api/issues/search?projectKeys=example_project_name&types=BUG');
no_vul=$(echo $vul | jq -r .total);
no_bug=$(echo $bug | jq -r .total);
echo "Total number of VULNERABILITIES are $no_vul"
echo "Total number of BUGS are $no_bug"
Here is the API documentation URL.
Related
Context
I wrote the following code to get the last n commits of a repository of a GitHub user/organisation:
# Get commits
commits_json=$(curl -H "Accept: application/vnd.github.v3+json" https://api.github.com/repos/$github_username/$github_repo_name/commits?per_page=1&page=1)
echo "commits_json=$commits_json"
echo ""
# Get the first commit.
readarray -t branch_commits_arr < <(echo "$commits_json" | jq ".[].sha")
echo "branch_commits_arr=$branch_commits_arr"
Issue
I noticed I get into the reported rate limits of 60 API calls per hour when I try to do this for all repositories in a GitHub user/organisation.
Attempt I
I tried the more general format to get the commit lists in a single API call:
curl -H "Accept: application/vnd.github.v3+json" https://api.github.com/repos/$some_user/commits?per_page=10&page=1
Which returned:
{ "message": "Not Found",
"documentation_url": "https://docs.github.com/rest/reference/repos#get-a-repository"
}
Attempt II
Another approach to get the data without triggering the API rate limit would be to parse the atom format of each repository, however, it seems like this is an undesirable hack/more boilerplate code than needed.
Question
Hence, I was wondering, how can one get a list/json of/containing all/the most recent 100/n commits across all the repositories of a GitHub user/organisation, using the GitHub API in Bash?
I have created a script that automatically backs up my GitHub repositories on a hard drive.
I use my Github username in combination with a personal access token to get authorized to Github. Now I've been reading a bit in their documentation about how to get ALL my repositories from the API (public & private), but I can only seem to get the public...
My script: https://github.com/TomTruyen/GitHub-Backup-Script/blob/main/github_backup_script.sh
From what I can understand on line 78, the url should return all my 'owned' repositories (which should include my privates ones)
repositories=$(curl -XGET -s https://"${GITHUB_USERNAME}":"${GITHUB_TOKEN}"#api.github.com/users/"${GITHUB_USERNAME}"/repos?per_page="${repository_count}" | jq -c --raw-output ".[] | {name, ssh_url}")
I have already enabled ALL repository scopes which should give me 'Full control of private repositories (and public)'
I'm out of ideas right now... am I doing something wrong?
NOTE: I'm trying to get my private repositories as a USER, not as an organization
NOTE: ${GITHUB_USERNAME} & ${GITHUB_TOKEN} are variables that I have of course filled in, in my script
You're calling the /users endpoint, but looking at List repositories for the authenticated user it looks like you should be calling /user/repos.
By default this will return all repositories, both public and private, for the currently authenticated user. You'll also need to correctly handle pagination (unless you know for sure you have fewer than 100 repositories).
I was able to fetch a list of all my repositories using the following script:
#!/bin/sh
#
# you must set GH_API_USER and GH_API_TOKEN in your environment
tmpfile=$(mktemp curlXXXXXX)
trap "rm -f $tmpfile" EXIT
page=0
while :; do
let page++
curl -sf -o $tmpfile \
-u "$GH_API_USER:$GH_API_TOKEN" \
"https://api.github.com/user/repos?per_page=100&page=$page&visibility=all"
count=$(jq length $tmpfile)
if [[ $count = 0 ]]; then
break
fi
jq '.[]|.full_name' $tmpfile
done
visibility=all is the key thing here, guys. I spent hours but end up not getting what I wanted. It's been mentioned in the doc as well -
https://docs.github.com/en/rest/repos/repos#list-repositories-for-the-authenticated-user
I am having an issue when trying to upload an image to twitter using a bash script and twurl.
When I use a variable (which is storing the media_id of the image I want to upload) as the parameter for the "media_ids" that I appended to my status update, no photo gets posted and I receive this error message:
twurl -j -X POST -H upload.twitter.com "/1.1/media/upload.json" -f /root/$imgName -F media > mediaID.txt
local mediaID=$(egrep -o " [0-9]{19}" mediaID.txt)
twurl -X POST -H api.twitter.com "/1.1/statuses/update.json?status=The Astronomy Photo Of The Day (courtesy of NASA) for the day $date is \"$title\". For more details and info check out: https://apod.nasa.gov/apod/astropix.html&media_ids=$mediaID" | jq
“errors”: [
{
“code”: 44,
“message”: “media_ids parameter is invalid.”
}
]
However, when I use the actual media_id value instead of the variable ($mediaID) of the image as the parameter for the "media_ids" that gets appended to the status update, everything works as expected.
Is there some issue with using a variable as a parameter for media_id?
I am brand new to twurl and api's and therefore might be missing some basic point.
I would really appreciate any help or suggestions on this topic.
Thank you!
For a project I need to extract data from a lot of different blockchain GitHub profiles to a csv.
After browsing through the GitHub API I was able to achieve some of the necessary data being shown as txt/csv files using bash commands and jq.
Now doing all of this manually would probably take 7 days. I have a list of profiles i need to loop through saved as CSV.
The list looks like this --> https://docs.google.com/spreadsheets/d/1lFsewAYI7F8zSw7WPhI9E9WwR8f4G1clw1yjxY3wz_4/edit#gid=0
My approach so far to get all the repo names looks like this:
sample='[{"name":"0chain"},{"name":"0stateapp"},{"name":"0xcert"}]'
the csv belongs in here, I didn't know how to redirect it to that variable yet, but for testing purposes this was enough. If somebody knows how to, feel free to give a hint.
for row in $(echo "${sample}" | jq -r '.[] | #base64'); do
_jq()
{
echo ${row} | base64 --decode | jq -r ${1}
}
for GHUSER in $( echo $(_jq '.name')); do
curl -s https://api.github.com/users/$GHUSER/repos?per_page=100 | jq -r '.[]|.full_name'
done
done
The output looks like this:
0chain/0chain-token
0chain/client-sdk
0chain/docs
0chain/gorocksdb
0chain/hostadmin
0chain/rocksdb
0stateapp/ZSCoin
0xcert/0xcert
0xcert/conventions
0xcert/docs
0xcert/erc721-validator
0xcert/erc721-validator-api
0xcert/erc721-validator-ui
0xcert/erc721-website
0xcert/ethereum
0xcert/ethereum-crowdsale
0xcert/ethereum-dex
0xcert/ethereum-erc20
0xcert/ethereum-erc721
0xcert/ethereum-minter
0xcert/ethereum-utils
0xcert/ethereum-xcert
0xcert/ethereum-xcert-builder
0xcert/ethereum-zxc
0xcert/framework
0xcert/framework-cert-test
0xcert/nonfungiblealliance-www
0xcert/solidity-style-guide
0xcert/techpaper
0xcert/truffle
0xcert/web3.js
What I need to do is use all of the above values and generate a file that contains:
Github Profile (already stored in the attached sheet)
The Date when accessing this information
All the repositories belonging to that profile (code above but
filtered)
Now the Interesting part:
The commit history
number of commit (ID)
number of commit (ID)
Date of commit
Description of commit
person who commited
checks passed
checks failed
Almost the same needs to be done for closed and open pull requests although I think when solving the "problem" above solving the pull requests is the same strategy.
For the commits I'd do something like this:
for commits in $( $repoarray) do curl -i https://api.github.com/repos/$commits/commits | jq -r '.[]|.author.lgoin (and whatever els is needed)' done
basically this chart here needs to be filled
https://docs.google.com/spreadsheets/d/1mFXiohiWNXNP8CVztFA1PFF41jn3J9sRUhYALZShsPY/edit?usp=sharing
what I need help with:
storing my output from the first loop in a an array
loop through that array to get the number of commits
loop through that array to get the data to closed pull requests
loop through that array to get the data to open pull requests
Excuse my "noobish" question.
I'm using bash/jq and the GitHub API for the time.
I'd appreciate any kind of help.
I've ran into a strange issue. I'm trying to script my router to collect usage stats and other stuff. I'm making one cURL to the auth URL to get a valid session id, then another using that session id to the page I need.
Here is my script:
SESSION_ID=$(curl --silent -D - -X POST http://10.0.0.1/login.cgi -d'admin_username=admin&admin_password=admin' | grep 'SESSION' | sed 's/Set-Cookie: SESSION=//' | sed 's/; path=\///')
echo $SESSION_ID # 1234567890
curl -v -H "Cookie: SESSION=$SESSION_ID" http://10.0.0.1/modemstatus_dslstatus.html
If I manually take SESSION_ID and insert it in place of '"$SESSION_ID"' everything is dandy. cURL shows the headers (via -v) and they are correct. Running the command while manually inserting the session id produces identical headers.
I'm sure it's something small. Please teach me something :)
Check for carriage returns \r in your variables which wouldn't appear with a simple echo in some cases.