How to download artifacts from CircleCI via Powershell command Invoke-RESTMethod - windows

Question

I'm trying to get artifacts from CircleCI in Powershell and getting back an unfamiliar data format?
Powershell likes to auto-convert your API’s JSON response to a PSCustomObject. This normally would be what I want.
Here is an example of my attempts at getting clean data.
Add the necessary .NET assembly
Add-Type -AssemblyName System.Net.Http
Create the HttpClient object
$client = New-Object -TypeName System.Net.Http.Httpclient
Get the web content.
$task = $client.GetByteArrayAsync(“https://circleci.com/api/v1.1/project/$vcs_type/$username/$project/$build_number/artifacts?circle-token=$CIRCLE_TOKEN”)
Wait for the async call to finish
$task.wait();
results:
(({
:path “src./file1.txt”,
:pretty-path “src/file1.txt”,
:node-index 0,
:url “https://15-198716507-gh.circle-artifacts.com/0/src/file1.txt”
} {
:path “src/file2.txt”,
:pretty-path “src/file2.txt”,
:node-index 0,
:url “https://15-198716507-gh.circle-artifacts.com/0/src/file2.txt”
}…continued
As you can see this is not JSON or YAML. Let’s try the built-in PowerShell tools like Invoke-RestMethod.
Invoke-RESTMethod -uri https://circleci.com/api/v1.1/project/$vcs_type/$username/$project/$build_number/artifacts?circle-token=$CIRCLE_TOKEN -Method GET
**Output:**
({
:path “src/file1.txt”,
:pretty-path “src/file1.txt”,
:node-index 0,
:url “https://15-198716507-gh.circle-artifacts.com/0/src/file1.txt”
} {
:path “src/file2.txt”,
:pretty-path “src/file2.txt”,
:node-index 0,
:url “https://15-198716507-gh.circle-artifacts.com/0/src/file2.txt”
}…continued
Dang same output. I know from the Invoke-RestMethod documentation that PS sees JSON and auto converts it to a PS object. Maybe it's converting a data type I'm not familiar with? I found it odd that PowerShell was getting EDN type when every other attempt outside PowerShell was JSON.
Maybe they should have the API updated to reply to PS request with JSON by default.
What is wrong with PowerShell not getting JSON data?

Answer 1

It's EDN, didn't know this until CircleCI answered a question on this topic. So if you are using PowerShell to retrieve artifacts from CircleCI you definitely want to know this.
you need to pass a header specifying the data type returned.
(Accept: application/json)
A CircleCI Support member let me know that in PowerShell you have to specify an Accept header to receive the data in JSON. No wonder I'm getting weird output!
So trying again with the new accept JSON header we have this command below.
Working command to get data in JSON and have it auto-convert to a PSObject.
Invoke-RestMethod -Uri https://circleci.com/api/v1.1/project/$vcs_type/$username/$project/$build_number/artifacts?circle-token=$CIRCLE_TOKEN -Method GET -ContentType 'application/json' -UseBasicParsing -Header #{"Accept" = "application/json"}
OUTPUT
$response|select path,url
path
----
src.orig/file1.txt
src.orig/file2.txt
url
---
https://15-824975-gh.circle-artifacts.com/0/src.orig/file1.txt
https://15-824975-gh.circle-artifacts.com/0/src.orig/file2.txt
Using the PS commands Invoke-WebRequest/Invoke-RestMethod both will receive data in EDN format if you don't do the below. Yay now I can use the data as I see fit to download my artifacts.
Reply from CircleCI that got me the solution.
#burninmedia So what's being sent back is actually a data format called EDN. If you want to return JSON you'll need to pass a header specifying so (Accept: application/json). Thanks!
Here is a simple script I wrote to download all the artifacts. Please be sure you're setting the environment variables.
if ($USERNAME -eq $null) { write-host " please add required variable USERNAME" ;exit }
if ($VCS_TYPE -eq $null) { write-host " please add required variable VCS_TYPE" ;exit}
if ($CIRCLE_TOKEN -eq $null) { write-host " please add required variable CIRCLE_TOKEN" ;exit}
if ($BUILD_NUMBER -eq $null) { write-host " please add required variable BUILD_NUMBER" ;exit}
if ($PROJECT -eq $null) { write-host " please add required variable PROJECT" ;exit}
if ($BASEPATH -eq $null) { write-host " please add required variable BASEPATH" ;exit}
$response = Invoke-RestMethod -Uri https://circleci.com/api/v1.1/project/$VCS_TYPE/$USERNAME/$PROJECT/$BUILD_NUMBER/artifacts?circle-token=$CIRCLE_TOKEN -Method GET -ContentType 'application/json' -UseBasicParsing -Header #{"Accept" = "application/json"}
ForEach ($i in $response){
$PATH = $(Split-Path -Path "$($BASEPATH)\$($i.path)")
if (-Not ( Test-Path $PATH) ) {
write-host "Creating folder: $($PATH)"
New-Item -ItemType Directory -Force -Path "$($PATH)"
}
Write-Host "Saving artifact $($i.pretty_path) to file: $($BASEPATH)\$($i.path)"
Invoke-RestMethod "$($i.url)?circle-token=$($CIRCLE_TOKEN)" -UseBasicParsing -OutFile "$($BASEPATH)\$($i.path)"
}
Bash version
export CIRCLE_TOKEN=':your_token'
echo $(https://circleci.com/api/v1.1/project/$vcs-type/$username/$project/$build_number/artifacts?circle-token=$CIRCLE_TOKEN) > ./artifact_json
for ((i = 0 ; i <= $(jq -c '.[].url ' ./artifact_json|wc -l) ; i++));
do
path=$(jq -c ".[$i].path" ./artifact_json|tr -d '"');
url=$(jq -c ".[$i].url" ./artifact_json|tr -d '"');
pathdir=$(dirname "$path")
echo "URL: $url"
echo "path: $path"
echo "Pathdir: $pathdir"
[ -d $pathdir ] && mkdir -p "$pathdir" #check if folder exists if not mkdir
wget -o $path $url
done
rm ./artifact_json```