I would like to write a PowerShell script to download the GitHub repo in ZIP format by following this instruction:
https://docs.github.com/en/rest/reference/repos#contents
$Token = 'MyUserName:MyPAT'
$Base64Token = [System.Convert]::ToBase64String([char[]]$Token)
$Headers = #{
"Authorization" = 'Basic {0}' -f $Base64Token;
"accept" = "application/vnd.github.v3+json"
}
$Uri = "https://api.github.com/repos/{owner}/{repo}/zipball"
$r = Invoke-WebRequest -Headers $Headers -Uri $Uri -Method Get | Out-File "D:\MyRepo.zip"
The code did download the zip file but I got this error message when I tried to open the zip file:
D:\MyRepo.zip
The archive is either in unknown format or damaged
I am very new to PowerShell, any help is appreciated!
You may need to look more closely at download-a-repository-archive-zip instructions. It says the response will have a 302 redirect to the URL for downloading. Invoke-WebRequest will not automatically redirect, but it will provide the response headers.
If you change your last line to be:
$response = Invoke-WebRequest -Headers $Headers -Uri $Uri -Method Get
you can review the $response object's Headers and issue another Invoke-WebRequest with the same headers and the 302 Uri:
$RedirectedResponse = Invoke-WebRequest -Headers $Headers -Uri $RedirectedURI -Method Get
$RedirectedResponse.Content will have the encoded file contents that you can decode and write to your local filesystem.
EDIT: I got to a system where I had GitHub access and tested the script. I found that the first response had a byte array with the zip file contents. This functionality is too useful not to share! Here's a script that works to download a repo:
$user = 'bjorkstromm'
$repo = 'depends'
$uri = "https://api.github.com/repos/$user/$repo/zipball/"
if(!$cred){$cred = Get-Credential -Message 'Provide GitHub credentials' -UserName $user}
$headers = #{
"Authorization" = "Basic " + [convert]::ToBase64String([char[]] ($cred.GetNetworkCredential().UserName + ':' + $cred.GetNetworkCredential().Password))
"Accept" = "application/vnd.github.v3+json"
}
$response = Invoke-WebRequest -Method Get -Headers $headers -Uri $uri
$filename = $response.headers['content-disposition'].Split('=')[1]
Set-Content -Path (join-path "$HOME\Desktop" $filename) -Encoding byte -Value $response.Content
I am trying to create rest api to update configuration in all environments in urban code.
Is there any rest client or do we need write any custom code.?
How do I start? Your kind suggestions please
Are you asking about updating environment properties? If so. I do this with Powershell.
$webUrl = "https://ibm-ucd.myCompany.com"
$ucdApiUserName = "yourCLIAccount"
$ucdApiPass = "yourCLIpassword"
$appName = "MyApplication"
$environment = "ADT"
$propertyNewValue = "myNewValue"
$credential = New-Object System.Management.Automation.PSCredential ($ucdApiUserName,(ConvertTo-SecureString $ucdApiPass -AsPlainText -Force))
####################################################################
## Bypass Cert Issues with connecting to HTTPS API
####################################################################
$certData = [string][System.Net.ServicePointManager]::CertificatePolicy
if ($certData -ne "TrustAllCertsPolicy")
{
add-type #"
using System.Net;
using System.Security.Cryptography.X509Certificates;
public class TrustAllCertsPolicy : ICertificatePolicy {
public bool CheckValidationResult(
ServicePoint srvPoint, X509Certificate certificate,
WebRequest request, int certificateProblem) {
return true;
}
}
"#
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
}
$hash = [ordered] #{
"application" = "$AppName";
"environment" = "$environment";
"isSecure" = "false";
"name" = "myEnvProperty";
"value" = "$propertyNewValue"
}
$newValuesJson = $hash | ConvertTo-Json
Write-Host "Updating uDeploy environment properties for $environment"
$uri = "$WebUrl/cli/environment/propValue?"
Invoke-RestMethod -Method 'PUT' -ContentType "application/json" -Credential
$credential -uri $uri -Body $newValuesJson | Out-Null
With cURL, we can pass a username with an HTTP web request as follows:
$ curl -u <your_username> https://api.github.com/user
The -u flag accepts a username for authentication, and then cURL will request the password. The cURL example is for Basic authentication with the GitHub Api.
How do we similarly pass a username and password along with Invoke-WebRequest? The ultimate goal is to user PowerShell with Basic authentication in the GitHub API.
I am assuming Basic authentication here.
$cred = Get-Credential
Invoke-WebRequest -Uri 'https://whatever' -Credential $cred
You can get your credential through other means (Import-Clixml, etc.), but it does have to be a [PSCredential] object.
Edit based on comments:
GitHub is breaking RFC as they explain in the link you provided:
The API supports Basic Authentication as defined in RFC2617 with a few
slight differences. The main difference is that the RFC requires
unauthenticated requests to be answered with 401 Unauthorized
responses. In many places, this would disclose the existence of user
data. Instead, the GitHub API responds with 404 Not Found. This may
cause problems for HTTP libraries that assume a 401 Unauthorized
response. The solution is to manually craft the Authorization header.
Powershell's Invoke-WebRequest does to my knowledge wait for a 401 response before sending the credentials, and since GitHub never provides one, your credentials will never be sent.
Manually build the headers
Instead you'll have to create the basic auth headers yourself.
Basic authentication takes a string that consists of the username and password separated by a colon user:pass and then sends the Base64 encoded result of that.
Code like this should work:
$user = 'user'
$pass = 'pass'
$pair = "$($user):$($pass)"
$encodedCreds = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes($pair))
$basicAuthValue = "Basic $encodedCreds"
$Headers = #{
Authorization = $basicAuthValue
}
Invoke-WebRequest -Uri 'https://whatever' -Headers $Headers
You could combine some of the string concatenation but I wanted to break it out to make it clearer.
Use this:
$root = 'REST_SERVICE_URL'
$user = "user"
$pass= "password"
$secpasswd = ConvertTo-SecureString $pass -AsPlainText -Force
$credential = New-Object System.Management.Automation.PSCredential($user, $secpasswd)
$result = Invoke-RestMethod $root -Credential $credential
If someone would need a one liner:
iwr -Uri 'https://api.github.com/user' -Headers #{ Authorization = "Basic "+ [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes("user:pass")) }
Invoke-WebRequest follows the RFC2617 as #briantist noted, however there are some systems (e.g. JFrog Artifactory) that allow anonymous usage if the Authorization header is absent, but will respond with 401 Forbidden if the header contains invalid credentials.
This can be used to trigger the 401 Forbidden response and get -Credentials to work.
$login = Get-Credential -Message "Enter Credentials for Artifactory"
#Basic foo:bar
$headers = #{ Authorization = "Basic Zm9vOmJhcg==" }
Invoke-WebRequest -Credential $login -Headers $headers -Uri "..."
This will send the invalid header the first time, which will be replaced with the valid credentials in the second request since -Credentials overrides the Authorization header.
Tested with Powershell 5.1
I had to do this to get it to work:
$pair = "$($user):$($pass)"
$encodedCredentials = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes($Pair))
$headers = #{ Authorization = "Basic $encodedCredentials" }
Invoke-WebRequest -Uri $url -Method Get -Headers $headers -OutFile Config.html
Here is another way using WebRequest, I hope it will work for you
$user = 'whatever'
$pass = 'whatever'
$secpasswd = ConvertTo-SecureString $pass -AsPlainText -Force
$credential = New-Object System.Management.Automation.PSCredential($user, $secpasswd)
$headers = #{ Authorization = "Basic Zm9vOmJhcg==" }
Invoke-WebRequest -Credential $credential -Headers $headers -Uri "https://dc01.test.local/"
This is what worked for our particular situation.
Notes are from Wikipedia on Basic Auth from the Client Side. Thank you to #briantist's answer for the help!
Combine the username and password into a single string username:password
$user = "shaunluttin"
$pass = "super-strong-alpha-numeric-symbolic-long-password"
$pair = "${user}:${pass}"
Encode the string to the RFC2045-MIME variant of Base64, except not limited to 76 char/line.
$bytes = [System.Text.Encoding]::ASCII.GetBytes($pair)
$base64 = [System.Convert]::ToBase64String($bytes)
Create the Auth value as the method, a space, and then the encoded pair Method Base64String
$basicAuthValue = "Basic $base64"
Create the header Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==
$headers = #{ Authorization = $basicAuthValue }
Invoke the web-request
Invoke-WebRequest -uri "https://api.github.com/user" -Headers $headers
The PowerShell version of this is more verbose than the cURL version is. Why is that? #briantist pointed out that GitHub is breaking the RFC and PowerShell is sticking to it. Does that mean that cURL is also breaking with the standard?
another way is to use certutil.exe
save your username and password in a file e.g. in.txt as username:password
certutil -encode in.txt out.txt
Now you should be able to use auth value from out.txt
$headers = #{ Authorization = "Basic $((get-content out.txt)[1])" }
Invoke-WebRequest -Uri 'https://whatever' -Headers $Headers
I know this is a little off the OPs original request but I came across this while looking for a way to use Invoke-WebRequest against a site requiring basic authentication.
The difference is, I did not want to record the password in the script. Instead, I wanted to prompt the script runner for credentials for the site.
Here's how I handled it
$creds = Get-Credential
$basicCreds = [pscredential]::new($Creds.UserName,$Creds.Password)
Invoke-WebRequest -Uri $URL -Credential $basicCreds
The result is the script runner is prompted with a login dialog for the U/P then, Invoke-WebRequest is able to access the site with those credentials. This works because $Creds.Password is already an encrypted string.
I hope this helps someone looking for a similar solution to the above question but without saving the username or PW in the script
I'm trying to download a file from Linux server with Apache web server to Windows Server 2012 R2 using Windows PowerShell
Note: the URL is HTTPS
$source = "https://uri"
$destination = "C:\path\file.txt"
$username = "admin"
$password = "#dfkl!f" | ConvertTo-SecureString -asPlainText -Force
$cred = New-Object System.Management.Automation.PSCredential($username,$password)
Invoke-WebRequest $source -OutFile $destination -Credential $cred
Invoke-WebRequest : Authorization Required
This server could not verify that you are authorized to access the document requested. Either you supplied the wrong
credentials (e.g., bad password), or your browser doesn't understand how to supply the credentials required.
Apache/2.2.15 (CentOS) Server at url Port 443
When I type the credentials through browser i m able to download but through powershell it show bad credentials
I just tried this against one of my Apache/Linux boxes on a SSL page that uses Basic auth, and it seemed to work... Your mileage might vary...
$source = "https://Uri"
$destination = "C:\Foo\Bar"
$username = 'mylogin'
$password = 'reallgoodpassword'
$auth = 'Basic ' + [System.Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes($username+':'+$password ))
Invoke-WebRequest -Headers #{ "Accept" = "*/*"; "Authorization" = $auth } -Uri $source -Method Get
try to create the passwordstring with
$password = ConvertTo-SecureString "#dfkl!f" -AsPlainText -Force
I am following below to upload files to onedrive using powershell.
http://www.powershellmagazine.com/2014/04/07/using-the-windows-live-apis-from-powershell/
Invoke-RestMethod -Uri $Uri -Method Put -InFile $Path
where is a full path to a file
$Path= "C:\SkyDrive/ServerBackups/Webs/Webs/test.txt"
and
$Uri = "https://login.live.com/oauth20_desktop.srf?code=XXXXXX
throws a (404) Not found error.
The value of $Uri used in the Invoke-RestMethod is wrong. The OAuth endpoint you used in your script is for authentication rather than OneDrive operation.
About how to upload file to OneDrive, this can be split into three parts.
Register OneDrive application for OAuth authentication process
Call OneDrive authentication API to get access token
Use access token to call OneDrive API so as to upload file
After you created your OneDrive application, you can get Application Id, secret key and redirect Uri if the application is web type. Then use the script below with the three value.
# get authorize code
Function Get-AuthroizeCode
{
[CmdletBinding()]
Param
(
[Parameter(Mandatory=$true)][String]$ClientId,
[Parameter(Mandatory=$true)][String]$RedirectURI
)
# the login url
$loginUrl = "https://login.live.com/oauth20_authorize.srf?client_id=$ClientId&scope=onedrive.readwrite offline_access&response_type=code&redirect_uri=$RedirectURI";
# open ie to do authentication
$ie = New-Object -ComObject "InternetExplorer.Application"
$ie.Navigate2($loginUrl) | Out-Null
$ie.Visible = $True
While($ie.Busy -Or -Not $ie.LocationURL.StartsWith($RedirectURI)) {
Start-Sleep -Milliseconds 500
}
# get authorizeCode
$authorizeCode = $ie.LocationURL.SubString($ie.LocationURL.IndexOf("=") + 1).Trim();
$ie.Quit() | Out-Null
RETURN $authorizeCode
}
# get access token and refresh token
Function New-AccessTokenAndRefreshToken
{
[CmdletBinding()]
Param
(
[Parameter(Mandatory=$true)][String]$ClientId,
[Parameter(Mandatory=$true)][String]$RedirectURI,
[Parameter(Mandatory=$true)][String]$SecrectKey
)
# get authorize code firstly
$AuthorizeCode = Get-AuthroizeCode -ClientId $ClientId -RedirectURI $RedirectURI
$redeemURI = "https://login.live.com/oauth20_token.srf"
$header = #{"Content-Type"="application/x-www-form-urlencoded"}
$postBody = "client_id=$ClientId&redirect_uri=$RedirectURI&client_secret=$SecrectKey&code=$AuthorizeCode&grant_type=authorization_code"
$response = Invoke-RestMethod -Headers $header -Method Post -Uri $redeemURI -Body $postBody
$AccessRefreshToken = New-Object PSObject
$AccessRefreshToken | Add-Member -Type NoteProperty -Name AccessToken -Value $response.access_token
$AccessRefreshToken | Add-Member -Type NoteProperty -Name RefreshToken -Value $response.refresh_token
RETURN $AccessRefreshToken
}
Then you get your valid access token, you can build valid header for Invoke-RestMethod
# get autheticate header
Function Get-AuthenticateHeader
{
[CmdletBinding()]
Param
(
[Parameter(Mandatory=$true)][String]$AccessToken
)
RETURN #{"Authorization" = "bearer $AccessToken"}
}
Finally, you can pass the header to Invoke-RestMethod with upload rest api.
There have four different OneDrive upload rest Api you can call.
Simple item upload
Resumable item upload
Multipart item upload
Upload from Url
If your target file is not too big, means the length of the file is within 100MB, I highly recommend to use Simple item upload.
For big file, it is common to call Resumable item upload API
It is a big story, you can refer to this upload file to OneDrive sample for the complete script directly.
Hopefully you can get rid of the problem.
You can't PUT a file to the OAuth endpoint... A quick scan of the referenced article suggests you should be using a $Uri value that looks more like $wlApiUri. See http://msdn.microsoft.com/en-US/library/dn659726.aspx for a description of the URL scheme for this kind of operation.