Access web using Powershell and Proxy - proxy

Question

I can't seem to get access to a webpage using Powershell. I keep getting a "(407) Proxy Authentication Required". I've tried many things. I don't know what the proxy is or what kind of authentication it requires. The only thing I have access to is in IE it uses a script for configuring. I tried using some IPs from that, but no luck. Any ideas?
Here is one example of what I tried:
$wc = New-Object System.Net.WebClient
$wc.Headers.Add("User-Agent","Mozilla/4.0+")
$wc.Proxy = [System.Net.WebRequest]::DefaultWebProxy
$wc.Proxy.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials
$wc.DownloadString("http://stackoverflow.com")

Answer 1

I had a similar issue and resolved it with just two lines of powershell:
$browser = New-Object System.Net.WebClient
$browser.Proxy.Credentials =[System.Net.CredentialCache]::DefaultNetworkCredentials

Answer 2

I haven't seen anyone doing something like this but there is a way to do this as a "global setting" in your Powershell script (I remember doing this in C# before for local dev builds).
[System.Net.WebRequest]::DefaultWebProxy = [System.Net.WebRequest]::GetSystemWebProxy()
[System.Net.WebRequest]::DefaultWebProxy.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials
This way if you don't want to update all your WebClients with proxy details, you can just override the global setting (have to be done every time you run the script).
But this assumes that the current logged in Windows user is valid for the system-defined proxy server.
NOTE: I would say that this is only useful as a quick and dirty way to get a PS script working that wasn't proxy aware before (like Cake build).

Answer 3

If the proxy answers "407", "proxy authentication required", then the authentication is required:
$Username="Hugo"
$Password="abcdefgh"
$WebProxy = New-Object System.Net.WebProxy("http://webproxy:8080",$true)
$url="http://aaa.bbb.ccc.ddd/rss.xml"
$WebClient = New-Object net.webclient
$WebClient.Proxy=$webproxy
$WebClient.proxy.Credentials = New-Object System.Net.NetworkCredential($Username, $Password)
$path="C:\Users\hugo\xml\test.xml"
$WebClient.DownloadFile($url, $path)
Content now resides in "test.xml"

Answer 4

I know the question is specifically about Powershell 2.0, but I would like to share information on setting up a proxy server in Powershell Core (6+) because it's extremely hard to find elsewhere.
I agree with Dandré, that the best solution is to configure the default proxy server. I just had an issue with Powershell Core (7.1). I was trying exactly what Dandré suggests, but it didn't have any effect. After several hours of research and investigation, I have found out that Powershell Core is probably not using System.Net.WebRequest to make web requests anymore, but rather System.Net.Nett.HttpClient.
When I have configured the HttpClient's default proxy server, all web connections made by Powershell (Invoke-WebRequest, PowerShellGet's Find-Module, Install-Module, etc.) finally started to work.
Configure Default Proxy Server for Powershell Core
If you need to make the configuration permanent, just add the commands below to your Powershell profile.
Configure a specific default proxy server
[System.Net.Http.HttpClient]::DefaultProxy = New-Object System.Net.WebProxy('http://your-proxy')
If you need to set a specific port, add it to the proxy server URI: http://proxy:1234.
Configure credentials for authenticating proxy
In case it's an authenticating proxy, you need to set up the credentials to be used for proxy authentication. Use the following command to set the credentials of your domain account under which you're currently logged in to Windows:
[System.Net.Http.HttpClient]::DefaultProxy.Credentials = [System.Net.CredentialCache]::DefaultCredentials
If you need different credentials, you can use the Get-Credential cmdlet, but it's interactive so it's not an ideal solution to have in your Powershell profile. But I'm sure there're other ways.
[System.Net.Http.HttpClient]::DefaultProxy.Credentials = Get-Credential
Configure proxy server bypassing
If you just need to bypass the proxy server and use the direct connection, but Powershell is using the default system-wide proxy server, simply set the HttpClient's default proxy to null:
[System.Net.Http.HttpClient]::DefaultProxy = New-Object System.Net.WebProxy($null)
Add Proxy Configuration to Powershell Profile
To make the configuration permanent, simply add the commands you need to your Powershell profile. There're four of them (All users, all hosts; All users, current host; Current user, all hosts; Current user, current host), but I would use the "Current user, all hosts". To find out the location of a specific profile, use the $Profile variable with a profile name:
$Profile.CurrentUserAllHosts
It should print the path $Home\Documents\Powershell\profile.ps1.
If the profile doesn't exist yet, just create it and put the configuration there. It will be executed every time you execute a new Powershell Core (pwsh.exe) instance.
Configure Default Proxy using Environment Variable
An alternative solution is to use an environment variable. According to the documentation, HttpClient is actually using the following variables to initialize the default proxy if they're defined:
HTTP_PROXY for HTTP requests
HTTPS_PROXY for HTTPS requests
ALL_PROXY for both HTTP and HTTPS
NO_PROXY may contain a comma-separated list of hostnames excluded from proxying
An example usage:
ALL_PROXY='http://proxy:1234'
And if you need to pass credentials:
ALL_PROXY='http://username:password#proxy:1234'
This solution is supported by a wider range of applications, so if it's better or worse than the previous solution depends on what exactly you want to configure, just Powershell or also other applications.

Answer 5

If you use the following you'll receive a prompt to enter your credentials:
$client.Credentials = Get-Credential

Answer 6

I solved this with just three lines:
$proxy='http://username:password#IP:PORT'
$ENV:HTTP_PROXY=$proxy
$ENV:HTTPS_PROXY=$proxy

Answer 7

If you know the script - just download it, open with Notepad and find IP and port of your proxy server. As for authentication - most probably your windows credentials are used, so in theory you should be able to keep it empty, unless there's something suspicious in the script.

Answer 8

try adding cache credentials....
$domain = 'ChDom'
$Client = new-object System.Net.WebClient
$cc = New-object System.Net.CredentialCache
$urlObj = New-object System.Uri($url)
#these variables must be plaintext strings
$creds = New-object System.Net.NetworkCredential($Username,$Password)
#your auth might be different than mine
$cc.add($urlobj,"NTLM",$creds)
$client.Credentials = $cc
$Client.Downloadfile($url, C:\Temp\TestPage.html)

Answer 9

Option 1 - Setting Proxy Authentication via Code.
Another Authenticated Proxy Example With Credentials
# 1). Set your http proxy address and port
$proxy = New-Object System.Net.WebProxy("http://p.weshare.net:80",$true)
# 2). Set your http proxy credentials
$proxy.Credentials = New-Object System.Net.NetworkCredential("username", "password")
# 3). Set the global proxy
[System.Net.WebRequest]::DefaultWebProxy = $proxy
# 4). sets a value that indicates whether to bypass the proxy server for local addresses.
[System.Net.WebRequest]::DefaultWebProxy.BypassProxyOnLocal = $true
# 5). Call Invoke-WebRequest without -Proxy Tag ( as it pulls it from what you defined )
$reposnseText = Invoke-WebRequest https://www.oracle.com
Just wrapping it as a Function
Function Set-My-Proxy
{
param($proxy,$username,$password)
$proxy = New-Object System.Net.WebProxy($proxy,$true)
$proxy.Credentials = New-Object System.Net.NetworkCredential($username, $password)
[System.Net.WebRequest]::DefaultWebProxy = $proxy
[System.Net.WebRequest]::DefaultWebProxy.BypassProxyOnLocal = $true
}
Call the Function
Set-My-Proxy 'http://p.weshare.net:80' 'username' 'password'
Option 2 - Setting Proxy Authentication via Another Proxy
Credits & Limitations of solution
Install local proxy, e.g : squid for windows
After installing, open squid.conf file ( right mouse click on squid icon on task bar -> Open Squid Configuration
Put the following code
please provide authenticated proxy ip only ( in our example : 10.1.2.3 - not domain one )
http_access allow all
http_port 3128
coredump_dir /var/spool/squid3
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern (Release|Packages(.gz)*)$ 0 20% 2880
refresh_pattern . 0 20% 4320
cache_peer 10.1.2.3 parent 80 0 no-query default login=my_username:my_password
never_direct allow all
access_log none
cache_log none
Restart Squid Service
You now may call your local proxy, which will forward request to authenticated proxy and response back
Invoke-WebRequest -Uri 'https://stackoverflow.com/' -Proxy 'http://127.0.0.1:3128'