How to Proxy auth on PhantomJS (proxy script) - proxy

I go throught the intranet/internet using proxy auth.
I'm not familiar with automation throught a proxy or proxys, in IExplorer we set up the proxy on LAN settings in "Use automatic configuration script" with something like:
http://some-url/url/file.proxy
Uncheck "Automatically detect settings" and we don't set any in the "Proxy Server" section.
So we can go "out" (internet/intranet).I have a username/password so everytime I just open a new IE instance, I got a prompt for them. How should I set this values on PhantomJS to get access to the network/internet ? I jus can't make it work, everytime I try to get a screenshot from anypage I got a webpage screenshot related to the proxy auth.
I've tried set the full/script.proxy url in the proxy prop and username/password but didn't work. Hope someone can provide an example for my understanding. Also I'll appreaciate some resources/good-to-read articles.

Got it.
I take a look to the script I just mentioned in my question, and just got the proxy (ip:port) needed in phantomJS.
Basically the script do some decisions about what proxy to use based on the requested url and return the proxy ip.
So the PhantomJS docs it's preety straightforward, I wasn't understanding how my proxy was set (by script),if it's your case, you can copy/paste the script url into your browser so you can analyze it and retrieve the information you need to setup phantomJs
he code in the proxy script is kind of easy to read (if you have any programming experiencie).

Related

Chrome policy "ExternalProtocolDialogShowAlwaysOpenCheckbox" not working

I am using the library https://github.com/aik099/PhpStormProtocol/ to get links to my IDE (PHPStorm) from the browser (Chrome). Everytime xdebug prints a warning, it attaches a link with an href like this:
phpstorm://open?file={filepath}&line={line}
That works perfect. The problem is that every time I click in the link I get a pop-up like this:
I have tried following the steps I found here:
https://superuser.com/questions/1481851/disable-chrome-to-ask-for-confirmation-to-open-external-application-everytime
After enabling that policy in the register of windows, and validating it also in my browser I still have the same pop-up without the checkbox to allow this protocol forever.
The problem is, that you want to open your url handler over a "http" connection.
But you get only the square if you have a "https" connection.
You can solve your problem if you set a rule, that your "http" connection is trusted.
For chrome write "chrome://flags/" in your address bar and write your url into the option "Insecure origins treated as secure" and set this to enable.
enter image description here
The External​Protocol​Dialog​Show​Always​Open​Checkbox does work on https websites only.
You can instead add you protocol phpstorm://* to Chrome's URLAllowlist, for example via the Windows registry :
The user will not see the popup at all and the software will open automatically.

Postman can't reach localhost

I am in a corporate env so I have to use a proxy to reach servers. This works well in postman and in browsers. What I can't reach is localhost in postman but I can reach localhost in the browser.
I am running Postman for Linux Version 6.0.9. I have tried reaching localhost:9082/rest/myapi.... and 127.0.0.1:9082/rest/myapi with both global proxy and system proxy turned on and with either turned on and with non turned on. In all cases I am not able to reach localhost.
What I get as an response is an error page from the proxy server! Someway the call gets out on the network instead of being kept on my machine.
The postman console:
My request headers are:
Cache-Control →no-cache
Connection →Keep-Alive
Content-Length →986
Content-Type →text/html; charset=utf-8
Pragma →no-cache
Proxy-Connection →Keep-Alive
My response headers are:
cache-control:"no-cache"
pragma:"no-cache"
content-type:"text/html; charset=utf-8"
proxy-connection:"Keep-Alive"
connection:"Keep-Alive"
content-length:"986"
My response body is an html page.
How can I make a call to localhost work with postman?
I was having similar issue with HTTP calls to local ASP.NET Core Web API apps. Changing the proxy settings didn't fix it for me.
Finally fixed it by turning off File > Settings > General > SSL Certificate Verification
I have the same issue. What works for me is:
Open File -> Settings -> Proxy
Then, enable proxy and put 127.0.0.1 : 80, if your web server runs on 80.
In the git thread, they say it is a known issue, so hopefully it will get resolved soon.
I solved this by Turning off System proxy of Postman.
Open File -> Settings -> Proxy
Then Turn off Use System Proxy
#Menuka Ishan Answer is correct. But in mac only if we change the turn off proxy won't work. You need to turn off the SSL Certificate verification also you need to disable.
I found a temporary solution:
In terminal, go to the directory where postman is installed and add:
machine#dev:~/Documents/Postman$ export NO_PROXY=localhost,127.0.0.1
machine#dev:~/Documents/Postman$ ./Postman
This will make calls to localhost work.
I found a similar problem on Windows machines here
The problem mentioned as an issue here. You should just write a simple batch file like this:
set HTTP_PROXY=
set http_proxy=
set HTTPS_PROXY=
set https_proxy=
START %LOCALAPPDATA%\Postman\Update.exe --processStart "Postman.exe"
open a simple text file, save this as a ".bat" file, and just run it!
I also faced the same issue, but the trick was I implemented both the methods that were stated earlier in the stack.
First I turned off the postman proxy as shown in below image.
Click Settings -> Proxy
And then turned off the SSL certificate verification check button.
Click Settings -> General
And then tried to hit my local server and finally it worked.
While attempting to Get https://localhost:5001/WeatherForecast via PostMan while developing a basic .NET Core (3.1) Web API (on Ubuntu Linux)
I got the error:
Could not get any response There was an error connecting to
https://localhost:5001/WeatherForecast.
I didn't notice the actual solution in the things to try at first, but if you check the highlighted text in the image you will see it.
You can see that the option is turned on in PostMan by default:
I turned the SSL Certificate Verification off and tried again and it worked.
If you are under corporate proxy and you are deploying your webapplication in localhost then no need to set proxy configuration like
http://username:password#corporateProxyServer:ServerPort in postman (Global Proxy Configuration).
As the app is deployed in local machine just use the localhost configuration in setting -> Global Proxy Configuration as :
proxy server : 127.0.0.1
port : 8300 (this is the port where the web application is running )
My problem was that i was setting an invalid Header token and it gave me the error
Error: Header name must be a valid HTTP token ["Accept "]
Warning: This request did not get sent completely and might not have all the required system headers
Just check your Headers.
Adding another potential answer to this for people to check after they've tried the other ideas here.
I was able to solve my own problem by checking the Postman console (View -> Show Postman Console) and then examining the error from the request (in my case it was because I had a newline character on an auth token).
In my case, I had to precede the localhost with https://
These steps solved this issue after 1 whole day struggle:
1) HTTP_PROXY= proxy.company.com:port
HTTPS_PROXY=proxy.company.com:port
to user variables, not to System variables. Make sure the the case is all cap.
2) Updating newtwork driver, and turning on Windows Defender......Make sure you don't see no reds in here.
I was not able to make any calls from postman to docker container running Couchbase Server. Very frustrating. Turns out I had proxy setting configured to some aws instance which has been long deleted.
Goto setting and remove the proxy setting and it worked like a charm on localhost/127.0.0.1/mac laptop eno IP addr or 0.0.0.0
In my case, I was using django-tenants and had to add the domain for the tenant in etc/hosts to get it working on localhosts. My entry for 127.0.0.1 now looks like
127.0.0.1 localhost test.localhost
My problem was forgetting to select Environment. From the drop down where you see No Environment in the image, select the name of your environment.
Menuka Ishan's solution above saved my hours of pain! I left the Global Proxy Configuration switched ON and changed the Proxy Server to 127.0.0.1. Switched OFF the Use System Proxy. I already had the entry 127.0.0.1 localhost in my hosts file under the path C:\Windows\System32\drivers\etc.
If you have active proxy in your system then make sure Global Proxy Configuration and Use System Proxy are turned off. To do this go to Postman preferences > Proxy. You can also try sending a request in postman without typing localhost (e.g. :8080/send)
i think i also found a perfect fix.
First you turn on global proxy configurations, add your local machine Ip which is 127.0.0.1,then also add localhost in the bypass.
Also turn off SSL verification.
It worked for me
This worked for me, i dissable the parrameter host in the header.
HOST Disabled
If anyone is looking for a solution to WooCommerce local install, here are the steps to make this work:
Go to Wordpress Admin > WooCommerce > Settings > Advanced
Create new API key, give it a name
Copy the Consumer Key & Consumer Secret to
your text editor
Open Postman
Create new connection Set the URL for
testing (GET) : http://mylocalsite/wp-json/wc/v3/orders (change
"mylocalsite" to yours)
Click on Authorization tab
Now here is the main change: Although in WC Rest docs, it says to use the "Basic Auth" - IT WILL NOT WORK. Since usually your localhost is HTTP
and not HTTPS. So "Basic Auth" is not relevant in our case.
You will need to change the Authorization to OAuth 1.0
Set Consumer Key & Consumer Secret respectively as it shows in Woocommerce
Click "Send" - it should work now
If still you have problems, try one by one the following:
File > Settings > Turn off SSL Verification
File > Settings > Turn off "Automatically follow redirects"

Fail to create cookies while using ngrok with header rewrite

I use docker as my local dev environment and use the dinghy-http-proxy which adds a new TLD .docker to map request to a nginx-proxy container.
My websites are typically reached through an URL like http://devel.domain.com.docker.
I want to use ngrok to develop locally while accessing remote webhooks.
I successfully launched ngrok with the command:
ngrok http -host-header=rewrite devel.domain.com.docker 80
I can access the login form of my web application through the address http://randomsubdomain.ngrok.io.
However, I can't log in because it looks like the cookie session can't be set.
Indeed, cookies sessions are tried to be set for the domain devel.domain.com.docker but as we use randomsubdomain.ngrok.io in the browser they are blocked for security reasons.
How can I bypass this problem? Am I missing something in my configuration? Is ngrok the right tool for what I want to achieve?
Asked directly to ngrok.io support and got this answer:
No, you're not missing anything, that's just an unfortunate side effect of rewriting the host header. Host header rewriting only works for some applications because of complications like this (and others that involve javascript and cross-origin, etc). If possible, it's always much better to reconfigure your website to accept the ngrok.io host header.
However, I found a solution by checking if the request contains in the header x-original-host the domain ngrok.io, and then I alter the session mechanism (in PHP session_set_cookie_params) to use the x-original-host domain instead.
As mperrin said you have to alter php cookie session mechanism.
Reading from session_set_cookie_params:
Set cookie parameters defined in the php.ini file.
The effect of this function only lasts for the duration of the script.
Thus, you need to call session_set_cookie_params() for every request
and before session_start() is called.
The most important argument is $domain and to make ngrok work equally decent you can also use before session_start() the command ini_set() (see ini_set): ini_set('session.cookie_domain', 'xxx.ngrok.io');
It also took me hours to resolve for my custom hosting php platform but I knew that my auth subsystem should work under a valid hostname apart from localhost so I focused in how the cookies are set from my code.
Such kind of php environment settings should be set early by any decent php framework and that was one of my primary goals when I started building it (in my case I only have to change the value in a json text file at the server).

set proxy username and password in firefox

I am able to set the proxy host and port using the preferences menu, but I see no option to set the username/password.
When I browse, I get a message, proxy is refusing connections. I am using squid proxy.
How do I do this?
I had the exact same problem as you had.
And finaly there is no way to add user and password to the default firefox proxy settings, so we need an addon to handle this for us.
In the current version FoxyProxy is doing this without any problems:
https://addons.mozilla.org/en/firefox/addon/foxyproxy-standard/
So try out the Add-On and you will solve this.
br
Wikrie
Open Firefox.
In the address bar, type about:config. A warning appears.
Click I'll be careful. I promise! A variety of configuration options appears.
In the Filter box, type network.negotiate.
Double-click network.negotiate-auth.allow-proxies to toggle between true and false. The option must be set to false. (It seems counterintuitive, but you set it to false to make it work with the ISA proxy.)
Find the link for detailed understanding https://kb.globalscape.com/KnowledgebaseArticle10522.aspx
Perhaps you must fill the proxy host value with something like the below
username:password#proxy.thing.com

HTTP Basic Auth and Proxy for selenium-webdriver (ruby bindings)

I'm attempting to use the selenium-webdriver [ruby bindings][1] to access an internal web-site that requires a proxy to be configured, and HTTP Basic Auth.
I currently have:
require "selenium-webdriver"
driver = Selenium::WebDriver.for :firefox
driver.navigate.to "http://my-internal-site.com"
But this fails due to both the proxy and http auth issues. If I add my username and password to the URL (i.e. http://username:password#site.com) I can do basic authentication on another site that doesn't require the proxy, but this doesn't seem like an ideal solution.
Any suggestions?
Unfortunately doing http://username:password#site.com has been the standard way of doing but with more and more browsers blocking this approach. Patrick Lightbody of BrowserMob discussed in the company blog on how they get it to work.
Until there is full support for this across browsers for WebDriver (or Selenium), alternate option is to integrate w/ desktop GUI automation tools, where the desktop GUI tool will automate the HTTP authentication part. You can probably find some examples for this or file downloads, uploads if you google for things like "Selenium AutoIt", etc.
For a cross platform solution, replace AutoIt with Sikuli or something similar.
I tried the approach with AutoIt and it worked fine until Selenium 2.18.0,
because they implemented UnhandledAlertException, which will be thrown as soon
as the proxy login dialog pops up.
if you try to catch it, you end up with an driver=null, you would need to loop
the attempt to create a driver and trust into your AutoIt Script to kill the window.
If you're using Google-Chrome, try creating a custom extension and import it through ChromeOptions. It supports http(s) that wasn't supported by browsermob_proxy in Chrome. In-case of redirects testing, this is the only way that will help you as of now...
For details, check this post
https://devopsqa.wordpress.com/2018/08/05/handle-basic-authentication-in-selenium-for-chrome-browser/

Resources