I am new on htmlunit, and with almost No knowledge of programming.
In Centos - Webserver (for www.mydomain.com), i am trying to create a proxy.
I want my server to be used as proxy on request comming for www.mydomain.com and
send response as htmlsnapshot.
i saw some of the things like
SocketAddress addr = new InetSocketAddress("xxx.xxx.xx.xxx", 8888);
Proxy proxy = new Proxy(Proxy.Type.HTTP, addr); //or Proxy.Type.SOCKS
URL url = new URL("http://mydomain.com/test"); URConnection conn =
url.openConnection(proxy);
But i wonder where to setup that.
Can this be done entirely from apache..
i dont see any configuration file as such fo htmlunit.
BTW, i have installed htmlunit ( using jpackage repo)
HTMLUnit is GUI-Less browser for Java programs Where did you see the code that you have included. From memory I cannot recall the API even having a Proxy Class. You can certainly point the WebClient at a proxy by passing in a ProxyConfig; and that is only there to declutter the WebClient Class.
Are you looking for a Proxy server, or a way to simulate a browser?
Related
We are using OpenApi Specification v1.4.8. We have load balancer urls and domain url. Domain url http://market.place.com/productsservice/swagger-ui.html and load balancer url is http://7649263.path.aws.com/productsservice/swagger-ui.html.
Due some ongoing network issue with the domain, we have shifted to use load balancer url for testing the application. When I access swagger ui using the above load balancer url, I see the domain URL in servers dropdown(haven't added it anywhere). Thus the url in browser address bar and in Servers dropdown is different.
Since Swagger uses the base url from 'Servers' dropdown to fabricate paths for testing, and my domain is not working, I am not able to fire request from Swagger UI. The request work fine via postman when the url in browser that is load balancer url is used.
Is there some configuration which can make the Servers dropdown URL and Browser url base path same ? What we are trying to do is, the Servers dropdown needs to take the url as in the address bar of browser.
Probably you have openapi 3 configuration java class.
You should
new OpenAPI().addServersItem(...)
Edit your method like this.
private String ip="109.162.230.178"; //your ip
private String port="8587"; //your port
// if you have context add this line
private String context="tws"; //your context
#Bean
public OpenAPI customOpenAPI() {
return new OpenAPI()
.addServersItem(new Server().url("http://"+ip+":"+port+"/"+context))
...
}
now your UI Combobox is like this :
For example my server static ip is
109.162.230.178
and i have to use this Openapi 3 UI
from another computer outside of my local network in Internet.
I think of the one reason why you have this issue. You should check in code if io.swagger.v3.oas.models.servers.Server is being used in OpenAPI configuration-
OpenAPI oas = new OpenAPI();
......
......
String url = "/market.place.com/"
List<Server> servers = new ArrayList<Server>();
Server server = new Server();
server.setUrl(url);
servers.add(server);
oas.setServers(servers);
If you are relying on programmatic approach for openAPI configuration and you have something like above in your code then that may be the reason why you are seeing your domain URI. You can replace it with correct URI to make it work.
Does inbuilt WebSocket API in WSO2 APIM manager supports socket.io/nodejs based endpoint?
I have socket.io based websocket server (in NodeJS) And socket.io javascript client, in between we are using WSO2 API manager for authenticating websocket connections. But WSO2 fails to connect backend server and gives nothing in response.
I used access_token query parameter for passing Bearer token (from JavaScript client) and it seems working as no error. But WSO2 does not forward requests to endpoint.
Same works fine with native WebSocket server-client.
You can add the following logs to <AM_HOME>/repository/conf/log4j.properties to debug the issue further.
For APIM 3.x.x you can change it according to the log4j2 and add to log4j2.properties
log4j.logger.org.wso2.carbon.inbound.endpoint.protocol.websocket.InboundWebsocketSourceHandler=DEBUG log4j.logger.org.wso2.carbon.inbound.endpoint.protocol.websocket.InboundWebsocketResponseSender=DEBUG
log4j.logger.org.wso2.carbon.websocket.transport.WebSocketClientHandler=DEBUG
log4j.logger.org.wso2.carbon.websocket.transport.WebsocketTransportSender=DEBUG
Previous answer was for socket.io JS client, here is the same hack for socket.io python client.
Modify client.py file from /home/user/.local/lib/python3.6/site-packages/engineio
line no. 515
change this,
return ('{scheme}://{netloc}/{path}/?{query}'
'{sep}transport={transport}&EIO=3').format(...
to this,
return ('{scheme}://{netloc}/{path}?{query}'
'{sep}transport={transport}&EIO=3').format(...
Now, instead of directly modifying actual flow we can use conditional parameter.
if self.noslash == 'true':
return ('{scheme}://{netloc}/{path}?{query}'
'{sep}transport={transport}&EIO=3').format(...
else return original statement.
noslash parameter can be fetched from connect() function in both engineio and socketio library of python.
def connect(self, url, headers={}, transports=None,
engineio_path='engine.io', noslash=None):
self.noslash = noslash
Here is the sample connection string for python-socketio
sio.connect('http://localhost:9099', headers={'Authorization':'Bearer 90e8sf10-3s1w-495f-b20d-5a009f63193v'}, transports=['websocket'], socketio_path='/livefeed/v1', noslash='true')
finally I get rid of this issue. Yes, WSO2 APIM websocket supports socket.io js library that will be the first answer to my own query. More on that, here are some findings.
The actual reason for issue was request URL pattern used by socket.io library and WSO2 APIM. Socket.io JS library make the final URL with / at the end of resource (reference), which is not accepted by query separation logic used by WSO2.
So in simple, this is acceptable
ws://localhost:9099/livefeed/v1?EIO=4&transport=websocket
but not this,
ws://localhost:9099/livefeed/v1/?EIO=4&transport=websocket
Solution:
After few discussions with WSO2 team, it was clear that it is impossible to implement changes at their end. So little hack in socket.io.js file worked for me.
line number - 2535
changed this,
_this.opts.path = _this.opts.path.replace(/\/$/, "") + "/";
to this,
if(typeof opts.noslash!== 'undefined' && opts.noslash== "true"){
_this.opts.path = _this.opts.path.replace(/\/$/, "");
}else{
_this.opts.path = _this.opts.path.replace(/\/$/, "") + "/";
}
and created socket using extra argument.
var socket = io.connect("http://localhost:9099?access_token=90e8sf10-3s1w-495f-b20d-5a009f63193v", { transports: ['websocket', 'polling'], path: '/livefeed/v1', noslash: 'true' });
I know this is not a actual solution, but it worked for me.
AWS Elastic Load Balancer listening through HTTPS (443) using SSL and redirecting requests to EC2 instances through HTTP (80), with IIS hosting a .net webapi application, using swashbuckle to describe the API methods.
Home page of the API (https://example.com) has a link to Swagger documentation which can bee read as https://example.com/swagger/ui/index.html when you hove over on the link.
If I click on the link it redirects the request on the browser to http://example.com/swagger/ui/index.html which displays a Page Not Found error
but if I type directly in the browser URL https://example.com/swagger/ui/index.html then it loads Swagger page, but then, when expanding the methods an click on "Try it out", the Request URL starts with "http" again.
This configuration is only for Stage and Production environments. Lower environments don't use the load balancer and just use http.
Any ideas on how to stop https being redirected to http? And how make swagger to display Request URLs using https?
Thank you
EDIT:
I'm using a custom index.html file
Seems is a known issue for Swashbuckle. Quote:
"By default, the service root url is inferred from the request used to access the docs. However, there may be situations (e.g. proxy and load-balanced environments) where this does not resolve correctly. You can workaround this by providing your own code to determine the root URL."
What I did was provide the root url and/or scheme to use based on the environment
GlobalConfiguration.Configuration
.EnableSwagger(c =>
{
...
c.RootUrl(req => GetRootUrlFromAppConfig(req));
...
c.Schemes(GetEnvironmentScheme());
...
})
.EnableSwaggerUi(c =>
{
...
});
where
public static string[] GetEnvironmentScheme()
{
...
}
public static string GetRootUrlFromAppConfig(HttpRequestMessage request)
{
...
}
The way I would probably do it is having a main file, and generating during the build of your application a different swagger file based on the environnement parameters for schemes and hosts.
That way, you have to manage only one swagger file accross your environments, and you only have to manage a few extra environnement properties, host and schemes (if you don't already have them)
Since I don't know about swashbuckle, I cannot answer for sure at your first question (the redirect)
Is it impossible for people behind authenticate proxy (with domain) to use Selenium in Python, going through CNTML proxy on localhost?
Nothing seems to be working. Eventhough I put proxy settings to that:
myProxy = "localhost:3128"
proxy = Proxy({
'proxyType': ProxyType.MANUAL,
'httpProxy': myProxy,
'ftpProxy': myProxy,
'sslProxy': myProxy,
'noProxy': '' # set this value as desired
})
self.browser = webdriver.Firefox(proxy=proxy)
I also tried to create a custom profile for Firefox, configuring the CNTLM proxy inside it, forcing in python that profile for my Firefox webdriver, but nothing is working.
Is that even possible?
For my Java EE web application I have a problem. I need to redirect a page A to another page B, using Spring controller I write:
Page A is for my browser (in http):
http://MYPUBLICSERVERNAME/MYCONTEXT/A.do :
In A which is a Spring controller has the following code for redirection:
#RequestMapping(method = RequestMethod.GET, value = "/A.do")
public String searchProfiles(){
....
return "redirect:/B.html";
}
It works perfectly in http.
so I go to B :
http://MYPUBLICSERVERNAME/MYCONTEXT/B.html
The problem comes with HTTPS because with HTTPS a reverse proxy intercept the request :
This proxy is on the same machine and I get :
impossible to connect because when i go to :
httpS://MYPUBLICSERVERNAME/MYCONTEXT/A.do
i expected to be redirected to :
httpS://MYPUBLICSERVERNAME/MYCONTEXT/B.html
Instead the browser is redirected to
http://localhost/MYCONTEXT/B.html
I understand that for the server the client is the reverse proxy not of course the browser.
So my question is : how can the servlet get the original server name i mean the server name indicated in the web browser (MYPUBLICSERVERNAME) and the protocol (HTTPS if HTTPS).
The workaround i think is using a cookie feeded by the browser, because the browser only know MYPUBLICSERVERNAME but maybe more elegant solution exist ?
Of course if I know a spring function that can magically redirect with expected protocol and server name it's ok too but i didn't find it.
Thanks for your answer
you can use the redirect with the whole path and protocol
return "redirect:https://MYPUBLICSERVERNAME/MYCONTEXT/B.html"
I am not sure if you can use refferer header in request (does proxy change the refferer ?)
String referrer = request.getHeader("referer");