enable subdomain iframe access - spring

I am trying to display iframe from x.example.com on a web page hosted at y.example.com
Here are the settings that I have done so for
Tomcat:
<filter>
<filter-name>ClickJackFilterEnabled</filter-name>
<filter-class>org.apache.catalina.filters.HttpHeaderSecurityFilter</filter-class>
<init-param>
<param-name>antiClickJackingEnabled</param-name>
<param-value>false</param-value>
</init-param>
</filter>
<filter-mapping>
<filter-name>ClickJackFilterEnabled</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
Spring Security:
httpResponse.setHeader("Content-Security-Policy", "default-src 'self' *.example.com; style-src 'self' *.googleapis.com *.amazonaws.com 'unsafe-inline'; script-src 'self' 'unsafe-inline' 'unsafe-eval'; child-src 'self' *.example.com; font-src *;img-src 'self' *.amazonaws.com");
httpResponse.setHeader("Access-Control-Allow-Origin", "http://*.example.com");
When I open the page with embedded iframs, I am still getting this error:
Refused to display 'http://x.example.com/' in a frame because it set 'X-Frame-Options' to 'SAMEORIGIN'.
Uncaught SecurityError: Sandbox access violation: Blocked a frame at "http://y.example.com" from accessing a frame at "http://x.example.com". The frame being accessed is sandboxed and lacks the "allow-same-origin" flag.
When I checked the headers using curl, the header X-Frame-Options does not exist
This the the output of curl
* Rebuilt URL to: y.example.com/
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 127.0.0.1...
* Connected to y.example.com (127.0.0.1) port 80 (#0)
> GET / HTTP/1.1
> Host: y.example.com
> User-Agent: curl/7.47.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< Pragma: no-cache
< Expires: 0
< X-Application-Context: application:dev:8080
< Content-Security-Policy: default-src 'self' *.example.com; style-src 'self' *.googleapis.com *.amazonaws.com 'unsafe-inline'; script-src 'self' 'unsafe-inline' 'unsafe-eval'; child-src 'self' *.example.com; font-src *;img-src 'self' *.amazonaws.com
< Access-Control-Allow-Origin: http://*.example.com
< Content-Type: text/html;charset=UTF-8
< Content-Language: en-IN
< Transfer-Encoding: chunked
< Date: Wed, 20 Jul 2016 13:21:57 GMT
<
{ [8200 bytes data]
What am I missing?
UPDATE:
I tried to set
document.domain = "example.com"
On both the web pages, I am still getting the error
Refused to display 'http://x.example.com/' in a frame because it set 'X-Frame-Options' to 'SAMEORIGIN'
When I type
document.domain
in the javascript console, I am getting
"example.com"
On both the web pages. So the origin is same for both the pages.

x.example.com is the one sending the SAMEORIGIN header. y.example.com can't override this, because then there would no way to block an iframe include. A site must grant permission (by lack of an origin policy or list of sites with permission) to other sites to include its contents.
Check the headers coming from x.example.com and you should see the policy actually blocking the iframe.

Finally I was able solve the issue. The problem is probably with how browsers deal with X-Frame-Options header.
The concept behind setting
document.domain = "example.com"
is to make the origin same for both the web pages. Browsers allow setting origin to a parent domain, so a page at a.x.example.com can set value of document.domain to x.example.com or to example.com
Now, even after setting document.domain to example.com, I was getting the error
Refused to display 'http://x.example.com/' in a frame because it set 'X-Frame-Options' to 'SAMEORIGIN'
When I removed the X-Frame-Options header altogether, using
http.headers().frameOptions().disable();
In securityConfiguration.java, it worked!
I don't know the reason why browsers didn't honour the document.domain setting. Couldn't find anything on this in mozilla documentation, or anywhere else. Hope this helps someone.

Related

How can I get http response with 301 code using python sockets

I am sending a get request to any host using sockets tcp, but I keep on getting "301 Moved Permanently" from pages with https.
I have tried to do it by changing the port from 80 to 443.
I have tried with the ssl library as well.
But keep getting 301 code
This is the code
import socket
import click
#click.command()
#click.option("-h", "--host", prompt=True)
#click.option("-p", "--port", type=int, prompt=True, default=80)
def cli(host, port):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((host, port))
message = f"GET / HTTP/1.1\r\nHost:{host}\r\nConnection: close\r\n\r\n"
request = message.encode('utf-8')
sent = 0
while sent < len(request):
sent = sent + sock.send(request[sent:])
response = b""
while True:
chunk = sock.recv(4096)
if len(chunk) == 0: # If no more data received, quitting
break
response = response + chunk
response_decode = response.decode('latin-1')
sock.close()
print(response_decode)
This is the response when I try to connect to www.eltiempo.com by port 80
HTTP/1.1 301 Moved Permanently
Server: AkamaiGHost
Content-Length: 0
Location: https://www.eltiempo.com/
Cache-Control: max-age=120
Expires: Sat, 12 Feb 2022 18:24:28 GMT
Date: Sat, 12 Feb 2022 18:22:28 GMT
Connection: close
Server-Timing: cdn-cache; desc=HIT
Server-Timing: edge; dur=1
version: desktop
I get this error with port 443
chunk = sock.recv(4096)
ConnectionResetError: [Errno 104] Connection reset by peer
Please tell me how to improve my code to avoid this 301 code.

How can I extract a certain header from http response [Set-Cookie]

I've searched around google for this, but I haven't been able to fix this problem yet, so I'm sending a post http request to a certain website, and in return I get this headers
map[Cache-Control:[no-cache] Content-Type:[application/json; charset=utf-8] Date:[Mon, 19 Oct 2020 15:38:41 GMT] Expires:[-1] P3p:[CP="CAO DSP COR CURa ADMa DEVa OUR IND PHY ONL UNI COM NAV INT DEM PRE"] Pragma:[no-cache] Site-Machine-Id:[CHI1-WEB5027] Set-Cookie:[.ROBL=A6251611CFF4513169C4CED10EFDC2DE9444E207CA376B27B906306AC18BFB0ACA6F17381240973900B19186F3E46C6BC9C52B99BC040579110E87145209A47040B241FE2C702C18AEF12A1AC746812B22596BFDB33C24DF1D5CEC72705DAD266343EC259528D8B7617BBE17408A957DF7A1C2CC7AC9DD9CC05FF8F4831BCC1669FB5221A74E6DB5C8EE0ED7F8F4AFA3767CCC39D919A62C6800EFFFF812DED5325F68D36B410D86A0CAB1FB0B8A90ADD529BE75A2DAFD3EB59D86BBC831C3144E577357B8EB0C514D0433F0B8E69DA151E6BA2C63968B46184167CAE05FE6B4749DC0449C71BB80A1306C6699E9EBD79E4C6A348CC33418D3E0DC3E6F5; expires=Wed, 12-Oct-2050 15:38:42 GMT; path=/; HttpOnly .RBID=eyJhbGciOiJIUzI1NiJ9.eyJqdGkiOiIyY2Q5N2IyOS01MjgxLTRjMWQtYjgxMS03OTQzNWZkNzU0ZjkiLCJzdWIiOjcxNjQ3MzgxOH0.yg9EiXLF4VY2O7Eu5mTdbax60tMrodiPbADWwRwZMeo; expires=Thu, 17-Oct-2030 15:38:42 GMT; path=/; secure; HttpOnly Data=UserID=-733325636; expires=Fri, 06-Mar-2048 16:38:42 GMT; path=/ REventTrackerV2=CreateDate=10/19/2020 10:38:42 AM&rbxid=&browserid=65376450118; expires=Fri, 06-Mar-2048 16:38:42 GMT; path=/] Vary:[Accept-Encoding] X-Frame-Options:[SAMEORIGIN]]
I want to extract .ROBL from, [Set-Cookie] just doing res.Header.Get(".ROBL") doesn't seem to be doing the job.
I tried to do split := strings.Split(string(header), ";") but that panics on fail so it's not relaible
Is there any relaible ways to extract .ROBL from [Set-Cookie] in the header?
Cookies are sent with the Set-Cookie HTTP header, so you can't simply get them as Header.Get("cookie-name"). You would have to parse the Set-Cookie header values. But the standard lib does this for you:
Cookies sent by a server may be parsed using Response.Cookies(). It returns you a slice of cookies (http.Cookie), just iterate over them until you find the one you're looking for.
cookies := resp.Cookies()
for _, c := range cookies {
if c.Name == ".ROBL" {
fmt.Println(c)
fmt.Println(c.Value)
}
}
Also note that if you want cookie management, you should consider using a CookieJar. For details, see What is the difference between cookie and cookiejar?

Scrapy: Check if response is an image

I need check if response is an image.
For requirements of the work I need to generate the url of the photos that can exist or no and record the url that contains an image.
When the url generated doesn't show a photo the response of the website is an html when the body is:
<body>No File Found</body>
also the response.status =200
The response header doesn't have a valuable info for both results with image and No File Found
For instance
HTTP/1.1 200 OK
Cache-Control: no-cache, no-store, must-revalidate
Pragma: no-cache
Transfer-Encoding: chunked
Expires: 0
Server: Microsoft-IIS/8.5
X-Powered-By: ASP.NET
X-Frame-Options: AllowAll
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: *
Date: Tue, 13 Aug 2019 01:44:40 GMT
The way that I found to check if the response is an image for this case was:
try :
no_file_found = response.xpath("/html/body[contains(., 'No File Found')]")
except:
photo_url = response.url
photo = PhotoItem()
photo['id'] = id
photo['url'] = photo_url
yield photo
Because When the response is an image the line
no_file_found = response.xpath("/html/body[contains(., 'No File Found')]")
throw this exception:
raise NotSupported("Response content isn't text")
I know that this isn't an elegant solution , but for this context it works
Question
My question is If there is another way more elegant to solve this problem, that not use try to solve that.
Notice that I don't need to download the image just need to record the valid url
Any suggestion is welcome.
Thanks in advance!!!
The simplest way would probably be to just check the type of the response:
from scrapy.http.response.text import TextResponse
if not isinstance(response, TextResponse):
# it's probably an image; do image stuff

Drupal 7 & Varnish 4 - I always get X-Drupal-Cache: MISS but X-Cache: HIT

I have run into the same issue as this person: X-Drupal-Cache for Drupal 7 website always hits MISS, and can not find a way out.
I am running Drupal 7 - Pressflow
and
Varnish 4.0
When I curl I get this result:
TTP/1.1 200 OK
Date: Fri, 08 Jul 2016 17:45:08 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Set-Cookie: __cfduid=db5fd757e7485622ac16af86f292603f51467999908; expires=Sat, 08-Jul-17 17:45:08 GMT; path=/; domain=.adland.tv; HttpOnly
X-Content-Type-Options: nosniff
**X-Drupal-Cache: MISS**
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Cache-Control: public, max-age=86400
X-Content-Type-Options: nosniff
Content-Language: en
X-Generator: Drupal 7 (http://drupal.org)
Last-Modified: Fri, 08 Jul 2016 17:41:27 GMT
Vary: Accept-Encoding
X-Varnish: 196743 3
Age: 213
Via: 1.1 varnish-v4
**X-Cache: HIT**
X-Cache-Hits: 22
Server: cloudflare-nginx
CF-RAY: 2bf55922d49b23d8-IAD
isvarnishworking.com tells me: "You deserve a gold star, here you go: gold star badge"....
While the "Varnish Indicator Chrome Extension" suggested in the linked Drupal org thread, tells me Varnish missed, on every single page of my website regardless if I am logged in or not.
If I turn Drupal cache for anonymous users at admin/config/development/performance off, Varnish will not work at all. If I set different minimum cache lifetimes there, it makes no difference.
In my settings.php I have this:
$conf['varnish_version'] = 4;
$conf['reverse_proxy'] = True;
$conf['reverse_proxy_addresses'] = array('127.0.0.1');
$conf['page_cache_invoke_hooks'] = FALSE;
$conf['page_cache_maximum_age'] = 86400;
$conf['cache_backends'][] = 'sites/all/modules/varnish/varnish.cache.inc';
$conf['cache_class_cache_page'] = 'VarnishCache';
$conf['reverse_proxy_header'] = 'HTTP_X_FORWARDED_FOR';
$conf['omit_vary_cookie'] = True;
$conf['drupal_http_request_fails'] = FALSE;
and this
$conf['cache_backends'][] = 'sites/all/modules/filecache/filecache.inc';
$conf['cache_backends'][] = 'sites/all/modules/authcache/authcache.cache.inc';
$conf['cache_backends'][] = 'sites/all/modules/authcache/modules/authcache_builtin/authcache_builtin.cache.inc';
$conf['cache_class_cache_page'] = 'DrupalFileCache';
while this has been commented out from the Varnish config in settings.php because if I don't, Varnish fails:
//$conf['cache'] = 1;
//$conf['cache_lifetime'] = 01080;
I have turned off all modules that could interfere, such as captcha modules, and I will note that the statistics won't count node hits correctly now, so something is being cached...
The VCL I use is grabbed straight from this github master with minimum changes
How can I troubleshoot this X-Drupal-Cache: MISS issue?
Your backend is clearly sending cookies:
Set-Cookie: __cfduid=db5fd757e7485622ac16af86f292603f51467999908; expires=Sat, 08-Jul-17 17:45:08 GMT; path=/; domain=.adland.tv; HttpOnly
In default configuration, Varnish will not cache a object coming from the backend with a Set-Cookie header present. Also, if the client sends a Cookie header, Varnish will bypass the cache and go directly to the backend.

How to know the endtime of each request for each of user in Jmeter

I'm using Jmeter and would like to identify the endtime of each request for each user.
Please take a look my testplan:
Thread group: 2 users
loop:1
2 HTTP request (request_1, request_2)
Start testing Web performance, the View Result tree shows: 4 results (2 for request_1, 2 for request_2)
request_2: 1 passed and 1 failed. Look in request table of result tree, I see:
Thread Name: jp#gc - Stepping Thread Group 1-1
Sample Start: 2014-04-18 09:28:06 ICT
Load time: 1100554
Latency: 550450
Size in bytes: 408190
Headers size in bytes: 4774
Body size in bytes: 403416
Sample Count: 1
Error Count: 0
Response code: 200
Response message: OK
Response headers:
HTTP/1.1 200 OK
Date: Fri, 18 Apr 2014 02:28:15 GMT
Server: Apache
X-Powered-By: PHP/5.3.3
Set-Cookie: ls23166422738597439695-runtime-publicportal=h4knpfldt76e3kvmunrn5i4u16; path=/limesurvey/
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
P3P: CP="IDC DSP COR ADM DEVi TAIi PSA PSD IVAi IVDi CONi HIS OUR IND CNT"
Last-Modified: Fri, 18 Apr 2014 02:36:09 GMT
Cache-Control: post-check=0, pre-check=0
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=utf-8
HTTPSampleResult fields:
ContentType: text/html; charset=utf-8
DataEncoding: utf-8
The questions are:
How to identify the time which cause request_2 is fail ? and how to display the endtime of each request for each user ?
How to displays information in the log panel of Jmeter (enable DEbug log mode on GUI), like "This is error....due to..."
Besides, as in the log panel (active log debug in GUI), some time the log entries stop at Thread 1-n (n=1,2...), after that 30s, the log is continue showing. So, I wonder about this time, web server has error, and in this time, Jmeter still send request or waiting Web server response ?
Thanks.
It can be done via Beanshell Pre Processor which you can add as a child of any "interesting" request.Example code would look like:
import java.util.Date;
long end_time_ms = prev.getEndTime(); // obtain sampler end time (in milliseconds from 1st Jan 1970)
Date end_time_date = new Date(end_time_ms); //convert it to human-readable date if you prefer
String response_message = prev.getResponseMessage(); // get initial response message
StringBuilder response = new StringBuilder(); // initialize StringBuilder to construct new response
response.append(response_message); // add initial response message
response.append(System.getProperty("line.separator")); // add new line
response.append("Thread finished at: ").append(end_time_date); // add thread finish date
prev.setResponseMessage(response.toString()); // set new response message
log.info("Thread finished at:" + end_time_date"); // to print it to the log
See above for Beanshell code and image below for UI impact
Never use GUI for anything apart from developing or debugging tests. If you want to add something to the log use log.info("something"); as above or JMeter __log() function

Resources