Ajax generated content and Facebook scraping - ajax

I've built an AngularJS app and it contains views that could be considered different pages but the app is a single page app as in the page doesn't reload.
I've read up on using escaped fragment URLs to redirect search engines to snapshots of pages. My issue is when I try to share a page from my app on Facebook that it returns a 404, as can be seen using Facebook's open graph debug tool.
My app has a Larvel back-end feeding to an AngularJS front-end and my htaccess looks like this to redirect spiders and Facebook to the snapshots. As far as I knew this was working but I guess it's not.
<IfModule mod_rewrite.c>
<IfModule mod_negotiation.c>
Options -MultiViews
</IfModule>
RewriteEngine On
RewriteCond %{HTTP_HOST} ^www\.(.*)$ [NC]
RewriteRule ^(.*)$ http://%1/$1 [R=301,L]
RewriteCond %{QUERY_STRING} ^_escaped_fragment_=/?(.*)$
RewriteRule ^(.*)$ /%1? [NC,PT]
# Redirect Trailing Slashes...
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)/$ /$1 [L,R=301]
# Handle Front Controller...
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^ index.php [L]
</IfModule>
My URLS are formatted to the user like: domain.com/#!/bike/id/bike+name
and the snapshot URLs are the same just without the hash bang i.e domain.com/bike/id/bike+name
I'm really stunmped on this one. Any suggestions or pointers would be greatly appreciated.

Facebook crawlers don't execute any Javascript so they won't see/have no idea about your Angular routes. What I've done in my project is I've mirrored the routes that need the open-graph tags in the back-end.
If my server gets a request for a page with og-tags it will make the necessary API calls to get the data, and attach them to the index template. If my initial request is the a page with no og-tags, I just render the regular SPA index. (I'm using HTML5 mode for urls so could be a bit different for you)
Also note that I said initial request to the server. This means that your og-tags won't change as you navigate within your SPA, they'll just be the tags of the first page you requested. This is actually a non-issue because Facebook makes individuals request to your server.
Let me know if this is unclear, I'll try to explain better.

Related

Issues with RewriteRule on Generic Anchor Redirect

I want to redirect any request to the root of my site to an anchor on the index. So
https://example.com/foo
Gets sent to
https://example.com/#foo
I've written this .htaccess file (it also redirects http requests to https, that part works, but is included for completeness)
RewriteEngine On
RewriteRule ^/(.*) /#$1 [NE,R=302]
RewriteCond %{HTTPS} !=on
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]
Based on the discussion in this thread: mod_rewrite with anchor link this should work, but it's not matching for some reason. I tried out the rule given in that thread using this tool: http://htaccess.madewithlove.be/ and it doesn't seem to work there either.
I've tried clearing my cache and accessing in incognito mode, to no avail. Any help?

mod_rewrite messing with sessions

I've been having a problem with mod_rewrite and sessions. I've a csrf token that is stored in a session everytime you visit the page but the problem is that when you submit a form the mod_rewrite before submitting the post it loads the page twice. So the csrf token changes before the post data arrives and it does not validate the submit.
I've been looking everywhere, I've changed the htaccess multiple times but it doesnt changes anything, it keeps happening.
If I dissable the url rewrite it works perfectly.
Options +FollowSymLinks
RewriteEngine On
RewriteBase /
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^actualidad/(.*) /actualidad-info.php?slug=$1 [QSA,L]
RewriteRule ^campana/(.*) /campanas-info.php?slug=$1 [QSA,L]
RewriteRule ^evento/(.*) /evento-info.php?slug=$1 [QSA,L]

code igniter & mod_rewrite - one rewrite rule breaking another

I have a site built in codeigniter. We use short urls from our database & rewrite rules to redirect them to their full path.
For example,
RewriteRule ^secure-form$ form/contract/secure-form [L]
This works fine by itself. But I would like to use SSL on certain pages. I have edited the code so that if you go to one of these pages, all instances of http:// within the page are replaced with https:// but I need to rewrite the url to use it as well.
The pages all use the same template and all the content comes from the database so I can't just specify ssl on a particular directory.
The url's for the secure pages all start with 'secure' so I wrote the following rules and placed them above the other rewrites.
RewriteCond %{HTTPS} off
RewriteCond %{REQUEST_URI} ^/secure/?.*$
RewriteRule ^(.*)$ https://%{HTTP_HOST}/$1 [R=301,L]
RewriteCond %{HTTPS} on
RewriteCond %{REQUEST_URI} !^/secure/?.*$
RewriteRule ^(.*)$ http://%{HTTP_HOST}/$1 [R=301,L]
RewriteRule ^secure-form$ form/contract/secure-form [L]
RewriteRule ^secure-different-form$ form/contract/secure-different-form [L]
all other rewrite rules for specific pages follow
then the default rewrite further down...
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ index.php/$1 [L]
The problem is that when I add the rules to change the protocol, it ends up displaying 'form/contract/secure-form' in the url instead of 'secure-form'.
This renders the actual form on the page broken since it uses that url to build itself.
If I take out the rules that change the protocol, it displays secure-form in the url as it should, but the page is not secure.
What am I doing wrong?
----UPDATE----
Ooh, after over 20 hrs of searching, I think I finally have an answer. So, first time through, https is off & gets turned on. Then, because of the 301, it's run again & the page gets sent to form/contract/secure... But this time, https is on. Since the uri no longer STARTS with secure, it turns https off.
Hopefully, this will help someone else.

Insert Yii module string in URL using mod_rewrite for mobile browser

I want to rewrite URL to access Yii module when a mobile browser detected. In Yii, I make a new module called mobile. I don't use theming (which is the common method for mobile site) since I intent to implement quite different logics for normal browser and mobile browser user.
The mobile module will be accessed using http://localhost/project/mobile (or http://project/mobile in production).
The default .htaccess file for Yii is below
RewriteEngine on
RewriteBase /project/
#RewriteBase / #for production
# if a directory or a file exists, use it directly
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
# otherwise forward it to index.php
RewriteRule . index.php
The problem is I want to insert string mobile for every URL accessed by mobile browser.
for example
http://localhost/project/user -> http://localhost/project/mobile/user
http://localhost/project/login -> http://localhost/project/mobile/login
http://localhost/project/books/1 -> http://localhost/project/mobile/books/1
or for production.
http://project/user -> http://project/mobile/user
http://project/login -> http://project/mobile/login
http://project/books/1 -> http://project/mobile/books/1
Anyone knows what the new .htaccess for this will be?
I don't mind changing few variables for development and production server and for the mobile browser detection I guess I can use the rule from http://detectmobilebrowsers.com
Try this one
RewriteEngine On
RewriteBase /project/
# Redirect /project/*/ to /project/mobile/*/
RewriteCond %{REQUEST_URI} !^/project/mobile/.*$
RewriteCond %{HTTP_USER_AGENT} "android|blackberry|ipad|iphone|ipod|iemobile|opera|mobile|palmos|webos|googlebot-mobile" [NC]
RewriteRule ^(.*)$ /project/mobile/$1 [L,R]
# Forward request to Yii
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . index.php [L]
It was adapted from .htaccess showed in here http://ohryan.ca/blog/2011/01/21/modern-mobile-redirect-using-htaccess/
The easiest thing you can do here is group all mobile functionality under a "mobile" module, and use Yii's PreRouter functionality to redirect the user in Yii itself. This gives you a lot more control about everything.
class PreRouter
{
public function routeRequest($oEvent)
{
$oApp = $oEvent->sender;
if (<isMobile>)
$oApp->defaultModule = 'mobile';
}
}
In your config you add:
'onbeginRequest' => array('PreRouter', 'routeRequest'),
Under the main array (so NOT under modules, it's an app setting).
As said, this gives you a lot of control over everything. You can determine when the user is to be redirected (for example iPad is also a mobile but a lot of people want the regular site). With this method you can store for example in a cookie that they want to remain on normal site instead of mobile etc.
I know its not with rewrites as you requested but I still hope you find the answer usefull :)

How do I redirect to a page after a file download has been initiated with mod_rewrite?

Just as the title states. Say an individual accesses a file from my database, http://domain.com/database/file.zip. Once that file download has been initiated, I wish the browser to be redirected to the database directory again. Here's what I have so far:
RewriteEngine On
Options +FollowSymLinks
RewriteRule ^Database(.zip)$ http://domain.com/db/index.html [R=301,L]
But, I get a 500 error.
And if I am being too picky, it would be nice to ignore this function on links such as: &file=something.zip.
Either way, getting the first portion to work would be fantastic.
Edit!
Here is what worked for me in the end. Cheers!
RewriteEngine on
RewriteCond %{REQUEST_FILENAME} .*zip$|.*rar$|.*tar$|.*txt$ [NC]
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !domain\.com [NC]
RewriteRule ^(.*)$ /dl.php?url=%{REQUEST_FILENAME} [L]
Not really possible with mod_rewrite the way you have described: once the server has started delivering content (sent a 200 status code) there is no way to initiate a second response without a corresponding second request.
If you want to do this you'll have to do it on the client side: for example launch the download targetting a separate, hidden iframe and if the download starts then you can change the page location using window.location.

Resources