i am planing to start a full ajax site project, and i was wondering about SEO.
The site will have urls like www.mysite.gr/#/category1 etc
Can Google crawl the site.
Is something that i have to noticed about full ajax and SEO
Any reading suggestions are welcome
Thanks
https://stackoverflow.com/questions/768233/do-hashes-in-urls-affect-seo
You might want to read about so called progressive enhancement.
Google supports indexing of AJAX sites, but unfortunately it involves extra work for the developer. See http://code.google.com/web/ajaxcrawling/docs/getting-started.html
I don't think Google is capable of doing so (yet)
http://googlewebmastercentral.blogspot.com/2009/10/proposal-for-making-ajax-crawlable.html
However you can of course make your site usable with or without JavaScript. That way, browsers will have the full candy stuff and Google (and text browsers) still can navigation your site.
In addition to SEO, you also need to think about usability standards here. A site that is that reliant on AJAX isn't going to work for things like screen-readers as well as spiders. You need a system for graceful degreadation. A website that can't function without JavaScript isn't really a functioning website.
The search engines will spider the initial page load - what happens to the page (with ajax) after that is irrelevant to listings.
Google itself doesn't crawl ajax content but advice a mechanism for it. For this you first need to change # to #!
Whole process to SEO AJAX content is explained here along with simple asp.net code to start working on it.
Imagine having to hit the “refresh” button in your browser to update your Twitter feed rather than just hitting the button on the page itself and having it instantly update? These are the types of problems that AJAX solves, although it does come with its pitfalls. Google might claim it’s able to crawl and parse AJAX websites, yet it’s risky to just take its word for it and leave your website’s organic traffic up to chance. Even though Google can usually index dynamic AJAX content, it’s not always that simple. This guide covers some of the things that can go wrong and how you can make sure your AJAX website is crawlable: https://prerender.io/ajax-seo/
Related
Where I'm at: I've read Google's documentation regarding it's AJAX crawling, and I've searched around a bit in this website and others, but I'm quite confused, as it seems that all problems address the same issue: AJAX crawing with hashbangs?
I've developed an app which, among other purposes, let's the user search for locations worldwide, using an AJAX searcher quite similar to Google's, but my app uses exclusively the question mark in AJAX, instead of hashbang. Due to compatibility issues, changing it to the hashbang is not an option.
Not only am I largely confused by the fact that I could not find anyone else using the question mark instead of the hashbang, I'm also wondering if there is any documentation regarding my issue: how to let google bot crawl all my AJAX content when I'm using the question mark instead of a hashbang in my AJAX app.
The AJAX crawling schema was created explicitly for applications and websites using hashbang (#!) in the URL structure, because the fragment part of the URLs only exist on the client side; the URL rewriting in the specs, i.e. from #! to ?_escaped_fragment_= is meant to solve that.
Since most of the web is already making use of Javascript in a way or other, we (Google) needed a better solution, so we started executing Javascript in the pages we crawled and effectively render every page, just like a normal browser would. To quote our blogpost, Understanding web pages better:
In order to solve this problem, we decided to try to understand pages by executing JavaScript. It’s hard to do that at the scale of the current web, but we decided that it’s worth it. We have been gradually improving how we do this for some time. In the past few months, our indexing system has been rendering a substantial number of web pages more like an average user’s browser with JavaScript turned on.
You can also see what we "see" using Fetch as Google in Search Console (former Webmaster Tools); read more about the feature in our post titled Rendering pages with Fetch as Google
Before you do anything else, please try to fetch a few pages from your site with Fetch as Google. You might not have to do anything at all, it might actually work out of the box. And the good news is that it's not only Google that's rendering pages!
I have developed a website using angularjs and web api.
The problem is that the ajax rendered content is not crawable by google. And no one can find the website using google search.
After reading many articles regarding this issue, including:
This one with all links of explanation going out,
Google ajax crawling protocol, and also stack over flow question, I couldn't find the proper solution. Those that mention asp.net solutions, are talking about mvc, and I need only the simple REST by web api, other articles are not talking about asp.net.
Is there any simple explanation?
I'm the one who asked this same question long ago, so I will answer from my experience:
Firstly, if all your content are accessible via unique URIs (including the hashbang if you use it), modern search engines should index it just fine. In fact Google can index javascript generated content now. You can try that via the Google Webmaster tool and see how your site is indexed.
Secondly, there are libraries that help you to serve parsed content to search engines if you need to, but in my case I didn't bother much with it since Google is indexing js nicely.
I've seen others ask this question, and maybe I'm missing something or this is outdated, but I don't see why AngularJS needs to be an issue with SEO.
Say you have a landing page and it has a bunch of links. Assuming you're using html5 mode in AngularJS (and I'm not sure that's 100% necessary) and something like ng-route then the links on the landing page can work both as "angular" (JavaScript) links and "old school" (full page load) links.
If you're a human user you can click a link and it will do angular magic and adjust the content without loading the full page. Ok, all fine.
But if you instead copy the link and paste it in a new tab or new browser, it will still work - assuming you've set up routes correctly.
I'm not an SEO expert by any stretch of the imagination, but as I understand it, having links that load pages and having those pages have real and useful content is the core of SEO, and done this way, AngularJS should work fine. The key thing to check is if you copy and paste the link (not just click it) that it works.
First I think I understand what cloaking is, but what is it in detail?
My problem: I've a webapp created using wavemaker, so it's full of javascript and ajax calls. Therefore the google crawler can't see any of my content. My idea is now to make a different simple html page for users have javascript disabled and for the google crawler. This page contains a javascript block and a redirect like this:
<script language="javascript" type="text/javascript">
//redirect to the ajax page
window.location.href = 'http://www.myhomepage.com/index.html?page=about';
</script>
The redirect will only occur when a user browses this site and have javascript turned on. The google crawler will never be redirected. Both pages have the same content, but different URLs. Do you think this technic is cloaking?
I think all the points that were raised on Flash based web sites apply to this scenario.
You have 1 web site that uses technologies the Search engine crawlers can not read (fully \ correctly).
here is what Matt Cutts said:
"A good rule of thumb is to take a look at your site in a text browser like Links or an ancient browser with JavaScript/CSS/Flash turned off. If you can reach all your pages just by clicking regular links, your site should be pretty crawlable."
http://www.mattcutts.com/blog/solved-another-common-site-review-problem/
based on that, and other articles:
if your code will show Search engine crawlers the same content - i do not think this is cloaking
Even simpler:
You could make use of the <noscript> Tag, hence delivering content to users (and the google bot) that have javascript turned off. No need for an ugly redirect ...
Just use it like this:
<noscript>Your content for Javascript disabled browsers and bots here</noscript>
It's usually referred to as a black hat SEO trick.
However it can have other uses, generally however a good server application shouldn't need to use tricks like that.
I think this explains it better than me though ...
http://en.wikipedia.org/wiki/Cloaking
Clocking is a SEO unethical technique which not follow SEO rules and harmful for your site. It's also call black hat SEO Technic. In the technique the content presented to the search engine spider is different from that presented to the user's browser.
We have web applications elgifto.com, roadbrake.com in which we used AJAX at many places, especially to update major portions of a page. All the important functionality of elgifto.com was implemented using AJAX. Now we realize a few issues due to AJAX implementation.
All the content implemented using
AJAX is not available to the SEO
bots and it is hurting the page rank
of our site.
Users will not be able to bookmark
some of the pages as they are always
available through AJAX.
When we want to direct the user from
one page through an anchor link to
another page having AJAX, we find it
difficult.
So now we are thinking of removing AJAX for these pages and use it only for small functionality such as something similar to marking a question as favorite in SO. So before going ahead and removing, we want to know expert's opinion on this. Thanks.
The problem is not "AJAX" per se, but your implementation of it. Just as a for instance, you can fix the 'bookmark' problem like google maps does it: provide a generated link for each state of your webapp.
SEO can befixed by supplying various of these state-links to the crawlers, either organically trough links in your site, or by supplying a list (sitemap).
If you implement 2, you can fix 1 and 3 with those links.
In the end you must figure out if the effort is worth it, and if you are not overusing AJAX ofcourse, but the statements you've made are not set in stone at all.
I'm costantly developing ajax based websites, with no problems for SEO at all. You just have to use it in the best possible way.
For example, I have a website with normal links pointing to normal webpages (PHP pages), this for normal navigation if a user doesn't have JS enabled. But if a user has JS enabled, a script will change the links behavior, only fetching the content of the page needed.
This way you still have phisycal separated webpages with all their content, which will be indexed as normal.
Facebook is doing Ajax History (Back and Forward button) and Bookmark using #! instead of just # in the URL. Is it always a good idea to do that, because I was thinking that a usual anchor could interfere with the Ajax History mechanism to trigger it into processing a normal anchor.
So, the Ajax History function will only process a hash portion only when it sees #! instead of just #.
And is using ! compatible with major browsers? If Facebook is using !, a guess is that it may be fairly well supported.
See Google's Making AJAX Applications Crawlable
for a possible use case (don't know if this is why Facebook used this fragment).
Update: This answer has been superseeded by this article. It discusses the issues with the Hashbang (#!), Hashes (#) and the HTML5 History API (pushState, popState) and the solutions.
In regards to usability on your website, it doesn't matter and you can use anything you like.
In regards to search engine optimization, having it and not having it both provide different avenues to go down.
For instance, Facebook uses the ! according to the Google Proposal for Making Ajax Applications Crawlable. Adding the ! will tell google that it should listen in on that ajax request and add that url to search engine results. This is great for websites which have already implemented ajax, as all you need to do is add the !.
The downside of this is that it only solves the problem of making your ajax crawlable. It does not solve the problems of:
Keeping the URLs clean and consistent for Ajax and Non-Ajax users. Eg. you could end up with www.facebook.com/profile.php?pid=123#!profile.php?pid=123
Keeping the website accessible by Non-Ajax users.
Keeping the URLs the same for both Ajax and Non-Ajax users.
Requires some severely complicated server side changes for escaping and translation of states in regards to query strings.
It is not compatible with the new HTML5 PopState functionality which is designed to truly solve these problems.
For websites which don't currently use ajax for everything, due to the problems above it is far better NOT using the Google Proposal as it is only workaround for sites like facebook which went ajax crazy and needed a desperate solution to SEO. There are alternatives which solve more of these problems (and with the HTML5 PopState now available can solve all the problems). One such alternative is jQuery Ajaxy (as seen on balupton.com) which works by simply upgrading your website into a ajax application, while keeping the experience for a Ajax-Enabled rich and interactive and continuing to work perfectly for Ajax-Disabled users.