We have multiple sites indexed in our GSA. For site abc.com, everything works perfectly. The start URL is the home page, and links from there are followed properly by GSA and the entire site is indexed.
For site def.com, the setup is the same (start URL, follow patterns) and the links are valid. The home page gets indexed but the links from there are not followed. But if I index a linked page directly by recrawling it, it gets added. License limit is not the issue - the number of crawled documents is well below the limit. I have also made sure the pages are not skipped due to "do not follow patterns". The site does not have a robots.txt file, so no exclusions there either. There are no noindex/nofollow/googleon/googleoff tags.
So why is GSA unable to follow links?
It depends on how def.com renders the content. If the links are built in run time for the site def.com at the browser end, GSA would not be able to replicate it to understand the links on the page.
Hit the def.com home page on your browser and look for anchor tags.<a href= />.
If it is present obviously GSA will follow the urls and index the same.
You can also check the crawl diagnostics (Number of links on this page to crawled pages) of the def.com home page to know whether GSA is able to discover links from your home page.
If GSA is not discovering links from the home page then go ahead and create jump page for def.com website and index that with noindex robots meta tag.
Hope it helps.
Related
As a new magento developer I'm using a theme to develop my store. I want to change the URL key of CMS page about. I followed the steps written here but I can't find my layout file for about page on the theme directory or base directory for change XML URL tag.
How can I edit the layout file? Can anyone help me?
Actually that tutorial is a little confusing because there, they are showing an example of changing a URL for two different type of pages (CMS and system).
For changing the URL of a CMS page, like about in your case, it's enough that you proceed until point 12. In short:
First. you change the URL from: (main menu) CMS/Pages/Your_Page -> Url Key: newURL
Then, find the blocks that it is used (e.g footer) and change the URL accordingly.
Plus, in general working in Magento after changing things like page URLs you need to do a reindex and in case of using cache you have to refresh/remove your cached data. Both could be found under System tab (Cache Management & Index Management).
I have a simple question...I had an old store and people bookmarked specific product pages in their browsers as well as Google search points to some specific pages. I have now installed a new version of Magento (1.9.1) and some of these URLs will not longer work (they will show the standard 404 not found Magento page).
What would be the best way to forward all these hits to the front page?
Thank you for any suggestions!
Allysin :-*
Login to the admin, go to System>Configuration>Web and change the drop-down value for CMS No Route Page in the Default Pages section. You should be able to select your CMS Home Page.
Here is a reference with images
If manual setting doesnt help,you can try this.
I have been developing a web page named: directorioelectronico.com and I have specially issues now, I will be very grateful that someone can be help me.
The web page has been submitted in Google.com and now all the links that are in the homepage are listed in search results BUT some links ej. google.com/maps are not list in the home page (because it appears only when you select your municipality). How can I tell to Google that it exists (maybe without a sitemap.xlm because my links have metadata that is very important that google knows?
In advance, Thank you very much for you help.
My solution was that I create a route /sitemap.xml and put it in robots.txt too I created an html page with all the sitemap with cities, companies and all imoportant links.
I am working with a Magento install that I did not set up and am pretty green when it comes to its operation... There is a default page serving as the "home page" when there is already a page defined in the back-end as the "home page". The path in the URL reads as: DOMAIN/index.php but when I try to find said index file using google developer mode, I can find no such file that would generate the layout. I suppose I just don't have enough overall knowledge to figure out what is at work here. Can someone help me figure out why it's using the default home page instead of mine?
On most webshops I've worked on the CMS home page wasn't really used. Instead of the CMS home it can be easier to use a custom template to setup the homepage with static blocks and what not.
If this is also the case for you, there might be two places you can find this custom template:
In the layout xml files, search forcms_index_index?
In backend, navigate to the CMS home page and go to the Design tab
About the index.php part: this is because you probably because you haven't enabled mod_rewrite in your server configuration
I inherited a Joomla site that has a set up I'm not familiar with. The url of pages on the site are like
http://hufboxing.com/about.htm
http://hufboxing.com/training.htm
http://hufboxing.com/schedule.htm
I looked in the .htaccess file and do not see any rules that map .htm urls to the Joomla query string urls.
In the CMs, on each article are url alias. The corresponding URL alias to each of the urls above are:
http://hufboxing.com/about-us
http://hufboxing.com/team-training
http://hufboxing.com/the-gym-schedule
However, going to any of these alias gives a 404 page. Even if I attach a .htm extension to these urls, it gives 404 page.
Where do I go in Joomla to add new pages to the site and add new .htm urls?
Here is the website:
http://www.hufboxing.com
That is normal. They are based on your article and menu aliases. Joomla uses a database to store the text so you won't find pages. How do you want the urls to look? Also what version? To add pages login to administrator and go to create article. Then you may want to go to the menu manager and link. I recommend you review the beginner docs at http://docs.joomla.org.