I'm trying to extract the keyphrase someone used to land on a website. And since I have the Google Analytics script, I should be able to get the cookie name ___utmz and then look for the string "utmctr=...." to find what keywords.
However, in Firefox, no matter what, I always see (not provided)
utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided)
Is there a way to extract the keywords if someone visited using FF?
This has nothing to do with Google Analytics or Firefox but instead is a result of how keywords are handled for SSL searches with Google. In those cases, Google search will not pass along the keyword in the referral and instead sets the value to (not provided). For details see the keyword section in https://support.google.com/analytics/answer/1033173
Related
I have been struggling to get any XPath technique to work on octoparse and similar software. I'm now trying google sheets from reading posts here and can't get it to work either.
Input: A slideshare presentation url (eg https://www.slideshare.net/carologic/ai-and-machine-learning-demystified-by-carol-smith-at-midwest-ux-2017)
Intended output: Slideshare embed url (in this case: https://www.slideshare.net/slideshow/embed_code/key/wZudqqTdctjWXA)
I think this would be the way to get the output using google sheets: =importxml(A1,"//meta[#itemprop='embedURL']/#content")
It is not working for me (failure to fetch url). With Octoparse etc I just got a blank value.
I'm being daft here, no doubt. Any help would be useful.
It doesn't work because slideshare is owned by LinkedIN, and they have put in a lot of effort to ensure they cant be scraped, including google sheets. Before it was possible, but I believe they eventually caught on to the work around.
Just setup my CSE with Image Search enabled in the Basics tab, but still ain't getting results.
If I don't use searchType=image in the query it works, if I put it back it stops working with no errors, just no results.
Any inputs?
It seems there are two elements required to consume the Google search API, the configuration of the API key and credentials.
Setup the API key here https://console.developers.google.com/project
and create your own CSE here https://www.google.co.uk/cse/all
I made the mistake like a lot of people of ignoring the CSE side of the setup (it didn't seem logical to search only my own site).
You need to create a CSE though, point it to search any url (I don't think it matters greatly), and then where it says "Sites to search" chose "Search the entire web but emphasise included sites". Also then set image search to "On".
Once you've done this browse to your new CSE, the url should contain your CX code e.g. cx=014341056213260336986:v4fl8n8sabc, you then need to use this in your request query string.
Your API query should now start returning results.
Image search needs to be switched to ON at the Custom Search Setup->Basics
As in normal Google search page, Google returns me results specific to my location. As i am considering/assuming for this, it gets my ip address my request, find uses location for it and returns the results.
I am writing the proxy server for search engines. So from user browser i get requests for some search terms and server returns search results.
I am curious to know, if from Custom Search API can i get the such results based on some IP address i provide or anything like that, so that i can mock the user location in my proxy server and pretend the Google Custom Search to be that user and get the location specific results
PS.I am new to this search engine api world, so please understand me.
Yes, you can, but just for the country, not the exact location.
Here is the docs for parameters of a query request:
https://developers.google.com/custom-search/json-api/v1/reference/cse/list
Look at the 'gl' parameter.
I'm not aware of any way to specify the location more precisely then this.
After these many years I'm not sure if you still need it, but I found it hard to find a helpful answear, so here's the solution for whoever needs it.
You can achieve that behavior with the "?gl=" query param as mentioned here, and pass the value as one of the country codes.
Worked for me, hope it helps someone!
When I do a site-specific search on google.com:
site:http://one-month-of-chat-logs.github.io security
I get 12 results. I signed up for a custom search engine (cx: 015271449006306103053:mz6wkimeenc) and API key, and I get only 3 results when I run the same search:
$ curl 'https://www.googleapis.com/customsearch/v1?key=$MY_API_KEY&cx=015271449006306103053%3Amz6wkimeenc&q=security'
Why do the results differ? Is my API request actually querying something different than the search I performed on google.com?
This google page has what you are looking for https://support.google.com/customsearch/answer/70392?hl=en
your results are unlikely to match those returned by Google Web Search, for several reasons:
Even if a custom search engine is configured to search the entire web,
it’s designed to emphasize results from your own sites.
Your custom search engine doesn’t include Google Web Search features such as
Oneboxes, real-time results, universal search, social feaures, or
personalized results.
If your custom search engine includes more than
ten sites, the results may be from a subset of our index and may
differ from the results of a 'site:' search on Google.com.
I found that it is impossible to get the right results using Google APIs. Even if the search is only for one website, their search results are different if you use their UI vs use the API and pay for it. This is I guess, because google makes more money if they can show ads, while APIs are definitely only a face saving measure.
Since some of you are ok with a paid solution(#ihsan) you can try using a third party service like https://www.expertrec.com where you can control your crawl (so crawl depth is not a problem), ranking (adjust it the way you like), use the API or the full solution, with out any ads.
My last question was somewhat narrower than this one.
I am interested to know how to track search terms in general. How would you install Google Analytics to simple form with a submit button (assuming there is only one field)?
In Google Analytics, under Profile Settings enable site search (Do Track Site Search), and set the Query Parameter to the name of the input field. (And, of course, make sure that the tracking code was installed on the page the form's action points to.)
Try using the utm_term URL parameter in your URL you send to GA.
Example: utm_term=running+shoes
You can submit to GA a URL as described here:
http://www.google.com/support/forum/p/Google+Analytics/thread?tid=24abd1518783c26f&hl=en