So google pulled the SOAP API back in 2006, and depreciated the REST API last year. Even though the REST api still works fine, I was wondering if anyone here knows an alternative for the web search. I am not talking about Custom search API (that thing is horrible with a 100 queries/day limit).
Also, I am currently using the REST api for a custom application. I have noticed mixed (and very unreliable) results if I pass search operators such as inurl: or site: with my queries. Does anyone know if these even work with the REST Api?
The search parameters do work for Googles deprecated REST API. But search results often do not equal the ones from a regular Google search. Most likely because the two do not (and never will) use the same data index.
YAHOOs BOSS could be a (non-free) alternative
Microsoft does offer a simple and free restful API for BING
Related
Given any URL of a site I "own", in Google Search Console I can see this information:
I am particularly interested in the "last crawl date".
How do I get the same information with the API (Search Console API or Webmaster Tools API)?
You cannot. Not via the Google Search Console API, the Webmaster Tools API, or any other Google API for that matter. How Google can design their APIs so poorly is beyond me. Providing access to 100% of the features that you can access through the UI of the same service, is the #1 most basic requirement of an API, and they fail even at that.
There's this workaround (requesting https://webcache.googleusercontent.com/search?q=cache:<YOUR_URL>... and scraping the response contents), but you'll start getting "429 too many requests" pretty soon, so it's basically useless unless you only need to make, I don't know, maybe a request every few days.
In practice, there doesn't seem to be any other way than logging the crawler's visits yourself (recognizing it from the user-agent string, validating the IP maybe with a reverse lookup or just against a list).
When searching in Yammer it uses an /api/v2/search endpoint like this:
https://www.yammer.com/api/v2/search/models?search=test&start=0&size=20&model_types=threads&_network=xxxxxxx&relevance=DEFAULT
Compared to the v1 search endpoint this returns more data. We need this data. And we successfully tested using this v2 endpoint.
Now the question: Is this v2 search endpoint safe to use? Is it "official"? Maybe somebody from Yammer reads along - on their Help page it says:
We encourage you to post all programming questions to StackOverflow
with a #yammer hashtag
So here we are. Any help is appreciated.
The Yammer v2 search API endpoint is not yet officially supported for third-party use. This is because we anticipate the need to make changes to it as we build it out. There are no plans to lock developers out of experimenting with the v2 search API. However, when developing any important functionality against this API, automated testing should be used to detect any unannounced breaking changes that we make while we continue to build it out.
So basically, I have a spring API with MySQL database and I need to be able to perform search of a user by his name in front-end web app which is using my API. The trick is that I need a kind of google search so by typing, lets say, 3 letters, list of all users who's name starts from these 3 letters would be returned from the API to the web app. By typing or deleting any letter, list should change dynamically.
My question is how can implement this kind of "dynamic" search in my API. What should requests look like? What should the whole architecture of the process in the API look like?
There are a number of ways to pull this off. Coding this in your API is probably going to be the least efficient. I would recommend using SOLR or Lucene to index the data you want to search on. Your front-end will have some JS that will update your list when characters are entered or deleted in the text field.
JQuery provides a plugin that might be just what you're looking for from the UI.
https://jqueryui.com/autocomplete/
I'm new to web API design, so I've tried to learn best practices of web API design using these articles:
1.Microsoft REST API Guidelines
2.Web API Design-Crafting Interfaces that Developers Love from "Apigee"
Apigee is recommending web API developers to use these recommendations to have better APIs.
I quote here two of the recommendations:
I need C# code for implementing these recommendations in my Web APIs (in ASP Core) which is a back-end for native mobile apps and AngularJs web site.
Sweep complexity behind the ‘?’
Most APIs have intricacies beyond the base level of a resource. Complexities can include many states that can be updated, changed, queried, as well as the attributes associated with
a resource.
Make it simple for developers to use the base URL by putting optional states and attributes behind the HTTP question mark. To get all red dogs running in the park:
GET /dogs?color=red&state=running&location=park
Partial response allows you to give developers just the information they need.
Take for example a request for a tweet on the Twitter API. You'll get much more than a typical twitter app often needs - including the name of person, the text of the tweet, a timestamp, how often the message was re-tweeted, and a lot of metadata.
Let's look at how several leading APIs handle giving developers just what they need in
responses, including Google who pioneered the idea of partial response.
LinkedIn
/people:(id,first-name,last-name,industry)
This request on a person returns the ID, first name, last name, and the industry.
LinkedIn does partial selection using this terse :(...) syntax which isn't self-evident.
Plus it's difficult for a developer to reverse engineer the meaning using a search engine.
Facebook
/joe.smith/friends?fields=id,name,picture
Google
?fields=title,media:group(media:thumbnail)
Google and Facebook have a similar approach, which works well.
They each have an optional parameter called fields after which you put the names of fieldsyou want to be returned.
As you see in this example, you can also put sub-objects in responses to pull in other information from additional resources.
Add optional fields in a comma-delimited list
The Google approach works extremely well.
Here's how to get just the information we need from our dogs API using this approach:
/dogs?fields=name,color,location
Now I need C# code that handles these kind of queries or even more complex like this:
api/books/?publisher=Jat&Writer=tom&location=LA?fields=title,ISBN?$orderBy=location desc,writerlimit=25&offset=50
So web API users will be able to send any kind of requests they want with different complexities, fields, ordering,... based on their needs.
I want to gather google analytics data for daily/monthly report. All visitors, visitors for specific path and conversion rate. Setting for google analytics is already done (I think).
I searched ruby libraries, but many informations are out-of-dated and google api is sometimes update.
Now(2012-09-28) what library is popular? And How to get data? I searched ruby-toolbox, but both two libraries are still active?
Libralies
'garb'
https://github.com/vigetlabs/garb
'google-api-client'
http://code.google.com/p/google-api-ruby-client/
Purpose:
My first step: view data
My second step: edit data(Add note or notation)
thanks,
Google-Api-Client works fine, is an official client and is constantly updated, just check the commit logs and you will see. Note that this API is not for Google Analytics only but for any Google API that uses the Google Discovery API. You should try that.
Regardless of the client that you use make sure it's compatible with the latest version of the API. If it's built using the Google API Discovery service it's a plus because that means the client library will have support for new methods as soon as they are available. It's like an API for building API clients.
GARB on the other hand seems to be abandoned for a while and is probably missing out on several features that came recently.
Note that the Google Analytics API are currently read only. So no client library will support that since there's no API for writing annotations. There's a feature request for that though.