Perspective API: Proper way to send requests with auto-detection of language - google-api

Question

I am a bit confused on the proper way to send requests using Google's Perspective API.
Sending the following request works:
{"comment":{"text":"yo hamburger"},"languages":["en"],"requestedAttributes":{"TOXICITY":{}}}
In the documentation, it says, "...If you are using a production attribute, language is auto-detected if not specified in the request." So, I tried:
{"comment":{"text":"yo hamburger"},"requestedAttributes":{"TOXICITY":{}}}
And in response, I got a HTTP/1.0 400 Bad Request.
I also tried including all of the languages listed on the documentation page, like this:
{"comment":{"text":"yo hamburger"},"languages":["en","fr","es","de","it","pt"],"requestedAttributes":{"TOXICITY":{}}}
But that also gave me a response of HTTP/1.0 400 Bad Request.
Another attempt was made leaving the array of languages empty, like this:
{"comment":{"text":"yo hamburger"},"languages":[],"requestedAttributes":{"TOXICITY":{}}}
However, it still gave me a response of HTTP/1.0 400 Bad Request.
I was wondering, what is the proper way to send a request to the API and have it auto-detect language?

Answer 1

User x00 provided the path to the solution in the question's comment section. By using curl, I was able to see what was going on.
Here's what was happening:
In this first example, the system worked without error.
CURL:
curl -H "Content-Type: application/json" --data \
'{comment: {text: "yo hamburger"},
languages: ["en"],
requestedAttributes: {TOXICITY:{}} }' \
https://commentanalyzer.googleapis.com/v1alpha1/comments:analyze?key=[API_KEY]
RESPONSE:
{
"attributeScores": {
"TOXICITY": {
"spanScores": [
{
"begin": 0,
"end": 12,
"score": {
"value": 0.050692778,
"type": "PROBABILITY"
}
}
],
"summaryScore": {
"value": 0.050692778,
"type": "PROBABILITY"
}
}
},
"languages": [
"en"
],
"detectedLanguages": [
"tr",
"ja-Latn",
"de",
"en"
]
}
In this second example, the system was indeed auto-detecting language, but since "yo hamburger" was detected as Turkish, it could not provide a solution and instead sent a 400 as the response code.
CURL:
curl -H "Content-Type: application/json" --data \
'{comment: {text: "yo hamburger"},
requestedAttributes: {TOXICITY:{}} }' \
https://commentanalyzer.googleapis.com/v1alpha1/comments:analyze?key=[API_KEY]
RESPONSE:
{
"error": {
"code": 400,
"message": "Attribute TOXICITY does not support request languages: tr",
"status": "INVALID_ARGUMENT",
"details": [
{
"#type": "type.googleapis.com/google.commentanalyzer.v1alpha1.Error",
"errorType": "LANGUAGE_NOT_SUPPORTED_BY_ATTRIBUTE",
"languageNotSupportedByAttributeError": {
"detectedLanguages": [
"tr"
],
"attribute": "TOXICITY"
}
}
]
}
}
This next example is more mysterious to me, as the language field for the request is plural, "languages," so it seems you can provide more than one language. However, it said it couldn't support that.
CURL:
curl -H "Content-Type: application/json" --data \
'{comment: {text: "yo hamburger"},
languages:["en","fr","es","de","it","pt"],
requestedAttributes: {TOXICITY:{}} }' \
https://commentanalyzer.googleapis.com/v1alpha1/comments:analyze?key=[API_KEY]
RESPONSE:
{
"error": {
"code": 400,
"message": "Attribute TOXICITY does not support request languages: en,fr,es,de,it,pt",
"status": "INVALID_ARGUMENT",
"details": [
{
"#type": "type.googleapis.com/google.commentanalyzer.v1alpha1.Error",
"errorType": "LANGUAGE_NOT_SUPPORTED_BY_ATTRIBUTE",
"languageNotSupportedByAttributeError": {
"requestedLanguages": [
"en",
"fr",
"es",
"de",
"it"
],
"attribute": "TOXICITY"
}
}
]
}
}
In this next example, leaving the languages array empty also provided the auto-detection of language, but again, "yo hamburger" was detected as Turkish, so it could not provide a response.
CURL:
curl -H "Content-Type: application/json" --data \
'{comment: {text: "yo hamburger"},
languages:[],
requestedAttributes: {TOXICITY:{}} }' \
https://commentanalyzer.googleapis.com/v1alpha1/comments:analyze?key=[API_KEY]
RESPONSE:
{
"error": {
"code": 400,
"message": "Attribute TOXICITY does not support request languages: tr",
"status": "INVALID_ARGUMENT",
"details": [
{
"#type": "type.googleapis.com/google.commentanalyzer.v1alpha1.Error",
"errorType": "LANGUAGE_NOT_SUPPORTED_BY_ATTRIBUTE",
"languageNotSupportedByAttributeError": {
"detectedLanguages": [
"tr"
],
"attribute": "TOXICITY"
}
}
]
}
}
Noticing that Perspective API would not allow me to choose all of the languages that are provided for the TOXICITY report, I decided to try two languages. The response was the same. Apparently Perspective API rejects the request if multiple languages are specified. Perhaps naming the field "languages" was a thought for the future.
CURL:
curl -H "Content-Type: application/json" --data \
'{comment: {text: "yo hamburger"},
languages: ["en","fr"],
requestedAttributes: {TOXICITY:{}} }' \
https://commentanalyzer.googleapis.com/v1alpha1/comments:analyze?key=[API_KEY]
RESPONSE:
{
"error": {
"code": 400,
"message": "Attribute TOXICITY does not support request languages: en,fr",
"status": "INVALID_ARGUMENT",
"details": [
{
"#type": "type.googleapis.com/google.commentanalyzer.v1alpha1.Error",
"errorType": "LANGUAGE_NOT_SUPPORTED_BY_ATTRIBUTE",
"languageNotSupportedByAttributeError": {
"requestedLanguages": [
"en",
"fr"
],
"attribute": "TOXICITY"
}
}
]
}
}

Answer 2

Maybe you're using bad client library or other issue is causing the problem, Here is documentation about client library that in example language is auto-detected without problem. Check that and if not successful provide more details for further investigations.

Answer 3

As I said in the comments, general approach to these kind of issues: use curl. It helps a lot.
To sum up you findings:
auto-detection with a set of languages doesn't seem to work.
the correct way to send a request with auto-detection enabled is
{comment: {text: "some text"}, requestedAttributes: {TOXICITY:{}} }
but sometimes it fails on short texts, especially with slang inside.
So what can be done about it?
The easyest way is to assign some weight to Bad Requests (probably something around 0.5). Anyway, as a response you get the probability and not a definitive answer. So
toxicity score = 1 means "definitely toxic"
toxicity score = 0 means "not toxic at all"
and toxicity score = 0.5 means "we have no idea"
same thing goes for Bad Request - "you have no idea"
and you will get 0.5 from time to time, so you must deal somehow with comments of that score anyway. As well as with network errors etc.
But I would say that a probability of toxicity of a comment that result in LANGUAGE_NOT_SUPPORTED_BY_ATTRIBUTE is higher than 0.5. But it's up to you to decide on the exact number.
As auto-detection doesn't work well with short texts you can bump up probability of correct auto-detection by adding some context into you request: a couple of other comments in the thread, or better yet, a couple of other comments from the same user. Not too big ones and not too small ones.
Make three requests specifying a language. As far as I can tell TOXICITY works only with English, Spanish, and French. On github I've got this reply:
"TOXICITY is currently supported in English (en), Spanish (es), French (fr), German (de), Portuguese (pt), and Italian (it). We will work to remove the contradictions you identified."
Auto-detect by yourself before sending a request. That'll require some effort, but it shouldn't be too hard, given you have much more context available to you than is available to Perspective API (or any other third-party API)
Also
These kind of APIs are not supposed to stay unattended. Fine tuning and moderation on your part is required. Or else we'll end up in the worst-case scenario of algocracy :).
And I think it's a good idea in general to store statistics of toxicity of comments for a user... as well as some manual coefficient. Because for example: Mathematical formulas give high toxicity
I've posted an couple of issues on github, but no reply yet (whating for reply on the second issue). When/If I'll get them I'll update my answer with details.