Get Google Drive item id of file with known path - google-api

A file is located in a known path on Google Drive, for example:
/root/Myfiles/test.txt
How can I get the item-id of the file using the Google Drive V3 REST API (https://www.googleapis.com/drive/v3/files/)? In detail, I am not sure how to construct the query paramer q= for this.
Regards,

Unless you have the file id of MyFiles then your going to have to do this in two calls.
The first thing we will do is list all the directories in root.
This can be done using the Q parameter as you already know
By passing parents in 'root' and mimeType = 'application/vnd.google-apps.folder' and name ='Myfiles' I tell it that I am looking for a folder called Myfiles that has a parent folder of root.
curl \
'https://www.googleapis.com/drive/v3/files?q=parents%20in%20%27root%27%20and%20mimeType%20%3D%20%27application%2Fvnd.google-apps.folder%27%20and%20name%20%3D%27YouTube%27&key=[YOUR_API_KEY]' \
--header 'Authorization: Bearer [YOUR_ACCESS_TOKEN]' \
--header 'Accept: application/json' \
--compressed
The response from this will then look something like this
{
"kind": "drive#fileList",
"incompleteSearch": false,
"files": [
{
"kind": "drive#file",
"id": "1R_QjyKyvET838G6loFSRu27C-3ASMJJa",
"name": "Myfiles",
"mimeType": "application/vnd.google-apps.folder"
}
]
}
I know know the file id of the folder called Myfiles
Now i can do another call which i request a file within that directory id with the name of test.txt like this parents in '1R_QjyKyvET838G6loFSRu27C-3ASMJJa' and name = 'test.txt'
The code will then look something like this
curl \
'https://www.googleapis.com/drive/v3/files?q=parents%20in%20%271R_QjyKyvET838G6loFSRu27C-3ASMJJa%27%20and%20name%20%3D%20%27test.txt%27&key=[YOUR_API_KEY]' \
--header 'Authorization: Bearer [YOUR_ACCESS_TOKEN]' \
--header 'Accept: application/json' \
--compressed
The response
{
"kind": "drive#fileList",
"incompleteSearch": false,
"files": [
{
"kind": "drive#file",
"id": "1_BgrWKsjnZvayvr2kbdHzSzE3K2tNsWhntBsQwfrDOw",
"name": "test.txt",
"mimeType": "application/vnd.google-apps.document"
}
]
}

Summary
As #DalmTo said If you want to search for files within a specific folder you need to have that ID to search within it.
parents in Whether the parent’s collection contains the specified ID.
Which means that you should do two separate queries. One asking for the id of your folder and another looking for the file test.txt in that folder.
q: parents in "root" and mimeType = "application/vnd.google-apps.folder" and name = "Myfiles"
q: parents in "ID_FOLDER" and mimeType = "text/plain" and name = "test"
Example:
If you only have one file in your entire Drive that meets the required characteristics, you could do it in a single query:
q: name = "test" and mimeType = "text/plain"
Caution: If you have uploaded the file, Drive may have detected it as: application/octet-stream. Normally .txt files are detected as plain/text, for more information on MIME types and Drive API, you can check here for common MIME types and here for Drive specific types.
Alternative using Google Apps Script
Here is an example using Google Apps Script:
function findFile() {
var folderId;
var folderQuery = '"root" in parents and title = "Myfiles" and mimeType = "application/vnd.google-apps.folder"'
let folder = Drive.Files.list({
q: folderQuery
})
folderId = folder.items[0].id
let fileQuery = `parents in "${folderId}" and title = "test"`
var file = Drive.Files.list({
q: fileQuery
})
return file.items[0].id
}
Caution: Google Apps Script uses Drive API v2, in this case the query_term name becomes title
More Information
For a deeper understanding of how the Drive API works you can check Search for files guide:
A query string contains the following three parts:
query_term operator values
query_term is the query term or field to search upon.
operator specifies the condition for the query term.
values are the specific values you want to use to filter your search results
To keep in mind when used outside of a client library:
Note: These examples use the unencoded q parameter, where name = 'hello' is encoded as name+%3d+%27hello%27. Client libraries handle this encoding automatically.

Related

How to accept and validate this map in gRPC protoc?

I'm sending the following POST request to my gRPC application:
curl \
--request POST \
--header 'Content-Type: application/json' \
--data-raw '{
"mandatory-key1": "value1",
"mandatory-key2": {
"arbitrary-optional-key1": [
"b",
"c"
],
"arbitrary-optional-key2": [
"e"
]
}
}' \
'http://localhost:11000/MyEndpoint'
The value associated with mandatory-key-1 must be a non-empty string.
The value associated with mandatory-key-2 must be a map where all keys are strings and all values are lists of strings.
Now I have to model this request's data structure in the gRPC proto file.
I am thinking of doing something like this:
message MyRequestData {
// pairs represents that map that the user will send in to the MyEndpoint.
map<string, string> pairs = 1;
}
But this specification is not general enough. I need to know how to write this specification correctly.
Question 1: How can I write this specification so it accepts strings in the values and also lists of strings?
Question 2: How can I do validation such that I ensure pairs has keys mandatory-key1 and mandatory-key2 and nothing else?
Question 3: How can I do validation such that I ensure:
pairs has keys mandatory-key1 and mandatory-key2 and nothing else?
pairs[mandatory-key1"] has value which is a non-empty string?
pairs["mandatory-key2"] has value which is a map of <strings, list of non-empty strings>?
Protobufs don't provide (the) validation (that you need).
You'd need to code your validation when you use the protoc-generated sources.
Protobuf doesn't support repeated map values directly, but you can:
message Request {
string mandatory_key1 = 1;
map<string, Value> mandatory_key2 = 2;
}
message Value {
repeated string value = 1;
}

Elasticsearch spell check suggestions even if first letter missed

I create an index like this:
curl --location --request PUT 'http://127.0.0.1:9200/test/' \
--header 'Content-Type: application/json' \
--data-raw '{
"settings" : {
"number_of_shards" : 1
},
"mappings" : {
"properties" : {
"word" : { "type" : "text" }
}
}
}'
when I create a document:
curl --location --request POST 'http://127.0.0.1:9200/test/_doc/' \
--header 'Content-Type: application/json' \
--data-raw '{ "word":"organic" }'
And finally, search with an intentionally misspelled word:
curl --location --request POST 'http://127.0.0.1:9200/test/_search' \
--header 'Content-Type: application/json' \
--data-raw '{
"suggest": {
"001" : {
"text" : "rganic",
"term" : {
"field" : "word"
}
}
}
}'
The word 'organic' lost the first letter - ES never gives suggestion options for such a mispell (works absolutely fine for any other misspells - 'orgnic', 'oragnc' and 'organi'). What am I missing?
This is happening because of the prefix_length parameter: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-suggesters.html . It defaults to 1, i.e. at least 1 letter from the beginning of the term has to match. You can set prefix_length to 0 but this will have performance implications. Only your hardware, your setup and your dataset can show you exactly what those will be in practice in your case, i.e. try it :). However, be careful - Elasticsearch and Lucene devs set the default to 1 for a reason.
Here's a query which for me returns the suggestion result you're after on Elasticsearch 7.4.0 after I perform your setup steps.
curl --location --request POST 'http://127.0.0.1:9200/test/_search' \
--header 'Content-Type: application/json' \
--data-raw '{
"suggest": {
"001" : {
"text" : "rganic",
"term" : {
"field" : "word",
"prefix_length": 0
}
}
}
}'
You need to use the CANDIDATE GENERATORS with phrase suggester check this out from Elasticsearch in Action book page 444
Having multiple generators and filters lets you do some neat tricks. For instance, if
typos are likely to happen both at the beginning and end of words, you can use multi-
ple generators to avoid expensive suggestions with low prefix lengths by using the
reverse token filter, as shown in figure F.4.
You’ll implement what’s shown in figure F.4 in listing F.4:
■ First, you’ll need an analyzer that includes the reverse token filter.
■ Then you’ll index the correct product description in two fields: one
analyzed with the standard analyzer and one with the reverse analyzer.
From Elasticsearch docs
The following example shows a phrase suggest call with two generators: the first one is using a field containing ordinary indexed terms, and the second one uses a field that uses terms indexed with a reverse filter (tokens are index in reverse order). This is used to overcome the limitation of the direct generators to require a constant prefix to provide high-performance suggestions. The pre_filter and post_filter options accept ordinary analyzer names.
So you can achieve this by using the reverse analyzer with the post-filter and pre-filter
And as you can see they said:
This is used to overcome the limitation of the direct generators to require a constant prefix to provide high-performance suggestions.
Check this Figure from Elasticsearch In Action book I believe it will make the idea more clear.
A screenshot from the book explains how elastic search will give us the correct phrase
For more information refer to the docs
https://www.elastic.co/guide/en/elasticsearch/reference/6.8/search-suggesters-phrase.html#:~:text=The%20phrase%20suggester%20uses%20candidate,individual%20term%20in%20the%20text.
If explained the full idea then this will be a very long answer but I gave you the key and you can go and do your research about using the phrase suggester with multiple generators.

symfony + AJAX how to upload files embedded in javascript JSON objects?

I have a client uploading files embedded in JSON objects because each one of these files has attached metadata.
The problem is that I do not know how many files they will upload so I need something dynamic
at the moment I have a fileList javascript object that contains sub objects that looks like this:
{
file1: null
meta1: null,
meta2: null,
etc...
}
{
file2: null
meta1: null,
meta2: null,
etc...
}
I upload it like this:
formData.append('files', this.fileList);
but in symfony, I do not know how to process this particular situation
If I look in the profiler I get something like this :
files "[object Object],[object Object]"
and $files = $request->files->get('files'); gives null
Is there another way to decode the data? it is there but I can't read it :(
If you have a file to upload, one per metadata row, I'd send the file separately when it's selected in the form (I like BlueImp for this, but there are others) with Ajax and pend it to a directory (with a cleanup script that deletes them when 24hr old, S3 does this nicely). Then return that path to the form, put that in your JSON-ified body you submit, then you have your metadata, which happens to include it's pending filepath. Move that pending file on submit and keep that new reference when you save the row or whatever. Voila.
So:
files[][filename].onchange: [/* Upload file to pending, return pending path */]
With return path, pending/ad32sY3KJ.png, submit
"files": [
["file":"`pending/ad32sY3KJ.png`","meta1":"Root Beer"],
["file":"`pending/34dks3DWf.png`","meta1":"Cat"]
]
Then your router handler has an array, so Content-type: application/json works as expected, although you'd use:
$body = \json_decode($request->getContent(), true);
Instead.

Elasticsearch URI Search multiple fields

I can do a quick URI search like
GET twitter/tweet/_search?q=user:kimchy
Can I search multiple fields this way? For example, user:kimchy AND age:23?
What I tried 1 (error):
curl -XDELETE localhost:9200/myindex/
curl localhost:9200/myindex/mytype/1 -d '{"a":1,"b":9}'
curl localhost:9200/myindex/mytype/2 -d '{"a":9,"b":9}'
curl localhost:9200/myindex/mytype/3 -d '{"a":9,"b":1}'
Say I want just the document {"a":9, "b":9}, I tried
GET localhost:9200/myindex/_search?q=a:9&b:9
but I get error
{
error: {
root_cause: [{
type: "illegal_argument_exception",
reason: "request [/myindex/_search] contains unrecognized parameter: [b:9]"
}],
type: "illegal_argument_exception",
reason: "request [/myindex/_search] contains unrecognized parameter: [b:9]"
},
status: 400
}
What I tried 2 (works!):
GET localhost:9200/myindex/_search?q=a:9 AND b:9
The spaces are important. Alternatively, use %20.
Yes, you can. Try something like this:
GET twitter/tweet/_search?q=user:kimchy%20AND%20age:23
Note that if you URI decode this, it's equivalent to:
GET twitter/tweet/_search?q=user:kimchy AND age:23
Note that when you are using this REST endpoint like this, I think you are really taking advantage of something like the query_string_query. Refer to those docs to get an idea of the extent of the query string language and features available to you.

CouchDB Filtered Replication

Trying out filters for replication, I stumbled upon a problem.
While my filter is working as an entry in the _replicator database, I doesn't when using cURL.
The filter in the design document is:
{
"_id": "_design/partial",
"filters": {
"mobile": "function(doc, req) {
if (doc._attachments) {
var result = new Boolean(true);
for (attachment in doc._attachments) {
if (attachment.content_type == 'image/jpeg') {
return true;
}
if (doc._attachments.length > 1024) {
result = false;
}
}
return result;
} else {
return true;
}
}"
}
}
The cURL line:
curl -X POST http://admin:pass#192.168.178.13:5985/_replicate -d '{\"source\":\"http://admin:pass#192.168.2:5984/docs2\",\"target\":\"docs2_partial\",\"filter\":\"partial/mobile\",\"create_target\":true}' -H "Content-Type: application/json"
I created _design/partial document on both target and source, but all documents are being replicated. Even the one with an attached binary bigger than 1 MB.
Any help is appreciated!
The cURL reply is:
{"ok":true,"session_id":"833ff96d21278a24532d116f57c45f31","source_last_seq":32,"replication_id_version":2,"history":[{"session_id":"833ff96d21278a24532d116f57c45f31","start_time":"Wed, 17 Aug 2011 21:43:46 GMT","end_time":"Wed, 17 Aug 2011 21:44:22 GMT","start_last_seq":0,"end_last_seq":32,"recorded_seq":32,"missing_checked":0,"missing_found":28,"docs_read":28,"docs_written":28,"doc_write_failures":0}]}
Using either " instead of \" or " instead of ' the result is:
{"error":"bad_request","reason":"invalid UTF-8 JSON: [...]}
Now I think perhaps the logic of your filter function simply has a bug. Here is how I read your filter policy:
All docs that have no attachments pass
All docs that have an image/jpeg attachment pass
Docs with more than 1,024 attachments fail
In any other case, the docs pass
That sounds like perhaps an incorrect policy. Another way to restate this policy is "Docs with more than 1024 attachments fail, everything else passes." However since you wrote so much code, I suspect my summary is not the true policy.
Another quick note, on what looks like a bug. Given:
for (attachment in doc._attachments) { /* ... */ }
The attachment variable will be things like "index.html" or "me.jpeg", i.e. filenames. To get the attachment content-type, you need:
var type;
// This is WRONG
type = attachment.content_type; // type set to undefined
// This is RIGHT
type = doc._attachments[attachment].content_type; // type set to "text/html" etc.
To avoid this bug, you could change your code to make things more clear:
for (attachment_filename in doc._attachments) { /* ... */ }
Next, doc._attachments.length will tell you the number of attachments in the document, not for example the length of the current attachment. It is odd that you test for that inside the loop, because the expression will never change. Are you trying to test for attachment size instead?
What is the output from curl (i.e. from CouchDB)?
From your example, my first guess is that you have a quoting error. Inside single-quotes, you do not need to escape the double-quotes. Try removing all those backslashes. What happens?
If you are on Windows, the single quote is not valid in the shell. In that case, keep the backslashes and just change the single-quote to a double-quote.

Resources