In Google's documentation for the Drive REST API v3, they provide the following "example" for how to upload a file to Drive using the "simple upload" method:
POST /upload/drive/v3/files?uploadType=media HTTP/1.1
Host: www.googleapis.com
Content-Type: image/jpeg
Content-Length: number_of_bytes_in_file
Authorization: Bearer your_auth_token
JPEG data
I am trying to construct a curl POST request to upload a .csv to Drive. I have tried the following (as well as countless variations), with no success. It keeps returning with an Error 400 (bad request)!!! error:
curl -s "https://www.googleapis.com/upload/drive/v3/files?uploadType=media HTTP/1.1" -H "Authorization: Bearer $access_token" -H "Content-Type: $mime_type" -H "Content-Length: $file_size" -d "$file" -X POST
For reference, the variables are defined as:
file_size="$(du -b $file | awk '{ print $1 }')"
mime_type=$(file -ib "$file")
Anyway, Google doesn't explain the "JPEG data" part, and I suspect that is where my curl POST request is wrong.
I've found a bunch of relevant examples on both here and Github, but none of them worked for me (all of the ones I found were outdated, anyway). I also tried multipart uploading, but I also couldn't get that to work (and again, most of the examples here seemed outdated).
Related
As I made a batch file to update NirSoft tools, I had a strange experience using wget.
First I downloaded a text file with pad links:
wget http://www.nirsoft.net/pad/pad-links.txt --backups=20 --append-output=C:\Path\Update\LOG\Nirsoft\%Timestamp%_NirSoft.log
After, I used fart-js to delete rows I did not need from the pad-links.txt file. Also I used that program to change the download links to https://www.nirsoft.net/utils, and change the file extensions to .zip.
fart ".\pad-links.txt" "http://www.nirsoft.net/pad" "http://www.nirsoft.net/utils" | tee --append C:\Path\Update\LOG\Nirsoft\%Timestamp%_NirSoft.log
and
fart ".\pad-links.txt" ".xml" ".zip" | tee --append C:\Path\Update\LOG\Nirsoft\%Timestamp%_NirSoft.log
After, to download the programs, I used:
wget --timestamping --input-file=C:\Path\UtilSuit\NirLauncher\Download\pad-links.txt --append-output=C:\Path\Update\LOG\Nirsoft\%Timestamp%_NirSoft.log
Having a look at the log file I found out that not all programs are stored in this location. For example WirelessKeyView is stored in https://www.nirsoft.net/toolsdownload/wirelesskeyview.zip.
Trying to get this file with wget leads to downloaded corrupt files at size of 4kb. The same with cURL and aria2. When I download it with Mozilla, or IDM, I have no problems to get the file. So I tried out wget --auth-no-challenge or wget --header="Accept: text/html" --user-agent="Mozilla/5.0 …"
I also tried cliget, the wget/aria2/curl lines it produced while normal downloading with Mozilla.
wget --header 'Host: www.nirsoft.net' --user-agent 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:92.0) Gecko/20100101 Firefox/92.0' --header 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' --header 'Accept-Language: de,en-US;q=0.7,en;q=0.3' --referer 'https://www.nirsoft.net/utils/wirelesskeyview.html' --header 'Upgrade-Insecure-Requests: 1' --header 'Sec-Fetch-Dest: document' --header 'Sec-Fetch-Mode: navigate' --header 'Sec-Fetch-Site: same-origin' --header 'Sec-Fetch-User: ?1' --header 'DNT: 1' --header 'Sec-GPC: 1' 'https://www.nirsoft.net/toolsdownload/wirelesskeyview.zip' --output-document 'wirelesskeyview.zip'
I googled and found this reference for powershell, (same error), but cannot reproduce the working answer in batch, (I am not familiar with powershell scripting).
So how is is possible to download the single wirelesskey.zip file with wget/curl or aria2 in a batch script?
A workaround I found out is downloading it directly from the pad Panel but I want the .zip-file, including the updated .chm-file, and also the 64-bit versions, if available.
One more note, within my anti-virus tool the nirsoft site is exempted from scanning, so that is not the answer.
Any solutions?
Aah, this one is simple. If you look at the actual page downloaded, it's called "403.html". So, let's open it. The first thing that strikes you is this:
<title>Error 403: Missing HTTP referer in the HTTP request</title>
So, the server wants a Referer header. Sure, let's give it one:
$ wget --referer foo <URL>
And it downloads the zip file correctly as expected.
Now, really, the server should not be returning a HTTP 200 response with a file called 403. It really should have sent back a HTTP 403 response. But what can you do? There's broken servers everywhere
I am trying to get a response from Microsofts Office 365 Management API using bash curl commands-
I get the token like this-
TOKEN=$(curl -X POST "https://login.microsoftonline.com/$TENANTID/oauth2/token" -d "grant_type=client_credentials&client_id=$CLIENTID&client_secret=$ACCESSCODE&resource=$RESOURCEURL" | jq -r '.access_token')
And then use the token to fetch the data like this-
RESULT=`curl -X GET -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" https://manage.office.com/api/v1.0/$TENANTID/ServiceComms/Services`
I do get a token back with the first command, so that works fine and my tenant/client/resource strings are correct
But the second one always gives
{
"error":{
"code":"","message":"Authorization has been denied for this request."
}
}
I pretty sure I have access-
What am I missing?
Turns out the resource URL i was using did not match the URL i was requesting data from
This question is about the search-tweets-ruby client provided by twitter for use with their premium and enterprise API's.
I am following the instructions and run into a 'Bad Authentication data' error when running the app (via terminal - Mac OS X) to retrieve tweets with a single rule.
The 'bearer token' and 'dev environment' are correct because a Curl request works.
The following are the contents of my ./config/config.yaml file
auth:
app_token: my_generated_bearer_token
labels:
environment: my_dev_environment_name
options:
search_type: premium
archive: fullarchive
max_results: 500
write_mode: standard-out
out_box: ./output
I'm not sure what I'm missing here, but would appreciate an assist. I haven't worked with the Twitter API before, although I've reviewed the documentation before asking on SO.
Thanks, everyone.
Update:
The first sample call, from the provided link is:
$ruby ./search-app.rb -r "snow profile_region:colorado has:media".
This yields a 'bad authentication error'.
I provided the contents of my yaml file, because presumably that is the only difference between the Curl request and the client app, if the 'bearer token' and 'environment name' work with Curl.
curl --request POST \
--url https://api.twitter.com/1.1/tweets/search/30day/prod.json \
--header 'authorization: Bearer AAAAAAAAAAAAAAAAAAAAAMLheAAAAAAA0%2BuSeid%2BULvsea4JtiGRiSDSJSI%3DEUifiRBkKG5E2XzMDjRfl76ZC9Ub0wnz4XsNiRVBChTYbJcE3F' \
--header 'content-type: application/json' \
--data '{
"query":"from:TwitterDev lang:en",
"maxResults": "100",
"fromDate":"201811010000",
"toDate":"201811062359"
}'
There is no code.
I think the issue is that your YAML file is configured for the full-archive search endpoint, yet your CURL-based call is going to the 30-day search endpoint. If you update the YAML file to point to the '30day' endpoint, I suspect it will succeed.
From what I've read on https://graphql.org/learn/serving-over-http/ and Performance test for graphQL API, I can send a graphQL request over jmeter.
In the stack overflow example, a graphQL server was built with Apollo and then queried via Jmeter. I am just trying to do load testing and not build schemas or data sets, so I don't have Apollo on my local machine, and I'm also not sure that I would be testing the QA environment if I did try to do this all locally.
I can access our graphQL at https://proprietary-website/graphql/gql and if I run the following query, it returns the expected results:
query currentUserProfile {
currentUserProfile {
id
}
}
Now, I try to apply that to jmeter
I create a HTTP Request in my thread group with protocol: https, server name: proprietary-website, method: GET, path: /graphql/gql
I create an HTTP Header Manager and include the required Bearer Token and Content-Type headers
I create a listener to View Results Tree
Then I run a jmeter load test with 500 threads
I see the valid URL passed in the request body:
GET https://proprietary-website/graphql/gql
GET data:
{"query":{"currentUserProfile {id}"}}
[no cookies]
but I get back:
HTTP/1.0 400 Bad Request
Content-Type: text/html; charset=UTF-8
Referrer-Policy: no-referrer
Content-Length: 1555
Date: Thu, 12 Dec 2019 16:40:26 GMT
I am new to graphql, am I not allowed to send Jmeter requests to the URL my dev gave me and do I have to use the apollo server? If so, can someone point me to some examples?
I am on Windows 10
I also tried:
k6, but this requires docker and i have no admin rights on my computer
easygraphql-lt but when I ran a test script with #easygraphql-lt, I got dependency errors that I don't quite understand how to fix
I looked at easygraphql-load-tester, but this would require me to have access to the project but all i have is the website. I don't have a graphQL.js or an index.js to reference
UPDATE: I just saw that I can copy the query as a curl request so I pasted that request in command line and it gave me extra headers that I put in my jmeter HTTP Header manager:
$ curl 'https://proprietary-website/graphql/gql' -H 'Accept-Encoding: gzip, deflate, br' -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'Connection: keep-alive' -H 'DNT: 1' -H 'Origin: https://proprietary-website' -H 'Authorization: Bearer redacted' --data-binary '{"query":" query currentUserProfile { currentUserProfile { id }}"}'
But I still get a 400 back, so I think I need to send the query differently as it is showing appended as a --data-binary (not sure what that means). I tried some other SO suggestions, no luck so far
I also tried against an app out of the O'Reilly graphql book, snowtooth.herokuapp.com. This was simpler, so the curl request was simpler, and the information I pasted into jmeter was less, but I still got the error: GET query missing using Protocol: HTTP, server name: snowtooth.herokuapp.com, method: GET and Body Data: {"query":"query { allLifts { name}}"}, plus all the requisite headers in the Header Manager
Not sure what your proprietary website implementation is, however this http://snowtooth.herokuapp.com/ can be tested like:
HTTP GET request:
HTTP POST request:
Also be aware that since JMeter 5.1 it's possible to create a JMeter test plan from CURL command, this http://snowtooth.herokuapp.com/ gives me the following CURL command:
curl 'http://snowtooth.herokuapp.com/' -H 'Accept-Encoding: gzip, deflate, br' -H 'Content-Type: application/json' -H 'Accept: application/json' -H 'Connection: keep-alive' -H 'DNT: 1' -H 'Origin: http://snowtooth.herokuapp.com' --data-binary '{"query":"{allLifts{name}}"}' --compressed
And when I import it to JMeter I can execute the requests successfully:
Having a problem with CURL and the HTTP User and password Auth methods, it is not liking the exclamation mark, I've tried escaping the following ways:
Tried and failed...
/usr/bin/curl -u 'UserName\WithSlash:PasswordWithExclamation!' https://test.com/
/usr/bin/curl -u UserName\\WithSlash:PasswordWithExclamation\! https://test.com/
Not working for basic or digest if it matters (using --anyauth) ... getting 401 denied...
What am I doing incorrectly?
curl -u UserName\\WithSlash:PasswordWithExclamation\! http://....
works fine.
it sends
GET / HTTP/1.1
Authorization: Basic VXNlck5hbWVcV2l0aFNsYXNoOlBhc3N3b3JkV2l0aEV4Y2xhbWF0aW9uIQ==
User-Agent: curl/7.21.0
Host: teststuff1.com:80
Accept: */*
which is "UserName\WithSlash:PasswordWithExclamation!" in the auth string.
not that complicated, just use "". at least it works on Linux.
for example:
curl -u "username:passwdwithspecialchar" GET https://....
If you know the server supports Basic auth, you could set the header directly:
curl --header "Authorization: Basic $(base64 --wrap=0 credentials)" https://example.org
This way you can store the user and password (UserName\WithSlash:PasswordWithExclamation!) without any escaping in the credentials file you pass to the base64 command.