I am using Watson's Speech-To-Text Lite service and I am trying to find a way to automate the loading of new audio files to transcribe. I am very new to Bash and so I'm unclear of even the more rudimentary terms - so I'm finding the problem hard to find a solution for.
For a single use-case, I run the following file (my API key omitted with 'MY APIKEY')
curl -X POST -u "apikey: MY APIKEY" --header "Content-Type: audio/flac" --data-binary "#audiofile_1.flac" "https://gateway-lon.watsonplatform.net/speech-to-text/api/v1/recognize?model=en-US_BroadbandModel&speaker_labels=true" > C:/Users/outputpath/output_1.txt
What I am essentially trying to achieve is to overcome having to manually type and retype the names of the audio files and output. So if I had three (or more) audio files (i.e. audiofile_1, 2, and 3.flac), i would like to create an output file corresponding to each audio file - Some psuedo-code that might help explain what I mean would be
files = [file_1, file_2, file_3]
for file_x in files:
run curl command
save as output_x
You almost got it. You just need to learn some shell syntax:
files=("file_1" "file_2" "file_3")
for file_x in "${files[#]}"
do
curl -X POST -u "apikey: MY APIKEY" --header "Content-Type: audio/flac" --data-binary "#${file_x}" "https://gateway-lon.watsonplatform.net/speech-to-text/api/v1/recognize?model=en-US_BroadbandModel&speaker_labels=true" > "C:/Users/outputpath/${file_x}.txt"
done
First you create the files array with your list of files. Then, you iterate over those files and run the curl command on each of them.
Related
I'm having some hard time to work this one out. I need to feed GITLAP API with issues that are created based on a file that i have. Normally the output of the file is the following:
Microsoft xxx xxxxx - Remote Code Execution xxxxxx- April 2018 xxxxx Updates
Red Hat Enterprise xxxx - java-1.8.0-xxxxx Multiple xxxxxxx- RHSA-xxxxxx
So far so good, I already deal with this the following way:
while read in; do
curl --request POST --header "PRIVATE-TOKEN: xxxxxxxxxxxxx" https://gitlab.com/api/v3/projects/xxxxxx/issues?title="$in";
done < ~/input_file
The problem is now that i need to add a second variable to this because i need to introduce a description on each issue and now my input file change for the following:
Microsoft Malware Protection - Remote Code Execution Vulnerability - April 2018 Security Updates 40697
Red Hat Enterprise Linux 6 - java-1.8.0-openjdk Multiple Vulnerabilities - RHSA-2018:1188 40861
I would like to construct something like this:
while read in;
curl --request POST --header "PRIVATE-TOKEN: xxxxxxxxxxx" "https://gitlab.com/api/v4/projects/xxxxx/issues?title=$in&description=https://myspecialink.com/portal/notifications/show/$id";
done < ~/input_file
for example:
$in: must be everything except the bold number that i signalize above.
$id: must be only the numbers in bold above.
Can someone help me on point me the best way of achieve this?
Use bash parameter expansion operators to split the input.
while read in; do
id=${in##* } # Remove everything up to last space
in=${in% *} # Remove everything from last space
curl --request POST --header "PRIVATE-TOKEN: xxxxxxxxxxx" "https://gitlab.com/api/v4/projects/xxxxx/issues?title=$in&description=https://myspecialink.com/portal/notifications/show/$id";
done
I made a script that downloads several files located in my professional OneDrive. This script works perfectly from a French computer, a US computer but it can't work from a Japanese computer.
To permit you understand the problem, I will detail the program:
1- I establish the token system (I got inspired by Jay Lee detailed answer) and retrieve the token in the access_token variable.
2- To download the file, in my case I cannot use
curl -w %{time_total} https://graph.microsoft.com/v1.0/me/drive/items/01M...WU/content -H "Authorization: Bearer $access_token"
Thus, this how I proceed:
#I get the item properties
itemProperties=$(curl ${ODf1Mb} -H "Authorization: Bearer $access_token")
#In these properties I select the downloadUrl that will permit me to download the file
downloadUrl=$(echo -e "$itemProperties" | grep "#microsoft.graph.downloadUrl" | awk -F'[",]' '{ print $9 }')
#Finally I execute this URL storing the download time in a variable (I do all this stuff for this)
dload=$(curl -w %{time_total} ${downloadUrl} -H "Authorization: Bearer $access_token")
As I said at the begin, for French and US computers it will work but on the Japanese machine it doesn't. I do get the itemProperties and the downloadUrl but when I call the downloadUrl with CURL it seems that it cannot reach the server because I have this:
As we can see we do not even have the Total weight to be downloaded. As an element of comparison, this is the result in a French machine:
I know, there is a warning relating to command substitution but I haven't tried to fix it yet because it makes its job.
Note -> the downloadUrl has this format:
https://lpl-my.sharepoint.com/personal/{user}_{company infra domain}_com/_layouts/15/download.aspx?
I just cannot figure out what is the problem. I can access to the https://lpl-my.sharepoint.com through the browser so I don't think the server IP is banned.
Check your ping / traceroute to see if lpl-my.sharepoint.com resolves to the same network location.
Also, I have seen other folks run curl with -v to see verbose traces and see if what the difference is.
Setup
I'm Using curl in the following bash script to push a JSON file to a REST API running in tomcat sitting behind nginx.
while IFS= read -d '' -r file; do
base=$(basename "$file")
datetime=$(find $file -maxdepth 0 -printf "%TY/%Tm/%Td %TH:%TM:%.2TS")
curl -vX POST -H "Content-Type: application/json" -H "Cache-Control: no-cache" \
-d #"$file" -u vangeeij:eian12 \
"http://192.168.105.10/homeaccess/services/aCStats/uploadData?username=vangeeij&filename=$base&datetime=$datetime"
#sudo mv "$file" /home/vangeeij/acserver/resultsOld
done < <(sudo find . -type f -print0)
Problem
When running this script I get a http 400 response with curl error:
curl: (18) transfer closed with outstanding read data remaining
What I have tried
I have found 2 things. First running the same URL and body through Postman yields a successful POST.
I found that this error goes away when the last parameter is removed from the URL &datetime=$datetime
I have also found a few connections between this error and setting a curl option something like
curl_setopt($curl, CURLOPT_HTTPHEADER, array('Expect:'));
But I'm not sure where/how to set this exactly when using curl in a simple bash script
Question
What do I need to change in my curl command to get rid of the error and still be able to use all parameters?
UPDATE
Starting a new question, as further investigation has lead me to a better understanding of the problem.
New Question Link
The error has to do with the fact that the parameter datetime= ends up with text in it that needs to be URL encoded.
This was confirmed by replacing the variable with 2017%2F03%2F01%2008%3A50%3A56
and it worked.
So now the problem is, that I can't get --data-urlencode datetime=$datetime to work. It seems this just gets appended to the JSON data or something.
This error is being generated by the fact that the datetime= paramater is being passed in with non encoded non URL friendly characters... (eg. space).
The fix to this would be to find a way to convert the $datetime to a URLEncoded String.
eg. convert:
2017/03/01 08:50:56
TO
2017%2F03%2F01%2008%3A50%3A56
See the following discussion for one method to accomplish this.
Post JSON data to Rest with URLEncoded query paramaters
I am working on Windows10 and tried to load a geojson file into my couchdb via the "curl" command and a POST request in the cmd which looks like that:
C:\Program Files\cURL\bin>curl -d #path-to-my-data\data.geojson -H "Content-type: application/json" -X POST http://127.0.0.1:5984/_utils/database.html?-dbName-
and then I get the following error:
{"error":"method_not_allowed","reason":"Only GET,HEAD allowed"}
On http://couchdb-13.readthedocs.org/en/latest/api-basics/ it is said, that "If you use the an unsupported HTTP request type with a URL that does not support the specified type, a 405 error will be returned, listing the supported HTTP methods."
When I try that with a PUT request, I get the same error.
I validated the json with jsonlint so this should not be the problem.
I tried several tutorials like "Three Steps to CouchDB Heaven …" or "Export & Import a Database with CouchDB" but none of them seems to work.
So I am not sure, where the problem is. Do I need to make changes in my geojson file, or something else?
thanks for your help
The needed curl command just looks like that:
curl -H "Content-Type: application/json" -X POST http://localhost:5984/db -d #C:\Users\Name\Desktop\data.geojson
How do you POST a binary variable in curl bash?
#!/usr/bin/env bash
IMAGE=$(curl "http://www.google.com/images/srpr/logo3w.png")
curl --data-binary "$IMAGE" --request "POST" "http://www.somesite.com"
Curl seems to do corrupt the image when uploading.
Curl has the option to write response to disk and then read from it, but it'd be more efficient to do it solely in memory.
Try to eliminate the variable ... as follows:
curl "http://www.google.com/images/srpr/logo3w.png" | curl --data-binary - --request "POST" "http://www.somesite.com"
From the curl man page:
If you start the data with the letter #, the rest should be a file name to read the data from, or - if you want curl to read the data from stdin.
EDIT: From the man page, too:
--raw When used, it disables all internal HTTP decoding of content or transfer encodings and instead makes them passed on unaltered, raw. (Added in 7.16.2)
What happens, if applied on either or both sides?
I had a related problem, where I wanted to dynamically curl a file from a given folder.
curl --data-binary directory/$file --request "POST" "http://www.somesite.com"
did not work - uploaded the string "directory/myFile.jar" instead of the actual file.
Adding the # symbol
curl --data-binary #directory/$file --request "POST" "http://www.somesite.com" fixed it.