I'm trying to upload a file to s3 using this curl command (can't use awscli)
src_file=/path/to/file.png
dest_file=test_image.png
bucket=mybucket.com
s3Key="<key>"
s3Secret="<secret>"
contentsha=`openssl sha256 ${src_file} | awk '{print $NF}'`
curl https://${bucket}.s3.amazonaws.com/${dest_file} \
-H "x-amz-content-sha256: ${contentsha}" \
--aws-sigv4 "aws:amz:us-east-1:s3" \
--user "${s3Key}:${s3Secret}" \
--upload-file "${src_file}" \
--insecure
but I keep getting this error:
<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message>
What am I doing wrong?
Related
I'm trying to upload the data to aws s3 using following script and getting below error.
Example Script
#!/usr/bin/sh
file_to_upload="/home/sraj/Hello10.txt"
bucket="mybucket"
filepath="/${bucket}/${file_to_upload}"
contentType='application\/x-compressed-tar'
dateValue="`date +'%a, %d %b %Y %H:%M:%S %z'`"
signature_string="PUT\n\n${contentType}\n${dateValue}\n${filepath}"
s3_access_key=xxxxxxxxx
s3_secret_key=yyyyyyyyy
signature_hash=`echo -en ${signature_string} | openssl sha256 -hmac ${s3_secret_key} -binary | base64`
echo "${s3_access_key} : ${s3_secret_key} : ${signature_hash}"
curl -X PUT -T "${file_to_upload}" \
-H "Host: ${bucket}.s3.amazonaws.com" \
-H "Date: ${dateValue}" \
-H "Content-Type: ${contentType}" \
-H "Authorization: AWS ${s3_access_key}:${signature_hash}" \
https://${bucket}.s3.amazonaws.com/${file_to_upload}
I think this is related to I AM user KMS encryption option.
Kindly help me how to solve above issue. I'm not much idea about curl and s3 also.
It will very helpful for sample code.
Some of them mentioned this is related to Signature version4 issue but I don't have much idea about fix this one.
The error says Invalid Argument.
I found this link, I have no idea if it will help.
https://www.gyanblog.com/aws/how-upload-aws-s3-curl/#curl-the-savior`
The only difference I see is the dateValue.
I do not know if that matters, but just in case, I posted this.
# about the file
file_to_upload=<file you want to upload>
bucket=<your s3 bucket name>
filepath="/${bucket}/${file_to_upload}"
# metadata
contentType="application/x-compressed-tar"
dateValue=`date -R`
signature_string="PUT\n\n${contentType}\n${dateValue}\n${filepath}"
#s3 keys
s3_access_key=<your s3 access key>
s3_secret_key=<your s3 secret key>
#prepare signature hash to be sent in Authorization header
signature_hash=`echo -en ${signature_string} | openssl sha1 -hmac ${s3_secret_key} -binary | base64`
# actual curl command to do PUT operation on s3
curl -X PUT -T "${file_to_upload}" \
-H "Host: ${bucket}.s3.amazonaws.com" \
-H "Date: ${dateValue}" \
-H "Content-Type: ${contentType}" \
-H "Authorization: AWS ${s3_access_key}:${signature_hash}" \
https://${bucket}.s3.amazonaws.com/${file_to_upload}
I am following this documentation (https://www.vaultproject.io/api-docs/secret/consul) for creating consul secret engine, create role and get a token. But I am getting a bit of misleading error when trying to get the token. Any ideas or similar issues?
here are the steps that I execute:
list roles
curl \
--header "X-Vault-Token: s.xxx" \
--request LIST \
https://vault.service.brain.consul:8200/v1/consul/roles \
--key /opt/vault/tls/vault.key \
--cert /opt/vault/tls/vault.pem \
--cacert /opt/vault/tls/cachain.pem
output
{"request_id":"a9a892e9-dba0-f6ca-b181-b16718db281d","lease_id":"","renewable":false,"lease_duration":0,"data":{"keys":["consul-worker"]},"wrap_info":null,"warnings":null,"auth":null}
show role
curl \
--header "X-Vault-Token: s.xxx" \
https://vault.service.brain.consul:8200/v1/consul/roles/consul-worker \
--key /opt/vault/tls/vault.key \
--cert /opt/vault/tls/vault.pem \
--cacert /opt/vault/tls/cachain.pem
output
{"request_id":"6b3b6dd8-ff30-eb91-20c6-3c0bd105be30","lease_id":"","renewable":false,"lease_duration":0,"data":{"lease":0,"local":false,"max_ttl":0,"policies":["agent"],"token_type":"client","ttl":0},"wrap_info":null,"warnings":null,"auth":null}
get role token
curl \
--header "X-Vault-Token: s.xxx" \
https://vault.service.brain.consul:8200/v1/consul/creds/consul-worker \
--key /opt/vault/tls/vault.key \
--cert /opt/vault/tls/vault.pem \
--cacert /opt/vault/tls/cachain.pem
output
{"errors":["1 error occurred:\n\t* tls: failed to find any PEM data in certificate input\n\n"]}
how is it going?
So I used a bash script to create a remote repository using a password to access a endpoint like this:
NEWVAR="{\"name\":\"$githubrepo\",\"private\":\"true\"}"
curl -u $USERNAME https://api.github.com/user/repos -d "$NEWVAR"
However, GitHub is going to not allow developers to access endpoints using passwords anymore. So my question is how do I create a remote repository using a personal access token?
use --header to transmit authorization:
#!/usr/bin/env sh
github_user='The GitHub user name'
github_repo='The repository name'
github_oauth_token='The GitHub API auth token'
# Create the JSON data payload arguments needed to create
# a GitHub repository.
json_data="$(
jq \
--null-input \
--compact-output \
--arg name "$github_repo" \
'{$name, "private":true}'
)"
if json_reply="$(
curl \
--fail \
--request POST \
--header 'Accept: application/vnd.github.v3+json' \
--header "Authorization: token $github_oauth_token" \
--header 'Content-Type: application/json' \
--data "$json_data" \
'https://api.github.com/user/repos'
)"; then
# Save the JSON answer of the repository creation
printf '%s' "$json_reply" >"$github_repo.json"
printf 'Successfully created the repository: %s\n' "$github_repo"
else
printf 'Could not create the repository: %s\n' "$github_repo" >&2
printf 'The GitHub API replied with this JSON:\n%s\n' "$json_reply" >&2
fi
See my answer here for a featured implementation example:
https://stackoverflow.com/a/57634322/7939871
I am trying to write a Bash script on my server (My Server) that will grab a file from one remote server (Source) and copy it to a Dropbox account (Destination). I need to get the file from Source via SFTP and will be copying it to Destination using the Dropbox API (HTTPS). So far I can get the file with:
curl -k "sftp://Source/file.txt" --user "me:mypasswd" -o "/test/file.txt" --ftp-create-dirs
and then copy it to Dropbox with
curl -X POST https://content.dropboxapi.com/2/files/upload \
--header "Authorization: Bearer " \
--header "Dropbox-API-Arg: {\"path\": \"/path/to/file.txt\",\"mode\": \"add\",\"autorename\": true,\"mute\": false,\"strict_conflict\": false}" \
--header "Content-Type: application/octet-stream" \
--data-binary #/test/file.txt
I'm guessing the "right" way to do this is to pipe the file from Source directly to Destination, but I'm just not sure how to go about putting them together.
This is definitely not my area of expertise, so I don't even know where to start - nested CURL calls? If anyone could point me in the right direction, I'd be most appreciative.
UPDATE
Here's the whole curl command I'm running:
curl -X POST https://content.dropboxapi.com/2/files/upload \
--header "Authorization: Bearer $token" \
--header "Dropbox-API-Arg: {\"path\": \"/xfer/chef.txt\",\"mode\": \"add\",\"autorename\": true,\"mute\": false,\"strict_conflict\": false}" \
--header "Content-Type: application/octet-stream" \
--data-binary "$(curl -k "http://marketing.dave0112.com/file.txt" --user "me:mypasswd")"
I was having issues with CURL not supporting SFTP so I changed to HTTP while i get that end sorted out. Not sure if that affects anything.
You can replace this line :
--data-binary #/test/file.txt
with
--data-binary #<(curl -k "sftp://Source/file.txt" --user "me:mypasswd")
If problems, try :
--data-binary "$(curl -k "sftp://Source/file.txt" --user "me:mypasswd")"
I am using Stripe's Curl API to create subscriptions.
My bash script looks like this
#!/bin/bash
set pass MyPasword123
curl https://api.stripe.com/v1/customers -u sk_test_BQokikJOvBiI2HlWgH4olfQ2 -d card[number]=4242424242424242 -d card[exp_month]=12 -d card[exp_year]=2016 -d card[cvc]=123 -d plan=EarlyAdopter -d email=test#gmail.com
expect -re "Enter host password for user 'sk_test_BQokikJOvBiI2HlWgH4olfQ2:':"
send "${pass}\r"
This does now work.
How do I enter automatically a password for the Stripe Curl API?
Thanks...
The issue here is that you are only passing the API key but you're missing the : at the end. This is mentioned in the documentation about the Authentication. It should be like this:
curl https://api.stripe.com/v1/customers \
-u sk_test_XXX: \
-d card[number]=4242424242424242 \
-d card[exp_month]=12 \
-d card[exp_year]=2016 \
-d card[cvc]=123 \
-d plan=EarlyAdopter \
-d email=test#gmail.com