Use current working directory as path with AWS Cli - windows

I'm attempting to sync my bucket from my local directory using AWS Cli on Windows.
It works using the command
aws s3 sync C:\[long path name] s3://[bucket name]
I would prefer to replace the path name with something shorter or just associate it with the bucket. I've tried chdir and cd. Is there an easy way to do this?


If you wish to synchronize the current directory to an Amazon S3 bucket, use:
aws s3 sync . s3://[bucket name]
Or to sync to a directory within the S3 bucket:
aws s3 sync . s3://[bucket name]/[path]
The same syntax works on Windows and Linux.

Related

cloudformation validation not working locally on windows

I'm trying to validate the cloudformation template using AWS CLI on my windows machine locally.
The command is:
aws cloudformation validate-template --template-body file:///C:/AWS/template.json
But Im getting below error:
Error parsing parameter '--template-body': Unable to load param file file:///C:/AWS/template.json: [Errno 2] No such file or directory: 'file:///C:/AWS/template.json'

You can check permission of AWS directory and your template.json file as well.
Sometimes in Windows system directory files you have created in system drive (C://) are user permission restricted. So, It will not allow to access any created file easily.
2nd Way:
You can upload your template to any s3 bucket and then validate your file using s3 url.
For that, aws cli has valid permission for that operations.
Below is the command you can try with changing s3 URL by pointing it to your bucket and stored file:
aws cloudformation validate-template --template-url https://s3.amazonaws.com/cloudformation-templates-us-east-1/S3_Bucket.template

Is there any way to transfer windows files by automatically convert in zip format and then to aws s3 bucket in an automate way daily

Looking an automate process so that files will compress automatically and then transfered to aws s3 bucket from local system.

Just create a script that will:
Zip the files
Use the AWS Command-Line Interface (CLI) aws s3 cp command to copy the file to Amazon S3

aws s3 cli not working in window task scheduler

I try to run the following aws cli command in console it working correctly.
I have aws access key and secret configured.
aws s3 sync "C:\uploadfolder" s3://uploadfolder
However, when i run it inside windows task scheduler in windows 10 as well as windows server 2012, I got the following error:
cannot find the file specified 0x80070002
It does not seems like it is a corrupted profile because it does not work for both windows and other command is running as expected.
Is there any step that I miss out? or any other special command needed when run aws cli in window task scheduler.

Your cli command is attempting to sync a FILE called "uploadfolder". You need to change to the directory first, then run the command. Your command should instead be:
cd C:\uploadfolder
aws s3 sync . s3://uploadfolder/
This will recursively copy all files in your local directory that are not in your s3 bucket. If you would also like the sync command to delete files that are no longer in the local directory, you also need to add the --delete flag.
aws s3 sync . s3://uploadfolder/ --delete

Move files S3 to Ftb using bash script

I want to move files from Amazon s3 to ftp using bash script command...
I already tried
rsync --progress -avz -e ssh s3://folder//s3://folder/
Can anyone please suggest the correct command?
Thanks in advance

AWS built sync in their cli,
aws s3 sync ./localdir s3://mybucket
You can sync your local directory to remote bucket.
How to install aws cli?
https://docs.aws.amazon.com/cli/latest/userguide/installing.html
If you don't want to take the cli installation route, you can use docker to connect to a container, share your local directory to that container and perform the sync.
https://hub.docker.com/r/mesosphere/aws-cli/
Hope it helps.

You can't copy objects from S3 in that way because S3 is not an SSH service, it a file storage. So the easiest way is to mount the S3 bucket. Then you can use it like a normal volume and copy all files to the target.
You should do that on the target system otherwise you have to copy all the file over the third server or computer.
https://www.interserver.net/tips/kb/mount-s3-bucket-centos-ubuntu-using-s3fs/

Downloading folders from aws s3, cp or sync?

If I want to download all the contents of a directory on S3 to my local PC, which command should I use cp or sync ?
Any help would be highly appreciated.
For example,
if I want to download all the contents of "this folder" to my desktop, would it look like this ?
aws s3 sync s3://"myBucket"/"this folder" C:\\Users\Desktop

Using aws s3 cp from the AWS Command-Line Interface (CLI) will require the --recursive parameter to copy multiple files.
aws s3 cp s3://myBucket/dir localdir --recursive
The aws s3 sync command will, by default, copy a whole directory. It will only copy new/modified files.
aws s3 sync s3://mybucket/dir localdir
Just experiment to get the result you want.
Documentation:
cp command
sync command

Just used version 2 of the AWS CLI. For the s3 option, there is also a --dryrun option now to show you what will happen:
aws s3 --dryrun cp s3://bucket/filename /path/to/dest/folder --recursive

In case you need to use another profile, especially cross account. you need to add the profile in the config file
[profile profileName]
region = us-east-1
role_arn = arn:aws:iam::XXX:role/XXXX
source_profile = default
and then if you are accessing only a single file
aws s3 cp s3://crossAccountBucket/dir localdir --profile profileName

In the case you want to download a single file, you can try the following command:
aws s3 cp s3://bucket/filename /path/to/dest/folder

You've many options to do that, but the best one is using the AWS CLI.
Here's a walk-through:
Download and install AWS CLI in your machine:
Install the AWS CLI using the MSI Installer (Windows).
Install the AWS CLI using the Bundled Installer for Linux, OS X, or Unix.
Configure AWS CLI:
Make sure you input valid access and secret keys, which you received when you created the account.
Sync the S3 bucket using:
aws s3 sync s3://yourbucket/yourfolder /local/path
In the above command, replace the following fields:
yourbucket/yourfolder >> your S3 bucket and the folder that you want to download.
/local/path >> path in your local system where you want to download all the files.

sync method first lists both source and destination paths and copies only differences (name, size etc.).
cp --recursive method lists source path and copies (overwrites) all to the destination path.
If you have possible matches in the destination path, I would suggest sync as one LIST request on the destination path will save you many unnecessary PUT requests - meaning cheaper and possibly faster.

Question: Will aws s3 sync s3://myBucket/this_folder/object_file C:\\Users\Desktop create also the "this_folder" in C:\Users\Desktop?
If not, what would be the solution to copy/sync including the folder structure of S3? I mean I have many files in different S3 bucket folders sorted by year, month, day. I would like to copy them locally with the folder structure to be kept.

Resources