I am running a postgresql database on AWS ec2 machine and scheduling the backup of the database every night at 12:00 a.m. by crontab which executes a shell script containing all the commands to backup the data. The problem is, I want to do it without stopping the database service. Please help!
https://www.postgresql.org/docs/current/static/app-pgdump.html,
pg_dump is a utility for backing up a PostgreSQL database. It makes
consistent backups even if the database is being used concurrently.
pg_dump does not block other users accessing the database (readers or
writers).
You run it without stopping the service
Related
I have developed a shell script which connects to the AWS EKS Cluster, Execute a kubectl command, get the result as JSON string and insert into AWS RDS MySQL DB. Tested the script in AWS Cloud Shell which is fine there. Now, I want to schedule the script (using CRON Scheduler may be ) every 30 minutes. I am exploring options on where to schedule the Unix Shell Job. Below are the options I can see from web :-
From an AWS EC2 Instance
Using AWS Systems Manager
For #1, I can start an EC2 instance free tier and schedule the cron job.
For #2, not very sure on how AWS Systems Manager works.
Is there any other approach to schedule a shell job in AWS or GCP ?
I am using user-data of ec2 instance to power up my auto scale instances and run the application. I am running node js application.
But it is not working properly. I have debugged and checked the instance cloud monitor output. So it says
pm2 command not found
After reading and investigating a lot I have found that the path for the command as root is not there.
As EC2 user-data when it tries to run it finds the path
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
After ssh as ec2-user it is
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/ec2-user/.local/bin:/home/ec2-user/bin
After ssh as sudo su it is
/root/.nvm/versions/node/v10.15.3/bin:/sbin:/bin:/usr/sbin:/usr/bin
It works only for the last path.
So what is the way or script to run the command as root during launch of the instance provided by user-data?
All thought to start your application with userdata is not recommended, because as per AWS documentation they are not assuring that instance will only come up after successful execution of user data. Even if user data failed it will spin up your instance.
For your problem, I assume if you give the complete absolute path of the binary, It will work.
/root/.nvm/versions/node/v10.15.3/bin/pm2
Better solution for this approach, create a service file for your application startup and start application with systemd or service.
I have a bash script running on a server to capture and process the Logs.
The script is invoked by an utility for processing the Logs. After processing the Logs the
state of the Logs should be stored in db2 table located in some other server.
say, if i have shell script on 110.88.99.10 and db2 on 110.88.99.11..
I need save the processed result to db2.. Any suggestions?
It seems that you need to install the DB2 client: IBM data server access. Once you have installed, you configure the remote instance and database (catalog TCPIP Node and catalog database), and then you can integrate db2 commands (db2 insert ... - log result) in your script.
While trying to run a riak-admin backup riak#ec2-xxx.compute-1.amazonaws.com riak /home/user/backup.dat all on a remote machine (ec2 instance) I encounter the following error message
{"init terminating in do_boot",{{nocatch,{could_not_reach_node,'riak#ec2-xxx.compute-1.amazonaws.com'}},[{riak_kv_backup,ensure_connected,1,[{file,"src/riak_kv_backup.erl"},{line,171}]},{riak_kv_backup,backup,3,[{file,"src/riak_kv_backup.erl"},{line,40}]},{erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,572}]},{init,start_it,1,[]},{init,start_em,1,[]}]}}
I assume there's a connection / permission error since the same backup command will work if run locally on the instance (with a local node ip of course), I should note the server (Node.js) can remotely connect to that ip so the port is open and accessible 8098). Any advice on how to make the backup operational remotely?
It would appear that the riak-admin backup command doesn't work remotely - and certainly it's not something I've ever tried to do. I'd recommend setting up a periodic backup (via cron or similar) and then use rsync to get your backup file down to local.
Alternatively, you could try the following hacky untested idea for a single script.
#!/bin/bash
ssh ec2-xxx.compute-1.amazonaws.com "riak-admin backup riak#ip-local-ec2 /home/user/backup.dat all"
rsync -avP ec2-xxx.compute-1.amazonaws.com:/home/user/backup.dat .
need pretty trivial task
i have server, which in crontab every night will run "something" what will launch new EC2 instance, deploy there code (ruby script), run it, upon completion of the script shutdown the instance.
how to do it the best?
thanks.
Here's an approach that can accomplish this without any external computer/cron job:
EC2 AutoScaling supports schedules for running instances. You could use this to start an instance at a particular time each night.
The instance could be of an AMI that has a startup script that does the setup and running of the job. Or, you could specify a user-data script be passed to the instance that does this job for you.
The script could terminate the instance when it has completed running.
If you are running EBS boot instance, then shutdown -h now in your script will terminate the instance if you specify instance-initiated-shutdown-behavior of terminate.