I'm pretty decent with bash and UNIX commands, and the Terminal app, but I'm wondering, is there a way to make a "macro" (maybe that's the wrong word!) to automate some tasks?
For example, to get into my current project directory, I type:
$ cd ~/Documents/College/F13/CS362/lab3/os-lab-3
And then I immediately do
$ hg pull
(password)
And then
$ hg update
So is there a way to even automate that first step of the cd call? I've never tried anything like that before, not sure if what I'm going for is even possible.
Bonus: is there a way to have the macro enter my password when prompted as well? Security/privacy is not really an issue here, there's no thermonuclear codes hiding around in the repo.
The name generally used in this context (a macro in bash) is shell script. To automate the commands from your post you would need to create a file with a name like 'myscript.sh' with the following contents:
#!/bin/bash
cd ~/Documents/College/F13/CS362/lab3/os-lab-3
hg pull
hg update
The first line of the script is a hashbang. This is a special comment line which indicates that file should be executed using the indicated program. Bash in this case.
To run the script first ensure that it is executable by running the following command:
chmod +x myscript.sh
From there in terminal window just run the script by specifying the path to it:
./myscript.sh
Don't put passwords into shell scripts though! It is a bad habit. For authentication with Mercurial an ssh key pair is the best way to go. The ssh key can be loaded into OSX's keychain, so you won't have to type anything when you run the script. You'll also need to add the public portion of your ssh key pair to the list of authorized hosts on the remote Mercurial repository.
Related
I have a complicated prompt. Very. I use git-bash for windows.
SSH-ing in on gitbash sends me to the CMD prompt. I know to type ssh -t user#host "bash -l" to get a bash prompt. It works, however, I use a repository called gitstatus to speed up the parsing of the git commands, and sshing in using bash -l calls the PS1 set in /c/cygwin/etc/bash.bashrc (for Mintty), NOT ~/bash.bashrc (for GitBash). This initially seems fine, as I can just copy paste the code from the GitBash *.bashrc to cygwin's. However, the gitstatus repository only works on bash terminals, aka not Cygwin/Mintty, so the prompt when I ssh in appears quite slower in git repos (Mintty doesn't allow gitstatus to be sourced and speed up parsing, once again) than if I was not SSH-ed in and using the GitBash-sourced prompt.
Are there any work arounds for this? I have seen many simnilar questions here but none have provided me a solution.
If you're in my situation, just source the scripts (duh). If you can't execute foo in file x when it should do the same thing as bar in file y, just source bar from file x. It seems obvious but took me a moment.
ALSO: Make sure to run dos2unix.exe on admin command prompt to remove carriage returns every time you make a change.
I have a Perl program on server_B which uses Perl DBI and 5.010 and runs fine from the server_B terminal. I run it from a shell script which first prepares some arguments and then passes them to the Perl program, all works fine.
I need to run a shell script on server_A that will execute that script on server_B. This is because the Perl program creates several files that I want to SFTP back over to server_A. This is the script I'm running on server_A:
ssh server_B <<- EOF
perl/update.sh
EOF
There is some strange behavior which I'm trying to understand:
The script (update.sh) on server_B runs mysql, which is not installed on server_A (which is why I have to do this whole thing.) If I try to run it on server_B as-is, I can call mysql just like that. But when I run the above script (on server_A) to ssh into server_B and run that script, it doesn't recognize mysql unless I change the file (on server_B) to call the full path /opt/mysql/client/bin/mysql (even though that file is already on server_B with mysql installed) Does this mean server_B is picking up on the PATH variable from server_A instead of using my PATH variable from server_B? Is it trying to run my programs from server_A on the files on server_B? How and why??
If I make the change above it executes the script, but when it hits Perl it says
Perl v5.10.0- required - this is only v5.8.8
Again, 5.10 works fine on server_B but the version of Perl on server_A is 5.8.8.
So I got rid of use 5.010; because it actually wasn't necessary, but then I have a similar problem with my modules (DBI and DBD::mysql). I get:
Can't locate DBI.pm in #INC (#INC contains.. [my Perl PATH from server_A])
at perlfile.pl line 4
I was expecting the ssh heredoc call to update.sh (from server_A) to run exactly as update.sh does if I call it on server_B, but instead it seems like it's trying to use my programs from server_A on server_B, which I find weird. Can anyone help me understand why it's happening? I feel like I'm misunderstanding something fundamental about how ssh works.
server_A is AIX with ksh
server_B is AIX with bash
Edit - since some of you voted to the effect that I haven't done my research, here's what else I've tried. I didn't mention because I don't understand them fully, these are just guesses based on other SO posts & hunches. It'd be disingenuous if I gave the impression I knew what I was talking about.
If this is a duplicate, which question should I be looking at? If this is a "just read the manual situation", which one? What should I look for?
Read man ssh looking for clues related to environment variables, didn't find anything I understood
Tried running with -t
Tried running with -t -t
Did log in remotely with ssh and manually running it - this DOES work
Sourced my .bash_profile in the update script
Tried to re-assign PATH as the remote server's PATH when ssh
Tried using a different delimiter for the heredoc
Tried < instead of <<
Tried without the "-"
Edit 2 with Saigo's help below I determined that when in interactive ssh, if I echo $PATH I do get the target server's $PATH, but in a shell script I don't. That led me to this:
https://serverfault.com/questions/643333/different-bash-path-variables-when-using-ssh-script-vs-interactive-ssh
where I found out that scripted ssh doesn't call .bashrc, but interactive ssh does. So it looks like I was on the right track trying to source .bash_profile inside the scripted SSH heredoc, just need .bashrc not .bash_profile - however I don't have a .bashrc on the target server. I do have .profile but when I source that, I get an error stating it's for interactive bash sessions only. So now I'm just trying to find whatever file would contain my $PATH variable because it's apparently not .bashrc as there isn't one in there.
Edit 3 - tried hard-coding the PATH variable into a file and sourcing that and even then when I echo $PATH I get the origin server's PATH variable. It is reading the file in correctly, I also assigned another test variable and echoed that as part of the script. I tried sourcing /etc/profile and no luck.
I found a solution that works perfectly. I wasn't able to get it to work with ~/.bashrc, ~/.bash_profile, or ~/.ssh/rc but still not sure why it's not picking up my environment variables even with sourcing these.
Since it works when I manually ssh in and then run the commands one-by-one, I used these arguments to run ssh in a forced interactive login.
ssh server_B bash --login -i "~/perl/update.sh"
See these for more:
https://superuser.com/questions/564926/profile-is-not-loaded-when-using-ssh-ubuntu
https://unix.stackexchange.com/questions/46143/why-bash-unable-to-find-command-even-if-path-is-specified-properly
Hope this is useful for someone in the future. Thank you for your assistance Saigo.
I want to run a remote program via ssh which requires a certain environment. Thus before executing the program I source a specific file building up the environment. If I'm logged onto the machine directly this is no problem but when I execute the command via ssh
#!/bin/bash
foo=`ssh user#host "source ~/script.sh; ~/run/program"`
I get an error that indicates that the script was not sourced correctly. Do you know what I have to do in order to get the script sourced and the program executed in the same session?
EDIT:
I'm exporting the LD_LIBRARY_PATH with the script and the executable is complaining that it cannot find the shared object file. The default shell is bash. 'Session' is definitive not the right wording. I meant 'terminal environment'.
This may not be the cleanest way, but if you invoke bash with the interactive option (-i) and send commands through the standard input, it should work.
In particular,
foo=`ssh user#host bash -i <<EOF
source ~/script.sh
~/run/program
EOF`
It would be much easier if you have a script program_in_env.sh that does exactly the two steps you want:
#!/bin/bash
source ~/script.sh
~/run/program
Then you would just need to call ssh user#host program_in_env.sh.
Good luck.
Thank you for all your time and help. I found the issue. The basic idea of how to execute the remote program was right from the beginning. When testing my case locally on the machine, the current working directory was different. For some reason the cwd is important when sourcing this bash script.
I am using vimwiki as my local wiki and keep it in git in order to be able to sync it with various pcs. I am trying to automate the process of putting the generated HTML from vimwiki on my server so I can easily look stuff up.
My idea is to checkout the repository on a regular basis on the server and have shell script in place which calls vim and tells him to execute VimwikiAll2HTML, ending afterwards. I can then symlink the html folder somewhere or point nginx there or whatever.
I was able to figure out that I can directly execute a command when calling vim by using the -c parameter:
vim -c "VimwikiAll2HTML" -n index.wiki
This command automatically generates the correct HTML. However, I have to press a key and then quit vim (:q) in order to get back into the shell. It doesn't seem suited to be run inside a bash script run by cron? Can I change the command somehow in order to exit after the html generation finished? Or is there any other way I'm not aware of? I looked into the vimwiki plugin because I thought that it maybe uses an external library for HTML generation which I can call in my script but it seems that the plugin does everything by itself.
This command should work:
$ vim -c VimwikiAll2HTML -c q index.wiki
This is my script to automate the git push of my static blog.
When I run each command one by one in the Terminal, it works. Do I need to add delays, maybe it's going too fast. The pelican command (static website generator) takes quite a lot of time (2 seconds). Is the rest of the script crashing during that?
#!/bin/sh
dropbox
cd blog
pelican . -s /Users/Paul-Arthur/Desktop/Desktop/Dropbox/blog/pelican.conf.py -t subtle
cd output
git add .
git commit -m 'commit'
git push
Updated: Sorry, yeah dropbox is a custom command in my bash_profile (this is not the problem, it works I know ;) ).
Sadly, when I click my script, it executes (but does not work) extremely quickly so I cannot see the errors.
Here is the output from the calepin command. The errors are normal and I expect it to run with that. Do you think that this is the problem? If so what can I do?
familys-imac:blog Paul-Arthur$ pelican . -s /Users/Paul-Arthur/Desktop/Desktop/Dropbox/blog/pelican.conf.py -t subtle
ERROR: Skipping ./articles/aboutme.md: impossible to find informations about 'title'
ERROR: Skipping ./articles/static_sites.md: impossible to find informations about 'title'
familys-imac:blog Paul
It might be due to the « cd » command, since it's not a command, it is a builtin from the shell, and doesn't act like a command.
To debug it, try adding «pwd» command inside your script before and after the «cd» line, to be sure the working directory has change.
It could also be due to the shell you are using, in the shebang (first line of your script), you are using the /bin/sh script. Is it the good one ? When you do it in your shell, you are maybe using another like bash, dash, zsh etc.
To determine that, type that in your current shell :
which `echo $0`
You will get an answer like :
/bin/bash
or something like this. Use this in your shell script :
#!/bin/bash
And try again your script.
Good luck with your project.