I would like to setup a makefile that reads user input with Bash autocompletion, and based on that create a directory where some files would be copied into. Basically something like
$ make initializeProject
Enter destination: Dir<ectory>/Sub<directory>/ProjectDirectory
mkdir: created directory 'Directory/Subdirectory/ProjectDirectory'
'Templates/TemplateFile' -> 'Directory/Subdirectory/ProjectDirectory/TemplateFile'
where < X > stands for autocompleted parts.
As of now my makefile does not autocomplete, and it looks like this:
initProjectLaTeX:
#read -p "Enter destination: " destination; \
mkdir -pv $$destination; \
cp -iv ~/.templates/latex/* $$destination
I have tried this option, but it produces the error /bin/sh: 1: read: Illegal option -e.
How can I access Bash autocompletion here?
make uses /bin/sh as default shell and on many systems it's not the same as bash (e.g. on Debian is dash). I suspect that the -e option of read is a bash extension, that is why you get the error:
/bin/sh: 1: read: Illegal option -e
setting /bin/bash as the SHELL in the Makefile should work:
Makefile
SHELL := /bin/bash
initProjectLaTeX:
#read -e -p "Enter destination: " destination; \
mkdir -pv $$destination; \
cp -iv ~/tmp/*c $$destination
example
$ make initProjectLaTeX
Enter destination: foo/bar/baz
mkdir: created directory 'foo'
mkdir: created directory 'foo/bar'
mkdir: created directory 'foo/bar/baz'
'/home/marco/tmp/foo.c' -> 'foo/bar/baz/foo.c'
$ tree foo
foo
└── bar
└── baz
└── foo.c
2 directories, 1 file
$
completion
$ make initProjectLaTeX
Enter destination: foo
foo/ foo.c
Enter destination: foo
foo/ foo.c
Enter destination: foo
Rather than have make interact with the user, pass the desired destination to make.
Start with a modified recipe:
initProjectLaTeX:
mkdir -pn $(DESTINATION)
cp -iv ~/.templates/latex/* $(DESTINATION)
then run
make initProjectLaTeX DESTINATION=Dir<ectory>/Sub<directory>/ProjectDirectory
(bash allows completion for the value of an assignment.)
Related
I am trying to execute following lines-
sh """
#!/bin/bash
cd ${WORKSPACE}
cat << EOF > build.sh
#!/bin/bash
mkdir wsbuildlinux
cd wsbuildlinux
cmake3 ../../wiresharkbuild_linux
make
EOF
chmod 755 build.sh
# run container and build wireshark
./build-env/run-wireshark-env ./build.sh
"""
and here is the content of "build.sh"-
#!/bin/bash
mkdir wsbuildlinux
cd wsbuildlinux
cmake3 ../../wiresharkbuild_linux
make
EOF
chmod 755 build.sh
# run container and build wireshark
./build-env/run-wireshark-env ./build.sh
But I keep getting following error-
/opt/cvsdirs/crdbuilds/jenkins/slave/workspace/Wireshark_Wireshark_trunk#tmp/durable-f510f070/script.sh: line 17: warning: here-document at line 5 delimited by end-of-file (wanted `EOF')
I have no idea what this error means and how to resolve it. Could you please give some advice on this
The EOF token has to be at the beginning of the line.
sh """
#!/bin/bash
cd ${WORKSPACE}
cat << EOF > build.sh
#!/bin/bash
mkdir wsbuildlinux
cd wsbuildlinux
cmake3 ../../wiresharkbuild_linux
make
EOF
chmod 755 build.sh
# run container and build wireshark
./build-env/run-wireshark-env ./build.sh
"""
I have the following code on cron job, it runs but the code does not really do what it supposed to. It does not create the directory plus is does not do anything in the code. Please help check if the way I pointed to the directory is wrong.
#!/bin/bash
NAMEDATE=`date +%F_%H-%M`_`whoami`
NAMEDATE2=`date `
mkdir ~/home/u3811*****/domains/website.com/public_html/cron/backup/files/$NAMEDATE -m 0755
mysqldump -u u3811*****_boss -p"*******" u3811*****_data | gzip ~/home/u3811*****/domains/website.com/public_html/cron/backup/files/$NAMEDATE/db.sql.gz
echo "This is the database backup for website.com on $NAMEDATE2" |
mailx -a ~/home/u3811*****/domains/website.com/public_html/cron/backup/files/$NAMEDATE/db.sql.gz -s "website.com Database attached" -- mail#gmail.com
chmod -R 0644 ~/home/u3811*****/domains/website.com/public_html/cron/backup/files/$NAMEDATE/*
exit 0
Your NAMEDATE variable needs to be modified a bit, as shown below, for more information about variables in bash see this link
NAMEDATE=$(date +%F_%H-%M"_"$(whoami))
When you issue the mkdir command you will need to pass the -p option to create the complete directory structure if it doesn't exists.
mkdir -p ~/home/u3811numbers/domains/website.com/public_html/cron/backup/files/$NAMEDATE -m 0755
Also, the ~ character on Linux based distributions is used as a shortcut for the home directory of the user that invokes it so, in the line below the result is /home//home/u3811*****/domains/website.com/public_html/cron/backup/files/2020-09-04_23-13_ you can read more about it in here
In you last command before the exit, you might need to pass a wildcard (*) to avoid removing the executable bit on the directory, see below
chmod -R 0644 ~/home/u3811*****/domains/website.com/public_html/cron/backup/files/$NAMEDATE/
The final version of your script will look something like this.
#!/bin/bash
NAMEDATE=$(date +%F_%H-%M"_"$(whoami))
NAMEDATE2=date
mkdir -p ~/home/u3811******/domains/website.com/public_html/cron/backup/files/$NAMEDATE -m 0755
mysqldump -u u3811*****_boss -p"******" u3811*****_data | gzip > ~/home/u3811*****/domains/website.com/public_html/cron/backup/files/$NAMEDATE/db.sql.gz
echo "This is the database backup for website.com on $NAMEDATE2" | mailx -a ~/home/u3811*****/domains/website.com/public_html/cron/backup/files/$NAMEDATE/db.sql.gz -s "website.com Database attached" -- mail#gmail.com
chmod -R 0644 ~/home/u3811*****/domains/website.com/public_html/cron/backup/files/$NAMEDATE/*
To debug a bash script you can always pass the -x flag for more information take a look at this article
In a directory I have a config file with my db variables.
This file (db/database.ini) looks like this:
[PostgreSQL]
host=localhost
database=...
user=postgres
password=...
I have another file (db/create_stmts.sql) where I have all my raw create table statements, and i am trying to experiment the use of a Makefile to have a command like this:
make create-db from_file=db/create_stmts.sql
In order not to repeat myself, I thought of tailing the variables of db/database.ini to a file which I would then source, creating shell variables to pass to psql in the make file.
Here's my plan:
make-db:
# from_file: path to .sql file with all create statements to create the database where to insert
# how to run: make create-db from_file={insert path to sql file}
file_path=$(PWD)/file.sh
tail -n4 db/database.ini > file.sh && . $(file_path)
# -U: --user
# -d: --database
# -q: --quiet
# -f: --file
psql -U $(user) -d $(database) -q -f $(from_file) && rm file.sh
Which I run by: make create-db from_file=db/create_stmts.sql
Which gives me this message - from which i kindof understand that the sourcing just did not work.
#from_file: path to .sql file with all create statements to create the database where to insert
# how to run: make create-db from_file={insert path to sql file}
file_path=/home/gabriele/Desktop/TIUK/companies-house/file.sh
tail -n4 db/database.ini > file.sh && .
# -U: --user
# -d: --database
# -q: --quiet
# -f: --file
psql -U -d -q -f db/schema_tables.sql && rm file.sh
psql: FATAL: Peer authentication failed for user "-d"
Makefile:3: recipe for target 'create-db' failed
make: *** [create-db] Error 2
Any help?
Another solution, perhaps simpler to understand:
make-db:
file_path=$$PWD/file.sh; \
tail -n4 db/database.ini > file.sh && . $$file_path; \
psql -U $$user -d $$database -q -f $$from_file && rm file.sh
Note using ; and \ to convince make to run all commands in a single shell, and using $$ to escape the $ and use shell variable references.
The error is in the text, namely
psql -U -d -q -f db/schema_tables.sql && rm file.sh
This happens because the variables $(user) and $(database) aren't set. Every line within a target is executed in a sub shell. There is now way to use source like you would in a regular script.
You could create a file named database.mk in which you define these variables and use include database.mk at the top of your makefile to include them:
Makefile
CONFILE ?= database
include $(CONFILE).mk
test:
#echo $(user)
#echo $(database)
database.mk
user := user
database := data
If you want to parse the ini file you could do that as such
CONFILE := db/database.ini
make-db: _setup_con
echo $(user) $(database)
# your target
_setup_con:
$(eval user=$(shell grep "user=" $(CONFILE) | grep -Eo "[^=]*$$"))
$(eval database=$(shell grep "database=" $(CONFILE) | grep -Eo "[^=]*$$"))
# and so forward
I would make it more Make-way by using feature of automatic Makefile generation. Given that a configuration file is a simple properties file, its syntax is easily parseable by Make, it's sufficient to just get the lines with variables, i.e.:
include database.mk
database.mk: db/database.ini
grep -E '^\w+=\w+$$' $< > $#
.PHONY: create-db
create-db: $(from_file)
psql -U $(user) -d $(database) -q -f $<
Some additional notes:
create-db should be made .PHONY to avoid situation when nothing is done due to somebody creating (accidentally or not) a file named create-db,
by making create-db depending on from_file one can get a clean and readable error from make that a file does not exist instead of possibly cryptic error later.
Here is the story:
cd ~
mkdir bin
export PATH=$PATH:bin
mkdir -p projects
cd projects
echo 'hello world' > hello.sh
chmod +x hello.sh
ln -s hello.sh ~/bin/hello
hello
output:
-bash: hello: command not found
How I changed it:
ln -s hello.sh ~/bin
hello.sh
The output is more weird:
-bash: /home/qht/bin/hello.sh: Too many levels of symbolic links
I ls it to see what happened:
ls -l ~/bin/hello.sh
/home/qht/bin/hello.sh -> hello.sh
I figure it out, hello.sh reference itself. And hello before reference hello.sh which doesn't exist.
I fix it by:
ln -sf $PWD/hello.sh ~/bin/hello
ls ~/bin/hello
/home/qht/bin/hello -> /home/qht/projects/hello.sh
and it works, I also man ln to see if there is a convenient option to do that, this is what I found:
ln -sfr hello.sh ~/bin/hello
ls -l ~/bin/hello
/home/qht/bin/hello -> ../projects/hello.sh
And it works, the -r option did the work.
But I'm curious, if ln -r can automatically write the relative path data into symbolic links, Why doesn't there an option maybe -a to do the absolute path work.
Or, is relative path for links is more practical than absolute path?
Try this:
cd ~
mkdir bin
export PATH=$PATH:~/bin # Need absolute path to bin
mkdir -p projects
cd projects
echo 'echo "hello world"' > hello.sh # If the script is just hello world
# this will become an infinite loop
chmod +x hello.sh
ln -s "$PWD/hello.sh" ~/bin/hello # the symbolic link in this case
# needs to be a absolute path
hello
I am trying to create tar from a file, which contains list of other files and saving it to stdout.
let suppose there is a file called "files-to-create" which has path of other files like /home/abc.txt /home/def.txt and I want to create tar of abc.txt,def.txt.
my script contains:
exec 100>&1
tar cf - -T files-to-sync >&100
and I am calling the script and saving it to some other file like:
/script.sh > final_tar.tar
But while creating the tar I am getting error, can somebody help me out?
You can use the following script to reach your goal, let me know if something is unclear:
Prototype 1:
$ cat scriptTar.sh
#!/bin/bash
readonly HELP="$(basename "$0") <list_of_files> <output_tar>
this script will generate a tar file composed of all files present in <list_of_files> input file
the output tar file will be saved as <output_tar>
to run the script provide the input and output filenames"
readonly INPUT_LIST_FILE=$1
readonly OUTPUT_TAR_FILE=$2
if [ -z "$INPUT_LIST_FILE" -o -z "$OUTPUT_TAR_FILE" ]
then
echo $HELP;
exit 1;
fi
tar cf - -T $INPUT_LIST_FILE > $OUTPUT_TAR_FILE
exit $?
Folder content:
$ tree .
.
├── a
│ └── abc.txt
├── b
│ └── def.txt
├── c
│ └── ghj.txt
├── files-to-sync.in
└── scriptTar.sh
3 directories, 5 files
List file content:
$ cat files-to-sync.in
./a/abc.txt
./b/def.txt
./c/ghj.txt
Execution:
$ ./scriptTar.sh files-to-sync.in output.tar
tar file content:
$ tar -tvf output.tar
-rw-rw-r-- arobert/arobert 4 2018-02-22 16:50 ./a/abc.txt
-rw-rw-r-- arobert/arobert 4 2018-02-22 16:50 ./b/def.txt
-rw-rw-r-- arobert/arobert 4 2018-02-22 16:50 ./c/ghj.txt
Or use the following script if you really want to display it on stdout:
Prototype 2 via ssh:
#!/bin/bash
readonly HELP="ERROR: $(basename "$0") <list_of_files>
this script will generate to stdout a tar file composed of all files present in <list_of_files> input file
to run the script provide the input file and redirect the output to a file"
readonly INPUT_LIST_FILE=$1
if [ -z "$INPUT_LIST_FILE" ]
then
echo $HELP;
exit 1;
fi
tar cf - -T $INPUT_LIST_FILE
Execution via ssh:
$ ssh user#localhost "cd /home/user/test_tar/; ./scriptTar.sh files-to-sync.in" > output.tar
user#localhost's password:
Content of the tar generated:
tar -tf output.tar
./a/abc.txt
./b/def.txt
./c/ghj.txt
extracting the content:
tar xvf output.tar
./a/abc.txt
./b/def.txt
./c/ghj.txt
checking the files:
more ?/*.txt
::::::::::::::
a/abc.txt
::::::::::::::
abc
::::::::::::::
b/def.txt
::::::::::::::
abc
::::::::::::::
c/ghj.txt
::::::
However if I were you, I would not only generate a tar file but add some compression (tar.gz) and transfer the file with rsync to be able to restart the download from the point where it stopped in case of transfer error.
So the proper solution is
Case1: If you are passing the list of file as an argument
you can use this:
files-to-sync=$1
tar cf - -T files-to-sync
Case2: If you want to use absolute path for the list of file
you can use this:
tar cfP - -T /path/to/the/file
use -P in case of absolute path.