I have this sql command : insert into users(username, password) values ($username, $password)
I want to display this line for every user
this is my script
#!/bin/bash
for name in $(cat /etc/passwd | cut -d: -f1)
do
pass= sudo grep -w $name /etc/shadow | cut -d: -f2
echo 'insert into `users`(`username`, `password`) values ($name, $pass)'
done
But when i execute the script it doesn't do the substitution
As root (sudo -s):
#!/bin/bash
while read name; do
pass=$(grep -w "$name" /etc/shadow | cut -d: -f2)
echo "INSERT INTO \`users\`(\`username\`, \`password\`) VALUES ($name, $pass)"
done < <(cut -d: -f1 /etc/passwd)
Notes
If you are a bash beginniner, some good pointers to start learning :
FAQ,
Guide,
Ref,
bash hackers,
quotes,
Check your script
And avoid people recommendations saying to learn with tldp.org web site, the tldp bash guide -ABS in particular) is outdated, and in some cases just plain wrong. The BashGuide and the bash-hackers' wiki are far more reliable.
Learn how to quote properly in shell, it's very important :
"Double quote" every literal that contains spaces/metacharacters and every expansion: "$var", "$(command "$var")", "${array[#]}", "a & b". Use 'single quotes' for code or literal $'s: 'Costs $5 US', ssh host 'echo "$HOSTNAME"'. See
http://mywiki.wooledge.org/Quotes
http://mywiki.wooledge.org/Arguments
http://wiki.bash-hackers.org/syntax/words
Here I have fixed the issues with your script. Please check if it works now :
#!/bin/bash
while read name
do
pass=$( sudo grep -w $name /etc/shadow | awk -F':' '{print $2}' )
echo "insert into 'users'('username', 'password') values ($name, $pass)"
done <<< "$(awk -F':' '{print $1}' /etc/passwd))"
Regards!
Try this Shellcheck-clean pure Bash code, which needs to be run as root:
#! /bin/bash -p
while IFS=: read -r name pass _ ; do
printf "insert into users (username, password) values ('%s', '%s')\\n" \
"$name" "$pass"
done </etc/shadow
/etc/shadow should contain the same users as /etc/passwd, so the code doesn't use /etc/passwd.
See BashFAQ/001 (How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?) for an explanation of how while IFS=: read -r ... works. It also explains the use of _ as a "junk variable".
See the accepted, and excellent, answer to Why is printf better than echo? for an explanation of why the code uses printf instead of echo.
Related
I am trying to read a file using a while loop running as sudo, and then run some awk on each line. File owner is user2 and I am running as user1.
sudo sh -c 'while IFS= read -r line; do
echo $line | awk '{print $NF}'; done < /home/user2/test.txt'
I am having trouble with the single quotes. Double quotes does not work at both the places where I have single quotes. Is there any way to get the same command to work with some adjustments?
NOTE: I got the output using the following methods:
sudo cat /home/user2/test.txt |
while IFS= read line; do
echo $line | awk '{print $NF}'
done
sudo awk {'print $NF'} /home/user2/test.txt
I am trying to understand if there is any solution for using sudo, while and awk with single quotes all of them in a single line.
There are a few reasonable options:
sudo sh -c 'while IFS= read -r line; do echo "$line" | awk '"'"'{print $NF}'"'"';done < /home/user2/test.txt'
or (observing that this particular awk does not require single quotes):
sudo sh -c 'while IFS= read -r line; do echo "$line" | awk "{print \$NF}"; done < /home/user2/test.txt'
or (always a good choice to simplify things) write a script:
sudo sh -c '/path/to/script'
or:
sudo sh << \EOF
while IFS= read -r line; do echo "$line" |
awk '{print $NF}'; done < /home/user2/test.txt
EOF
Note that for all of these it would be good to get rid of the loop completely and do:
sudo sh -c 'awk "{printf \$NF}" /home/user2/test.xt'
but presumably this is a simplified version of the actual problem
You should definitely only use sudo for the code which absolutely needs the privileges. This would perhaps be one of the very few cases where a single cat makes sense.
sudo cat /home/user2/test.txt | awk '{ print $NF }'
As others have noted already, the while read loop is completely useless and quite inefficient here, but if you really insist on doing something like that, it's not hard to apply the same principle. (Notice also the fixed quoting.)
sudo cat /home/user2/test.txt |
while IFS= read -r line; do
echo "$line" # double quotes are important
done | awk '{ print $NF }'
Tangentially, you cannot nest single quotes inside single quotes. The usual solution to that is to switch one set of quotes to double quotes, and probably also then correspondingly escape what needs escaping inside the double quotes.
sudo sh -c 'while IFS= read -r line; do
echo "$line" | awk "{print \$NF}"
# double quotes and^ escape^ ^
done < /home/user2/test.txt'
Just to spell this out, the double quotes around the Awk script are weaker than single quotes, so the dollar sign in $1 needs to be backslashed to protect it from the (inner) shell.
... But as suggested above, the awk script doesn't need to run inside sudo at all here; and anyway, it's also more efficient to put it outside the done.
Suppose I have a string such like s=DNA128533_mutect2_filtered.vcf.gz. How could I extract the DNA128533 as an ID variable.
I tried
id=(cut -d_ -f1 <<< ${s}
echo $id
It seems not working. some suggestions? Thanks
No need to spend a sub-shell calling cut -d'_' -f1 and using bashism <<< "$s".
The POSIX shell grammar has built-in provision for stripping-out the trailing elements with variable expansion, without forking a costly sub-shell or using non-standard Bash specific <<<"here string".
#!/usr/bin/env sh
s=DNA128533_mutect2_filtered.vcf.gz
id=${s%%_*}
echo "$id"
You want to filter the DNA... part out of the filename. Therefore:
s="DNA128533_mutect2_filtered.vcf.gz"
id=$(echo "$s" | cut -d'_' -f1)
echo "$id"
If you want to use your way of doing it (with <<<), do this:
id=$(cut -d'_' -f1 <<< "$s")
echo "$id"
Your command has some syntax issues, like you are missing ).
And you want the output of the command to be stored in variable id, so you have to make it run via the $( ) syntax.
IFS is the bash way delimiter, we can cut string as below:
IFS='_' read -r -a array <<< "a_b_c_d"
echo "${array[0]}"
i would like to create a bash script that creates the md5-hash of a string.
BUT the string can contain special characters and spaces. How can i get it that such a string is readable for eg. md5sum?
I have written the following script as "md5.sh":
#!/bin/bash
echo -n $1 | md5sum | awk '{print $1}'
But if i use "./md5.sh " (with a space at the end) this will not be recognized.
Also quoted cannot be read from the script. And a single " will end in an prompt...
Hope someone can help me :)
First, you need to quote the parameter in the script (and use printf; the world would be a better place if people forgot echo existed):
printf '%s' "$1" | md5sum | awk '{print $1}'
Second, to pass an actual space as the argument, it must be quoted as well so that the shell doesn't discard it:
./md5.sh " "
I wanted to print the name from the entire address by shell scripting. So user1#12.12.23.234 should give output "user1" and similarly 11234#12.123.12.23 should give output 11234
Reading from the terminal:
$ IFS=# read user host && echo "$user"
<user1#12.12.23.234>
user1
Reading from a variable:
$ address='user1#12.12.23.234'
$ cut -d# -f1 <<< "$address"
user1
$ sed 's/#.*//' <<< "$address"
user1
$ awk -F# '{print $1}' <<< "$address"
user1
Using bash in place editing:
EMAIL='user#server.com'
echo "${EMAIL%#*}
This is a Bash built-in, so it might not be very portable (it won't run with sh if it's not linked to /bin/bash for example), but it is probably faster since it doesn't fork a process to handle the editing.
Using sed:
echo "$EMAIL" | sed -e 's/#.*//'
This tells sed to replace the # character and as many characters that it can find after it up to the end of line with nothing, ie. removing everything after the #.
This option is probably better if you have multiple emails stored in a file, then you can do something like
sed -e 's/#.*//' emails.txt > users.txt
Hope this helps =)
I tend to use expr for this kind of thing:
address='user1#12.12.23.234'
expr "$address" : '\([^#]*\)'
This is a use of expr for its pattern matching and extraction abilities. Translated, the above says: Please print out the longest prefix of $address that doesn't contain an #.
The expr tool is covered by Posix, so this should be pretty portable.
As a note, some historical versions of expr will interpret an argument with a leading - as an option. If you care about guarding against that, you can add an extra letter to the beginning of the string, and just avoid matching it, like so:
expr "x$address" : 'x\([^#]*\)'
I have a configuration file that contains lines like "hallo;welt;" and i want to do a grep on this file.
Whenever i try something like grep "$1;$2" my.config or echo "$1;$2 of even line="$1;$2" my script fails with something like:
: command not found95: line 155: =hallo...
How can i tell bash to ignore ; while evaluating "..." blocks?
EDIT: an example of my code.
# find entry
$line=$(grep "$1;$2;" $PERMISSIONSFILE)
# splitt line
reads=$(echo $line | cut -d';' -f3)
writes=$(echo $line | cut -d';' -f4)
admins=$(echo $line | cut -d';' -f5)
# do some stuff on the permissions
# replace old line with new line
nline="$1;$2;$reads;$writes;$admins"
sed -i "s/$line/$nline/g" $TEMPPERM
my script should be called like this: sh script "table" "a.b.*.>"
EDIT: another, simpler example
$test=$(grep "$1;$2;" temp.authorization.config)
the temp file:
table;pattern;read;write;stuff
the call sh test.sh table pattern results in: : command not foundtable;pattern;read;write;stuff
Don't use $ on the left side of an assignment in bash -- if you do it'll substitute the current value of the variable rather than assigning to it. That is, use:
test=$(grep "$1;$2;" temp.authorization.config)
instead of:
$test=$(grep "$1;$2;" temp.authorization.config)
Edit: also, variable expansions should be in double-quotes unless there's a good reason otherwise. For example, use:
reads=$(echo "$line" | cut -d';' -f3)
instead of:
reads=$(echo $line | cut -d';' -f3)
This doesn't matter for semicolons, but does matter for spaces, wildcards, and a few other things.
A ; inside quotes has no meaning at all for bash. However, if $1 contains a doublequote itself, then you'll end up with
grep "something";$2"
which'll be parsed by bash as two separate commands:
grep "something" ; other"
^---command 1----^ ^----command 2---^
Show please show exactly what your script is doing around the spot the error is occurring, and what data you're feeding into it.
Counter-example:
$ cat file.txt
hello;welt;
hello;world;
hell;welt;
$ cat xx.sh
grep "$1;$2" file.txt
$ bash -x xx.sh hello welt
+ grep 'hello;welt' file.txt
hello;welt;
$
You have not yet classified your problem accurately.
If you try to assign the result of grep to a variable (like I do) your example breaks.
Please show what you mean. Using the same data file as before and doing an assignment, this is the output I get:
$ cat xx.sh
grep "$1;$2" file.txt
output=$(grep "$1;$2" file.txt)
echo "$output"
$ bash -x xx.sh hello welt
+ grep 'hello;welt' file.txt
hello;welt;
++ grep 'hello;welt' file.txt
+ output='hello;welt;'
+ echo 'hello;welt;'
hello;welt;
$
Seems to work for me. It also demonstrates why the question needs an explicit, complete, executable, minimal example so that we can see what the questioner is doing that is different from what people answering the question think is happening.
I see you've provided some sample code:
# find entry
$line=$(grep "$1;$2;" $PERMISSIONSFILE)
# splitt line
reads=$(echo $line | cut -d';' -f3)
writes=$(echo $line | cut -d';' -f4)
admins=$(echo $line | cut -d';' -f5)
The line $line=$(grep ...) is wrong. You should omit the $ before line. Although it is syntactically correct, it means 'assign to the variable whose name is stored in $line the result of the grep command'. That is unlikely to be what you had in mind. It is, occasionally, useful. However, those occasions are few and far between, and only for people who know what they're doing and who can document accurately what they're doing.
For safety if nothing else, I would also enclose the $line values in double quotes in the echo lines. It may not strictly be necessary, but it is simple protective programming.
The changes lead to:
# find entry
line=$(grep "$1;$2;" $PERMISSIONSFILE)
# split line
reads=$( echo "$line" | cut -d';' -f3)
writes=$(echo "$line" | cut -d';' -f4)
admins=$(echo "$line" | cut -d';' -f5)
The rest of your script was fine.
It seems like you are trying to read a semicolon-delimited file, identify a line starting with 'table;pattern;' where table is a string you specify and pettern is a regular expression grep will understand. Once the line is identified you wish to replaced the 3rd, 4th and 5th fields with different data and write the updated line back to the file.
Does this sound correct?
If so, try this code
#!/bin/bash
in_table="$1"
in_pattern="$2"
file="$3"
while IFS=';' read -r -d$'\n' tuple pattern reads writes admins ; do
line=$(cut -d: -f1<<<"$tuple")
table=$(cut -d: -f2<<<"$tuple")
# do some stuff with the variables
# e.g., update the values
reads=1
writes=2
admins=12345
# replace the old line with the new line
sed -i'' -n $line'{i\
'"$table;$pattern;$reads;$writes;$admins"'
;d;}' "$file"
done < <(grep -n '^'"${in_table}"';'"${in_pattern}"';' "${file}")
I chose to update by line number here to avoid problems of unknown characters in the left hand of the substitution.