for example our bash script's name is masodik and there is a text.txt with these lines:
qwer
qw2qw
12345
qwert432
Then I write ./masodik text.txt and i got
qw2qw
12345
I tried it many ways and I dont know why this is not working
#!/bin/bash
for i in read u ; do
echo $i $u | grep '^[a-zA-Z0-9][a-zA-Z0-9][0-9]'
done
$ grep -E '^.{2}[0-9]' text.txt
qw2qw
12345
, and in script it could be something like:
#!/bin/sh
grep -E '^.{2}[0-9]' "$1"
To print lines whose third character is a digit:
grep ^..[0-9] text.txt
^ matches the start of the line. The dot . matches any character. [0-9] matches any digit.
You can do it with awk quite easily as well:
awk '/^..[0-9]/' file
Result
With your input in file:
$ awk '/^..[0-9]/' file
qw2qw
12345
(sed works as well, sed -n '/^..[0-9]/p' file)
The problem with the code here:
#!/bin/bash
for i in read u ; do
echo $i $u | grep '^[a-zA-Z0-9][a-zA-Z0-9][0-9]'
done
...is that the for syntax is wrong:
read u is treated as a word list. So the $u variable is never set, so $u stays empty.
The for loop will run twice -- the 1st time $i will be set to the string "read", the 2nd time $i will be set to the string "u". Since neither string contains a number, the grep returns nothing.
The code never reads text.txt.
See Sasha Khapyorsky's answer for actual working code.
If for some odd reason all external utils, (grep, awk, etc.), are forbidden, this pure POSIX code would work:
#!/bin/sh
while read u ; do
case "$u" in
[a-zA-Z0-9][a-zA-Z0-9][0-9]*) echo "$u" ;;
esac
done
If perl is installed into the system then shell script will look like
#!/bin/bash
perl -e 'print if /^.{2}\d/' text.txt
Related
I want to make a script that can do the following automatically:
grep 'string1' file.txt | grep 'string2' | grep 'string3' ... | grep 'stringN'
The idea is that the script can be run like this:
myScript.sh file.txt string1 string2 string3 ... stringN
and the script has to return all the lines of file.txt that contain all the strings.
For instance, if file.txt looks like this:
hello world
hello world run
hello planet world
And I can make a grep like this:
grep hello file.txt | grep world
and I get:
hello world
hello world run
hello planet world
I want to make a script that makes this automatically, with an undefined number of strings as parameters.
I found that it is hard to achieve this, since the number of strings can be variable. First, I tried to create an array called args like this in myScript.sh:
#!/bin/bash
args=("$#")
with the purpose of storing the arguments. I know that the ${args[0]} is going to be my file.txt and the rest are the strings that I need to use in the distinct greps, but I don't know how to proceed and if this is the best approach to solve the problem. I would appreciate any suggestion about how to program this.
sed is capable of doing this perfectly with a single process, and avoids these eval shenanigans. The resulting script is actually quite simple.
#!/bin/sh
file=$1
shift
printf '\\?%s?!d\n' "$#" |
sed -f - "$file"
We generate a line of sed script for each expression; if the expression is not (!) found, we delete (d) this input line, and start over with the next one.
This assumes your sed accepts - as the argument to -f to read the script from standard input. This is not completely portable; you would perhaps need to store the generated script in a temporary file instead if this is a problem.
This uses ? as the internal regex separator. If you need a literal ? in one of the patterns, you will need to backslash-escape it. In the general case, creating a script which finds an alternative separator which is in none of the search expressions would perhaps be possible, but at that point, I'd move to a proper scripting language (Python would be my preference) instead.
You can generate the pattern of operation and save it in a variable:
pattern="$(printf 'grep %s file.txt' "$1"; printf ' | grep %s' "${#:2}" ; printf '\n')"
and then
eval "$pattern"
Example:
% cat file.txt
foo bar
bar spam
egg
% grep_gen () { pattern="$(printf 'grep %s file.txt' "$1"; printf ' | grep %s' "${#:2}" ; printf '\n')"; eval "$pattern" ;}
% grep_gen foo bar
foo bar
You can create the command in a loop and then use eval to evaluate it.
This is using cat so you can group all the grep.
#! /bin/bash
file="$1"
shift
args=( "$#" )
cmd="cat '$file'"
for a in "${args[#]}"
do
cmd+=' | '
cmd+="grep '$a'"
done
eval $cmd
An eval-free alternative:
#!/bin/bash
temp1="$(mktemp)"
temp2="$(mktemp)"
grep "$2" "$1" > temp1
for arg in "${#:3}"; do
grep "$arg" temp1 > temp2
mv temp2 temp1
done
cat temp1
rm temp1
mktemp generates a temporary file with a unique name and returns its name; it should be widely available.
The loop then executes grep for each argument and renames the second temp file for the next loop.
This is the optimization of Diego Torres Milano's code and the answer to my original question:
#! /bin/bash
file=$1
shift
cmd="cat '$file'"
for 'a' in "$#"
do
cmd+=" | grep '$a'"
done
eval $cmd
How do i iterate through a file and print the first word only. The line is colon separated. example
root:01:02:toor
the file contains several lines. And this is what i've done so far but it does'nt work.
FILE=$1
k=1
while read line; do
echo $1 | awk -F ':'
((k++))
done < $FILE
I'm not good with bash-scripting at all. So this is probably very trivial for one of you..
edit: variable k is to count the lines.
Use cut:
cut -d: -f1 filename
-d specifies the delimiter
-f specifies the field(s) to keep
If you need to count the lines, just
count=$( wc -l < filename )
-l tells wc to count lines
awk -F: '{print $1}' FILENAME
That will print the first word when separated by colon. Is this what you are looking for?
To use a loop, you can do something like this:
$ cat test.txt
root:hello:1
user:bye:2
test.sh
#!/bin/bash
while IFS=':' read -r line || [[ -n $line ]]; do
echo $line | awk -F: '{print $1}'
done < test.txt
Example of reading line by line in bash: Read a file line by line assigning the value to a variable
Result:
$ ./test.sh
root
user
A solution using perl
%> perl -F: -ane 'print "$F[0]\n";' [file(s)]
change the "\n" to " " if you don't want a new line printed.
You can get the first word without any external commands in bash like so:
printf '%s' "${line%%:*}"
which will access the variable named line and delete everything that matches the glob :* and do so greedily, so as close to the front (that's the %% instead of a single %).
Though with this solution you do need to do the loop yourself. If this is the only thing you want to do with the variable the cut solution is better so you don't have to do the file iteration yourself.
I have a file that looks like this:
some random
text
00ab46f891c2emore random
text
234324fc234ba253069
and yet more text
only one line in the file contains only hex characters (234324fc234ba253069), how do I extract that? I tried sed -ne 's/^\([a-f0-9]*\)$/\1/p' file I used line start and line end (^ and &) as delimiters, but I am obviously missing something...
Grep does the job,
$ grep '^[a-f0-9]\+$' file
234324fc234ba253069
Through awk,
$ awk '/^[a-f0-9]+$/{print}' file
234324fc234ba253069
Based on the search pattern given, awk and grep prints the matched line.
^ # start
[a-f0-9]\+ # hex characters without capital A-F one or more times
$ # End
sed can make it:
sed -n '/^[a-f0-9]*$/p' file
234324fc234ba253069
By the way, your command sed -ne 's/^\([a-f0-9]*\)$/\1/p' file is working to me. Note, also, that it is not necessary to use \1 to print back. It is handy in many cases, but now it is too much because you want to print the whole line. Just sed -n '/pattern/p' does the job, as I indicate above.
As there is just one match in the whole file, you may want to exit once it is found (thanks NeronLeVelu!):
sed -n '/^[a-f0-9]*$/{p;q}' file
Another approach is to let printf decide when the line is hexadecimal:
while read line
do
printf "%f\n" "0x"$line >/dev/null 2>&1 && echo "$line"
done < file
Based on Hexadecimal To Decimal in Shell Script, printf "%f" 0xNUMBER executes successfully if the number is indeed hexadecimal. Otherwise, it returns an error.
Hence, using printf ... >/dev/null 2>&1 && echo "$line" does not let printf print anything (redirects to /dev/null) but then prints the line if it was hexadecimal.
For your given file, it returns:
$ while read line; do printf "%f\n" "0x"$line >/dev/null 2>&1 && echo "$line"; done < a
234324fc234ba253069
Using egrep you can restrict your regex to select lines that only match valid hex characters i.e. [a-fA-F0-9]:
egrep '^[a-fA-F0-9]+$' file
234324fc234ba253069
Hy,
Can someone help me with splitting mac addresses from a log file? :-)
This:
000E0C7F6676
should be:
00:0E:0C:7F:66:76
Atm i split this with OpenOffice but with over 200 MAC Address' this is very boring and slow...
It would be nice if the solution is in bash. :-)
Thanks in advance.
A simple sed script ought to do it.
sed -e 's/[0-9A-F]\{2\}/&:/g' -e 's/:$//' myFile
That'll take a list of mac addresses in myFile, one per line, and insert a ':' after every two hex-digits, and finally remove the last one.
$ mac=000E0C7F6676
$ s=${mac:0:2}
$ for((i=1;i<${#mac};i+=2)); do s=$s:${mac:$i:2}; done
$ echo $s
00:00:E0:C7:F6:67:6
Pure Bash. This snippet
mac='000E0C7F6676'
array=()
for (( CNTR=0; CNTR<${#mac}; CNTR+=2 )); do
array+=( ${mac:CNTR:2} )
done
IFS=':'
string="${array[*]}"
echo -e "$string"
prints
00:0E:0C:7F:66:76
$ perl -lne 'print join ":", $1 =~ /(..)/g while /\b([\da-f]{12})\b/ig' file.log
00:0E:0C:7F:66:76
If you prefer to save it as a program, use
#! /usr/bin/perl -ln
print join ":" => $1 =~ /(..)/g
while /\b([\da-f]{12})\b/ig;
Sample run:
$ ./macs file.log
00:0E:0C:7F:66:76
imo, regular expressions are the wrong tool for a fixed width string.
perl -alne 'print join(":",unpack("A2A2A2A2A2A2",$_))' filename
Alternatively,
gawk -v FIELDWIDTHS='2 2 2 2 2 2' -v OFS=':' '{$1=$1;print }'
That's a little funky with the assignment to change the behavior of print. Might be more clear to just print $1,$2,$3,$4,$5,$6
Requires Bash version >= 3.2
#!/bin/bash
for i in {1..6}
do
pattern+='([[:xdigit:]]{2})'
done
saveIFS=$IFS
IFS=':'
while read -r line
do
[[ $line =~ $pattern ]]
mac="${BASH_REMATCH[*]:1}"
echo "$mac"
done < macfile.txt > newfile.txt
IFS=$saveIFS
If your file contains other information besides MAC addresses that you want to preserve, you'll need to modify the regex and possibly move the IFS manipulation inside the loop.
Unfortunately, there's not an equivalent in Bash to sed 's/../&:/' using something like ${mac//??/??:/}.
a='0123456789AB'
m=${a:0:2}:${a:2:2}:${a:4:2}:${a:6:2}:${a:8:2}:${a:10:2}
result:
01:23:45:67:89:AB
I'm trying to get this script to basically read input from a file on a command line, match the user id in the file using grep and output these lines with line numbers starting from 1)...n in a new file.
so far my script looks like this
#!/bin/bash
linenum=1
grep $USER $1 |
while [ read LINE ]
do
echo $linenum ")" $LINE >> usrout
$linenum+=1
done
when i run it ./username file
i get
line 4: [: read: unary operator expected
could anyone explain the problem to me?
thanks
Just remove the [] around read line - they should be used to perform tests (file exists, string is empty etc.).
How about the following?
$ grep $USER file | cat -n >usrout
Leave off the square brackets.
while read line; do
echo $linenum ")" $LINE
done >> usrout
just use awk
awk -vu="$USER" '$0~u{print ++d") "$0}' file
or
grep $USER file |nl
or with the shell, (no need to use grep)
i=1
while read -r line
do
case "$line" in
*"$USER"*) echo $((i++)) $line >> newfile;;
esac
done <"file"
Why not just use grep with the -n (or --line-number) switch?
$ grep -n ${USERNAME} ${FILE}
The -n switch gives the line number that the match was found on in the file. From grep's man page:
-n, --line-number
Prefix each line of output with the 1-based line number
within its input file.
So, running this against the /etc/passwd file in linux for user test_user, gives:
31:test_user:x:5000:5000:Test User,,,:/home/test_user:/bin/bash
This shows that the test_user account appears on line 31 of the /etc/passwd file.
Also, instead of $foo+=1, you should write foo=$(($foo+1)).