If I need to start grepping a file from line num 1293 all the way to the end of the file how can I do that?
More detailed info in case it helps:
I am trying to whip a quick function in my bashrc that lets me quickly search vim snippet files for a particular snippet echoing the snippet name and associated command(s) to screen. So I have no probs getting the line num for the snippet name and even printing out the command on the following line num. But if the snippet is a multi-line command then I need to grep for the next line beginning with snippet "^snippet " and then return all lines between, but I cannot find any details how I can go about getting grep to start its search starting from a particular line num.
A secondary question is how in a .bashrc function can I exit the function early? When I use the 'exit' command
currently commented out in the funct below the terminal itself exits/closes rather than just exiting the funct.
function vsls() {
if [[ "$2" =~ ^(html|sh|vim)$ ]] ; then
sPath="$2".snippets
elif [[ "$2" =~ ^(html|sh|vim).snippets$ ]] ; then
sPath="$2"
else
echo "\nExiting. You did not enter a recognized vim snippets file name."
# exit 69
fi
lnN=$(more $HOME/.vim/snippets/"$sPath"|grep -nm 1 $1|sed -r 's/^([0-9]*):.*$/\1/') ; echo "\$lnN: ${lnN}"
cntr="$lnN"
sed -n "$cntr"p "$HOME/.vim/snippets/$sPath"
((cntr++))
sed -n "$cntr"p "$HOME/.vim/snippets/$sPath"
}
#chepner
I don't know why (lack of know-how likely) but without specifying 'more' I get a permissions error:
03:43 ~ $ fLNum=$($HOME/.vim/snippets/"$sPath"|grep -nm 1 tdotti|sed -r 's/^([0-9]*):.*$/\1/') ; echo "\$fLNum: ${fLNum}"
bash: /home/user/.vim/snippets/html.snippets: Permission denied
$fLNum:
03:43 ~ $ fLNum=$(more $HOME/.vim/snippets/"$sPath"|grep -nm 1 tdotti|sed -r 's/^([0-9]*):.*$/\1/') ; echo "\$fLNum: ${fLNum}"
$fLNum: 1293
Now working as desired:
I stuck with sed since I feel most comfortable using sed. I have used the -n print opt before, but not too often so it totally escaped my mind to try something like that.
function vsls() {
if [[ "$2" =~ ^(html|sh|vim)$ ]] ; then
sPath="$2".snippets
elif [[ "$2" =~ ^(html|sh|vim).snippets$ ]] ; then
sPath="$2"
else
echo "\nExiting. You did not enter a recognized vim snippets file name."
# exit 69
fi
fLNum=$(more $HOME/.vim/snippets/"$sPath"|grep -nm 1 "snippet $1"|sed -r 's/^([0-9]*):.*$/\1/') ; echo "\$fLNum: ${fLNum}" #get line number of the snippet name searched, entered as input $1
((tLNum1 = fLNum+=1)) ; echo "\$tLNum1: ${tLNum1}" # tmpLineNum is next line num from which to start next grep search for lineNum of next snippet entry to determine where commands of desired snippet end
tLNum2=$(sed -n "${tLNum1},$ p" $HOME/.vim/snippets/"$sPath"|grep -nm 1 "snippet"|sed -r 's/^([0-9]*):.*$/\1/') ; echo "\$tLNum2: ${tLNum2}" #lineNum of next 'snippet entry'
let sLNum=tLNum2+fLNum sLNum-=1 ; let sLNum-=1 ; echo "\$sLNum: ${sLNum}" #tmpLineNum2 is not actual line num in file, but rather the number of lines since the start of the second search, that is necessarily somewhere within the file: so if second search begins on line 1294, for all intents and purpose actual line num 1294 is line 1 of the new (second) search; therefore I need to add the tLNum2 with fLNum to determine actual lineNum in the of the next snippet entry
echo ""
sed -n "${fLNum},${sLNum} p" "$HOME/.vim/snippets/$sPath"
echo ""
}
But it is curious why I needed to do:
let sLNum=tLNum2+fLNum sLNum-=1 ; let sLNum-=1
to get the correct line number of the second grep search. I only got lucky fooling around, b/c I would have thought:
let sLNum=tLNum2+fLNum sLNum-=1
or:
let sLNum=tLNum2+fLNum ; let sLNum-=1
should have done the trick; that is, secondLineNum = tmpLNum2 + firstLineNum and then secondLineNum - 1. But the result would never end up 1 less but always equal to tLNum+fLNum. It would be good to learn why that did not work as expected.
But its working. so thanks.
Or with sed like this:
sed -n "1293,$ p" yourfile | grep xyz
Or, if the line number is in a variable called line:
sed -n "${line},$ p" yourfile | grep xyz
Or, if you want your grep to find nothing in the first 1292 lines, but still report the correct line number if you are using grep -n, you can just get the (empty) hold buffer for grep to look at for lines 1 to 1292
sed "1,1292g" yourfile | grep -n xyz
awk is better suited for this
awk '/search_pattern/ && NR > 1292' filename
tail -n +1293 file | grep ....
Related
I want my script to find a file (in the current directory) with the first line equal to START. Then that file should have FILE <file_name> as the last line. So I want to extract the <file_name> - I use tail for this. It works ok for standard file names but cracks for nonstandard file names like a a or a+b-c\ = e with tail reporting tail option used in invalid context -- 1
Here is the beginning of the script:
#!/bin/bash
next_stop=0;
# find the first file
start_file=$(find . -type f -exec sed '/START/F;Q' {} \;)
mv "$start_file" $start_file # << that trick doesn't work
if [ ! -f "$start_file" ]
then
echo "File with 'START' head not found."
exit 1
else
echo "Found $start_file"
fi
# parse the last line of the start file
last_line=$(tail -1 $start_file) # << here it crashes for hacky names
echo "last line: $last_line"
if [[ $last_line == FILE* ]] ; then
next_file=${last_line#* }
echo "next file from last line: $next_file"
elif [[ $last_line == STOP ]] ; then
next_stop=true;
else
echo "No match for either FILE or STOP => exit"
exit 1
fi
I tried to embrace the find output with braces this way
mv "$start_file" $start_file
but it doesn't help
This error is occur to the character of the escape.
You should write it start_file variable in quotes.
last_line=$(tail -1 $start_file) --> last_line=$(tail -1 "$start_file")
For you two examples, you need to escape space and egual in file name (with \ character), and escape escape character too.
So a a have to be a\ a when passing to tail, and a+b-c\ = e have to be a+b-c\\\ \=\ e.
You can use sed to make this replacement.
This example give you an better and easier way to make this replacement :
printf '%q' "$Strange_filename"
I'd like to understand bash a bit better as I'm apparently horrible at it...
I'm trying to generate a sequence of constant width integers, but then test them to do something exceptional for particular values. Like so:
for n in $(seq -w 1 150)
do
# The next line does not work: doit.sh: line 9: XX: command not found
#decval= $( echo ${n} | sed 's/^0//g' | sed 's/^0//g' )
#if [[ ${decal} -eq 98 ]] ; then
if [[ $( echo ${n} | sed 's/^0//g' | sed 's/^0//g' ) -eq 98 ]] ; then
echo "Do something different for 98"
elif [[ $( echo ${n} | sed 's/^0//g' | sed 's/^0//g' ) -eq 105 ]] ; then
echo "Do something different for 98"
fi
done
This script works for my purposes, but if I try and make the assignment 'decval= $(…' I get an error 'command not found'. I don't understand this, can someone explain?
Also, is there an improvement I can make to this script if I have a large number of exceptions to prevent a long list of if ; then elif … ?
The problem is in the space between = and $:
decval= $(…
You should write without spaces:
decval=$(...
Because, if you write the space, your shell reads decval= as declval="" and treats the result of $(echo...) as the name of a command to execute, and obviously it doesn't find the command.
Also (just a small optimization), you can write:
sed 's/^0\+//'
instead of
sed 's/^0//g' | sed 's/^0//g'
Here:
0\+ means 0 one or more times;
g is removed, because g means replace all occurences in the string, and you have only one occurence (^ can be only one time in a string).
Also, you can check your variable even with leading zeros, without sed:
[[ "$n" =~ "0*98" ]]
I have this scenarios:
File Content:
10.1.1.1
10.1.1.2
10.1.1.3
10.1.1.4
I want sed or awk so that when i cat the file every time new line is returned.
like
First iteration:
cat ip | some magic
10.1.1.1
Second iteration returns
10.1.1.2
Third iteration returns
10.1.1.3
Fourth iteration returns
10.1.1.4
and after n number of iterations, it returns to line 1
Fifth iteration returns:
10.1.1.1
Can we do it using sed or awk.
You will need to store the line number in a file and increment it with modulus at each invocation.
get_line () {
if [[ ! -e /var/local/get_line.next ]]
then
if [[ ! -e /var/local ]]
then
mkdir -p /var/local
fi
line_no=1
else
line_no=$(< /var/local/get_line.next)
fi
file_length=(wc -l < ip_file)
if ((file_length == 0))
then
echo "Error: Data file is empty" >&2
return 1
fi
if ((line > file_length))
then
line=1
fi
sed -n "$line_no{p;q}" ip_file
echo "$((++line_no))" > /var/local/get_line.next
}
This is in the form of a function which you can incorporate in a script. Feel free to change the location of the get_line.next file. Note that permissions will need to be correct to read or write the files or to create the directory, if necessary.
You will not need to use cat.
You can't do this with cat. You also can't seek on a pipe so you can't use a pipe ..
You can do this with a nested while loop
while ((1))
do
while read line
do
echo "$line"
done <somefile
done
I'm trying to parse a csv file I made with Google Spreadsheet. It's very simple for testing purposes, and is basically:
1,2
3,4
5,6
The problem is that the csv doesn't end in a newline character so when I cat the file in BASH, I get
MacBook-Pro:Desktop kkSlider$ cat test.csv
1,2
3,4
5,6MacBook-Pro:Desktop kkSlider$
I just want to read line by line in a BASH script using a while loop that every guide suggests, and my script looks like this:
while IFS=',' read -r last first
do
echo "$last $first"
done < test.csv
The output is:
MacBook-Pro:Desktop kkSlider$ ./test.sh
1 2
3 4
Any ideas on how I could have it read that last line and echo it?
Thanks in advance.
You can force the input to your loop to end with a newline thus:
#!/bin/bash
(cat test.csv ; echo) | while IFS=',' read -r last first
do
echo "$last $first"
done
Unfortunately, this may result in an empty line at the end of your output if the input already has a newline at the end. You can fix that with a little addition:
!/bin/bash
(cat test.csv ; echo) | while IFS=',' read -r last first
do
if [[ $last != "" ]] ; then
echo "$last $first"
fi
done
Another method relies on the fact that the values are being placed into the variables by the read but they're just not being output because of the while statement:
#!/bin/bash
while IFS=',' read -r last first
do
echo "$last $first"
done <test.csv
if [[ $last != "" ]] ; then
echo "$last $first"
fi
That one works without creating another subshell to modify the input to the while statement.
Of course, I'm assuming here that you want to do more inside the loop that just output the values with a space rather than a comma. If that's all you wanted to do, there are other tools better suited than a bash read loop, such as:
tr "," " " <test.csv
cat file |sed -e '${/^$/!s/$/\n/;}'| while IFS=',' read -r last first; do echo "$last $first"; done
If the last (unterminated) line needs to be processed differently from the rest, #paxdiablo's version with the extra if statement is the way to go; but if it's going to be handled like all the others, it's cleaner to process it in the main loop.
You can roll the "if there was an unterminated last line" into the main loop condition like this:
while IFS=',' read -r last first || [ -n "$last" ]
do
echo "$last $first"
done < test.csv
I have the code in my bash scripts that works unstable:
# check every line of check_list file presents in my_prog output
MY_LIST=`./my_prog`
for l in $(cat check_list); do
if ! echo -n "$MY_LIST" | grep -q -x "$l"; then
die "Bad line: '$l'"
fi
done
This piece of code of my huge scripting pool shows "Bad line: 'smthng'" with probability around 1/5000. I wasn't able to represent this event by the naked script but only in my huge scripting pool.
However this code seems to work very fine:
# check every line of check_list file presents in my_prog output
./my_prog > my_list
for l in $(cat check_list); do
if ! grep -q -x "$l" "my_list"; then
die "Bad line: '$l'"
fi
done
The reason why I don't like the second statement is that its use an intermediate file "my_list".
What could be a problem of unstable working of the first statement?
Instead of calling grep for every line in your check_list, you can run one awk program:
awk '
FILENAME == ARGV[1] {check_list[$0]; next}
$0 in check_list {
print "bad line: " $0
exit 1
}
' check_list <(./my_prog)
Or, see if there are any common lines between your program's output and your check_list:
common=$( comm -12 <(sort -u check_list) <(./my_prog | sort -u) )
if [ -n "$common" ]; then
echo "bad lines: "
echo "$common"
die
fi
I don't know what's wrong with the first version but you can easily eliminate the creation of a temporary file.
Note the you'll have to correct the logic, i did not really understand that, probably you'll want to update a variable in the inner loop and decide whether to die after the inner loop.
for i in $*; do
for l in $(cat check_list); do
if ! echo "$i" | grep -q -x "$l"; then
die "Bad line: '$i', '$l'"
fi
done
done | ./my_prog