I am trying to remove rows based on 4th column value is equal to 7 and 5th column value less than 30 from my text file using awk.
Here is my text file
1 1 2017 7 00 00 95.197469 112.803277
1 1 2017 7 05 00 94.139040 113.255244
1 1 2017 7 10 00 93.084220 113.715022
1 1 2017 7 15 00 92.033141 114.182867
1 1 2017 7 20 00 90.985940 114.659045
1 1 2017 7 25 00 89.500772 115.143830
1 1 2017 7 30 00 88.574990 115.637504
1 1 2017 7 35 00 87.614221 116.140360
1 1 2017 7 40 00 86.633495 116.652701
1 1 2017 7 45 00 85.642547 117.174839
1 1 2017 7 50 00 84.647055 117.707097
1 1 2017 7 55 00 83.650410 118.249809
1 1 2017 8 00 00 82.654745 118.803319
1 1 2017 8 05 00 81.661486 119.367982
1 1 2017 8 10 00 80.671646 119.944164
1 1 2017 8 15 00 79.685987 120.532243
1 1 2017 8 20 00 78.705118 121.132609
1 1 2017 8 25 00 77.729550 121.745662
1 1 2017 8 30 00 76.759731 122.371816
1 1 2017 8 35 00 75.796072 123.011494
1 1 2017 8 40 00 74.838956 123.665132
1 1 2017 8 45 00 73.888755 124.333179
1 1 2017 8 50 00 72.945832 125.016092
1 1 2017 8 55 00 72.010551 125.714342
1 1 2017 9 00 00 71.083276 126.428408
With awk:
awk '$4!=7 || $5>=30 {print}' file
Output:
1 1 2017 7 30 00 88.574990 115.637504
1 1 2017 7 35 00 87.614221 116.140360
1 1 2017 7 40 00 86.633495 116.652701
1 1 2017 7 45 00 85.642547 117.174839
1 1 2017 7 50 00 84.647055 117.707097
1 1 2017 7 55 00 83.650410 118.249809
1 1 2017 8 00 00 82.654745 118.803319
1 1 2017 8 05 00 81.661486 119.367982
1 1 2017 8 10 00 80.671646 119.944164
1 1 2017 8 15 00 79.685987 120.532243
1 1 2017 8 20 00 78.705118 121.132609
1 1 2017 8 25 00 77.729550 121.745662
1 1 2017 8 30 00 76.759731 122.371816
1 1 2017 8 35 00 75.796072 123.011494
1 1 2017 8 40 00 74.838956 123.665132
1 1 2017 8 45 00 73.888755 124.333179
1 1 2017 8 50 00 72.945832 125.016092
1 1 2017 8 55 00 72.010551 125.714342
1 1 2017 9 00 00 71.083276 126.428408
Perhaps a more concise awk
Removing line where field 4 equals 7 and field 5 less than 30
$ awk '!($4==7 && $5<30)' case_file_48485025
Output
1 1 2017 7 30 00 88.574990 115.637504
1 1 2017 7 35 00 87.614221 116.140360
1 1 2017 7 40 00 86.633495 116.652701
1 1 2017 7 45 00 85.642547 117.174839
1 1 2017 7 50 00 84.647055 117.707097
1 1 2017 7 55 00 83.650410 118.249809
1 1 2017 8 00 00 82.654745 118.803319
1 1 2017 8 05 00 81.661486 119.367982
1 1 2017 8 10 00 80.671646 119.944164
.
.
How it works
By default awk prints a line if it sees a non-zero value as its command
!($4==7 && $5<30) would evaluate to zero when your condition is met and hence awk doesn't print.
This might work for you (GNU sed):
sed '/^.........7.[0-2]/d' file
Delete a line that has a 7 in the 4th column and 0 to 2 as the first character of the 5th column.
Related
I have a bed file named coverage.bed. When I execute head coverage.bed, this is the beginning of what outputs:
chr start end . . strand length CG CA CT CC TG AG GG
chr1 3000380 3000440 . . + 172 0 2 9 2
chr1 3000492 3000552 . . + 172 0 1 9 1
chr1 3000593 3000653 . . + 1055 0 4 7 4
However, when I view the file using gedit coverage.bed, I see that this is the correct first 3 lines:
chr start end . . strand length CG CA CT CC TG AG GG
chr1 3000380 3000440 . . + 172 0 2 9 1 3 5 2
chr1 3000492 3000552 . . + 172 0 1 9 2 8 1 1
chr1 3000593 3000653 . . + 1055 0 4 7 3 6 5 4
Why is this happening? A python script outputted this file-- could it be possible that there is something wrong with the code that would lead to this error?
Edit: the output of sed -n 2p coverage.bed | hexdump -C is:
00000000 63 68 72 31 09 33 30 30 30 33 38 30 09 33 30 30 |chr1.3000380.300|
00000010 30 34 34 30 09 2e 09 2e 09 2b 09 31 37 32 09 30 |0440.....+.172.0|
00000020 09 32 09 39 09 31 09 33 09 35 09 32 0d 0a |.2.9.1.3.5.2..|
0000002e
This is my table wys_attendence:
id studid adate amonth ayear acls_id attendence
1 28 29 07 2015 10 1
2 31 29 07 2015 10 0
4 32 29 07 2015 10 1
5 28 30 07 2015 10 0
6 31 30 07 2015 10 1
7 32 30 07 2015 10 1
9 28 31 07 2015 10 1
10 31 31 07 2015 10 1
11 32 31 07 2015 10 1
13 28 01 08 2015 10 1
14 31 01 08 2015 10 0
15 32 01 08 2015 10 1
17 28 02 08 2015 10 0
18 31 02 08 2015 10 1
19 32 02 08 2015 10 1
21 28 03 08 2015 10 1
22 31 03 08 2015 10 1
23 32 03 08 2015 10 0
24 28 04 08 2015 10 1
25 31 04 08 2015 10 1
26 32 04 08 2015 10 0
I want to check if values between the t_adates is in the table, and to display values from between the t_adates
if I am select 29/07/2015 to 03/08/0215.
The output I get is incorrect. This is what I get:
studid
28
31
32
not display dates and corresponding values
But I want it like this:
studid 29 30 31 01 02 03
28 1 0 1 1 0 1
31 0 1 1 0 1 1
32 1 1 1 1 1 0
my controller code is here
`$startdate_exploded = explode("/",Input::get('curdate'));
$enddate_exploded = explode("/",Input::get('enddate'));
$curdate = Input::get('curdate');
$enddate = Input::get('enddate');
$teacher = WysTeacher::all();
$attendance = DB::table('wys_teacherattendances')
->whereBetween('t_adate', array($startdate_exploded[0], $enddate_exploded[0]))
->whereBetween('t_amonth', array($startdate_exploded[1], $enddate_exploded[1]))
->whereBetween('t_ayear', array($startdate_exploded[2], $enddate_exploded[2]))
->groupBy('t_adate')
->get();
$teacher_daysattend=WysTeacherattendance::whereBetween('t_adate', array($startdate_exploded[0], $enddate_exploded[0]))
->whereBetween('t_amonth', array($startdate_exploded[1], $enddate_exploded[1]))
->whereBetween('t_ayear', array($startdate_exploded[2], $enddate_exploded[2]))
->get();`
my view.blade.php code is here
<table class="table table-hover" id="datatable">
#foreach($attendance as $attendances)
<th>{{$attendances->t_adate}}</th>
#endforeach
</tr>
#foreach($teacher as $teachers)
<tr>
<td>{{$teachers->tname}}</td>
#foreach($teacher_daysattend as $teacher_daysattends)
#if($teachers->user_id == $teacher_daysattends->t_auserid)
#if($teacher_daysattends->t_attendance == 1 )
<td><font color="green">1</font></td>
#elseif($teacher_daysattends->t_attendance == 0)
<td><font color="red">0</font></td>
#endif
#endif
#endforeach
</tr>
#endforeach
</table>
How can I modify my query to achieve the above result as well as check all day ,month ad year.if iam select dates (29/07/2015 to 03/08/2015 ) are in the database or not if in database then day display only once and display all details from database?
This is my table wys_attendence:
id studid adate amonth ayear acls_id attendence
1 28 02 07 2015 10 1
2 31 02 07 2015 10 0
4 32 02 07 2015 10 1
5 28 30 07 2015 10 0
6 31 30 07 2015 10 1
7 32 30 07 2015 10 1
9 28 31 07 2015 10 1
10 31 31 07 2015 10 1
11 32 31 07 2015 10 1
13 28 06 08 2015 10 1
14 31 06 08 2015 10 0
15 32 06 08 2015 10 1
17 28 07 08 2015 10 0
18 31 07 08 2015 10 1
19 32 07 08 2015 10 1
21 28 08 08 2015 10 1
22 31 08 08 2015 10 1
23 32 08 08 2015 10 0
24 28 12 08 2015 10 1
25 31 12 08 2015 10 1
26 32 12 08 2015 10 0
I want to check if values between the t_adates is in the table, and to display values from between the t_adates
if I am select 1/07/2015 to 31/08/0215.
The output I get is incorrect. This is what I get:
studid 2/07/2015 06/08/2015 07/08/2015 08/08/2015 30/07/205 31/07/2015
28 1 1 0 1 0 1
31 0 0 1 1 1 1
32 1 1 1 0 1 1
not display values are incorrect order
But I want it like this:
studid 2/07/2015 30/07/205 31/07/2015 06/08/2015 07/08/2015 08/08/2015
28 1 0 1 1 0 1
31 0 1 1 0 1 1
32 1 1 1 1 1 0
my controller code is here
`$startdate_exploded = explode("/",Input::get('curdate'));
$enddate_exploded = explode("/",Input::get('enddate'));
$attendence_tbl = WysAttendancename::where('cls_id',$id)->first();
$wys_attendence_table = $attendence_tbl->attendance_name;
$attendance = DB::table($wys_attendence_table)
->whereBetween('adate', array($startdate_exploded[0], $enddate_exploded[0]))
->whereBetween('amonth', array($startdate_exploded[1], $enddate_exploded[1]))
->whereBetween('ayear', array($startdate_exploded[2], $enddate_exploded[2]))
->groupBy('adate')
->get();
$stud_attend = DB::table($wys_attendence_table)
->whereBetween('adate', array($startdate_exploded[0], $enddate_exploded[0]))
->whereBetween('amonth', array($startdate_exploded[1], $enddate_exploded[1]))
->whereBetween('ayear', array($startdate_exploded[2], $enddate_exploded[2]))
->get();`
my view.blade.php is
`<td>Student Name</td>
#foreach($attendance as $attendances)
<td><font size="-1">{{$attendances->adate}}-{{$attendances->amonth}}-{{$attendances->ayear}}</font></td>
#endforeach
</tr>
#foreach($students as $student)
#if($student->studcls == $id)
<tr>
<td>{{$student->studname}}</td>
#foreach($stud_attend as $stud_attends)
#if($student->id == $stud_attends->studid)
#if($stud_attends->attendence == 1)
<td><font color="green" size="3">p</font></td>
#elseif($stud_attends->attendence == 0)
<td><font color="red" size="3">a</font></td>
#endif
#endif
#endforeach
</tr>
#endif
#endforeach`
How can I modify my query to achieve the above result as well as check all dates (01/07/2015 to 31/08/2015 ) are in the database or not if in database then day display only once and display all details in from database?
If you want to group records by date then use groupBy() as explained on http://laravel.com/docs/4.2/queries#selects
$attendance = DB::table('wys_teacherattendances')
->where('t_amonth', $amonth)
->where('t_ayear',$ayear)
->groupBy('t_adate')
->get();
I am writing a code to parse MFT of NTFS. I`m trying analyse Data Run of non residental $INDEX_ALLOCATION attrib:
11 01 2C 11 02 FE 11 00
9F 0B 21 01 DB 00 21 01
D9 00 21 01 E0 00 21 01
F6 00 21 01 10 01 00 F1
After regroup I see problem in Data Run No 3:
DataRun 1: 11 01 2C
DataRun 2: 11 02 FE
DataRun 3: 11 00 9F <- what does mean "00" ?
I tried analyse it using Active Disk Editor 3 and this software decompose it to:
DataRun 3: 11 00 9F 0B
In my opinion header of DataRun 3 ("11") mean 1 length and 1 offset so there should be 2 bytes after header, but there are 3 bytes.
Any idea?
I have an application that decodes data from a magnetic stripe reader. But, I'm having difficulty getting my calculated LRC check byte to match the one on the cards. If I were to grab 3 cards each with 3 tracks, I would guess the algorithm below would work on 4 of the 9 tracks in those cards.
The algorithm I'm using looks like this (C#):
private static char GetLRC(string s, int start, int end)
{
int result = 0;
for (int i = start; i <= end; i++)
{
result ^= Convert.ToByte(s[i]);
}
return Convert.ToChar(result);
}
This is an example of track 3 data that fails the check. On this card, track 2 matched, but track 1 also failed.
0 1 2 3 4 5 6 7 8 9 A B C D E F
00 3 4 4 4 4 4 4 4 4 4 4 5 5 5 5 5
10 5 5 5 5 5 6 6 6 6 6 6 6 6 6 6 7
20 7 7 7 7 7 7 7 7 7 8 8 8 8 8 8 8
30 8 8 8 9 9 9 9 9 9 9 9 9 9 0 0 0
40 0 0 0 0 0 0 0 1 2 3 4 1 1 1 1 1
50 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3
60 3 3 3 3 3 3 3 3
The sector delimiter is ';' and it ends with a '?'.
The LRC byte from this track is 0x30. Unfortunately, the algorithm above computes an LRC of 0x00 per the following calculation (apologies for its length. I want to be thorough):
00 ^ 3b = 3b ';'
3b ^ 33 = 08
08 ^ 34 = 3c
3c ^ 34 = 08
08 ^ 34 = 3c
3c ^ 34 = 08
08 ^ 34 = 3c
3c ^ 34 = 08
08 ^ 34 = 3c
3c ^ 34 = 08
08 ^ 34 = 3c
3c ^ 34 = 08
08 ^ 35 = 3d
3d ^ 35 = 08
08 ^ 35 = 3d
3d ^ 35 = 08
08 ^ 35 = 3d
3d ^ 35 = 08
08 ^ 35 = 3d
3d ^ 35 = 08
08 ^ 35 = 3d
3d ^ 35 = 08
08 ^ 36 = 3e
3e ^ 36 = 08
08 ^ 36 = 3e
3e ^ 36 = 08
08 ^ 36 = 3e
3e ^ 36 = 08
08 ^ 36 = 3e
3e ^ 36 = 08
08 ^ 36 = 3e
3e ^ 36 = 08
08 ^ 37 = 3f
3f ^ 37 = 08
08 ^ 37 = 3f
3f ^ 37 = 08
08 ^ 37 = 3f
3f ^ 37 = 08
08 ^ 37 = 3f
3f ^ 37 = 08
08 ^ 37 = 3f
3f ^ 37 = 08
08 ^ 38 = 30
30 ^ 38 = 08
08 ^ 38 = 30
30 ^ 38 = 08
08 ^ 38 = 30
30 ^ 38 = 08
08 ^ 38 = 30
30 ^ 38 = 08
08 ^ 38 = 30
30 ^ 38 = 08
08 ^ 39 = 31
31 ^ 39 = 08
08 ^ 39 = 31
31 ^ 39 = 08
08 ^ 39 = 31
31 ^ 39 = 08
08 ^ 39 = 31
31 ^ 39 = 08
08 ^ 39 = 31
31 ^ 39 = 08
08 ^ 30 = 38
38 ^ 30 = 08
08 ^ 30 = 38
38 ^ 30 = 08
08 ^ 30 = 38
38 ^ 30 = 08
08 ^ 30 = 38
38 ^ 30 = 08
08 ^ 30 = 38
38 ^ 30 = 08
08 ^ 31 = 39
39 ^ 32 = 0b
0b ^ 33 = 38
38 ^ 34 = 0c
0c ^ 31 = 3d
3d ^ 31 = 0c
0c ^ 31 = 3d
3d ^ 31 = 0c
0c ^ 31 = 3d
3d ^ 31 = 0c
0c ^ 31 = 3d
3d ^ 31 = 0c
0c ^ 31 = 3d
3d ^ 31 = 0c
0c ^ 32 = 3e
3e ^ 32 = 0c
0c ^ 32 = 3e
3e ^ 32 = 0c
0c ^ 32 = 3e
3e ^ 32 = 0c
0c ^ 32 = 3e
3e ^ 32 = 0c
0c ^ 32 = 3e
3e ^ 32 = 0c
0c ^ 33 = 3f
3f ^ 33 = 0c
0c ^ 33 = 3f
3f ^ 33 = 0c
0c ^ 33 = 3f
3f ^ 33 = 0c
0c ^ 33 = 3f
3f ^ 33 = 0c
0c ^ 33 = 3f
3f ^ 3f = 00 '?'
If anybody can point out how to fix my algorithm, I would appreciate it.
Thanks,
PaulH
Edit:
So that you can see if I'm accidentally missing any bytes in my LRC calculation or including the wrong ones (the final '.' is actually a '\r'). The complete data from all three tracks:
0 1 2 3 4 5 6 7 8 9 A B C D E F
00 % U V W X Y Z 0 1 2 3 4 5 6 7 8
10 9 9 A B C D E F G H I J K L M N
20 O P Q R S T U V W X Y Z 1 2 3 0
30 1 2 3 4 5 6 7 8 9 A B C D E F G
40 H I J K L M N O P Q R S T ? 3 ;
50 1 2 3 4 5 6 7 1 2 3 4 5 6 7 8 9
60 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
70 6 7 8 9 0 ? 5 ; 3 4 4 4 4 4 4 4
80 4 4 4 5 5 5 5 5 5 5 5 5 5 6 6 6
90 6 6 6 6 6 6 6 7 7 7 7 7 7 7 7 7
A0 7 8 8 8 8 8 8 8 8 8 8 9 9 9 9 9
B0 9 9 9 9 9 0 0 0 0 0 0 0 0 0 0 1
C0 2 3 4 1 1 1 1 1 1 1 1 1 1 2 2 2
D0 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3
E0 ? 0 .
The GetLRC() algorithm re-instrumented as suggested to only XOR bytes that appear an odd number of times:
private static char GetLRC(string s, int start, int end)
{
int result = 0;
byte cur_byte = Convert.ToByte(s[start]);
int count = 0;
for (int i = start; i <= end; i++)
{
byte b = Convert.ToByte(s[i]);
if (cur_byte != b)
{
if (count % 2 != 0)
{
result ^= cur_byte;
}
cur_byte = b;
count = 0;
}
++count;
}
if (count % 2 != 0)
{
result ^= cur_byte;
}
return Convert.ToChar(result);
}
The calculation steps taken by the new GetLRC() function:
00 ^ 3b = 3b ';'
3b ^ 33 = 08
08 ^ 31 = 39
39 ^ 32 = 0b
0b ^ 33 = 38
38 ^ 34 = 0c
0c ^ 33 = 3f
3f ^ 3f = 00 '?'
Question: Does the LRC byte come from the card itself or is it being added by the reader firmware? (i.e. perhaps this is a firmware bug)
Can I make a suggestion? Store your data as run lengths and only do the xor if the run length is odd - and then only do it once (runLength & 0x01) times. That will get rid of a ton of the worthless bit work and make it clearer on what is occuring. Doing that yields:
Run Lengths:
(01,3b)(01,33)(10,34)(10,35)(10,36)(10,37)(10,38)(10,39)(10,30)
(01,31)(01,32)(01,33)(01,34)(10,31)(10,32)(09,33)(1,3f)
Doing the even/odd thing gives:
3b ^ 33 ^ 31 ^ 32 ^ 33 ^ 34 ^ 33 ^ 3f
08-->39-->0B-->38-->0C-->3F-->00
Much simpler and cleaner to look at. My guess is that looking at your data, that there is an extra 30 somewhere in your data stream or 1 short. Adding that extra 30 gets you your answer:
3b ^ 33 ^ 31 ^ 32 ^ 33 ^ 34 ^ 33 ^ 30 ^ 3F
08-->39-->0B-->38-->0C-->3F-->0F-->30
Beyond that, I'll keep digging...
Can you add some asserts or other validation to your input parameters? I'd hate to see out of bounds start/end causing excitement and/or a null string. Also, is there a possibility of an off by one with start end? Inclusive/exclusive data range? That could account for an extra 0x030 at the end of your data from a 0 stored at the end of your track 3 being converted to a 0x30. Also, is there any possibility of having either corrupt data or a corrupt LRU? Obviously, this is the kind of thing your check is trying to catch. Perhaps it caught something?
Algorithm about LRC is corrected, but the format of data to calculate LRC maybe wrong.
(it depends on your MSR reader)
There are two format of track define by ANSI/ISO (Alpha and BCD).
The coding of binary is different to ASCII.
In this case, start sentinel is ';' ,so the format should be BCD.
(Alpha start sentinel is '%')
LRC is use "Real track data" to calculate (not include parity bit),
Convert rule
ASCII to BCD ->(ASCII - 0x30)
--Data Bits-- Parity
b1 b2 b3 b4 b5 Character Function
0 0 0 0 1 0 (0H) Data
1 0 0 0 0 1 (1H) "
0 1 0 0 0 2 (2H) "
1 1 0 0 1 3 (3H) "
0 0 1 0 0 4 (4H) "
1 0 1 0 1 5 (5H) "
0 1 1 0 1 6 (6H) "
1 1 1 0 0 7 (7H) "
0 0 0 1 0 8 (8H) "
1 0 0 1 1 9 (9H) "
0 1 0 1 1 : (AH) Control
1 1 0 1 0 ; (BH) Start Sentinel
0 0 1 1 1 < (CH) Control
1 0 1 1 0 = (DH) Field Separator
0 1 1 1 0 > (EH) Control
1 1 1 1 1 ? (FH) End Sentinel
In your sample,
Convert ASCII track data to BCD format.
Use BCD data to calculate LRC, the result is 0x00.
Then convert LRC(BCD to ASCII), finally got LRC = 0x30.
P.S. ASCII convert to Alpha
if(bASCII >= 0x20 && bASCII <= 0x5B)
{
return(bASCII - 0x20);
}
else if(bASCII >= 0x5C && bASCII <= 0x5F)
{
return(bASCII - 0x1F);
}
Your algorithm doesn't match the LRC algorithm in Wikipedia's article. Are you sure you're using the correct algorithm?