I need a script to send stdout to journald and to the console parallel. For the journald lines I need to sanitize the messages before persisting.
I have a dummy example to show my issue:
ping google.com | tee >( sed 's/seq/SEQ/' | systemd-cat -t 'my-ping')
When I have sed & systemd-cat the messages to journald are delayed and they arrive only after stopping the ping process.
Example:
$ ping google.com | tee >( sed 's/seq/SEQ/' | systemd-cat -t 'my-ping')
PING google.com (216.58.197.238) 56(84) bytes of data.
64 bytes from nrt13s49-in-f14.1e100.net (216.58.197.238): icmp_seq=1 ttl=40 time=240 ms
64 bytes from nrt13s49-in-f14.1e100.net (216.58.197.238): icmp_seq=2 ttl=40 time=240 ms
64 bytes from nrt13s49-in-f14.1e100.net (216.58.197.238): icmp_seq=3 ttl=40 time=240 ms
64 bytes from nrt13s49-in-f14.1e100.net (216.58.197.238): icmp_seq=4 ttl=40 time=240 ms
64 bytes from nrt13s49-in-f14.1e100.net (216.58.197.238): icmp_seq=5 ttl=40 time=240 ms
^C
The messages are going in all at once (see timestamp):
journalctl -f | grep my-ping
Aug 17 06:03:40 hostname my-ping[30555]: PING google.com (216.58.197.238) 56(84) bytes of data.
Aug 17 06:03:40 hostname my-ping[30555]: 64 bytes from nrt13s49-in-f14.1e100.net (216.58.197.238): icmp_SEQ=1 ttl=40 time=240 ms
Aug 17 06:03:40 hostname my-ping[30555]: 64 bytes from nrt13s49-in-f14.1e100.net (216.58.197.238): icmp_SEQ=2 ttl=40 time=240 ms
Aug 17 06:03:40 hostname my-ping[30555]: 64 bytes from nrt13s49-in-f14.1e100.net (216.58.197.238): icmp_SEQ=3 ttl=40 time=240 ms
Aug 17 06:03:40 hostname my-ping[30555]: 64 bytes from nrt13s49-in-f14.1e100.net (216.58.197.238): icmp_SEQ=4 ttl=40 time=240 ms
Aug 17 06:03:40 hostname my-ping[30555]: 64 bytes from nrt13s49-in-f14.1e100.net (216.58.197.238): icmp_SEQ=5 ttl=40 time=240 ms
It seems this behavior only presents when I use both if any one of them is left out everything works as expected.
Do you have any pointers what can be the issue and how to get over it?
Related
I want to specify 25 off 1GB hugepages on our Centos 7 system, which has 48GB RAM.
I have specified the following boot parameters:
hugepagesz=1G hugepages=25 default_hugepagesz=1G
but after boot the system reports:
$ cat /proc/meminfo | grep Huge
AnonHugePages: 0 kB
HugePages_Total: 43
HugePages_Free: 43
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 1048576 kB
I find that:
$ sudo cat /etc/sysctl.conf | grep nr
vm.nr_hugepages = 200
I tried setting vm.nr_hugepages = 25 directly and via a custom tuned profile (that includes 'balanced') but still vm.nr_hugepages gets set to 200.
So something, somewhere is overriding the value I set. What could this be?
when I use bash to write a temp test file on liunx terminal.
echo text_content>file1
if set length of text_content 4096 char(random char from [a-Z]) long.
the result file1 ends up use two 4K blocks. and one inode.
test#instance-7:~/notes/rust$ du -csh file1
8.0K file1
8.0K totaldu
But why it used two 4K blocks? I mean, Isn't one 4K block is enough for it?
if I set the length of the text_content 4095 char long, it used only one 4K block.
why it's using more blocks for it needed? or I'm missing something?
here are some disk info for my liunx machine.
test#instance-7:~/notes/rust$ sudo fdisk -l /dev/sda
Disk /dev/sda: 30 GiB, 32212254720 bytes, 62914560 sectors
Disk model: PersistentDisk
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gptDisk identifier: 35BD657D-931E-497E-A86C-D3D7C4F6BD2A
Try this experiment:
write cat > file1 and hit Enter,
then write this and then hit Ctrl-D twice without hitting Enter first;
write cat > file2 and hit Enter,
then write this, and then hit Enter and then Ctrl-D.
finally run diff file1 file2 and ls -l file[12]
I have a string from which I would like to extract all the ip addresses. I'm very new to bash. Please help
#!/usr/bin/env bash
IP=$(ping -c 25 x.x.x.255)
"$IP" will have the following text. Please suggest a way to get only the ip addresses like x.x.x.252, x.x.x.141 ......
PING x.x.x.255 (x.x.x.255): 56 data bytes
64 bytes from x.x.x.252: icmp_seq=0 ttl=64 time=0.111 ms
64 bytes from x.x.x.141: icmp_seq=0 ttl=255 time=2.200 ms
64 bytes from x.x.x.197: icmp_seq=0 ttl=64 time=70.087 ms
64 bytes from x.x.x.108: icmp_seq=0 ttl=64 time=70.136 ms
--- x.x.x.255 ping statistics ---
25 packets transmitted, 25 packets received, +12 duplicates, 0.0% packet loss
You could use grep's -E and -o options:
$ ips=$(ping -c 4 example.com | grep -Eo '\s([0-9]+\.){3}[0-9]+\s')
$ echo $ips
93.184.216.34 93.184.216.34 93.184.216.34 93.184.216.34
The regex matches a space \s, followed by a capturing group consisting of one or more digits [0-9]+ and a dot \. repeated three times {3}, followed by [0-9]+\s. Information about grep's options can be found in man grep.
This question already has answers here:
Use tee (or equivalent) but limit max file size or rotate to new file
(7 answers)
Closed 7 years ago.
Given a bash script running ping -c 25 google.com | tee /home/user/myLogFile.log
And the output file /home/user/myLogFile.log containing:
PING google.com (117.102.117.238) 56(84) bytes of data.
64 bytes from 117.102.117.238: icmp_seq=1 ttl=61 time=12.7 ms
64 bytes from 117.102.117.238: icmp_seq=2 ttl=61 time=61.1 ms
(...)
64 bytes from 117.102.117.238: icmp_seq=25 ttl=61 time=7.11 ms
--- google.com ping statistics ---
25 packets transmitted, 25 received, 0% packet loss, time 24038ms
rtt min/avg/max/mdev = 5.573/11.293/61.102/11.210 ms
How to limit the maximum lines in the log file, and if that maximum is reached, the file is reset and keep the next output saved?
Ok, I think you can do something like:
ping -c 25 google.com | tee >(split -d -b 100000 - /home/user/myLogFile.log)
I am capturing traffic using:
tcpdump -i <interface> -nn -s0 -w ike2.pcap
Then I am reading the captured file using:
tcpdump -vvv -l -r ike2.pcap
While reading, the first packet is getting displayed as:
07:22:33.320142 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], length: 296) 10.0.0.1.isakmp > 10.0.0.2.isakmp: [udp sum ok] isakmp 2.0 msgid cookie ->: phase 1 I #34[]:
(#33)
(#34)
(#40)
(#41)
(#41)
I need to get the details of this packet. In this ISAKMP IKEv2 packet, I am interested to extract the values of 'Encryption Algorithm' and 'Integrity Algorithm' (i.e 'ENCR_3DES' and 'AUTH_HMAC_MD5_96')
I can view the values if I inspect the packet in wireshark. But, I have to do this from a shell script, so I cannot use wireshark. I need to get these values from tcpdump read command itself.
I am assuming there might be some way to print the encryption and integrity algorithms used from tcpdump read command. Could you please help to figure this out.
Captured packet:
Could find the way to do it as you are expecting with only tcpdump, but has #user862787 said use tshark like:
# tshark -V -r somecapfile.pcap
Frame 1: 196 bytes on wire (1568 bits), 196 bytes captured (1568 bits)
Encapsulation type: Linux cooked-mode capture (25)
Arrival Time: May 10, 2017 02:00:34.811347000 CDT
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1494399634.811347000 seconds
[Time delta from previous captured frame: 0.000000000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 0.000000000 seconds]
Frame Number: 1
Frame Length: 196 bytes (1568 bits)
Capture Length: 196 bytes (1568 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: sll:ethertype:ip:sctp:m3ua:sccp:tcap:gsm_map]
The -V does what you want!
I can view the values if I inspect the packet in wireshark. But, I have to do this from a shell script, so I cannot use wireshark.
But you could perhaps use TShark.
tcpdump -r file with some other flags (-X, for example).