I have a server with installed hwraid (megaclisas) https://hwraid.le-vert.net/wiki/DebianPackages
the sample output looks like:
-- Controller information --
-- ID | H/W Model | RAM | Temp | BBU | Firmware
c0 | PERC H310 Mini | 0MB | 59C | Absent | FW: 20.13.3-0001
-- Array information --
-- ID | Type | Size | Strpsz | Flags | DskCache | Status | OS Path | CacheCade |InProgress
c0u0 | RAID-10 | 3272G | 64 KB | RA,WT | Default | Optimal | /dev/sda | None |None
-- Disk information --
-- ID | Type | Drive Model | Size | Status | Speed | Temp | Slot ID | LSI ID
c0u0s0p0 | HDD | SEAGATE ST900MM0006 LS0AS0N3Bxxx | 837. Gb | Online, Spun Up | 6.0Gb/s | 31C | [32:0] | 0
c0u0s0p1 | HDD | SEAGATE ST900MM0006 LS0AS0N3Bxxx | 837. Gb | Online, Spun Up | 6.0Gb/s | 31C | [32:1] | 1
c0u0s1p0 | HDD | SEAGATE ST900MM0006 LS0AS0N3Bxxx | 837. Gb | Online, Spun Up | 6.0Gb/s | 28C | [32:2] | 2
c0u0s1p1 | HDD | SEAGATE ST900MM0006 LS0AS0N3Bxxx | 837. Gb | Online, Spun Up | 6.0Gb/s | 30C | [32:3] | 3
c0u0s2p0 | HDD | SEAGATE ST900MM0006 LS0AS0N3Bxxx | 837. Gb | Online, Spun Up | 6.0Gb/s | 29C | [32:4] | 4
c0u0s2p1 | HDD | SEAGATE ST900MM0006 LS0AS0N3Bxxx | 837. Gb | Online, Spun Up | 6.0Gb/s | 31C | [32:5] | 5
c0u0s3p0 | HDD | SEAGATE ST900MM0006 LS0AS0N3Bxxx | 837. Gb | Online, Spun Up | 6.0Gb/s | 30C | [32:7] | 7
c0u0s3p1 | HDD | SEAGATE ST900MM0006 LS0AS0N3Bxxx | 837. Gb | Online, Spun Up | 6.0Gb/s | 28C | [32:6] | 6
What I want to achieve is to grep Status value if it is not equal to Optimal or Online and then pipe to email. The problem I have here is to how to get that using sed or awk.
Here is a solution how to proceed
data=$(mktemp)
externalprogram > $data
RESULT=$(
grep "| RAID" $data | sed -n '/Optimal/!p'
grep "| HDD" $data | sed -n '/Online,/!p'
)
rm $data
echo "$RESULT"
explanation
grep interested lines
print if search pattern was not found
Related
I'm trying to receive a signal whith a center frequency equal to 2e8Hz and a bandwith slightly smaller than 2e5Hz with my USRP X310.
I use the rx_sampletofile.cpp func of uhd 3.10.1 as shown :
./rx_sampletofile --file test.bin --duration --rate 4e8 --nsamps 4e8 1 --freq 2e8 --type float --bw 8e7 --skip-lo
When I look at the terminal, everything is executed without error or warnings but I see that the actual rx frequency isn't changed afterall ...
Setting RX Rate: 20.000000 Msps...
Actual RX Rate: 20.000000 Msps...
Setting RX Freq: 200.000000 MHz...
Setting RX LO Offset: 0.000000 MHz...
Actual RX Freq: 0.000000 MHz...
Setting RX Bandwidth: 0.250000 MHz...
Actual RX Bandwidth: 0.250000 MHz...
I tried to change RX Freq to lower freq and also--lo-offset but it always stays at RX Freq = 0Hz so if you have any idea I'll take it.
Thanks
It appears that my issue is finally related to the daughterboards because the drivers don't detect them well.
Daughterboard Issue :
_____________________________________________________
| | /
| | | RX Dboard: A
| | | ID: Unknown (0x0095)
| | | Serial: 31F94F3
| | | _____________________________________________________
| | | /
| | | | RX Frontend: 0
| | | | Name: Unknown (0x0095) - 0
| | | | Antennas:
| | | | Sensors:
| | | | Freq range: 0.000 to 0.000 MHz
| | | | Gain Elements: None
| | | | Bandwidth range: 0.0 to 0.0 step 0.0 Hz
| | | | Connection Type: IQ
| | | | Uses LO offset: No
| | | _____________________________________________________
| | | /
| | | | RX Codec: A
| | | | Name: ads62p48
| | | | Gain range digital: 0.0 to 6.0 step 0.5 dB
| | _____________________________________________________
| | /
| | | RX Dboard: B
| | | _____________________________________________________
| | | /
| | | | RX Frontend: 0
| | | | Name: Unknown (0xffff) - 0
| | | | Antennas:
| | | | Sensors:
| | | | Freq range: 0.000 to 0.000 MHz
| | | | Gain Elements: None
| | | | Bandwidth range: 0.0 to 0.0 step 0.0 Hz
| | | | Connection Type: IQ
| | | | Uses LO offset: No
| | | _____________________________________________________
| | | /
| | | | RX Codec: B
| | | | Name: ads62p48
| | | | Gain range digital: 0.0 to 6.0 step 0.5 dB
| | _____________________________________________________
Your UHD is too old for your hardware revision of the TwinRX daughterboard.
The only solution is to use a more modern version of UHD. This will also require you to load a more modern version of the FPGA image.
I have the following simplified schematic for my system.
GPIO4
+-----------------------------+
| GPIO3 |
| +------------------------+ |
| | GPIO2 | |
| | +-------------------+ | |
| | | GPIO1 | | |
| | | +--------------+ | | |
| | | | | | | |
+-+--+--+--+-+ | | | |
I2C1 +-------------+ | | | | |
| | ++-+-+-++
I2C2 +-------------+ | I2C | |
| SWITCH +-----------+ CPU |
I2C3 +-------------+ | | |
| | +-------+
I2C4 +-------------+ |
+------------+
What is the best way to implement/configure this in Linux and why? Is it possible to somehow just configure this in the device tree exporting 4 i2c devices in the user-space and whenever someone wants to write/read to one of those 4 i2c buses the corresponding GPIO is asserted?
Is it only possible to be implemented in a kernel module/driver?
From the comments:
There is an "i2c-mux-gpio" driver which should help with this. – #Ian Abbott Sep 12 '19 at 11:35
Do we need to think about underlying cluster while designing nifi templates?
Here is my simple flow
+-----------------+ +---------------+ +-----------------+
| | | | | |
| READ FROM | | MERGE | | PUT HDFS |
| KAFKA | | FILES | | |
| +-----------------------> | +---------------------> | |
| | | | | |
| | | | | |
| | | | | |
+-----------------+ +---------------+ +-----------------+
I have 3 nodes cluster.. When system is running I check "cluster" menu and see only master node is utilizing sources, other cluster nodes seems idle... The question is in such a cluster should I design template according to cluster or nifi should do the load balancing.
I saw one of my colleagues created remote processors for each node on cluster and put a load balancer in front of these within template, is it required? (like below)
+------------------+
| | +-------------+
| REMOTE PROCESS | | input port |
+----> | GROUP FOR | | (rpg) |
| | NODE 1 | +-------------+
| | | |
| | | |
| +------------------+ v
+-----------------+ +-----------------+ RPG
| | | | | +--------------+
| READ FROM | | | | | |
| KAFKA | | LOAD BALANCER | | +------------------+ | MERGE FILES |
| +-------------> | +-------------> | | | |
| | | | | | REMOTE PROCESS | | |
| | | | | | GROUP FOR | | |
| | | | | | NODE 2 | | |
+-----------------+ +-----------------+ RPG | | +--------------+
| +------------------+ |
| |
| v
|
| +-------------------+ +---------------+
| | | | |
| | REMOTE PROCESS | | PUT HDFS |
+-----> | GROUP FOR | | |
| NODE 3 | | |
| | | |
| | | |
+-------------------+ +---------------+
And what is the use-case for load-balancer except remote clusters, can I use load-balancer to split traffic into several processors to speedup the operation?
Apache NiFi does not do any automatic load balancing or moving of data, so it is up to you to design the data flow in a way that utilizes your cluster. How to do this will depend on the data flow and how the data is being brought into the cluster.
I wrote this article once to try and summarize the approaches:
https://community.hortonworks.com/articles/16120/how-do-i-distribute-data-across-a-nifi-cluster.html
In you case with Kafka, you should be able to have the flow run as shown in your first picture (without remote process groups). This is because Kafka is a data source that will allow each node to consume different data.
If ConsumeKafka appears to be running on only one node, there could be a couple of reasons for this...
First, make sure ConsumeKafka is not scheduled for primary node only.
Second, figure out how many partitions you have for your Kafka topic. The Kafka client (used by NiFi) will assign 1 consumer to 1 partition, so if you have only 1 partition then you can only ever have 1 NiFi node consuming from it. Here is an article to further describe this behavior:
http://bryanbende.com/development/2016/09/15/apache-nifi-and-apache-kafka
In reading RFC 4733, it doesn't clearly state whether the event duration should not increment in the final 3 e-bits. It seems the important information in the event is the m-bit, timestamp, and e-bit. If the event duration does increment in the final 3 e-bits, would it make sense to consider each of the 3 e-bits as seperate events and triplicate the tones? Or should the first e-bit received be the end of the event and the last 2 ebits be disgarded? I have a wireshark capture that shows the event duration incrementing in the 3 ebits and I am tyring to make sense of this.
Given that the final packet of the event may be transmitted three times, the duration field should monotonically increase. In the discussion in the comments we thus see three packets, each with the E bit set, and durations of 720, 800 and 880. This indicates that the packets are sent 80ms apart, because the duration field in the packet indicates that the event "has so far lasted as long as indicated by this parameter".
However, it's still a single event, so your playout of the event should last for the duration of the first packet you receive.
For example, you're seeing three packets arrive, but if the first packet (with duration 720) didn't arrive, you'd see the second packet (with duration 800), and you should play the tone for 800ms.
That said, I'd expect the sender to send the end packet with the same duration, rather than what you're seeing. That might be a bug in the sender. (Transmission must cause an increment in duration, but this is retransmission.)
The sender is clearly breaking the RFC since
the 'E' bit should be set when the event has ended
the duration is increased according to the duration of the event
If the duration is still increasing then clearly the event has not ended but if the E bit was set the event has ended - i.e. contradiction
On the other hand (from 2.5.2.2)
once the receiver has received the end of the event it should stop playing the tone.
A receiver SHOULD NOT restart a tone once playout has stopped.
The receiver MAY determine on the basis of retained history and the timestamp and event code of the current packet that it corresponds to an event already played out and lapsed. In that case, further reports for the event MUST be ignored
i.e. You can tell that the event has already played out from the timestamp and should not repeat the event in this case
The example in the RFC 4733 is given in Table 5
https://datatracker.ietf.org/doc/rfc4733/
+-------+-----------+------+--------+------+--------+--------+------+
| Time | Event | M | Time- | Seq | Event | Dura- | E |
| (ms) | | bit | stamp | No | Code | tion | bit |
+-------+-----------+------+--------+------+--------+--------+------+
| 0 | "9" | | | | | | |
| | starts | | | | | | |
| 50 | RTP | "1" | 0 | 1 | 9 | 400 | "0" |
| | packet 1 | | | | | | |
| | sent | | | | | | |
| 100 | RTP | "0" | 0 | 2 | 9 | 800 | "0" |
| | packet 2 | | | | | | |
| | sent | | | | | | |
| 150 | RTP | "0" | 0 | 3 | 9 | 1200 | "0" |
| | packet 3 | | | | | | |
| | sent | | | | | | |
| 200 | RTP | "0" | 0 | 4 | 9 | 1600 | "0" |
| | packet 4 | | | | | | |
| | sent | | | | | | |
| 200 | "9" ends | | | | | | |
| 250 | RTP | "0" | 0 | 5 | 9 | 1600 | "1" |
| | packet 4 | | | | | | |
| | first | | | | | | |
| | retrans- | | | | | | |
| | mission | | | | | | |
| 300 | RTP | "0" | 0 | 6 | 9 | 1600 | "1" |
| | packet 4 | | | | | | |
| | second | | | | | | |
| | retrans- | | | | | | |
| | mission | | | | | | |
If you look at the last 2, you see they both have the "E" bit set, and that the duration is set to 1600 (8000 * 200/1000).
I hope that helps!
So I'm stuck. I am working on a credit system with expirations. Similar to credit card miles but not exactly. By the way I am sorry for the book ahead but I needed to add enough detail to help get the whole picture.
What I need is a system where a user accumulates credits for doing activities. But they can also spend these credits on activities. The credits should expire after 30 days if they are not used. I seem to be stuck on how to accurately calculate this in a batch that will run every night. Any ideas in any language would be greatly appreciated as I seem to be stuck on just one minor detail that I can't get around. Here is an example of the data:
7/1: +5 - user signs up
7/2: +5 - user interacts with system
7/2: -3 - user purchases activity
7/3: +5 - user interacts with system
So at this point the user has received 15 credits and has spent 3. Leaving him with a total of 12 credits. (At least I got basic math down :P)
I should add that currently we are playing with the idea of having two fields: last processed, next processed. So these values at this time assuming it was a new sign up are:
Last Processed Date: 7/1
Next Process Date: 8/1
So now 8/1 comes around. The batch starts and looks at all credits that are older than 30 days. Which at this point is 5.
This is where it starts to get fuzzy.
Then the system should look at all the credits that have been spent in the last 30 days to see if they are using any credits. Because they should only expire if they haven't been used. So there are 3. So I then deduct the user 2 credits because that is the difference of credits earned older than 30 days and what has been spent. So I finish the batch and set the dates accordingly for the next day. Now assuming they haven't spent anymore I start the calculation over of credits earned older than 30, which is 5 and credits spent which again is 3. But I obviously don't want to consider the 3 credits that I considered yesterday. What is a good approach to not include those 3 credits again for consideration.
That is where I am stuck.
We are thinking about writing a debit record for the expired credits so we can track them but having a hard time seeing how I can use it in this calculation.
If you read this far thank you. If you even make a somewhat effort in the answer I will at a minimum give you an up vote for effort.
EDIT:
Ok #Greg mentioned something that I forgot to address. The idea of putting a flag on the credits considered. A valid point but not one that can work because of the following scenario:
Let's say that on a particular day a user spends 10 credits. But the expired credits that the batch is considering only accumulated to 5. Well he should still have 5 more credits left over to not have expired because he spent more than a single expiration. So the flag wouldn't work because we would have skipped those 5 extra credits. Hope that makes sense?
For every user of the system keep an array, that stores information about the amount of credits available to the user for the next 30 consecutive days
For example the data for some user might look like this
8 |
7 | |
6 | | | |
5 | | | | | | | | | | |
4 | | | | | | | | | | | | | | | | |
3 | | | | | | | | | | | | | | | | | | | | | | | |
2 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
1 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
-------------------------------------------------------------
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
^ ^ ^
| \_ |
today tomorrow in 15 days
Every time the user earns some credits, You increase amounts for all days by the number of credits earned. For example if the user earns 2 credits the table changes as follows. It's like rising the whole graph up.
10 |
9 | |
8 | | | |
7 | | | | | | | | | | |
6 | | | | | | | | | | | | | | | | |
5 | | | | | | | | | | | | | | | | | | | | | | | |
4 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
3 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
2 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
1 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
-------------------------------------------------------------
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
^ ^ ^
| \_ |
today tomorrow in 15 days
If The user has x credits today and spends y credits, You decrease the amount of credits available to him to x - y, for every day he has an amount greater than x - y. For days he has no more than x - y, the amount stays the same. It's like cutting the top of the graph off. For example if the user spends 3 credits the graph changes to
7 | | | | | | | | | | |
6 | | | | | | | | | | | | | | | | |
5 | | | | | | | | | | | | | | | | | | | | | | | |
4 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
3 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
2 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
1 | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
-------------------------------------------------------------
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
^ ^ ^
| \_ |
today tomorrow in 15 days
Every day You shift the graph to the left to model expiring credits. The user will have the following amounts tomorrow
7 | | | | | | | | | |
6 | | | | | | | | | | | | | | | |
5 | | | | | | | | | | | | | | | | | | | | | | |
4 | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
3 | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
2 | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
1 | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
-------------------------------------------------------------
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
^ ^ ^
| \_ |
today tomorrow in 15 days
I wouldn't consider trying to process the data as you present it. Instead, you should keep track of how many credits the user has, and when they expire. That way you keep track of which credits were used when the purchase is made, instead of trying to work it all out later.
So when the user signs up, they have:
5 credits expiring on 8/1
After interacting with the system the next day:
5 credits expiring on 8/1
5 credits expiring on 8/2
After purchasing something:
2 credits expiring on 8/1
5 credits expiring on 8/2
And so on.
Assuming you run this batch on a daily basis, you can have a table that keeps track of all the credits they earned, and the credits they used (negative credits).
At the beginning of the next month, your job is simply to find out which of the credits earned on the first day were not spent during the month.
The number of credits earned on the first day - the credits they spent all of last month. If the number is positive, they have some credits that need to expired. So simple add a record in the table with a negative credit. This will zero-out the unused credits.
The next day, repeat the process by seeing how many credits they earned on the second day minus the sum of all the credits they earned in the last month, taking into account the record with the negative credits you created the previous day.
How about adding a flag to the expenditures? If the flag is not set, then you can include that expenditure in the batch, if necessary. If you do use the expenditure to offset an expiration, then you set the flag. Next time through, you'll ignore that expenditure because the flag is set.
Use a debit record to record normal expenditures. When the monthly batch job runs, it can calculate the total debits which are less than or equal to the expiring credits. If there are credits to expire, simply insert an appropriate debit record (appropriate == to cancel the excess, in your application). In this way, any 'running total' code which examines only credits and debits will reach the same balance that your batch code intended.
One approach to this problem is to store only the transactions, not the balance. Then you always calculate the balance in real time when needed. Here's the data:
Date : Amount : Expiries
7/1 : +5 : 7/31
7/2 : +5 : 8/1
7/2 : -3 : never
7/3 : +5 : 8/2
The balance at any time is simply the total of all transactions that have not yet expired. No need to run any batch processes.
Regarding Julians reply (that I can't comment to yet), I'm dealing with just the same problem and Julians approach won't work because that would result the account being able to go negative.
If the user didn't use the service for one month, on 8/4 the account balance would be -3 and one activity worth of 5 would bring the balance to 2, not to 5 as it should.