I'm analysing how long time it takes to write 1 MB to a eMMC. It is supposed to happen during powerfail, thus time is critical.
I found that the write can be done in 40 ms, which is fine.
However randomly (one out of several hundreds) it takes more than 200 ms, which is bad.
Why does these long write times happen. Is it Linux, or is it the eMMC which is doing some housekeeping. I suspect the eMMC, but I have not been able to find any articles about this. Any references would be appreciated.
I run this test script in my init script before anything else is launched, thus the script is alone with all the resources
dd bs=4096 count=128 if=/dev/zero of=/tmp/e2
t=0
count=0
while [ $t -lt 20 ]
do
t=$(time dd if=/tmp/e2 of=/rootfs/data/test-nvram bs=4M 2>&1)
t=$(echo $t | awk /real/'{print $9}' | tr -d 's.' )
echo $count $t
count=`expr $count + 1`
done
I have also tried to write directly to /dev/mmcblk0p1 with the same result, thus the file system does not seem to matter. Doing a lttng on a full system seems to indicate, that the CPU is waiting for an interrupt from the eMMC.
My setup is:
ARMv7 LS1021A dual core, 1GHz
Linux kernel 4.14.34
eMMC: Micron MTFC8GAKAJCN-4M IT
Related
I'd like to run a bash script, for use on a Raspberry Pi, that says "if the seconds of the current time is exactly 00 or 30, then do X".
I've googled and found some suggestions to use cron, but I think there'd be a small delay at the start which I want to avoid.
Thanks
If you don't like the delay of cron, which is mostly for background stuff, you could loop in the foreground:
while true; do
d=$(date +%S)
if [ $d -eq 0 -o $d -eq 30 ]; then
# command here
date +%S.%N
# replace the above command with whatever you want
sleep 5
else
sleep 0.001
fi
done
The Linux date command can check the current system clock quite quickly. I've used this loop to print the nanosecond timer with data to demonstrate the low latency. That is, on my system, I get:
30.001057483
00.003022980
30.003011572
and so on.
I'm working on a Raspberry Pi in bash script and I wanted to know if it was possible to determine how many times the RPi rebooted. The point is that my program is doing something and if I reach 3 reboots it starts doing something else.
I already found this https://unix.stackexchange.com/questions/131888/is-there-a-way-to-tell-how-many-times-my-computer-has-rebooted-in-a-24-hour-peri
but the problem is that it gives me a number that can't be modified easily.
Any ideas ?
Thanks for the clarification.
last reboot | grep ^reboot | wc -l
That's the number of reboots your system did. Since your program will not "survive" a reboot, I assume that you want the number of reboots since your program ran the first time. So you want to store the number of reboots the first time around, and read it back on (the first and) subsequent starts:
if [[ ! -e ~/.reboots ]]
then
echo $(last reboot | grep ^reboot | wc -l) > ~/.reboots
fi
INITIAL_REBOOTS=$(cat ~/.reboots)
# Now you can check if the *current* number of reboots
# is larger than the *initial* number by three or more:
REBOOTS=$(last reboot | grep ^reboot | wc -l)
if [[ $(expr $REBOOTS - $INITIAL_REBOOTS) -ge 3 ]]
then
echo "Three or more reboots"
else
echo "Less than three reboots"
fi
The above lacks all kinds of finesse and error checking (e.g. in case someone has tampered with ~/.reboots), but is meant as proof of concept only.
I have 3000 very quick jobs to run that on average take 2/3 seconds.
The list of jobs is in a file, and I want to control how many I have open.
However, the process of starting a job in background (& line) seems to take some time itself, therefore, some jobs are already finishing before "INTOTAL" amount have got started...
Therefore, I am not using my 32 core efficiently.
Is the a better approach than the one below?
#!/bin/sh
#set -x
INTOTAL=28
while true
do
NUMRUNNING=`tasklist | egrep Prod.exe | wc -l`
JOBS=`cat jobs.lst | wc -l`
if [ $JOBS -gt 0 ]
then
MAXSTART=$(($INTOTAL-$NUMRUNNING))
NUMTOSTART=$JOBS
if [ $NUMTOSTART -gt $MAXSTART ]
then
NUMTOSTART=$MAXSTART
fi
echo 'Starting: '$NUMTOSTART
for ((i=1;i<=$NUMTOSTART;i++))
do
JOB=`head -n1 jobs.lst`
sed -i 1d jobs.lst
/Prod $JOB &
done
sleep 2
fi
sleep 3
done
You may want to have a look at parallel, which you should be able to install on Cygwin according to the release notes. Then running the tasks in parallel can be as easy as:
parallel /Prod {} < jobs.lst
See here for an example of this in its man page (and have a look through the plethora of examples for more about the many options it has).
To control how many jobs to run at a time use the -j flag. By default it will run 1 job per core at a time, so 32 for you. To limit to 16 for instance:
parallel -j 16 /Prod {} < jobs.lst
Somebody else has written (TM) some bash script that forks very many sub-processes. It needs optimization. But I'm looking for a way to measure "how bad" the problem is.
Can I / How would I get a count that says how many sub-processes were forked by this script all-in-all / recursively?
This is a simplified version of what the existing, forking code looks like - a poor man's grep:
#!/bin/bash
file=/tmp/1000lines.txt
match=$1
let cnt=0
while read line
do
cnt=`expr $cnt + 1`
lineArray[$cnt]="${line}"
done < $file
totalLines=$cnt
cnt=0
while [ $cnt -lt $totalLines ]
do
cnt=`expr $cnt + 1`
matches=`echo ${lineArray[$cnt]}|grep $match`
if [ "$matches" ] ; then
echo ${lineArray[$cnt]}
fi
done
It takes the script 20 seconds to look for $1 in 1000 lines of input. This code forks way too many sub-processes. In the real code, there are longer pipes (e.g. progA | progB | progC) operating on each line using grep, cut, awk, sed and so on.
This is a busy system with lots of other stuff going on, so a count of how many processes were forked on the entire system during the run-time of the script would be of some use to me, but I'd prefer a count of processes started by this script and descendants. And I guess I could analyze the script and count it myself, but the script is long and rather complicated, so I'd just like to instrument it with this counter for debugging, if possible.
To clarify:
I'm not looking for the number of processes under $$ at any given time (e.g. via ps), but the number of processes run during the entire life of the script.
I'm also not looking for a faster version of this particular example script (I can do that). I'm looking for a way to determine which of the 30+ scripts to optimize first to use bash built-ins.
You can count the forked processes simply trapping the SIGCHLD signal. If You can edit the script file then You can do this:
set -o monitor # or set -m
trap "((++fork))" CHLD
So fork variable will contain the number of forks. At the end You can print this value:
echo $fork FORKS
For a 1000 lines input file it will print:
3000 FORKS
This code forks for two reasons. One for each expr ... and one for `echo ...|grep...`. So in the reading while-loop it forks every time when a line is read; in the processing while-loop it forks 2 times (one because of expr ... and one for `echo ...|grep ...`). So for a 1000 lines file it forks 3000 times.
But this is not exact! It is just the forks done by the calling shell. There are more forks, because `echo ...|grep...` forks to start a bash to run this code. But after it is also forks twice: one for echo and one for grep. So actually it is 3 forks, not one. So it is rather 5000 FORKS, not 3000.
If You need to count the forks of the forks (of the forks...) as well (or You cannot modify the bash script or You want it to do from an other script), a more exact solution can be to used
strace -fo s.log ./x.sh
It will print lines like this:
30934 execve("./x.sh", ["./x.sh"], [/* 61 vars */]) = 0
Then You need to count the unique PIDs using something like this (first number is the PID):
awk '{n[$1]}END{print length(n)}' s.log
In case of this script I got 5001 (the +1 is the PID of the original bash script).
COMMENTS
Actually in this case all forks can be avoided:
Instead of
cnt=`expr $cnt + 1`
Use
((++cnt))
Instead of
matches=`echo ${lineArray[$cnt]}|grep $match`
if [ "$matches" ] ; then
echo ${lineArray[$cnt]}
fi
You can use bash's internal pattern matching:
[[ ${lineArray[cnt]} =~ $match ]] && echo ${lineArray[cnt]}
Mind that bash =~ uses ERE not RE (like grep). So it will behave like egrep (or grep -E), not grep.
I assume that the defined lineArray is not pointless (otherwise in the reading loop the matching could be tested and the lineArray is not needed) and it is used for other purpose as well. In that case I may suggest a little bit shorter version:
readarray -t lineArray <infile
for line in "${lineArray[#]}";{ [[ $line} =~ $match ]] && echo $line; }
First line reads the complete infile to lineArray without any loop. The second line is process the array element-by-element.
MEASURES
Original script for 1000 lines (on cygwin):
$ time ./test.sh
3000 FORKS
real 0m48.725s
user 0m14.107s
sys 0m30.659s
Modified version
FORKS
real 0m0.075s
user 0m0.031s
sys 0m0.031s
Same on linux:
3000 FORKS
real 0m4.745s
user 0m1.015s
sys 0m4.396s
and
FORKS
real 0m0.028s
user 0m0.022s
sys 0m0.005s
So this version uses no fork (or clone) at all. I may suggest to use this version only for small (<100 KiB) files. In other cases grap, egrep, awk over performs the pure bash solution. But this should be checked by a performance test.
For a thousand lines on linux I got the following:
$ time grep Solaris infile # Solaris is not in the infile
real 0m0.001s
user 0m0.000s
sys 0m0.001s
This following script is used for running parallel subprocess in bash,which is slightly changed from Running a limited number of child processes in parallel in bash?
#!/bin/bash
set -o monitor # means: run background processes in a separate processes...
N=1000
todo_array=($(seq 0 $((N-1))))
max_jobs=5
trap add_next_job CHLD
index=0
function add_next_job {
if [[ $index -lt ${#todo_array[#]} ]]
then
do_job $index &
index=$(($index+1))
fi
}
function do_job {
echo $1 start
time=$(echo "scale=0;x=$RANDOM % 10;scale=5;x/20+0.05" |bc);sleep $time;echo $time
echo $1 done
}
while [[ $index -lt $max_jobs ]] && [[ $index -lt ${#todo_array[#]} ]]
do
add_next_job
done
wait
The job is choosing a random number in 0.05:0.05:5.00 and sleep that much second.
For example, with N=10, a sample out put is
1 start
4 start
3 start
2 start
0 start
.25000
2 done
5 start
.30000
3 done
6 start
.35000
0 done
7 start
.40000
1 done
8 start
.40000
4 done
9 start
.05000
7 done
.20000
5 done
.25000
9 done
.45000
6 done
.50000
8 done
which has 30 lines in total.
But for big N such as 1000,the result can be strange.One run gives 2996 lines of ouput,with 998 lines with start ,999 with done ,and 999 with float number.644 and 652 is missing in start,644 is missing in done.
These test are runned on an Arch Linux with bash 4.2.10(2).Similar results can be produced on debian stable with bash 4.1.5(1).
EDIT:I tried parallel in moreutils and GNU parallel for this test.Parallel in moreutils has the same problem.But GNU parallel works perfect.
I think this is just due to all of the subprocesses inheriting the same file descriptor and trying to append to it in parallel. Very rarely two of the processes race and both start appending at the same location and one overwrites the other. This is essentially the reverse of what one of the comments suggests.
You could easily check this by redirecting through a pipe, such as with your_script | tee file because pipes have rules about atomicity of data delivered by single write() calls that are smaller than a particular size.
There's another question on SO that's similar to this (I think it just involved two threads both quickly writing numbers) where this is also explained but I can't find it.
The only thing I can imagine is that you're running out of resources; check "ulimit -a" and look for "max user processes". If that's less then the number of processes you want to spawn, you will end up with errors.
Try to set the limits for your user (if you're not running as root) to a higher limit. On Redhatish systems you can do this by:
Adding that line to /etc/pam.d/login:
session required pam_limits.so
Adding the following content to /etc/security/limits.conf:
myuser soft nproc 1000
myuser hard nproc 1024
where "myuser" is the username who is granted the right, 1000 the default value of "max user processes" and 1024 the maximum number of userprocesses. Soft- and hard-limit shouldn't be too much apart. It only says what the user is allowed to set himself using the "ulimit" command in his shell.
So the myuser will start with a total of a 1000 processes (including the shell, all other spawned processes), but may raise it to 1024 using ulimit:
$ ulimit -u
1000
$ ulimit -u 1024
$ ulimit -u
1024
$ ulimit -u 2000
-bash: ulimit: max user processes: cannot modify limit: Operation not permitted
A reboot is not required, it works instantly.
Good luck!
Alex.