When I enter the command ps -ef |grep sharatds , I get a list of processes.
sharatds 13164 13163 0 20:53 pts/2 00:00:00 [bt.C.256] <defunct>
sharatds 13165 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13199 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13233 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13267 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13301 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13335 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13369 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13403 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13437 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13471 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13505 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13539 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13573 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13607 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13641 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13675 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13709 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13743 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13777 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13811 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13845 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13879 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
sharatds 13913 13163 0 20:53 pts/2 00:00:00 [rsh] <defunct>
I want to kill all the processes which have the last column as defunct .
Can anybody help me with a script ?
This will do:
ps -ef | grep sharatds | awk '{print $2}' | xargs kill
I usually do something like this:
kill $(ps -ef |grep sharatds|awk '{print $2}')
Edit: Wait! Those are defunct processes. They are already dead, and cannot be killed further! The parent process will have to run wait() to read their statuses so that they can be cleaned up and removed from the process table.
Related
I want to run a fortran code called orbits_01 on SLURM. I want to run multiple jobs simultaneously (i.e. parallelize over multiple cores). After running multiple jobs, each orbits_01 program will call another executable called optimizer, and the optimizer will constantly call another Python script called relax.py. When I submitted the jobs to SLURM by sbatch python main1.py, the jobs failed to even call the optimizer. However, the whole scheme works fine when I ran locally. The local process status is shown below:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
shuha 39395 0.0 0.0 161540 3064 ? S Oct22 0:19 sshd: shuha#pts/72
shuha 39396 0.0 0.0 118252 5020 pts/72 Ss Oct22 0:11 \_ -bash
shuha 32351 0.3 0.0 318648 27840 pts/72 S 02:08 0:00 \_ python3 main1.py
shuha 32968 0.0 0.0 149404 1920 pts/72 R+ 02:10 0:00 \_ ps uxf
shuha 32446 0.0 0.0 10636 1392 pts/72 S 02:08 0:00 ../orbits_01.x
shuha 32951 0.0 0.0 113472 1472 pts/72 S 02:10 0:00 \_ sh -c ./optimizer >& log
shuha 32954 0.0 0.0 1716076 1376 pts/72 S 02:10 0:00 \_ ./optimizer
shuha 32955 0.0 0.0 113472 1472 pts/72 S 02:10 0:00 \_ sh -c python relax.py > relax.out
shuha 32956 99.6 0.0 749900 101944 pts/72 R 02:10 0:02 \_ python relax.py
shuha 32410 0.0 0.0 10636 1388 pts/72 S 02:08 0:00 ../orbits_01.x
shuha 32963 0.0 0.0 113472 1472 pts/72 S 02:10 0:00 \_ sh -c ./optimizer >& log
shuha 32964 0.0 0.0 1716076 1376 pts/72 S 02:10 0:00 \_ ./optimizer
shuha 32965 0.0 0.0 113472 1472 pts/72 S 02:10 0:00 \_ sh -c python relax.py > relax.out
shuha 32966 149 0.0 760316 111992 pts/72 R 02:10 0:01 \_ python relax.py
shuha 32372 0.0 0.0 10636 1388 pts/72 S 02:08 0:00 ../orbits_01.x
shuha 32949 0.0 0.0 113472 1472 pts/72 S 02:10 0:00 \_ sh -c ./optimizer >& log
shuha 32950 0.0 0.0 1716076 1376 pts/72 S 02:10 0:00 \_ ./optimizer
shuha 32952 0.0 0.0 113472 1472 pts/72 S 02:10 0:00 \_ sh -c python relax.py > relax.out
shuha 32953 100 0.0 749892 101936 pts/72 R 02:10 0:03 \_ python relax.py
I have a main Python script called main1.py, which does a for loop to run multiple orbits_01 jobs at the same time. Then the main script will wait for all jobs to finish. Here 3 parent orbits_01 jobs are running in parallel, and each parent job has multiple child jobs. The heavy computations are done by the python code relax.py, so each job should be able to run only using one core. I want to know what is the best way to submit and parallelize multiple parent jobs with multiple child jobs over all cores in one node on SLURM?
I have data like this:
Jul29 16:52
Jul30 19:06
Jul31 17:04
Aug1 17:22
Aug2 18:53
Aug3 21:44
Aug4 22:56
Aug6 17:01
Aug8 02:19
Aug8 16:49
Aug9 16:37
Aug10 21:09
Aug12 05:24
Aug12 17:09
Aug14 16:39
Aug16 16:41
Aug4 22:56
Aug6 17:01
Aug8 02:19
Aug8 16:49
Aug9 16:37
Aug10 21:09
Aug12 05:24
Aug12 17:09
Aug14 16:39
Aug16 16:41
Aug4 22:56
Aug6 17:01
Aug8 02:19
Aug8 16:49
Aug9 16:37
Aug10 21:09
Aug16 20:24
Aug16 19:09
Aug16 18:39
Aug16 16:41
I want to take out the duplicates, sort by the first column, then maintain that order and sort by the second column. Like the following:
Jul01 11:00
Aug01 12:00
Aug02 12:40
Aug03 10:00
Aug03 11:00
Aug03 13:00
I have this code:
cat filename | awk '!a[$0]++'
This only sorts the first column and something random happens to the second column. Any ideas?
When I tried cat ming | sort -k1M -k1d -k2V, I get this:
Jul29 16:52
Jul30 19:06
Jul31 17:04
Aug10 21:09
Aug10 21:09
Aug10 21:09
Aug1 17:22
Aug12 05:24
Aug12 05:24
Aug12 17:09
Aug12 17:09
Aug14 16:39
Aug14 16:39
Aug16 16:41
Aug16 16:41
Aug16 16:41
Aug16 18:39
Aug16 19:09
Aug16 20:24
Aug2 18:53
Aug3 21:44
Aug4 22:56
Aug4 22:56
Aug4 22:56
Aug6 17:01
Aug6 17:01
Aug6 17:01
Aug8 02:19
Aug8 02:19
Aug8 02:19
Aug8 16:49
Aug8 16:49
Aug8 16:49
Aug9 16:37
Aug9 16:37
Aug9 16:37
sort -u -k1.1,1.3M -k1.4n -k2V filename
-u
delete duplicate lines
-k1.1,1.3M
sort each line from word 1, character 1 to word 1, character 3 in month mode
-k1.4n
sort each line from word 1, character 4 until end of word 1 by numeric value
-k2V
sort second word in "version number" mode, which works well for the timestamp
you can use the following:
sort -k1M -k1.4n -k2V abcss | uniq
explanation:
k1M : does a month sort on the 1st column
k1.4n : does an numeric sort to get the columns in order
k2V : does a version sort on the second column to get timestamp right
The output will be:
Jul29 16:52
Jul30 19:06
Jul31 17:04
Aug1 17:22
Aug2 18:53
Aug3 21:44
Aug4 22:56
Aug6 17:01
Aug8 02:19
Aug8 16:49
Aug9 16:37
Aug10 21:09
Aug12 05:24
Aug12 17:09
Aug14 16:39
Aug16 16:41
Aug16 18:39
Aug16 19:09
Aug16 20:24
I'm using golang to call pppd and then kill it after a while. However I got a lot of defunct proccesses in this way.
This is how I run pppd
exec.Command("sh", "-c", "pppd call vpn").CombinedOutput()
This is how I kill it.
exec.Command("sh", "-c", "pkill pppd").CombinedOutput()
Then I got a lot of this
root 31541 23536 0 10:54 ? 00:00:00 [pppd] <defunct>
root 31929 23356 0 10:55 ? 00:00:00 [pptpgw] <defunct>
root 31933 23356 0 10:55 ? 00:00:00 [pptpcm] <defunct>
root 31940 23356 0 10:55 ? 00:00:00 [pppd] <defunct>
root 31993 23536 0 10:55 ? 00:00:00 [pptpgw] <defunct>
root 31997 23536 0 10:55 ? 00:00:00 [pptpcm] <defunct>
root 31998 23536 0 10:55 ? 00:00:00 [pppd] <defunct>
root 32012 23356 0 10:55 ? 00:00:00 [pptpgw] <defunct>
root 32016 23356 0 10:55 ? 00:00:00 [pptpcm] <defunct>
root 32017 23356 0 10:56 ? 00:00:00 [pppd] <defunct>
root 32070 23536 0 10:56 ? 00:00:00 [pptpgw] <defunct>
root 32074 23536 0 10:56 ? 00:00:00 [pptpcm] <defunct>
root 32075 23536 0 10:56 ? 00:00:00 [pppd] <defunct>
root 32083 23356 0 10:56 ? 00:00:00 [pptpgw] <defunct>
root 32087 23356 0 10:56 ? 00:00:00 [pptpcm] <defunct>
root 32089 23356 0 10:56 ? 00:00:00 [pppd] <defunct>
root 32131 23536 0 10:57 ? 00:00:00 [pptpgw] <defunct>
root 32135 23536 0 10:57 ? 00:00:00 [pptpcm] <defunct>
root 32148 23536 0 10:57 ? 00:00:00 [pppd] <defunct>
root 32160 23356 0 10:57 ? 00:00:00 [pptpgw] <defunct>
root 32164 23356 0 10:57 ? 00:00:00 [pptpcm] <defunct>
root 32165 23356 0 10:57 ? 00:00:00 [pppd] <defunct>
root 32177 23536 0 10:57 ? 00:00:00 [pptpgw] <defunct>
root 32181 23536 0 10:57 ? 00:00:00 [pptpcm] <defunct>
How can I avoid defunct processes.
These "zombie" processes are created when a process has finished, but the parent has not read their exit status via the wait system call.
I would guess that all you need to do is call (*Cmd).Wait() on every command structure you create. Obviously This will be less straight forward than you may like, since you probably don't want to call Wait on the first command until after the second command is finished.
EDIT: As is pointed out in the comments, (*Cmd).CombinedOutput() calls (*Cmd).Run(), which calls (*Cmd).Wait()... So the above is wrong. The real answer in this case is that for some reason sh isn't cleaning up, and so the solution is to cut out the midle man and do the call like so:
exec.Command("pppd", "call", "vpn").CombinedOutput()
That'll teach me to read the docs a little closer next time...
A simpler way to cancel your command would be to use exec.CommandContext. e.g.
ctx, cancel := context.WithCancel(context.Background())
exec.CommandContext(ctx, "pppd", "call", "vpn").CombinedOutput()
// in some other goroutine...
cancel()
Maybe this would solve your zombie problem?
run subprocess in a new thread
go exec.Command("sh", "-c", "pppd call vpn").CombinedOutput()
kill subprocess
exec.Command("pkill", "pppd").CombinedOutput().CombinedOutput()
I'm running a validation software and I want all of the output sent to a text file and have the results of multiple files placed/appended to the same file. I thought my code was working, but I just discovered I'm only getting the results from 1 file output to the text file.
java -jar /Applications/epubcheck-3.0.1/epubcheck-3.0.1.jar ~/Desktop/Validator/*.epub 2>&1 | tee -a ~/Desktop/Validator/EPUBCHECK3_results.txt
open ~/Desktop/Validator/EPUBCHECK3_results.txt
EDIT
When I run the same .jar file using Windows command line it will process a batch of files and appeand the results appropriately. I would just do this, but it would mean having to switch work stations and transferring files to validate them. I would like to get this running through the Unix shell on my Mac system so that I don't have to do unnecessary work. Command line that IS working below:
FOR /f %%1 in ('dir /b "C:\Users\scrawfo\Desktop\epubcheck\drop epubs here\*.epub"') do (
echo %%1 >> epubcheck.txt
java -jar "C:\Users\scrawfo\Desktop\epubcheck\epubcheck-3.0.jar" "C:\Users\scrawfo\Desktop\epubcheck\drop epubs here\%%1" 2>> epubcheck.txt
echo. >> epubcheck.txt)
notepad epubcheck.txt
del epubcheck.txt
syntax provided by you is correct there might be some problem with java output or something Try Executing it without redirection
cat test
Output:-
This is Test File ...............
Next Executed Command with same syntax
ps l 2>&1 | tee -a test
Output:-
F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME
COMMAND 4 0 3287 1 20 0 4060 572 n_tty_ Ss+ tty2
0:00 /sbin/mingetty /dev/tty2 4 0 3289 1 20 0 4060 572
n_tty_ Ss+ tty3 0:00 /sbin/mingetty /dev/tty3 4 0 3291
1 20 0 4060 576 n_tty_ Ss+ tty4 0:00 /sbin/mingetty
/dev/tty4 4 0 3295 1 20 0 4060 576 n_tty_ Ss+ tty5
0:00 /sbin/mingetty /dev/tty5 4 0 3297 1 20 0 4060 572
n_tty_ Ss+ tty6 0:00 /sbin/mingetty /dev/tty6 4 0 19086
1 20 0 4060 572 n_tty_ Ss+ tty1 0:00 /sbin/mingetty
/dev/tty1 4 0 20837 20833 20 0 108432 2148 wait Ss pts/0
0:00 -bash 4 0 21471 20837 20 0 108124 1036 - R+ pts/0
0:00 ps l 0 0 21472 20837 20 0 100908 664 pipe_w S+ pts/0
0:00 tee -a test
Checked File
cat test
Output:-(Appended properly)
This is Test File ...............
F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND 4 0
3287 1 20 0 4060 572 n_tty_ Ss+ tty2 0:00
/sbin/mingetty /dev/tty2 4 0 3289 1 20 0 4060 572
n_tty_ Ss+ tty3 0:00 /sbin/mingetty /dev/tty3 4 0 3291
1 20 0 4060 576 n_tty_ Ss+ tty4 0:00 /sbin/mingetty
/dev/tty4 4 0 3295 1 20 0 4060 576 n_tty_ Ss+ tty5
0:00 /sbin/mingetty /dev/tty5 4 0 3297 1 20 0 4060 572
n_tty_ Ss+ tty6 0:00 /sbin/mingetty /dev/tty6 4 0 19086
1 20 0 4060 572 n_tty_ Ss+ tty1 0:00 /sbin/mingetty
/dev/tty1 4 0 20837 20833 20 0 108432 2148 wait Ss pts/0
0:00 -bash 4 0 21471 20837 20 0 108124 1036 - R+ pts/0
0:00 ps l 0 0 21472 20837 20 0 100908 664 pipe_w S+ pts/0
0:00 tee -a test
How would you go about using a single command to empty multiple text files in terminal?
My thought was that you could use something along these lines...:
find /var/log/apache2/*log -exec `echo > '{}'` \;
I know it's simple enough to create a shell script that would easily do that:
echo "#!/bin/sh\n\necho \"\" > \"$1\"" > /usr/local/bin/empty && chmod +x /usr/local/bin/empty
...
find /var/log/apache2/*.log -exec empty {} \;
But is it possible to do this without creating your own script in a similar manner?
#!/bin/bash
for file in /var/log/apache2/*.log; do
: > "$file"
done
Or the one-liner version
for file in /var/log/apache2/*.log; do : > "$file"; done
If you need to recurse into subdirs, then you can use the following two options:
Bash 4.X
shopt -s globstar; for file in /var/log/apache2/**.log; do : > "$file"; done
Posix
find /var/log/apache2 -type f -name "*.log" -exec sh -c 'for f; do : > "$f"; done' _ {} +
Or use sed:
sed Q -i *
replacing file contents in place by emptiness.
update with explanation
sed can do all kinds of replacements, e.g. using regular expressions:
$ cat /etc/passwd | sed -e s/^[^:]*/USER/
with the pattern saying "substitute anything until : with USER", giving:
USER:x:0:0:root:/root:/bin/bash
USER:x:1:1:daemon:/usr/sbin:/bin/sh
USER:x:2:2:bin:/bin:/bin/sh
etc etc
Adding -i in the mix, sed can edit files in-place, so you probably NEVER want to do this:
$ sed -e s/^[^:]*/USER/ -i /etc/passwd
(Note that on e.g. Mac OS X, you need to put in an extra argument after -i to provide the "backup suffix", which is used to make backups before sed does its magic on your files)
Now the quest is for the shortest sed script to lose all input, which is either d or Q. d would delete all input (and then output nothing), Q would quit immediately (and output nothing). Q is presumably fastest.
Then, the sarnold example would look like:
$ ls -l
-rw-r--r-- 1 sarnold sarnold 10161 2011-12-30 17:47 alternatives.log
-rw-r----- 1 sarnold sarnold 50976 2011-12-30 17:47 auth.log
-rw-r--r-- 1 sarnold sarnold 759 2011-12-30 17:47 boot.log
$ sed Q -i *.log
$ ls -l
-rw-r--r-- 1 sarnold sarnold 0 2011-12-30 17:47 alternatives.log
-rw-r----- 1 sarnold sarnold 0 2011-12-30 17:47 auth.log
-rw-r--r-- 1 sarnold sarnold 0 2011-12-30 17:47 boot.log
$
You can also use : >(tee *.log) to spawn the tee(1) program to perform the truncation:
$ ls -l
total 1284
-rw-r--r-- 1 sarnold sarnold 10161 2011-12-30 17:47 alternatives.log
-rw-r----- 1 sarnold sarnold 50976 2011-12-30 17:47 auth.log
-rw-r--r-- 1 sarnold sarnold 759 2011-12-30 17:47 boot.log
-rw-r--r-- 1 sarnold sarnold 41377 2011-12-30 17:47 bootstrap.log
-rw-r----- 1 sarnold sarnold 0 2011-12-30 17:47 daemon.log
-rw-r--r-- 1 sarnold sarnold 73075 2011-12-30 17:47 dpkg.log
-rw-r--r-- 1 sarnold sarnold 2914 2011-12-30 17:47 fontconfig.log
-rw-r----- 1 sarnold sarnold 152295 2011-12-30 17:47 kern.log
-rw-r----- 1 sarnold sarnold 0 2011-12-30 17:47 lpr.log
-rw-r----- 1 sarnold sarnold 950 2011-12-30 17:47 mail.log
-rw-r--r-- 1 sarnold sarnold 212115 2011-12-30 17:47 pm-powersave.log
-rw-r--r-- 1 sarnold sarnold 413195 2011-12-30 17:47 pm-suspend.log
-rw-r--r-- 1 sarnold sarnold 0 2011-12-30 17:47 pycentral.log
-rw-r----- 1 sarnold sarnold 0 2011-12-30 17:47 ufw.log
-rw-r----- 1 sarnold sarnold 0 2011-12-30 17:47 user.log
-rw-r--r-- 1 sarnold sarnold 210426 2011-12-30 17:47 Xorg.0.log
-rw-r--r-- 1 sarnold sarnold 93985 2011-12-30 17:47 Xorg.1.log
$ : >(tee *.log)
$ ls -l
total 0
-rw-r--r-- 1 sarnold sarnold 0 2011-12-30 17:47 alternatives.log
-rw-r----- 1 sarnold sarnold 0 2011-12-30 17:47 auth.log
-rw-r--r-- 1 sarnold sarnold 0 2011-12-30 17:47 boot.log
-rw-r--r-- 1 sarnold sarnold 0 2011-12-30 17:47 bootstrap.log
-rw-r----- 1 sarnold sarnold 0 2011-12-30 17:47 daemon.log
-rw-r--r-- 1 sarnold sarnold 0 2011-12-30 17:47 dpkg.log
-rw-r--r-- 1 sarnold sarnold 0 2011-12-30 17:47 fontconfig.log
-rw-r----- 1 sarnold sarnold 0 2011-12-30 17:47 kern.log
-rw-r----- 1 sarnold sarnold 0 2011-12-30 17:47 lpr.log
-rw-r----- 1 sarnold sarnold 0 2011-12-30 17:47 mail.log
-rw-r--r-- 1 sarnold sarnold 0 2011-12-30 17:47 pm-powersave.log
-rw-r--r-- 1 sarnold sarnold 0 2011-12-30 17:47 pm-suspend.log
-rw-r--r-- 1 sarnold sarnold 0 2011-12-30 17:47 pycentral.log
-rw-r----- 1 sarnold sarnold 0 2011-12-30 17:47 ufw.log
-rw-r----- 1 sarnold sarnold 0 2011-12-30 17:47 user.log
-rw-r--r-- 1 sarnold sarnold 0 2011-12-30 17:47 Xorg.0.log
-rw-r--r-- 1 sarnold sarnold 0 2011-12-30 17:47 Xorg.1.log
$
Invoke the shell executable in -exec.