I understand the impacts and functions of the kernel processkswapd.
As the output of ps -elf | grep swapd, I found kswapd is started by kthreadd. But how is it started step by step? Where's the extract related source code?
Here is the output of ps -elf | grep swapd:
$ ps -elf | head -n 1; sudo ps -elf | grep -i kswapd
F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD
1 S root 46 2 0 80 0 - 0 kswapd 11:42 ? 00:00:00 [kswapd0]
You see, the PID of the kernel process kthreadd is 2:
F S UID PID PPID C PRI NI ADDR SZ WCHAN STIME TTY TIME CMD
1 S root 2 0 0 80 0 - 0 kthrea 6/2 00:00:00 [kthreadd]
In addition, I can't find a binary program with the same name throughout the rootfs. For details, see below:
$ cat /proc/46/cmdline
#outputs nothing
sudo find / -iname kswapd 2>/dev/null
#outputs nothing
I think mm/vmscan.c has all or most of the answers you're looking for.
If you're asking how kswapd is initialized, the file contains kswapd_init().
If you're asking how kswapd is woken up by a process that needs more memory, the file contains wakeup_kswapd().
You can use a combination of grep, printk, and dump_stack() commands to step through the instructions executed before and aaft
I am trying to see the number of active puma threads on my server.
I can not see it through ps:
$ ps aux | grep puma
healthd 2623 0.0 1.8 683168 37700 ? Ssl May02 5:38 puma 2.11.1 (tcp://127.0.0.1:22221) [healthd]
root 8029 0.0 0.1 110460 2184 pts/0 S+ 06:34 0:00 grep --color=auto puma
root 18084 0.0 0.1 56836 2664 ? Ss May05 0:00 su -s /bin/bash -c puma -C /opt/elasticbeanstalk/support/conf/pumaconf.rb webapp
webapp 18113 0.0 0.8 83280 17324 ? Ssl May05 0:04 puma 2.16.0 (unix:///var/run/puma/my_app.sock) [/]
webapp 18116 3.5 6.2 784992 128924 ? Sl May05 182:35 puma: cluster worker 0: 18113 [/]
As in the configuration I have:
threads 8, 32
I was expecting to see at least 8 puma threads?
To quickly answer the question, the number of threads used by a
process running on a given PID, can be obtained using the
following :
% ps -h -o nlwp <pid>
This will just return you the total number of threads used by the
process. The option -h removes the headers and the option -o nlwp
formats the output of ps such that it only outputs the Number of Light Weight Processes (NLWP) or threads. Example, when only a single process puma is running and its PID is obtained with pgrep, you get:
% ps -h -o nlwp $(pgrep puma)
4
What is the difference between process, thread and light-weight process?
This question has been answered already in various places
[See here, here and the excellent geekstuff
article]. The quick, short and ugly version is :
a process is essentially any running instance of a program.
a thread is a flow of execution of the process. A process
containing multiple execution-flows is known as multi-threaded
process and shares its resources amongst its threads (memory,
open files, io, ...). The Linux kernel has no knowledge of what
threads are and only knows processes. In the past,
multi-threading was handled on a user level and not kernel
level. This made it hard for the kernel to do proper process
management.
Enter lightweight processes (LWP). This is essentially the
answer to the issue with threads. Each thread is considered to
be an LWP on kernel level. The main difference between a
process and an LWP is that the LWP shares resources. In other words, an Light Weight Process is kernel-speak for what users call a thread.
Can ps show information about threads or LWP's?
The ps command or process status command provides information
about the currently running processes including their corresponding
LWPs or threads. To do this, it makes use of the /proc directory
which is a virtual filesystem and regarded as the control and
information centre of the kernel. [See here and here].
By default ps will not give you any information about the LWPs,
however, adding the option -L and -m to the command generally does
the trick.
man ps :: THREAD DISPLAY
H Show threads as if they were processes.
-L Show threads, possibly with LWP and NLWP columns.
m Show threads after processes.
-m Show threads after processes.
-T Show threads, possibly with SPID column.
For a single process puma with pid given by pgrep puma
% ps -fL $(pgrep puma)
UID PID PPID LWP C NLWP STIME TTY STAT TIME CMD
kvantour 2160 2876 2160 0 4 15:22 pts/39 Sl+ 0:00 ./puma
kvantour 2160 2876 2161 99 4 15:22 pts/39 Rl+ 0:14 ./puma
kvantour 2160 2876 2162 99 4 15:22 pts/39 Rl+ 0:14 ./puma
kvantour 2160 2876 2163 99 4 15:22 pts/39 Rl+ 0:14 ./puma
however, adding the -m option clearly gives a nicer overview. This
is especially handy when multiple processes are running with the same
name.
% ps -fmL $(pgrep puma)
UID PID PPID LWP C NLWP STIME TTY STAT TIME CMD
kvantour 2160 2876 - 0 4 15:22 pts/39 - 0:44 ./puma
kvantour - - 2160 0 - 15:22 - Sl+ 0:00 -
kvantour - - 2161 99 - 15:22 - Rl+ 0:14 -
kvantour - - 2162 99 - 15:22 - Rl+ 0:14 -
kvantour - - 2163 99 - 15:22 - Rl+ 0:14 -
In this example, you see that process puma with PID 2160 runs with 4
threads (NLWP) having the ID's 2160--2163. Under STAT you see two different values Sl+ and 'Rl+'. Here the l is an indicator for multi-threaded. S and R stand for interruptible sleep (waiting for an event to complete) and respectively running. So we see that 3 of the 4 threads are running at 99% CPU and one thread is sleeping.
You also see the total accumulated CPU time (44s) while a single thread only runs for 14s.
Another way to obtain information is by directly using the format
specifiers with -o or -O.
man ps :: STANDARD FORMAT SPECIFIERS
lwp lightweight process (thread) ID of the dispatchable
entity (alias spid, tid). See tid for additional
information. Show threads as if they were processes.
nlwp number of lwps (threads) in the process. (alias thcount).
So you can use any of lwp,spid or tid and nlwp or thcount.
If you only want to get the number of threads of a process called
puma, you can use :
% ps -o nlwp $(pgrep puma)
NLWP
4
or if you don't like the header
% ps -h -o nlwp $(pgrep puma)
4
You can get a bit more information with :
% ps -O nlwp $(pgrep puma)
PID NLWP S TTY TIME COMMAND
19304 4 T pts/39 00:00:00 ./puma
Finally, you can combine the flags with ps aux to list the threads.
% ps aux -L
USER PID LWP %CPU NLWP %MEM VSZ RSS TTY STAT START TIME COMMAND
...
kvantour 1618 1618 0.0 4 0.0 33260 1436 pts/39 Sl+ 15:17 0:00 ./puma
kvantour 1618 1619 99.8 4 0.0 33260 1436 pts/39 Rl+ 15:17 0:14 ./puma
kvantour 1618 1620 99.8 4 0.0 33260 1436 pts/39 Rl+ 15:17 0:14 ./puma
kvantour 1618 1621 99.8 4 0.0 33260 1436 pts/39 Rl+ 15:17 0:14 ./puma
...
Can top show information about threads or LWP's?
top has the option to show threads by hitting H in the interactive mode or by launching top with top -H. The problem is that it lists the threads as processes (similar to ps -fH).
% top
top - 09:42:10 up 17 days, 3 min, 1 user, load average: 3.35, 3.33, 2.75
Tasks: 353 total, 3 running, 347 sleeping, 3 stopped, 0 zombie
%Cpu(s): 75.5 us, 0.6 sy, 0.5 ni, 22.6 id, 0.0 wa, 0.0 hi, 0.8 si, 0.0 st
KiB Mem : 16310772 total, 8082152 free, 3662436 used, 4566184 buff/cache
KiB Swap: 4194300 total, 4194300 free, 0 used. 11363832 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
868 kvantour 20 0 33268 1436 1308 S 299.7 0.0 46:16.22 puma
1163 root 20 0 920488 282524 258436 S 2.0 1.7 124:48.32 Xorg
...
Here you see that puma runs at about 300% CPU for an accumulated time of 46:16.22. There is, however, no indicator that this is a threaded process. The only indicator is the CPU usage, however, this could be below 100% if 3 threads are "sleeping"? Furthermore, the status flag states S which indicates that the first thread is asleep. Hitting H give you then
% top -H
top - 09:48:30 up 17 days, 10 min, 1 user, load average: 3.18, 3.44, 3.02
Threads: 918 total, 5 running, 910 sleeping, 3 stopped, 0 zombie
%Cpu(s): 75.6 us, 0.2 sy, 0.1 ni, 23.9 id, 0.0 wa, 0.0 hi, 0.2 si, 0.0 st
KiB Mem : 16310772 total, 8062296 free, 3696164 used, 4552312 buff/cache
KiB Swap: 4194300 total, 4194300 free, 0 used. 11345440 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
870 kvantour 20 0 33268 1436 1308 R 99.9 0.0 21:45.35 puma
869 kvantour 20 0 33268 1436 1308 R 99.7 0.0 21:45.43 puma
872 kvantour 20 0 33268 1436 1308 R 99.7 0.0 21:45.31 puma
1163 root 20 0 920552 282288 258200 R 2.0 1.7 124:52.05 Xorg
...
Now we see only 3 threads. As one of the Threads is "sleeping", it is way down the bottom as top sorts by CPU usage.
In order to see all threads, it is best to ask top to display a specific pid (for a single process):
% top -H -p $(pgrep puma)
top - 09:52:48 up 17 days, 14 min, 1 user, load average: 3.31, 3.38, 3.10
Threads: 4 total, 3 running, 1 sleeping, 0 stopped, 0 zombie
%Cpu(s): 75.5 us, 0.1 sy, 0.2 ni, 23.6 id, 0.0 wa, 0.0 hi, 0.7 si, 0.0 st
KiB Mem : 16310772 total, 8041048 free, 3706460 used, 4563264 buff/cache
KiB Swap: 4194300 total, 4194300 free, 0 used. 11325008 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
869 kvantour 20 0 33268 1436 1308 R 99.9 0.0 26:03.37 puma
870 kvantour 20 0 33268 1436 1308 R 99.9 0.0 26:03.30 puma
872 kvantour 20 0 33268 1436 1308 R 99.9 0.0 26:03.22 puma
868 kvantour 20 0 33268 1436 1308 S 0.0 0.0 0:00.00 puma
When you have multiple processes running, you might be interested in hitting f and toggle PGRP on. This shows the Group PID of the process. (PID in ps where PID in top is LWP in ps).
How do I get the thread count without using ps or top?
The file /proc/$PID/status contains a line stating how many threads
the process with PID $PID is using.
% grep Threads /proc/19304/status
Threads: 4
General comments
It is possible that you do not find the process of another user
and therefore cannot get the number of threads that process is using. This could be due to the mount options of /proc/ (hidepid=2).
Used example program:
#include <omp.h>
#include <stdio.h>
#include <stdlib.h>
int main (int argc, char *argv[]) {
char c = 0;
#pragma omp parallel shared(c) {
int i = 0;
if (omp_get_thread_num() == 0) {
printf("Read character from input : ");
c = getchar();
} else {
while (c == 0) i++;
printf("Total sum is on thread %d : %d\n", omp_get_thread_num(), i);
}
}
}
compiled with gcc -o puma --openmp
If you are just looking for number of threads that are spawned by the process, you can see the number of task folders created under /proc/[pid-of-process]/task because each thread creates a folder under this path. So counting the number of folders would be sufficient.
In fact the ps utility itself reads the information from this path, a file /proc/[PID]/cmdline which is represented in a more readable way.
From Linux Filesystem Hierarchy
/proc is very special in that it is also a virtual file-system. It's sometimes referred to as a process information pseudo-file system. It doesn't contain 'real' files but run-time system information (e.g. system memory, devices mounted, hardware configuration, etc). For this reason it can be regarded as a control and information center for the kernel. In fact, quite a lot of system utilities are simply calls to files in this directory.
All you need to get the PID of the process puma, use ps or any utility of your choice
ps aux | awk '/[p]uma/{print $1}'
or more directly use pidof(8) - Linux man page which gets you the PID directly given the process name as input
pidof -s puma
Now that you have the PID to count the number of task/ folders your process had created use the find command
find /proc/<PID>/task -maxdepth 1 -type d -print | wc -l
ps aux | grep puma will give you list of process for only puma. You need to find out , how many threads are running by the particular Process. May be this will help you:
ps -T -p 2623
You need to provide process id, for which you want to find out number of threads. Make sure you are providing accurate process id.
Using ps and wc to count puma threads:
ps --no-headers -T -C puma | wc -l
The string "puma" can be replaced as desired. Example, count bash threads:
ps --no-headers -T -C bash | wc -l
On my system that outputs:
9
The code in the question, ps aux | grep puma, has a few grep related problems:
It returns grep --color=auto puma, which isn't a puma thread at all.
Similarly any util or command with the string "puma", e.g. a util called notpuma, would be matched by grep.
I found "htop" to be an excellent solution. Just toggle "tree view" and you can view each puma-worker and the threads under that worker.
Number of puma threads for each worker:
ps aux | awk '/[p]uma/{print $2}' | xargs ps -h -o nlwp
Sample output:
7
59
59
61
59
60
59
59
59
I'm a little bit confused, i changed my time on one file with the shell command :
touch -t = touch -t 201606012135 trial01
But after the ls -lt, I got this :
-rw-r--r-- 1 CharleyD staff 87 1 jun 2016 trial01
drwxr-xr-x 15 CharleyD staff 510 3 apr 12:57 Hybrid_proj
Why the shell doesn't write the hours like the "Hybrid_proj" directory for the "trial01" ? The trial01 file have the hour : 21:35, so itsn't empty.
Indeed, I search to get this in output :
-rw-r--r-- 1 CharleyD staff 87 1 jun 21:35 trial01
drwxr-xr-x 15 CharleyD staff 510 3 apr 12:57 Hybrid_proj
How I can do this ?
Thx a lot buddies to enlighten my way ! ;)
If a file is not from the current year, ls defaults to showing the year instead of the time. The time is still correctly set, just formatted differently.
To always show the full time, with GNU ls, you can use ls --time-style=long-iso -l:
$ ls --time-style=full-iso -l
total 0
-rw-r--r-- 1 user user 0 2017-04-04 13:20 newfile
-rw-r--r-- 1 user user 0 2016-04-03 12:34 oldfile
With BusyBox ls, you can use -e:
$ busybox ls -e
total 0
-rw-r--r-- 1 user user 0 Tue Apr 4 13:20:42 2017 newfile
-rw-r--r-- 1 user user 0 Sun Apr 3 12:34:00 2016 oldfile
With macOS ls, you can use -lT:
$ ls -lT
total 0
-rw-r--r-- 1 user group 0 Apr 4 13:19:35 2017 myfile
-rw-r--r-- 1 user group 0 Apr 3 12:34:00 2016 oldfile
In each case, you get a long timestamp with the same format for older and newer files.
Use the -T option if your ls supports it.
This is a very strange situation. I'm on OS X 10.11.6
I have an old tty still hanging around (ttys001) but I don't know how to access it and why its still there. It simply does not have any window on the os x desktop. I'm on ttys000.
$ tty
/dev/ttys000
This means I'm currently on ttys000
$ w
22:01 up 15 days, 7:47, 3 users, load averages: 1.65 1.43 1.45
USER TTY FROM LOGIN# IDLE WHAT
Sidharth console - 30Jul16 15days -
Sidharth s000 - 13:48 - w
Sidharth s001 - Thu13 9:12 -bash
I can understand the login from console (it happens automatically) but where is this s001 (i.e. ttys001) coming from -- I can't switch to it -- I don't see any os x terminal windows corresponding to ttys001.
USER PID PPID PGID SESS JOBC STAT TT TIME COMMAND
root 30994 30725 30994 0 0 Ss s000 0:00.04 login -pf Sidharth
Sidharth 30995 30994 30995 0 1 S s000 0:00.33 -bash
root 32409 30995 32409 0 1 R+ s000 0:00.01 ps aj
root 26065 1 26065 0 0 Ss+ s001 0:00.04 login -pfl Sidharth /bin/bash -c exec -la bash /bin/bash
Sidharth 26066 26065 26065 0 0 S+ s001 0:00.28 -bash
Sidharth 29465 26066 26065 0 0 S+ s001 0:00.00 -bash
These are the various processes with associated ttys. Again, I can't understand the life of me what 26065, 26066 and 29465 (all associated with 26065) are doing/why are they there.
Some observations: the parent of 30944 is 30725 which is the Mac Terminal application (this makes sense). But equally interesting is that the parent of 26065 (corresponding to login -pfl Sidharth /bin/bash -c exec -la bash /bin/bash is launchd i.e. pid 1)
I've noticed stuff like this before: there is usually an old ttys but its not visible.
Nope I'don't have any additional tabs open in my os x terminal program that could cause this
My question is this: Why is my ttys001 inaccessible? How do I "get to" ttys001
I'm a newbie to shell programming and I'd like to find the IP address from the process ID. Right now, I'm able to get the PID for a specific process from :
vmname=$1
pid=`ps aux | grep $vmname | awk 'NR==1{printf("%s\n", $2) }'`
echo $pid
The above method returns the PID but how do I get the port from the pid? If I get the port, is there a command to get the IP address as well?
I'm using Ubuntu 11.04 and the above script is actually trying to find out the IP of a virtual machine running on KVM using this method.
Thanks!
You can employ the lsof utility. It gives the list of open files for a process. Use lsof -p pid . You need to grep on the output to get the port values for eg. something like this - lsof -p pid| grep TCP. This will list all the ports opened or connected to by the process. Refer to the manual of the utility. For most systems the utility comes pre-bundled with your OS. However, if it is not pre-bundled then you need to install this utility.
The PID and the computer's IP Address are two completely unrelated things.
PID stands for Process ID, and it's a handle for the OS to keep track of your program, among other things.
IP address is related to a network interface. Most computers have one or two of these (in the case of ethernet card/wireless device.)
Anyway, one way to get your computer's IP address is something similar to the following...There are quite possibly better ways to do it and I just don't know 'em...
$ ifconfig eth0
eth0 Link encap:Ethernet HWaddr 60:eb:69:96:da:87
inet addr:192.168.1.112 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::62eb:69ff:fe96:da87/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:876533 errors:0 dropped:0 overruns:0 frame:0
TX packets:560999 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:229205080 (229.2 MB) TX bytes:136756800 (136.7 MB)
Interrupt:40 Base address:0x8000
$ ifconfig eth0 | grep "inet addr"
inet addr:192.168.1.112 Bcast:192.168.1.255 Mask:255.255.255.0
$ ifconfig eth0 | grep "inet addr" | cut -d ":" -f 2
192.168.1.112 Bcast
$ ifconfig eth0 | grep "inet addr" | cut -d ":" -f 2 | cut -d " " -f 1
192.168.1.112
So the last command will get you what you want inside your script. The rest are just there to show you how I built up to the last command.
Before I start lsof should be used as suggested by #Drona if lsof and if root/su/sudo access is available.
For completness I was investigating this for getting the IP address of currently logged in chrooted SFTP users for a nagios script I did not want to have to create a sudoers rule for.
Easy way (not as easy as lsof and needs root but for completeness)
Step 1
$ ps -ef | grep ssh
UID PID PPID C STIME TTY TIME CMD
root 2479 14186 0 17:05 ? 00:00:00 sshd: sftpuser [priv]
1008 2481 2479 0 17:06 ? 00:00:00 sshd: sftpuser#notty
1008 2482 2481 0 17:06 ? 00:00:00 sshd: sftpuser#internal-sftp
root 2483 14186 0 17:06 ? 00:00:00 sshd: ttyuser [priv]
ttyuser 2485 2483 0 17:06 ? 00:00:00 sshd: ttyuser#pts/0
Above you can see the PID for the ssh users (added the ps columns for easier interpretation)
Step 2
sudo lsof -p 2481 | grep TCP
sshd 2481 root 3u IPv4 29176657 0t0 TCP 192.168.1.2:44156 (ESTABLISHED)
Alternative (more complex has the possibility of not needing rood)
Step 2 - Requires root access but is optional
$ sudo ls -l /proc/2481/fd
total 0
lrwx------ 1 root root 64 Jul 3 17:07 0 -> /dev/null
lrwx------ 1 root root 64 Jul 3 17:07 1 -> /dev/null
lr-x------ 1 root root 64 Jul 3 17:07 11 -> pipe:[29209918]
lrwx------ 1 root root 64 Jul 3 17:07 2 -> /dev/null
lrwx------ 1 root root 64 Jul 3 17:07 3 -> socket:[29209894]
lrwx------ 1 root root 64 Jul 3 17:07 5 -> socket:[29211080]
lr-x------ 1 root root 64 Jul 3 17:07 6 -> pipe:[29209915]
l-wx------ 1 root root 64 Jul 3 17:07 7 -> pipe:[29209915]
l-wx------ 1 root root 64 Jul 3 17:07 8 -> pipe:[29209916]
lr-x------ 1 root root 64 Jul 3 17:07 9 -> pipe:[29209917]
Step 3
$ fgrep 29209894 /proc/2481/net/tcp
8: 0101A8C0:0016 0201A8C0:B0B0 ...
here fgrep uses the number on the socked and the PID to extract the information.
The important information is 0101A8C0:0016 and 0201A8C0:B0B0. The first relates to ther server and the second is the connected client where the first part (split by the colon) is the hexadecimal representation of the reversed IP address and the second is the hexadecimal representation of the port. i.e
0101A8C0 -> 1.1.168.192 -> 192.168.1.1. If you know the port the server is listening on you can skip Step 2 and use the following instead of Step 3.
Step 2 + 3 Replacement when knowing the server port - if no root is availalble
in this case as I was checking for SFTP connections on the standard ssh port of 22 (in hex 0016)
$ fgrep 0016 /proc/2481/net/tcp
8: 0101A8C0:0016 0201A8C0:B0B0 ...