We’ve all learned that a process has an ID (the so-called PID for Process IDentifier), that a process can have one or more threads, and that each thread also has an ID (the so-called TID for Thread IDentifier). At least, this is what we have been taught in computer science classes, and what we read in books and blogs.

How surprised would you be to learn that processes and even threads don’t really exist in Linux?

Well, at least they don’t exist in the kernel. They are mere abstractions for humans in userland. Follow me, it’s gonna be a fun trip!

The ps Command#

Let’s spawn a simple gedit process and call ps to get details about it:

$ gedit &                                 
[1] 73357
$ ps -L 73357
    PID     LWP TTY      STAT   TIME COMMAND
  73357   73357 pts/4    SNl    0:00 gedit
  73357   73380 pts/4    SNl    0:00 gedit
  73357   73381 pts/4    SNl    0:00 gedit
  73357   73383 pts/4    SNl    0:00 gedit

According to the man pages, the -L option “shows threads, possibly with LWP and NLWP columns”. What are LWP and NLWP? The man pages say that LWP is the:

light weight process (thread) ID of the dispatchable entity (alias spid, tid). See tid for additional information.

NLWP is described as the:

number of lwps (threads) in the process (alias thcount)

It doesn’t appear here in ps’s output.

So the first column identifies the process (the PID) and the second column identifies the threads (the TID). Here, we are fully in userland. We do have a process, gedit, with PID 73357, and it does have threads.

But did you notice that the first line has the same value for PID and TID? This is not a coincidence. For each process, the first TID has the same value as the PID. Why? Is it a convention to give the process and the first thread the same identifier? Not exactly. In fact, it’s the first leak of the process/thread abstractions.

Let’s go back to the man pages and have a look at the description of a TID:

the unique number representing a dispatchable entity (alias spid, tid). This value may also appear as: a process ID (pid); a process group ID (pgrp); a session ID for the session leader (sid); a thread group ID for the thread group leader (tgid); and a tty process group ID for the process group leader (tpgid).

From this description, we can understand that everything is somehow a thread because all other identifiers are in fact TIDs. In particular, a PID is a TID. This is because processes don’t really exist in Linux. Processes are groups of threads, and each group has a leader. The TID of the leader is used as the TGID (Thread Group ID) and TGID is the PID. The man pages confirm this last claim by describing a PID as:

a number representing the process ID (alias tgid).

Let’s follow the alias and see the definition of a TGID:

a number representing the thread group to which a task belongs (alias pid). It is the process ID of the thread group leader.

Notice that a new term has come into play: task. Save it for later, it will be back.

We can tune our ps command to display the exact columns we want and confirm the connections we made by following the aliases in the man pages:

$ ps -L -o pid,tgid,tid,lwp,cmd 73357 
    PID    TGID     TID     LWP CMD
  73357   73357   73357   73357 gedit
  73357   73357   73380   73380 gedit
  73357   73357   73381   73381 gedit
  73357   73357   73383   73383 gedit

Inspection of Kernel Data#

Let’s leave the blurry abstractions of ps and look at the data that the kernel exposes in /proc. This pseudo-filesystem is where ps gets its information: it reads these files and displays them in a human-friendly way, as its source code shows.

The first file to look at is status, where we find the TGID (which is the PID):

$ cat /proc/73357/status
Name:   gedit
...
Tgid:   73357
...
Pid:    73357
PPid:   68096
...

The second is the task directory, with one entry per… task. Remember this term from the description of a TGID in the man pages? It’s getting more concrete right now. We can list the tasks in the “process”:

$ ls /proc/73357/task
73357  73380  73381  73383

We find the same TIDs displayed by ps. We can inspect the status file of each of these tasks:

$ grep -E '^Pid:|Tgid:' /proc/73357/task/*/status
/proc/73357/task/73357/status:Tgid:     73357
/proc/73357/task/73357/status:Pid:      73357
/proc/73357/task/73380/status:Tgid:     73357
/proc/73357/task/73380/status:Pid:      73380
/proc/73357/task/73381/status:Tgid:     73357
/proc/73357/task/73381/status:Pid:      73381
/proc/73357/task/73383/status:Tgid:     73357
/proc/73357/task/73383/status:Pid:      73383

Here, strangely enough, the term TID doesn’t appear, and PID is used instead. This is not a quirk of /proc: it comes from the kernel code itself, as we will see in the next section.

Processes were already gone (*), but now threads are gone too. The kernel knows about tasks, each with a PID and member of a group of tasks, identified by a TGID, which is the PID of the task that is the leader of the group.

(*) = Well, except that /proc is short for “process(es)”…

Jump Into The Kernel Source Code#

The scheduler of the Linux kernel only knows one thing: tasks. They are represented in C by the struct task_struct type in sched.h.

We can clearly see that a task_struct object has a PID and a TGID, but no TID:

	pid_t				pid;
	pid_t				tgid;

This is why the status files in /proc show Pid: and not Tid:. They are generated by the kernel directly from these two fields.

We may wonder why the field is named pid instead of tid (for Task ID maybe).

The answer is history. Thread groups were added later, as the man pages of clone() explain in the description of the CLONE_THREAD flag:

Thread groups were a feature added in Linux 2.4 to support the POSIX threads notion of a set of threads that share a single PID. Internally, this shared PID is the so-called thread group identifier (TGID) for the thread group. Since Linux 2.4, calls to getpid(2) return the TGID of the caller.

Before that, tasks were just processes, each with its own PID. Hence pid. When thread groups came, the kernel kept the field and added tgid next to it.

It’s funny to see that in the source code of ps, the field is named tid, for “task ID”. Since the comment says the data comes from the kernel structure, pid would have been a better name to match the kernel naming:

// Basic data structure which holds all information we can get about a process.
// (unless otherwise specified, fields are read from /proc/#/stat)
//
// Most of it comes from task_struct in linux/sched.h
//
typedef struct proc_t {
    int
        tid,            // (special)       task id, the POSIX thread ID (see also: tgid)

Conclusion#

The scheduler of the Linux kernel has neither threads nor processes. It has tasks, represented by task_struct objects.

The first abstraction is threads, which are tasks seen from userland. The second abstraction is processes, which are groups of tasks.

Can we actually say that “processes and even threads don’t really exist in Linux”? Yes and no: it depends on the point of view. Down in the kernel, they really don’t exist as such. Up at user level, the abstractions we use are processes and threads.

This quote from Stack Overflow is a good summary:

Linux uses a 1-1 threading model, with (to the kernel) no distinction between processes and threads – everything is simply a runnable task.