Understanding Unix UIDs

Have you ever tried to dwell into the mysteries of Unix’s User IDs? Well if you have, I guess you probably found yourself with a headache. Every conversation I recall having in the office about edge cases and weird scenarios related with UIDs concluded with the phrase “Is this even possible?”.

Today, we’ll try to make some sense out of it, including some interesting examples & suggestions. But first, we owe ourselves a quick recap to paint a complete picture.

Part 1: What are User Identifiers (UIDs)?

Unix-based operating systems identify users registered with the system using UIDs. Every user has a unique UID. The textual representation of a user exists only in the user-space whereas the kernel is aware only of the UID.

When we create a new user (e.g. using useradd), the system automatically allocates a new UID and assigns it to the user.

UID values

• UIDs are stored inside uid_t which is a 32bit unsigned int
• UID -1 (which is actually 4,294,967,294), represents an invalid UID
• UID 0 is the superuser
• UIDs 1–99 are statically allocated by the system
• UIDs 100–499 are reserved for dynamically allocated users by the system and administrators. These days, most systems reserve the extended range 100–999 for this purpose.

Listing system users

If we want to list all the users on the system, we can examine /etc/passwd. Every line in this file represents a user in the system, and uses the following format:

<username>:<password>:<user-id>:<group-id>:<comment>:<home>:<shell>

• Username: The textual representation of the user’s login name. It’s length is limited to 32 chars, and it should not contain capital letters.
• Password: Today, a hint about the whereabouts of the user password. Older versions used this value differently.
• User ID: The ID of the user
• Group ID: The user’s primary group ID
• Comment field (a.k.a GECOS or GCOS): Optional and used only for informational purposes.
• Home: This is the user’s home directory: the initial directory where the user is placed after logging in.
• Shell: This is the program to run at login (if empty, use /bin/sh). If set to a nonexistent executable, the user will be unable to login.

  A standard trick is to put /usr/sbin/nologin for users that should never open a shell, but still allow other applications and other users re-authenticate as this user (Using methods we will discuss in the following sections)

If you want to read some more about this file, make sure to check out the man page passwd(5)

Part 2: Process UIDs & Permissions

Now that we’ve discussed UIDs on their own, let’s try to understand their relationship with Unix processes.

The POSIX standard prescribes that every process has three different UIDs at any given point in time:

• Real UID (ruid): This ID determines who owns the process.
• Effective UID (euid): This is the actual UID that defines the process’ permissions when accessing shared resources such as message queues, shared memory, and semaphores.

  On most UNIX systems, this UID also determines the permissions when accessing files, whereas Linux uses the filesystem UID described below for this task.

• Saved set-UID (suid): This UID is used in set-user-ID programs to save the effective ID that was set when the program was executed. Such program can gain and drop privileges by switching its effective UID back and forth between the values in its real UID and saved set-UID.

Linux systems add an additional UID to processes:

• File system UID (fsuid): This ID is used to determine permissions for accessing files.

From this moment on, all examples will revolve around Linux systems, so get used to seeing the file-system UID.

If you want to read the small letters, make sure to check out the man page credentials(7).

Listing process’ UIDs

To list a process’ UIDs, we can examine its status file, found at /proc/<pid>/statsu.

For example, if we have a vim process running as PID 12345, we can examine it using head /proc/12345/status. The output will resemble this:

Name:   vim
State:  S (sleeping)
Tgid:   12345
Pid:    12345
PPid:   12222
TracerPid:      0
Uid:    1000    1000    1000    1000
Gid:    1000    1000    1000    1000
FDSize: 3
Groups:

As you can see, there are four different values under the Uid key. The values are real, effective, saved set, and filesystem UIDs in respective order.

Shell UIDs

The shell we are using is also a par-excellence process, so it also has four different UIDs. Because checking your own identity is a fairly common operation, the shell gives us easy access to this information:

• We can get our user name using whoami
• We can get our real and effective UIDs using $UID and $EUID respectively

One other way to get our UIDs is to read it from our own status file using a simple trick. When inside a shell, we can read our own PID using $$, so we can use the command grep Uid /proc/$$/status to get the UID line from our own status file:

Uid:    1000    1000    1000    1000

Using grep Uid is more elegant than head because it prints only the UIDs, though both would have worked of course.

But what happens if we want to change our current UID? What if we want to perform certain operations as a different user? For that, the shell has a substitute user command: su

The command accepts a user name, and substitutes the active user for the current shell. Of course, this is an over-simplification of the tool, because the process might involve authentication, multiple parameters, etc. If you want to know what other options are there, check out the man page su(1).

I know you probably want to scream: “What about sudo you bastard!?”, but I urge you to wait and keep on reading. Some readers might still be lacking certain pieces of information required to understand how sudo works.

Executed Process UIDs

When a new process is executed (e.g. via execve), the system has to set its UIDs. In most cases, the decision-making process is very simple: The kernel takes the effective UID of the executing process and uses it for all four UIDs. Let’s see an example:

> echo $EUID
1001
> vim &
> grep Uid /proc/$!/status
Uid:    1001    1001    1001    1001

Running a command with a succeeding & tells the shell to run it in a sub-shell at the background. The shell does not wait for the command to finish, and the return status is 0. Then, just like we can get our own PID inside a shell using $$, we can get the PID of the background process using $!.

As you can see, everything is pretty straight-forward. The effective UID of the user executing vim was used for all UID values of the newly-created process, just like expected.

Alas(!) there is a feature called SUID that make things more complicated and we’ll discuss thoroughly down the road.

Part 3: File UIDs & Permissions

There are some special file permissions that break the fairly straight-forward flow from the last example. To cover these unique capabilities and understand how they work and why we need them, we first have to make sure we have all the relevant prior knowledge.

Who owns this file and what are its permissions

In Unix, everything is a file. And every file has an owner and a certain set of permissions. The owner is simply one of the UIDs registered with the system and the permissions are best described as a bitmask that defines which operations are allowed and by what user:

          OWNER GROUP OTHER
          |---| |---| |---|
        - r w x r w x r w x
        | | | | | | | | | |
        | | | | | | | | | +----> Others can execute
        | | | | | | | | +------> Others can write
        | | | | | | | +--------> Others can read
        | | | | | | |
        | | | | | | +----------> Group members can execute
        | | | | | +------------> Group members can write
        | | | | +--------------> Group members can read
        | | | |
        | | | +----------------> Owner can execute
        | | +------------------> Owner can write
        | +--------------------> Owner can read
        |
        +----------------------> File type

From this point on we will ignore the ‘File-type’ bit. But, we will say two things: 1. It contains a single letter that represents the type, e.g: d for directory or l for link. 2. Permissions may have different meanings for different file types.

Each bit in the mask may be switched on or off. When it’s switched on, the given user[s] has permission to perform the operation, otherwise, he can’t. When the bit is switched off, its symbolic representation is -.

To see a file’s owner and permissions, one can use ls -l <file> (a.k.a ll <file> ) or stat <file> and just look for a similarly looking string. For example, here is an what we see when we execute ls -l /bin/bash:

-rwxr-xr-x 1 someuser myusers 1113504 Apr  4  2018 /bin/bash

• File owner is someuser
• File group is myusers
• Owner can read, write and execute this file
• Group members can read and execute this file, but not write to it
• Other members can read and execute this file, but not write to it

Understanding directory permissions

Having a firm grasp of file permissions, we can discuss the subtle nuances of directory permissions. Unlike a file, the idea of executing a directory is quite absurd, and writing to it is not very intuitive as well. Thus, when talking about directories, these permission bits have a different meaning:

• Read permission: Permitted users can enumerate the file names inside a given directory. However, the file name is the only accessible property.
• Execute permission: Permitted users can access files of which they know of. If there is an executable ssh in the a directory apps , they won’t be able to run ls -l apps , but could still execute apps/ssh , or ls apps/ssh (Given they have appropriate permissions on the file itself).
• Write permissions: Permitted users can create, delete or modify any files and subdirectories. Here, in contrast to the execute permission, these actions are allowed even if the file or subdirectory is owned by another user.

When we say permitted users, we mean owner/group/others according to the permissions mask.

Numeric representation of the permissions mask

These last few paragraphs we used the symbolic representation for permissions, but a very common alternative representation is a numeric one.

To convert the symbolic representation to a numeric one, we ignore the file-type and consider every group of read-write-execute permissions as an octal number where r is the MSB, and x is the LSB. Here is the numeric value for /bin/bash:

-rwxr-xr-x 1 someuser myusers 1113504 Apr  4  2018 /bin/bash
 rwx --------> 4 + 2 + 1 = 7
    r-x -----> 4 + 0 + 1 = 5
       r-x --> 4 + 0 + 1 = 5
           ==> The numerical mode is 0755

Noticed the 0 up at front? A common practice for representing an octal number is adding a zero as a prefix.

Changing file owner

The tool for changing a file owner is called chown. Just like any other Unix tool, it is highly configurable, but the most banal use is as simple as it gets:

# Set root user as the owner of the file
chown root:root /path/to/my/exec
# Set someuser as the owner and myusers as the group of the file
chown someuser:myusers /path/to/my/exec

Changing file permissions

The tool for this job is chmod. It supports both symbolic and numeric representations, and basically is very simple.

When using numeric mode, you simply have to specify the new mode you want for the file. The new mode must configure the all the permissions at once, for example:

# Make readable, writable and executable by anyone
chmod 0777 /bin/bash
# Make readable and executable by owner only
chmod 0600 ~/.ssh/authorized_keys

Omitted numbers are considered as leading zeros, i.e 7 is 007.

When using textual mode, you can configure the permission in a more subtle ways. The configurations we set is comprised from:

• Which users’ access to the file will be changed: Owner a.k.a user (u), Group members(g), Others (o), or all users (a). When this bit is not specified, the default is all users (a).
• An operator that defines the type of change, e.g. setting new permissions (+) or clearing them (-)
• The affected permissions, e.g. read (r) or write (w)

Here are some examples:

chmod g+x /bin/bash # Add execution permissions for group members
chmod u-w /bin/bash # Remove write permissions for owner
chmod  +r /bin/bash # Add read permissions for all users

Part 4: Set owner User ID (SETUID a.k.a SUID) bit

Finally, we get to the juicy stuff. Now we are well prepared to talk about the first special file permissions we were so eager to explore.

This special file permission, allows applications to run with a predefined UID no matter which user actually executed them.

How does this work? When an executable file with the SUID bit is executed, the kernel automatically sets the effective and saved set UIDs to the file owner’s UID, whereas the real UID will contain the UID of the actual user that executed the application.

Let’s see an example. Assuming we have an application called printuids in our folder, that simply prints the UID values when we run it.

> whoami
user1
> echo $EUID
1000
> ls -l printuids
-rwxrwxrwx 1 user1 user1 8448 Dec 15 21:07 printuids
> ./printuids
ruid[1000], euid[1000], suid[1000]

The code for the printuids can be found further down, under Gettings UIDs.

At the moment, we didn’t set the SUID permission bit, so no surprises here. But, what will happen if we set the SUID permission bit for this file? To accomplish this goal, we first have to set the file owner (Remember, the system will run the SUID executable using the owner’s UID):

> sudo chown root:root printuids
> ls -l printuids
-rwsrwxrwx 1 **root root** 8448 Dec 15 21:07 printuids

Now, we’ll set the SUID bit for the executable file:

> sudo chmod u+s printuids
> ls -l printuids
-rw**s**rwxrwx 1 root root 8448 Dec 15 21:07 printuids

Did you notice the weird s permission (In bold) where one would expect to find an x ? This is how the OS lets us know that the SUID is set for this file. Now, let’s finally execute the file and see what happens:

> ./printuids
ruid[1000], euid[0], suid[0]

As expected, the effective and saved set UIDs are 0, because the file owner is root, whose UID is 0.

sudo (superuser do)

Remember my earlier comment regarding sudo? Well, now we can fully understand how this application works.

sudo is an application for performing operations as the superuser, and allows advanced management of permissions for users on the system using the /etc/sudoers file.

The executable is owned by root and has the SUID bit set. Whenever a user tries to run a command using sudo, his permissions are validated using the sudoers file and if the request is granted, the operation is performed with elevated privileges.

One of the options for sudo is the -i switch, that, when run without parameters, grants us access to a new shell running under the superuser.

Part 5: Sticky-bit

Some more juicy stuff around the corner. This is another esoteric file permission you won’t get to meet too much. Actually, many even mistake the sticky bit for the SUID bit, but that can’t be farther from the truth.

The sticky bit is irrelevant nowadays, and was more relevant on older (I mean, really older, like 1970s) systems. When this bit was set for an executable file, the program’s text image was saved on the swap device so it will load more quickly. Since this issue no longer persists today with the advancement of technology, I would consider this feature as obsolete.

But, lets say that we really, really, really wanted to set the sticky bit on a file. Is it possible? Yes, it is! It’s as simple as setting the SUID bit, we just have to use a different letter :)

> touch testfile
> chmod +t testfile
> ls -l testfile
-rw-rw-r-**T** 1 root root   0 Dec 19 18:32 testfile

Do you see the T bit? This is the sticky bit right there, laying around uselessly.

Any question pops up into your mind right now? Well, one should. Looking at the permission bits, how can we know if the file is executable by others? The trick is very simple. If the file is not executable by others, it says T. If the file is executable, it says t .

> chmod +x testfile
> ls -l testfile
-rw-rw-r-**t** 1 root root   0 Dec 19 18:32 testfile

Come on now, why do we even care about this antiquated bit? Cuz some smart-ass developer decided to reuse this bit for directories, with a totally different meaning. It is so different, that it even earned itself a different name: ‘Restriction deletion flag’, and as you probably guessed by now, it has something to do with deleting stuff.

Consider the /tmp directory. It is kinda special, because we want to let anyone & anywhere access this directory anytime. It seems that any application on the machine can do as it wishes in this bottomless pithole.

Is it though? Well NO! Have you every tried deleting or renaming a file owned by a different user? Let’s see what happens:

> whoami
user1
> cd /tmp
> ls -l
-rwxrwxrwx 1 user2 user2    0 Dec 19 18:22 user2.txt
> rm user2.txt
rm: cannot remove 'user2.txt': Operation not permitted
> mv user2.txt user1.txt
mv: cannot move 'user2.txt' to 'user1.txt': Operation not permitted

Wait, what!? user2.txt has rwx permissions for others! That means for user1 as well! Why are we not permitted to do this operation?

This is where the restricted deletion flag comes into play. Let’s look at the permission bits of the /tmp directory:

> ls -l /
drwxrwxrw**t**  1 root root   4096 Dec 19 18:28 tmp

Do you see the “sticky-bit”? This is the glorious flag protecting us from thieves and outlaws. Setting this bit for a directory is just as easy as it is for regular files.

Part 6: Getting and Setting UIDs during run-time

This last part is focused mainly for programmers that write system applications and want to familiarize themselves with the tool-set provided by the operating system. And believe it or not, this is widely used in most Unix services everybody heard of. Some popular examples are:

• An application that checks if it was executed by a specific user (e.g superuser) and exits if it wasn’t.
• A privileged server process that creates child processes to handle incoming client connections, but doesn’t want the child handling process to be privileged as well. So it drops the child’s permissions right after creating (fork -ing) it.

The examples will be written in C, just because they truly describe the system-calls. Every other API in every other low/high-level language is probably some facade for these calls.

Getting UIDs

We’ll start with the easier part. These system-calls have no implications on the executing process whatsoever. You call the method, get the answer, and that’s it.

uid_t getuid(void); // get the real UID
uid_t geteuid(void); // get the effective UID

Remember the printuids from the SETUID file permission part? This application uses getresuid , which is a GNU extension over the basic POSIX API (This is why we need the #define _GNU_SOURCE ). Here is the source code:

#define _GNU_SOURCE
#include <unistd.h> // for getresuid & uid_t
#include <stdio.h> // for printf
#include <errno.h> // for perror

int main(int argc, char **argv) {
  uid_t ruid, euid, suid;
  if (getresuid(&ruid, &euid, &suid)) {
    perror("getresuid");
    return 1;
  }

  printf("ruid[%u], euid[%u], suid[%u]\n", ruid, euid, suid);
  return 0;
}

Setting UIDs

Now, for the intricate part. Here we also have three different system-calls, but unlike the getters, they each have its own rules and side affects.

The explanations in this section rely heavily on the man pages. Don’t be surprised if you find some familiar quotes.

Before we start talking about actually changing things, we should first comprehend the rules of changing our UID. Lucky for us, we can sum it up using a couple of simple rules:

• The superuser (UID 0) can change any of its UIDs to any arbitrary UID. Yes, one can change its UID to a value not registered as a valid user.
• An unprivileged user can change any of its UIDs to one of: the current real UID, the current effective UID or the current saved set-user-ID

Setting a uid by a superuser might not always succeed (e.g. because of resource limits for a certain user), so make sure to always check the return value!

setuid

int setuid(uid_t _uid_);

When executed by an unprivileged user, sets the effective user ID of the calling process.

When executed by the superuser (euid is 0), all process-related user ID’s are set to_uid_. After this has occurred, it is impossible for the program to regain root privileges. If you want to drop privileges temporarily, use seteuid instead.

seteuid

int seteuid(uid_t _euid_);

Sets the effective user ID of the calling process.

On Linux, seteuid() is implemented as library function that calls setreuid

setreuid

int setreuid(uid_t _ruid_, uid_t _euid_);

Sets real and effective user IDs of the calling process.

Supplying a value of -1 for either the real or effective user ID forces the system to leave that ID unchanged.

setresuid

int setresuid(uid_t _ruid_, uid_t _euid_, uid_t _suid_);

Just like getresuid, this useful system-call is a GNU extension, so we have to add #define _GNU_SOURCE before #include <unistd.h>.

This call sets the real UID, the effective UID, and the saved set-UID of the calling process. the file-system UID is always set to the same value as the (possibly new) effective UID.

Just like setreuid, if one of the arguments equals -1, the corresponding value is not changed.

setfsuid

int setfsuid(uid_t _fsuid_);

The system call setfsuid() changes the value of the caller’s filesystem user ID, the user ID that the Linux kernel uses to check for all accesses to the filesystem.

Normally, the file-system UID shadows the effective UID, in fact, every call that changes the effective UID will also change the file-system UID to keeps the flow consistent with other POSIX-based systems.

This operation is essential only for applications such as the Linux NFS, where we want handle files on behalf of the operating user, without changing the actual effective UID of the application.

Thanks for sticking around for this long. I never imagined that this article will end up being so comprehensive when I started writing it, but I hope this will be useful for those who are newer to the subject.