Linux Syscalls in Assembly

Assembly is human-readable machine code and as a result there are common functions that wouldn’t make sense to code from scratch every time.

A great example of this is printing to the screen; doing this in pure assembly would be tedious and add a great deal of code.

To facilitate common but complex operations, operating systems use system calls, or ‘syscalls’. A syscall is a globally available function made available by the operating system.

Syscalls in Windows: You will see shortly that the examples in this article focus on Linux-based syscall examples. Both Windows and Linux have syscalls available, but Windows syscalls are considered somewhat unstable, as they aren’t guaranteed not to change over time. As a result, a Windows syscall may not work after a new release. For this reason, most assembly programmers working in Windows don’t use syscalls. However, they do use similar methods that are similar in principle, including calling ‘wrapper’ C functions from kernel32.dll.

You can find details for Windows syscalls here.

What is a Syscall?

A syscall is a function provided by the operating system, which we can access in our assembly code. Syscalls are referenced by a number, which must be stored in the rax register.

A syscall may take in arguments that we provide in other registers. We provide the first argument in the rdi register, the second in rsi, and the third in rdx.

When we make the system call, we do so with the ‘syscall‘ instruction. Let’s see the common example of making an exit syscall, which exits the program. The exit syscall has is numbered 60, and takes in one argument that specifies the exit code.

Here’s what this looks like:

mov rax, 60 ; exit syscall is number 60
mov rdi, 0  ; exit code of 0 indicates success
syscall

That’s it! The ‘syscall’ instruction executes the system call. It looks at the rax register, finding the value of ’60’ indicating that we want to exit the program. Then it exits the program, providing the user the exit code specified.

This simple example shows how easy it is to make syscalls. Windows works similarly, but the syscall numbers are subject to change between releases.

Getting the Syscall Numbers

In Linux, we can get the syscall numbers by reading the contents of the unistd_64.h system file. We can do this in bash using the following command:

cat /usr/include/x86_64-linux-gnu/asm/unistd_64.h

We can see that the read syscall is numbered 0, write is number 1, open is 2, and close is 3.

To find the exit syscall, we can pipe our command to grep and search for ‘exit’:

cat /usr/include/x86_64-linux-gnu/asm/unistd_64.h | grep exit

The exit syscall is number 60, which we saw earlier!

x86-64 Assembly Syscall Registers

We’ve seen that the rax register is used to store the syscall number and rdi is used for the first argument. The following table shows the remaining registers and their roles in system calls:

Register	Description
rax	Syscall Number
rbx	Callee Saved
rdi	1st arg
rsi	2nd arg
rdx	3rd arg
rcx	4th arg
r8	5th arg
r9	6th arg

The first six arguments have designated registers, as shown above. If we need to use more arguments, we can add them to the stack.

Identifying Syscall Arguments

Now we know how to look syscalls and where their arguments need to go. Before using the syscall, we still need to identify the function arguments that we need to supply for it work properly. We can do this using section 2 of the manual ‘man’ pages, which contains system calls.

We can see this using the command ‘man man’:

man man

In order to access the man page for a specific syscall, we can use the ‘-s 2’ argument:

man -s 2 <syscall>

This gives us a ton of information about the syscall, including the arguments that we need to provide for the syscall to work properly.

Let’s take a look at the man page for the exit syscall:

man -s 2 exit

We can see that the _exit() function has one argument, an integer ‘status’.

We can also learn a lot more about the syscall by reading the man page entry!