Learn Introduction to Lsof
Table of Contents
Under Linux, many objects are considered a file, regardless of whether the object is actually a file, device, directory, or socket. Listing a file is easy, there is the shell built-in
ls for that. But what if a user wanted to see which files are currently opened by the web server process? Or if that user wanted to find out which files are opened in a certain directory? That’s where
lsof comes into play. Imagine
lsof as a
ls with the addition of “open files”.
Please note that while the BSD’s have a different utility for this job,
fstat, several other flavors of Unix (Solaris, for example) also possess
lsof. The options and flags are different on the other platforms, as well as the look of the output, but generally the knowledge in this article should be applicable for them too.
First, let’s take a look at the format of
lsof output and how it is to be read. The usual output of
lsof without any parameters would resemble the following. This has been trimmed for readability.
COMMAND PID TID USER FD TYPE DEVICE SIZE/OFF NODE NAME init 1 root cwd DIR 254,1 4096 2 / init 1 root rtd DIR 254,1 4096 2 / init 1 root txt REG 254,1 36992 7077928 /sbin/init init 1 root mem REG 254,1 14768 7340043 /lib/x86_64-linux-gnu/libdl-2.13.so init 1 root mem REG 254,1 1603600 7340040 /lib/x86_64-linux-gnu/libc-2.13.so init 1 root mem REG 254,1 126232 7340078 /lib/x86_64-linux-gnu/libselinux.so.1 init 1 root mem REG 254,1 261184 7340083 /lib/x86_64-linux-gnu/libsepol.so.1 init 1 root mem REG 254,1 136936 7340037 /lib/x86_64-linux-gnu/ld-2.13.so init 1 root 10u FIFO 0,14 0t0 4781 /run/initctl
These columns mean the following:
- COMMAND – The process that an open file belongs to, in this example everything is related to
- PID – The process identification number of said process.
- USER – The user that the process runs under. For
init, it’s almost always
- FD – The file descriptor of the file, the most common being:
cwd– The current working directory (you might notice the similarity to the
pwdcommand which prints the current working directory).
rtd– The root directory of a process.
text file, this can either mean a configuration file related to the process or the “source code” related to (or belonging to) the process.
mem– A so called “memory mapped file”, that means a segment of virtual memory (read: RAM) that has been assigned to a file.
- A number – the number represents the actual file descriptor, the character after the number is the mode in which the file is opened:
u– Read and write.
- TYPE – Specifies the actual type of the file, the most common are:
REG– A regular file.
DIR– A directory.
FIFO– First in, first out.
- DEVICE – The major and minor number of the device that holds the file.
- SIZE – The size of the file, in bytes.
- NODE – The inode number of the file.
- NAME – The name of the file.
This might be a little bit overwhelming for now, but if you work with
lsof a few times, it will quickly sink into your brain.
As mentioned above, the output of
lsof has been shortened here. Without any arguments or filters,
lsof produces hundreds of lines of output which will only leave you confused.
There are two basic approaches to solve that problem:
- Use one or more of the
lsofcommand line options to narrow down the results.
- Pipe the output through, for example,
While the latter option may sound more comfortable since you won’t have to memorize the
lsof command line options, it’s generally not as flexible and efficient, so we’ll stick to the first one.
Let’s imagine that you want to open a file with your favorite text editor, and that the text editor tells you that it can only be opened in read-only mode because another program is already accessing it.
lsof will help you find out who the perpetrator is:
This will produce an output similar to this:
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME vim 2679 root 5w REG 254,1 121525 6035622 /root/lsof.txt
Apparently, you forgot to close and older session! A very similar problem happens when you try to unmount an NFS share and
umount tells you it can’t because something is still accessing the mounted folder. Again,
lsof can help you with identifying the culprit:
lsof +D /path/to/your/directory/
Notice the trailing slash, that’s important. Otherwise
lsof will assume you mean a regular file. Don’t be confused by the
+ in front of the flag –
lsof has so many command line options that it needs
+ in addition to the more common
-. The output would look like this:
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME mocp 5637 music 4r REG 0,19 10147719 102367344 /home/Music/RMS_GNU_SONG.ogg
That means that the process
mocp, with the PID
5637, belonging to the user
music has opened a file called
RMS_GNU_SONG.ogg. However, even after closing that process, there is still a problem – the NFS volume can’t be unmounted.
lsof has a
-c flag that displays files opened an arbitrary process name.
lsof -c mocp
That would produce an output looking like this:
mocp 9383 music 4r REG 0,19 10147719 102367344 /home/Music/ANOTHER_RMS_GNU_SONG.ogg
In this example, there is another instance of
mocp running, preventing you from unmounting the share. After shutting down that process, you want to make sure that the user
music has no other potentially problematic files open.
lsof has a
-u flag for showing files opened by a specific user. Remember, a file isn’t always just a regular file on your hard disk!
lsof -u music
You can also pass several users, separated by commas:
lsof -u music,moremusic
An important note on the default behavior of
lsof: the results are OR-based, which means that you will see file results opened by processes that are owned by either the user
music, or the user
moremusic. If you wanted to see results matching processes that are owned by both users, then you would have to pass the flag
lsof -au music, moremusic
Since both of the users are in the group
musicusers, then you can also list files based on group:
lsof -g musicusers
You can also combine command line flags:
lsof -u music,moremusic -c mocp or lsof -u ^music +D /home/Music
In the last line, we added another special flag –
^, which stands for a logical NOT. If the output is empty after running that command, then the unmounting will most likely be successful.
In the previous examples, we mostly looked at regular files. How about sockets and network connections?
To list all current network connections
lsof has the
The output looks similar to what we’ve seen so far…
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME owncloud 3509 myuser 25u IPv4 44946 0t0 TCP strix.local:34217->myserver.itweb.services.com:https (ESTABLISHED) firefox 3612 myuser 82u IPv4 49663 0t0 TCP strix.local:43897->we-in-f100.1e100.net:https (ESTABLISHED) ssh 3784 myuser 3u IPv4 10437 0t0 TCP strix.local:51416->someserver.in:ssh (ESTABLISHED) wget 4140 myuser 3w IPv4 45586 0t0 TCP strix.local:54460->media.ccc.de:http (CLOSE_WAIT)
… except for one difference: instead of file names or directories, the column
NAME now shows connection information. Each connection consists of the following parts:
- Local hostname.
- Source port of the connection.
- Destination DNS name.
- Destination port.
- Status of the connection.
As with many other tools, you may opt-out of resolving DNS names and ports (
-P, respectively). The flag
-i takes additional parameters. You can specify whether or not to show
icmp connections or certain ports:
lsof -i :25 or lsof -i :smtp
Again, parameters can be combined. The following example…
lsof -i tcp:80
… will only show you TCP connections using port 80. You may also combine it with the options that you already know from “classic” files:
lsof -a -u httpd -i tcp
This will show you all TCP connections opened by the user
httpd. Note the
-a flag, which changes the default behavior of
lsof (as mentioned earlier). As with most command-line tools, you can go extremely deep. The following will only show you TCP connections whose state is “ESTABLISHED”:
lsof -i -s TCP:ESTABLISHED
At this point, you should have a basic understanding on how
lsof works, along with some common use cases. For further reading, see the manpage of
lsof on your system.
Do you need help setting up this on your own service?
Please contact us and we’ll provide you the best possible quote!