How to Find Large Files and Folders Consuming Most of Your Space

How to Find Large Files and Folders Consuming Most of Your Space

If your server (or computer) is running low on webspace without any obvious reason, here's you could find any large files and folders that may be exhausting your disk space.

Before we start looking for large files, here is how you could check your current disk usage:

root@bobby:~$ df -h

The output would look something like:

Filesystem Size Used Avail Use% Mounted on
/dev/xvda 40G 7.8G 30G 21% /

In my case I am using 7.8G out of 40GB available. Basically the df command reports the file system disk space usage.

We would focus on the /home folder as this is practically the folder which your users would be storing their files at.

The easiest way to start is to simply look for files larger than 500MB for example. The command that you would need to use is very simple:

root@bobby:~$ find /home -type f -size +500M -exec ls -lh {} \;

The output should look something like:

-rw-r--r-- 1 root root 1.8G May 13 15:43 /home/user1/large-file.txt
-rw-r--r-- 1 root root 1.8G May 13 15:42 /home/bobby/large-file.txt
-rw-r--r-- 1 root root 1.8G May 13 15:43 /home/bobby/public_html/wp-content/core.dump
-rw-r--r-- 1 root root 1.8G May 13 15:42 /home/bobby/public_html/error_log

Depending on the size of your disk, the command might take a while to complete. You can then go ahead and delete any of the large files that you do not need.

NOTE: If needed you can change the "-size +500M" part to a convenient for you value.

If the find command does not help, we might have to look for folders containing a lot of individual but small files. Probably you all know that ls -lah would not give you any information regarding the size of the folders.

root@bobby:~$ ls -lah /home/
total 28K
drwxr-xr-x 7 root root 4.0K Feb 17 08:31 .
dr-xr-xr-x 19 root root 4.0K Jan 10 21:49 ..
drwx--x--x 3 user1 user1 4.0K Dec 4 20:47 user1
drwx--x--x 6 bobby bobby 4.0K Mar 23 14:34 bobby
drwxr-xr-x 3 root root 4.0K Nov 13 17:44 root
drwx------ 2 testuser testuser 4.0K Feb 17 08:31 testuser
drwx--x--x 4 user2 user2 4.0K Jan 18 13:49 user2

In order to get the size of those folders we would simply use the du command.

du - estimate file space usage


root@bobby:~$ cd /home/
root@bobby:/home$ du -h --max-depth=1
16K ./testuser
28K ./user
16K ./root
1.8G ./user2
5.0G ./bobby
7.1G .

I can see that the user called "bobby" is using the most of my webspace, so we would cd in that folder and run the du command one more time:

root@bobby:/home$ cd bobby/
root@bobby:/home/bobby$ du -h --max-depth=1
76K ./.npm
4.0K ./script
5.0G ./public_html
12K ./.ssh
5.0G .


We would need to do that a few more times until we get to the folder that is using those 5GB:

root@bobby:/home/bobby$ cd public_html/
root@bobby:/home/bobby/public_html$ du -h --max-depth=1
8.0K ./test
4.9G ./wp-content
4.0K ./
14M ./wp-includes
7.4M ./wp-admin
16K ./node
5.0G .


Then one more time:

root@bobby:/home/bobby/public_html$ cd wp-content/
root@bobby:/home/bobby/public_html/wp-content$ du -h --max-depth=1
3.6M ./themes
252K ./plugins
4.0K ./upgrade
4.9G ./large-folder
4.9G .


And again:

root@bobby:/home/bobby/public_html/wp-content/large-folder$ du -h --max-depth=1
4.9G .


We've found the folder! We can see how many files are in the folder by running this command:

root@bobby:/home/bobby/public_html/wp-content/large-folder$ ls -lah | wc -l


Basically you could check those files and delete them if they are not required.

Hope that this helps and if you have any questions you can feel free to contact me at any time.


The Real-Time Data Platform for Developers

Buy me a coffeeBuy me a coffee