Linux/Doppelte Dateien

Find and Remove Duplicate Files

Whether you’re using Linux on your desktop or a server, there are good tools that will scan your system for duplicate files and help you remove them to free up space.

Solid graphical and command-line interfaces are both available.

Duplicate files are an unnecessary waste of disk space.

After all, if you really need the same file in two different locations you could always set up a symbolic link or hard link, storing the data in only one location on disk.

FSlint

fdupes

dupeGuru

DupeGuru

How To Find And Delete Duplicate Files In Linux

I always backup the configuration files or any old files to somewhere in my hard disk before edit or modify them, so I can restore them from the backup if I accidentally did something wrong.

But the problem is I forgot to clean up those files and my hard disk is filled with a lot of duplicate files after a certain period of time.
I feel either too lazy to clean the old files or afraid that I may delete an important files.
If you’re anything like me and overwhelming with multiple copies of same files in different backup directories, you can find and delete duplicate files using the tools given below in Unix-like operating systems.

A word of caution

Please be careful while deleting duplicate files.

If you’re not careful, it will lead you to accidental data loss.
I advice you to pay extra attention while using these tools.

Find And Delete Duplicate Files In Linux

For the purpose of this guide, I am going to discuss three utilities namely

Rdfind
Fdupes
FSlint

These three utilities are free, open source and works on most Unix-like operating systems.

Rdfind

How to Find Duplicate Files in Linux and Remove Them

Brief: FSlint is a great GUI tool to find duplicate files in Linux and remove them.

FDUPES also find the files with same name in Linux but in the command line way.

If you have this habit of downloading everything from the web like me, you will end up having multiple duplicate files.

Most often, I can find the same songs or a bunch of images in different directories or end up backing up some files at two different places.
It’s a pain locating these duplicate files manually and deleting them to recover the disk space.

If you want to save yourself from this pain, there are various Linux applications that will help you in locating these duplicate files and removing them.

In this article, we will cover how you can find and remove these files in Ubuntu.

Note: You should know what you are doing.

If you are using a new tool, it’s always better to try it in a virtual directory structure to figure out what it does before taking it to root or home folder.
Also, it’s always better to backup your Linux system!

FSlint: GUI tool to find and remove duplicate files

FSlint helps you search and remove duplicate files, empty directories or files with incorrect names.

It has a command-line as well as GUI mode with a set of tools to perform a variety of tasks.

To install FSlint, type the below command in Terminal.

sudo apt install fslint

Open FSlint from the Dash search.

FSlint includes a number of options to choose from.

There are options to find duplicate files, installed packages, bad names, name clashes, temp files, empty directories etc.
Choose the Search Path and the task which you want to perform from the left panel and click on Find to locate the files.
Once done, you can select the files you want to remove and Delete it.

You can click on any file directory from the search result to open it if you are not sure and want to double check it before deleting it.

You can select Advanced search parameters where you can define rules to exclude certain file types or exclude directories which you don’t want to search.

FDUPES: CLI tool to find and remove duplicate files

FDUPES is a command line utility to find and remove duplicate files in Linux.

It can list out the duplicate files in a particular folder or recursively within a folder.
It asks which file to preserve before deletion and the noprompt option lets you delete all the duplicate files keeping the first one without asking you.

Installation on Debian / Ubuntu

sudo apt install fdupes

Installation on Fedora

dnf install fdupes

Once installed, you can search duplicate files using the below command:

fdupes /path/to/folder

For recursively searching within a folder, use -r option

fdupes -r /home

This will only list the duplicate files and do not delete them by itself.

You can manually delete the duplicate files or use -d option to delete them.

fdupes -d /path/to/folder

This won’t delete anything on its own but will display all the duplicate files and gives you an option to either delete files one by one or select a range to delete it.

If you want to delete all files without asking and preserving the first one, you can use the noprompt -N option.

In the above screenshot, you can see the -d command showing all the duplicate files within the folder and asking you to select the file which you want to preserve.

Final Words

There are many other ways and tools to find and delete duplicate files in Linux.

Personally, I prefer the FDUPES command line tool; it’s simple and takes no resources.

How do you deal with the finding and removing duplicate files in your Linux system? Do tell us in the comment section.