Linux/Doppelte Dateien

Find and Remove Duplicate Files

Whether you’re using Linux on your desktop or a server, there are good tools that will scan your system for duplicate files and help you remove them to free up space.

Solid graphical and command-line interfaces are both available.

Duplicate files are an unnecessary waste of disk space.

After all, if you really need the same file in two different locations you could always set up a symbolic link or hard link, storing the data in only one location on disk.

FSlint

fdupes

dupeGuru

DupeGuru

How To Find And Delete Duplicate Files In Linux

I always backup the configuration files or any old files to somewhere in my hard disk before edit or modify them, so I can restore them from the backup if I accidentally did something wrong.

But the problem is I forgot to clean up those files and my hard disk is filled with a lot of duplicate files after a certain period of time.
I feel either too lazy to clean the old files or afraid that I may delete an important files.
If you’re anything like me and overwhelming with multiple copies of same files in different backup directories, you can find and delete duplicate files using the tools given below in Unix-like operating systems.

A word of caution

Please be careful while deleting duplicate files.

If you’re not careful, it will lead you to accidental data loss.
I advice you to pay extra attention while using these tools.

Find And Delete Duplicate Files In Linux

For the purpose of this guide, I am going to discuss three utilities namely

Rdfind
Fdupes
FSlint

These three utilities are free, open source and works on most Unix-like operating systems.

Rdfind

FSlint

FSlint is yet another duplicate file finder utility that I use from time to time to get rid of the unnecessary duplicate files and free up the disk space in my Linux system.

Unlike the other two utilities, FSlint has both GUI and CLI modes.
So, it is more user-friendly tool for newbies.
FSlint not just finds the duplicates, but also bad symlinks, bad names, temp files, bad IDS, empty directories, and non stripped binaries etc.

Installing FSlint

FSlint is available in AUR, so you can install it using any AUR helpers.

$ yay -S fslint

On Debian, Ubuntu, Linux Mint:

$ sudo apt-get install fslint

On Fedora:

$ sudo dnf install fslint

On RHEL, CentOS:

$ sudo yum install epel-release $ sudo yum install fslint

Once it is installed, launch it from menu or application launcher.

This is how FSlint GUI looks like.

Datei:Bild6.png

FSlint interface

As you can see, the interface of FSlint is user-friendly and self-explanatory.

In the Search path tab, add the path of the directory you want to scan and click Find button on the lower left corner to find the duplicates.
Check the recurse option to recursively search for duplicates in directories and sub-directories.
The FSlint will quickly scan the given directory and list out them.

Datei:Bild8.pngfslint GUI

From the list, choose the duplicates you want to clean and select any one of them given actions like Save, Delete, Merge and Symlink.

In the Advanced search parameters tab, you can specify the paths to exclude while searching for duplicates.

Datei:Bild7.png

fslint advanced search

FSlint command line options

FSlint provides a collection of the following CLI utilities to find duplicates in your filesystem:* findup — find DUPlicate files

findnl — find Name Lint (problems with filenames)
findu8 — find filenames with invalid utf8 encoding
findbl — find Bad Links (various problems with symlinks)
findsn — find Same Name (problems with clashing names)
finded — find Empty Directories
findid — find files with dead user IDs
findns — find Non Stripped executables
findrs — find Redundant Whitespace in files
findtf — find Temporary Files
findul — find possibly Unused Libraries
zipdir — Reclaim wasted space in ext2 directory entries

All of these utilities are available under /usr/share/fslint/fslint/fslint location.

For example, to find duplicates in a given directory, do:

$ /usr/share/fslint/fslint/findup ~/Downloads/

Similarly, to find empty directories, the command would be:

$ /usr/share/fslint/fslint/finded ~/Downloads/

To get more details on each utility, for example findup, run:

$ /usr/share/fslint/fslint/findup --help

For more details about FSlint, refer the help section and man pages.

$ /usr/share/fslint/fslint/fslint --help $ man fslint

Resources

How to Find Duplicate Files in Linux and Remove Them

Brief: FSlint is a great GUI tool to find duplicate files in Linux and remove them.

FDUPES also find the files with same name in Linux but in the command line way.

If you have this habit of downloading everything from the web like me, you will end up having multiple duplicate files.

Most often, I can find the same songs or a bunch of images in different directories or end up backing up some files at two different places.
It’s a pain locating these duplicate files manually and deleting them to recover the disk space.

If you want to save yourself from this pain, there are various Linux applications that will help you in locating these duplicate files and removing them.

In this article, we will cover how you can find and remove these files in Ubuntu.

Note: You should know what you are doing.

If you are using a new tool, it’s always better to try it in a virtual directory structure to figure out what it does before taking it to root or home folder.
Also, it’s always better to backup your Linux system!

FSlint: GUI tool to find and remove duplicate files

FSlint helps you search and remove duplicate files, empty directories or files with incorrect names.

It has a command-line as well as GUI mode with a set of tools to perform a variety of tasks.

To install FSlint, type the below command in Terminal.

sudo apt install fslint

Open FSlint from the Dash search.

FSlint includes a number of options to choose from.

There are options to find duplicate files, installed packages, bad names, name clashes, temp files, empty directories etc.
Choose the Search Path and the task which you want to perform from the left panel and click on Find to locate the files.
Once done, you can select the files you want to remove and Delete it.

You can click on any file directory from the search result to open it if you are not sure and want to double check it before deleting it.

You can select Advanced search parameters where you can define rules to exclude certain file types or exclude directories which you don’t want to search.

FDUPES: CLI tool to find and remove duplicate files

FDUPES is a command line utility to find and remove duplicate files in Linux.

It can list out the duplicate files in a particular folder or recursively within a folder.
It asks which file to preserve before deletion and the noprompt option lets you delete all the duplicate files keeping the first one without asking you.

Installation on Debian / Ubuntu

sudo apt install fdupes

Installation on Fedora

dnf install fdupes

Once installed, you can search duplicate files using the below command:

fdupes /path/to/folder

For recursively searching within a folder, use -r option

fdupes -r /home

This will only list the duplicate files and do not delete them by itself.

You can manually delete the duplicate files or use -d option to delete them.

fdupes -d /path/to/folder

This won’t delete anything on its own but will display all the duplicate files and gives you an option to either delete files one by one or select a range to delete it.

If you want to delete all files without asking and preserving the first one, you can use the noprompt -N option.

In the above screenshot, you can see the -d command showing all the duplicate files within the folder and asking you to select the file which you want to preserve.

Final Words

There are many other ways and tools to find and delete duplicate files in Linux.

Personally, I prefer the FDUPES command line tool; it’s simple and takes no resources.

How do you deal with the finding and removing duplicate files in your Linux system? Do tell us in the comment section.

Quellen