Diskussion:Fdupes

fdupes

The fdupes command isn’t usually installed by default, but it’s available in many Linux distribution’s repositories.

• It’s a simple command-line tool.
• This is probably the most convenient, quickest tool you can use if you want to find duplicate files in an environment where you only have access to a Linux command line, not a graphical user interface.

Using it is simple.

• Just run the fdupes command followed by the path to a directory.
• So, fdupes /home/chris would list all duplicate files in the directory /home/chris — but not in subdirectories! The fdupes -r /home/chris command would recursively search all subdirectories inside /home/chris for duplicate files and list them.

This tool won’t automatically remove anything, it will just show you a list of duplicate files.

• You can then delete the duplicate files by hand, if you like.
• You can also run the command with the -d switch to have it help you delete files.
• You’ll be prompted to choose the files you want to preserve.

Fdupes

Fdupes is yet another command line utility to identify and remove the duplicate files within specified directories and the sub-directories.  It is free, open source utility written in C programming language.

• Fdupes identifies the duplicates by comparing file sizes, partial MD5 signatures, full MD5 signatures, and finally performing a byte-by-byte comparison for verification.

Similar to Rdfind utility, Fdupes comes with quite handful of options to perform operations, such as:* Recursively search duplicate files in directories and sub-directories

• Exclude empty files and hidden files from consideration
• Show the size of the duplicates
• Delete duplicates immediately as they encountered
• Exclude files with different owner/group or permission bits as duplicates
• And a lot more.

Installing Fdupes

Fdupes is available in the default repositories of most Linux distributions.

On Arch Linux and its variants like Antergos, Manjaro Linux, install it using Pacman like below.

$ sudo pacman -S fdupes

On Debian, Ubuntu, Linux Mint:

$ sudo apt-get install fdupes

On Fedora:

$ sudo dnf install fdupes

On RHEL, CentOS:

$ sudo yum install epel-release $ sudo yum install fdupes

Usage

Fdupes usage is pretty simple.

• Just run the following command to find out the duplicate files in a directory, for example ~/Downloads.

$ fdupes ~/Downloads

Sample output from my system:

/home/sk/Downloads/Hyperledger.pdf /home/sk/Downloads/Hyperledger(1).pdf

As you can see, I have a duplicate file in /home/sk/Downloads/ directory.

• It shows the duplicates from the parent directory only.
• How to view the duplicates from sub-directories? Just use -r option like below.

$ fdupes -r ~/Downloads

Now you will see the duplicates from /home/sk/Downloads/ directory and its sub-directories as well.

Fdupes can also be able to find duplicates from multiple directories at once.

$ fdupes ~/Downloads ~/Documents/ostechnix

You can even search multiple directories, one recursively like below:

$ fdupes ~/Downloads -r ~/Documents/ostechnix

The above commands searches for duplicates in “~/Downloads” directory and “~/Documents/ostechnix” directory and its sub-directories.

Sometimes, you might want to know the size of the duplicates in a directory.

• If so, use -S option like below.

$ fdupes -S ~/Downloads 403635 bytes each: /home/sk/Downloads/Hyperledger.pdf /home/sk/Downloads/Hyperledger(1).pdf

Similarly, to view the size of the duplicates in parent and child directories, use -Sr option.

We can exclude empty and hidden files from consideration using -n and -A respectively.

$ fdupes -n ~/Downloads $ fdupes -A ~/Downloads

The first command will exclude zero-length files from consideration and the latter will exclude hidden files from consideration while searching for duplicates in the specified directory.

To summarize duplicate files information, use -m option.

$ fdupes -m ~/Downloads 1 duplicate files (in 1 sets), occupying 403.6 kilobytes

To delete all duplicates, use -d option.

$ fdupes -d ~/Downloads

Sample output:

[1] /home/sk/Downloads/Hyperledger Fabric Installation.pdf [2] /home/sk/Downloads/Hyperledger Fabric Installation(1).pdf

Set 1 of 1, preserve files [1 - 2, all]:

This command will prompt you for files to preserve and delete all other duplicates.

• Just enter any number to preserve the corresponding file and delete the remaining files.
• Pay more attention while using this option.
• You might delete original files if you’re not be careful.

If you want to preserve the first file in each set of duplicates and delete the others without prompting each time, use -dN option (not recommended).

$ fdupes -dN ~/Downloads

To delete duplicates as they are encountered, use -I flag.

$ fdupes -I ~/Downloads

For more details about Fdupes, view the help section and man pages.

$ fdupes --help $ man fdupes

Also read:* Duplicate Photos Fixer: Organize Your Photo Library Well

FDUPES: CLI tool to find and remove duplicate files

FDUPES is a command line utility to find and remove duplicate files in Linux.

• It can list out the duplicate files in a particular folder or recursively within a folder.
• It asks which file to preserve before deletion and the noprompt option lets you delete all the duplicate files keeping the first one without asking you.

Installation on Debian / Ubuntu

sudo apt install fdupes

Installation on Fedora

dnf install fdupes

Once installed, you can search duplicate files using the below command:

fdupes /path/to/folder

For recursively searching within a folder, use -r option

fdupes -r /home

This will only list the duplicate files and do not delete them by itself.

• You can manually delete the duplicate files or use -d option to delete them.

fdupes -d /path/to/folder

This won’t delete anything on its own but will display all the duplicate files and gives you an option to either delete files one by one or select a range to delete it.

• If you want to delete all files without asking and preserving the first one, you can use the noprompt -N option.

In the above screenshot, you can see the -d command showing all the duplicate files within the folder and asking you to select the file which you want to preserve.

Final Words

There are many other ways and tools to find and delete duplicate files in Linux.

• Personally, I prefer the FDUPES command line tool; it’s simple and takes no resources.

How do you deal with the finding and removing duplicate files in your Linux system? Do tell us in the comment section.