|
|
| (19 dazwischenliegende Versionen desselben Benutzers werden nicht angezeigt) |
| Zeile 1: |
Zeile 1: |
| == Find and Remove Duplicate Files ==
| | [[Kategorie:Linux/Datei]] |
| Whether you’re using Linux on your desktop or a server, there are good tools that will scan your system for duplicate files and help you remove them to [https://www.howtogeek.com/185173/4-ways-to-free-up-disk-space-on-linux/ free up space].
| |
| * Solid graphical and command-line interfaces are both available.
| |
| | |
| Duplicate files are an unnecessary waste of disk space.
| |
| * After all, if you really need the same file in two different locations you could always set up a symbolic link or hard link, storing the data in only one location on disk.
| |
| | |
| === FSlint ===
| |
| FSlint is available in various Linux distributions’ software repositories, including Ubuntu, Debian, Fedora, and Red Hat.
| |
| * Just fire up your package manager and install the “fslint” package.
| |
| * This utility provides a convenient graphical interface by default, but it also includes command-line versions of its various functions.
| |
| * Like many Linux applications, the FSlint graphical interface is just a front-end that uses the FSlint commands underneath.
| |
| | |
| Don’t let that scare you away from using FSlint’s convenient graphical interface, though.
| |
| * By default, it opens with the Duplicates pane selected and your home directory as the default search path.
| |
| * All you have to do is click the Find button and FSlint will find a list of duplicate files in directories under your home folder.
| |
| * Use the buttons to delete any files you want to remove, and double-click them to preview them.
| |
| | |
| Note that the command-line utilities aren’t in your path by default, so you can’t run them like typical commands.
| |
| * On Ubuntu, you’ll find them under /usr/share/fslint/fslint.
| |
| * So, if you wanted to run the entire fslint scan on a single directory, here are the commands you’d run on Ubuntu:
| |
| | |
| <syntaxhighlight lang="bash" highlight="1" line>
| |
| cd /usr/share/fslint/fslint
| |
| ./fslint /path/to/directory
| |
| </syntaxhighlight>
| |
| | |
| This command won’t actually delete anything.
| |
| * It will just print a list of duplicate files — you’re on your own for the rest.
| |
| | |
| === fdupes ===
| |
| [[fdupes]]
| |
| | |
| === dupeGuru===
| |
| [[DupeGuru]]
| |
| | |
| == How To Find And Delete Duplicate Files In Linux ==
| |
| I always backup the configuration files or any old files to somewhere in my hard disk before edit or modify them, so I can restore them from the backup if I accidentally did something wrong.
| |
| * But the problem is I forgot to clean up those files and my hard disk is filled with a lot of duplicate files after a certain period of time.
| |
| * I feel either too lazy to clean the old files or afraid that I may delete an important files.
| |
| * If you’re anything like me and overwhelming with multiple copies of same files in different backup directories, you can find and delete duplicate files using the tools given below in Unix-like operating systems.
| |
| | |
| ; A word of caution
| |
| Please be careful while deleting duplicate files.
| |
| * If you’re not careful, it will lead you to [https://www.ostechnix.com/prevent-files-folders-accidental-deletion-modification-linux/ accidental data loss].
| |
| * I advice you to pay extra attention while using these tools.
| |
| | |
| === Find And Delete Duplicate Files In Linux ===
| |
| For the purpose of this guide, I am going to discuss three utilities namely
| |
| # Rdfind
| |
| # Fdupes
| |
| # FSlint
| |
| | |
| These three utilities are free, open source and works on most Unix-like operating systems.
| |
| | |
| ===== Rdfind =====
| |
| [[Rdfind]]
| |
| | |
| ===== FSlint =====
| |
| '''FSlint''' is yet another duplicate file finder utility that I use from time to time to get rid of the unnecessary duplicate files and free up the disk space in my Linux system.
| |
| * Unlike the other two utilities, FSlint has both GUI and CLI modes.
| |
| * So, it is more user-friendly tool for newbies.
| |
| * FSlint not just finds the duplicates, but also bad symlinks, bad names, temp files, bad IDS, empty directories, and non stripped binaries etc.
| |
| | |
| '''Installing FSlint'''
| |
| | |
| FSlint is available in [https://aur.archlinux.org/packages/fslint/ AUR], so you can install it using any AUR helpers.
| |
| | |
| $ yay -S fslint
| |
| | |
| On Debian, Ubuntu, Linux Mint:
| |
| | |
| $ sudo apt-get install fslint
| |
| | |
| On Fedora:
| |
| | |
| $ sudo dnf install fslint
| |
| | |
| On RHEL, CentOS:
| |
| | |
| $ sudo yum install epel-release
| |
| $ sudo yum install fslint
| |
| | |
| Once it is installed, launch it from menu or application launcher.
| |
| | |
| This is how FSlint GUI looks like.
| |
| | |
| [[Image:Bild6.png|top]]
| |
| | |
| FSlint interface
| |
| | |
| As you can see, the interface of FSlint is user-friendly and self-explanatory.
| |
| * In the '''Search path''' tab, add the path of the directory you want to scan and click '''Find''' button on the lower left corner to find the duplicates.
| |
| * Check the recurse option to recursively search for duplicates in directories and sub-directories.
| |
| * The FSlint will quickly scan the given directory and list out them.
| |
| | |
| [[Image:Bild8.png|top]]fslint GUI
| |
| | |
| From the list, choose the duplicates you want to clean and select any one of them given actions like Save, Delete, Merge and Symlink.
| |
| | |
| In the '''Advanced search parameters''' tab, you can specify the paths to exclude while searching for duplicates.
| |
| | |
| [[Image:Bild7.png|top]]
| |
| | |
| fslint advanced search
| |
| | |
| '''FSlint command line options'''
| |
| | |
| FSlint provides a collection of the following CLI utilities to find duplicates in your filesystem:* '''findup''' — find DUPlicate files
| |
| * '''findnl''' — find Name Lint (problems with filenames)
| |
| * '''findu8''' — find filenames with invalid utf8 encoding
| |
| * '''findbl''' — find Bad Links (various problems with symlinks)
| |
| * '''findsn''' — find Same Name (problems with clashing names)
| |
| * '''finded''' — find Empty Directories
| |
| * '''findid''' — find files with dead user IDs
| |
| * '''findns''' — find Non Stripped executables
| |
| * '''findrs''' — find Redundant Whitespace in files
| |
| * '''findtf''' — find Temporary Files
| |
| * '''findul''' — find possibly Unused Libraries
| |
| * '''zipdir''' — Reclaim wasted space in ext2 directory entries
| |
| | |
| All of these utilities are available under '''/usr/share/fslint/fslint/fslint''' location.
| |
| | |
| For example, to find duplicates in a given directory, do:
| |
| | |
| $ /usr/share/fslint/fslint/findup ~/Downloads/
| |
| | |
| Similarly, to find empty directories, the command would be:
| |
| | |
| $ /usr/share/fslint/fslint/finded ~/Downloads/
| |
| | |
| To get more details on each utility, for example '''findup''', run:
| |
| | |
| $ /usr/share/fslint/fslint/findup --help
| |
| | |
| For more details about FSlint, refer the help section and man pages.
| |
| | |
| $ /usr/share/fslint/fslint/fslint --help
| |
| $ man fslint
| |
| | |
| ; Resources
| |
| * [https://rdfind.pauldreik.se/ Rdfind Website]
| |
| * [https://github.com/pauldreik/rdfind Rdfind GitHub Repository]
| |
| * [https://github.com/adrianlopezroche/fdupes Fdupes GitHub Repository]
| |
| * [http://www.pixelbeat.org/fslint/ FSlint Website]
| |
| * [https://github.com/pixelb/fslint FSlint GitHub Repository]
| |
| | |
| == How to Find Duplicate Files in Linux and Remove Them ==
| |
| ''Brief: FSlint is a great GUI tool to find duplicate files in Linux and remove them.
| |
| * FDUPES also find the files with same name in Linux but in the command line way. ''
| |
| | |
| If you have this habit of downloading everything from the web like me, you will end up having multiple duplicate files.
| |
| * Most often, I can find the same songs or a bunch of images in different directories or end up backing up some files at two different places.
| |
| * It’s a pain locating these duplicate files manually and deleting them to recover the disk space.
| |
| | |
| If you want to save yourself from this pain, there are various Linux applications that will help you in locating these duplicate files and removing them.
| |
| * In this article, we will cover how you can find and remove these files in Ubuntu.
| |
| | |
| ''Note: You should know what you are doing.
| |
| * If you are using a new tool, it’s always better to try it in a virtual directory structure to figure out what it does before taking it to root or home folder.
| |
| * Also, '''it’s always better to [https://itsfoss.com/backup-restore-linux-timeshift/ backup your Linux system]!'''''
| |
| | |
| === FSlint: GUI tool to find and remove duplicate files ===
| |
| FSlint helps you search and remove duplicate files, empty directories or files with incorrect names.
| |
| * It has a command-line as well as GUI mode with a set of tools to perform a variety of tasks.
| |
| | |
| To install FSlint, type the below command in Terminal.
| |
| | |
| sudo apt install fslint
| |
| | |
| Open FSlint from the Dash search.
| |
| | |
| FSlint includes a number of options to choose from.
| |
| * There are options to find duplicate files, installed packages, bad names, name clashes, temp files, empty directories etc.
| |
| * Choose the Search Path and the task which you want to perform from the left panel and click on Find to locate the files.
| |
| * Once done, you can select the files you want to remove and Delete it.
| |
| | |
| You can click on any file directory from the search result to open it if you are not sure and want to double check it before deleting it.
| |
| | |
| You can select '''Advanced search parameters''' where you can define rules to exclude certain file types or exclude directories which you don’t want to search.
| |
| | |
| === FDUPES: CLI tool to find and remove duplicate files ===
| |
| FDUPES is a command line utility to find and remove duplicate files in Linux.
| |
| * It can list out the duplicate files in a particular folder or recursively within a folder.
| |
| * It asks which file to preserve before deletion and the noprompt option lets you delete all the duplicate files keeping the first one without asking you.
| |
| | |
| ==== Installation on Debian / Ubuntu ====
| |
| sudo apt install fdupes
| |
| | |
| ==== Installation on Fedora ====
| |
| dnf install fdupes
| |
| | |
| Once installed, you can search duplicate files using the below command:
| |
| | |
| fdupes /path/to/folder
| |
| | |
| For recursively searching within a folder, use -r option
| |
| | |
| fdupes -r /home
| |
| | |
| This will only list the duplicate files and do not delete them by itself.
| |
| * You can manually delete the duplicate files or use ''-d'' option to delete them.
| |
| | |
| fdupes -d /path/to/folder
| |
| | |
| This won’t delete anything on its own but will display all the duplicate files and gives you an option to either delete files one by one or select a range to delete it.
| |
| * If you want to delete all files without asking and preserving the first one, you can use the noprompt ''-N'' option.
| |
| | |
| In the above screenshot, you can see the ''-d'' command showing all the duplicate files within the folder and asking you to select the file which you want to preserve.
| |
| | |
| ==== Final Words ====
| |
| There are many other ways and tools to find and delete duplicate files in Linux.
| |
| * Personally, I prefer the FDUPES command line tool; it’s simple and takes no resources.
| |
| | |
| How do you deal with the finding and removing duplicate files in your Linux system? Do tell us in the comment section.
| |
| | |
| === Quellen ===
| |
| * [https://itsfoss.com/find-duplicate-files-linux/ https://itsfoss.com/find-duplicate-files-linux/]
| |
| * [https://www.howtogeek.com/201140/how-to-find-and-remove-duplicate-files-on-linux/ https://www.howtogeek.com/201140/how-to-find-and-remove-duplicate-files-on-linux/]
| |
| * [https://www.ostechnix.com/how-to-find-and-delete-duplicate-files-in-linux/ https://www.ostechnix.com/how-to-find-and-delete-duplicate-files-in-linux/]
| |