Find duplicates by size first, then hash. Never the other way
The method that takes seconds instead of hours, plus the folders you must never let a duplicate finder touch.
Duplicates accumulate in a predictable way: the same photos imported twice, a folder copied before a reinstall, an archive extracted beside itself. They are real space, and the tools that find them are also the tools most likely to delete something that only looked like a copy.
The method that makes it fast
Two files of different sizes cannot be identical. That single fact turns an hour of hashing into a few seconds, because only groups of files that already share an exact size need to be read at all.
$root = "$env:USERPROFILE"
# 1. Group by exact size. Only groups with more than one member can contain duplicates.
$candidates = Get-ChildItem $root -Recurse -File -Force -EA SilentlyContinue |
Where-Object Length -gt 1MB |
Group-Object Length | Where-Object Count -gt 1
# 2. Hash only those, and group again on the hash.
$dupes = $candidates | ForEach-Object { $_.Group } |
Get-FileHash -Algorithm SHA256 -EA SilentlyContinue |
Group-Object Hash | Where-Object Count -gt 1
# 3. What the copies are costing, largest first
$dupes | ForEach-Object {
$size = (Get-Item $_.Group[0].Path).Length
[PSCustomObject]@{
MB = [math]::Round($size * ($_.Count - 1) / 1MB, 1)
Copies = $_.Count
Path = $_.Group[0].Path
}
} | Sort-Object MB -Descending | Select-Object -First 25The MB column is what you would recover by keeping one copy, not the total size of the group. That distinction matters when deciding whether the exercise is worth the risk.
Where to point it, and where never to
| Folder | Run it here | Why |
|---|---|---|
| Pictures, Videos, Documents | Yes | Where real duplicates accumulate |
| Downloads | Yes | The same installer fetched repeatedly |
C:\Windows and Program Files | Never | Identical files there are meant to be identical |
node_modules, package caches | Never | Duplication is how the tool works |
| OneDrive or Drive folders | Careful | A deletion syncs everywhere immediately |
The trap: files that are already one copy
A hard link makes one set of bytes appear under two names. A duplicate finder reports it as two identical files, and deleting one frees nothing, because there was only ever one copy. Windows itself is built from these, which is why the component store misleads every size tool: hard links and junctions.
# LinkCount greater than 1 means other names point at the same data
(Get-Item 'C:\path\to\file' -Force).LinkCountDeleting safely
- Sort by recovered space and work on the top ten only. The long tail is rarely worth the risk.
- Keep the copy in the folder you would look in first, not the one with the shortest path.
- Delete to the Recycle Bin, never with a permanent delete, so a mistake is a right click away.
- Empty the bin a week later, once you have used the machine normally.
A photo library is the usual reason people start this, and there the catalogue matters as much as the files: removing a duplicate behind an editor's back leaves a broken entry, which is covered in photo editors on Windows.
Common questions
How do I find duplicate files on Windows?
Group files by exact size first, then hash only the groups with more than one member. Two files of different sizes cannot be identical, so this avoids reading almost everything.
Are duplicate file finders safe?
The finding is safe; the automatic deleting is where damage happens. Review the largest groups by hand, delete to the Recycle Bin, and never run one over Windows, Program Files or package caches.
Why does deleting a duplicate free no space?
Because the two names may point at the same data through a hard link. Check with (Get-Item path -Force).LinkCount: anything above 1 means other names share the same bytes.
Where do duplicates usually come from?
Photos imported twice, a folder copied before a reinstall, archives extracted beside themselves, and the same installer downloaded repeatedly.