Remove Duplicate Lines
Paste a list and keep only unique lines, in the original order or sorted, with a count of what was removed.
Processed locally in your browser/ Your data stays in your browser.
What is Remove Duplicate Lines?
Duplicate-line removal turns a list with repeats into a list of distinct entries: email lists, keyword lists, log excerpts, URLs, IDs. Each line is compared as a whole; the first (or last) occurrence is kept.
You decide how strict the comparison is: case-sensitive or not, with or without trimming spaces around each line, and whether empty lines count as duplicates. The notes tell you how many lines were removed and how many remain.
How does it work?
- Paste your list, one item per line.
- Choose case sensitivity, trimming, and whether to keep the first or the last occurrence of each duplicate.
- Optionally keep empty lines untouched or sort the result A → Z, then copy the unique lines.
Common use cases
- Deduplicating a mailing list or a list of domains before import.
- Cleaning a keyword export from an SEO tool that contains repeated phrases.
- Reducing a log excerpt to the distinct messages.
- Merging two lists pasted together and keeping every entry once.
Examples
Try this input in the tool above:
apple banana Apple cherry banana cherry date
apple banana Apple cherry date
Privacy
Remove Duplicate Lines runs entirely in your browser. The text or files you provide are processed on your device and are not uploaded, logged or stored on our servers.
Limitations
Lines are compared exactly (after the optional trim and case folding). "foo" and "foo." are different lines, and invisible differences such as non-breaking spaces make lines distinct.
Frequently asked questions
Which occurrence is kept?
By default the first one, so the original order is preserved. Choose "last occurrence" to keep the final copy of each line instead.
Why are two identical-looking lines not merged?
They probably differ by trailing spaces, a tab, a non-breaking space or letter case. Turn on trimming or case-insensitive mode, or clean the text first.
Related tools
Sort Lines
Order any list one item per line: A to Z, Z to A, natural, numeric or by length.
Text Tools
Remove Empty Lines
Strip blank lines from pasted text in one click, or keep a single blank line between paragraphs.
Text Tools
Trim Lines
Remove the spaces and tabs at the start and end of each line of a list or block of text.
Text Tools
Reverse Lines
Flip a list upside down: the last line becomes the first, the second-to-last becomes the second, and so on.
Text Tools
Shuffle Lines
Randomize the order of any list with one click, using a fair shuffle that every arrangement is equally likely to result from.
Text Tools
CSV Deduplicator
Find and remove duplicate rows in a CSV, comparing whole rows or only the columns you choose.
JSON & Data
Text Diff – Compare Two Texts
Paste an original and a modified text to see exactly what was added and removed, at the level of detail you choose.
Text Tools
Line Counter
Paste a text, a log or a list and get the number of lines, how many are empty or repeated, and which line is the longest.
Text Tools