Paste a list, get it back with each line appearing once — or flip it around and see only the lines that repeated, and how often. It is the job a spreadsheet does in four clicks and a script does in one line, except neither is to hand when you have a list in your clipboard.
The two questions this answers
They look like the same job and they are not. "Give me a clean list" and "tell me what repeated" want opposite outputs, and the second is usually the one that matters.
- Each line once — the deduplicated list, in the original order. What you want before importing, comparing or sending.
- Only the repeated lines — every value that occurred more than once, with a count. This is how you find the duplicate email in an export, the double-booked SKU, the log line that fired forty times.
Why two identical-looking lines can be treated as different
Most surprises with deduplication come from characters you cannot see, and both of the common ones are handled before anything is compared.
- Line endings. A file written on Windows ends every line with a carriage return and a newline. Split on the newline alone and every line keeps an invisible carriage return, so nothing matches anything. All three conventions are recognised here.
- Unicode spelling. The é in café can be one character, or an e followed by a combining accent. They are indistinguishable on screen and unequal to a plain comparison — a common cause of a list that still has duplicates in it after being deduplicated. Both spellings are normalised before comparing.
- Case and padding. Turn on "ignore case" or "ignore surrounding spaces" and Apple, apple and " apple " become one entry. The line you get back is the original, spacing and capitals intact — only the comparison is relaxed.
First or last
When several lines collapse into one, something has to decide which survives. Keeping the first preserves the order you pasted. Keeping the last is what you want when a later row carries the newer version of the same record — a second export appended to the first, say, where the newest state is at the bottom.
Either way the surviving lines stay in their original relative order, so the result still reads like the list you started with rather than a reshuffle.
Counts, and getting them somewhere useful
The summary under the tool reports lines in, lines out, how many were removed, and how many distinct values occurred more than once. That last number is the one worth reading: two thousand lines becoming nineteen hundred tells you little, but "eleven values repeated" tells you how much work is ahead.
Turning on "append the count to each line" produces a tab-separated result, which pastes straight into a spreadsheet as two columns. It is off by default because most of the time the thing you want is a clean list to paste somewhere else.
FAQ
Does this keep the original order?
Yes, unless you ask it to sort. The surviving lines appear in the order they did in your input, which means the result is still recognisably your list. Sorting is a separate option because it destroys that.
How do I find out which entries are duplicated, rather than removing them?
Switch "what to show" to "only the repeated lines". You get one row per value that occurred more than once, and turning on the count option adds how many times each occurred.
Why does it still show duplicates after deduplicating?
Almost always something invisible differs: trailing whitespace, different capitalisation, or two Unicode spellings of the same accented letter. Turn on "ignore surrounding spaces" and "ignore case" — the accent case is normalised for you already.
Is there a size limit?
No fixed one. Everything runs in your browser, so the practical limit is your device’s memory; lists of tens of thousands of lines are comfortable. Very large files are better handled by a spreadsheet or a command line, which can stream rather than hold everything at once.
Is my list uploaded anywhere?
No. The comparison happens in your browser and nothing is transmitted, logged or stored — which is the reason a list of customer emails or internal identifiers is safe to paste here.