Geek asks and answers

Python - quick way to remove duplicates in this list?

You know to flip the list:

a = ["hello", "hello", "hi", "hi", "hey"]

to list:

b = ["hello", "hi", "hey"]

You just do it like this:

b = list(set(a))

He is fast and pythonic.

But what if I need to include this list:

a = [["hello", "hi"], ["hello", "hi"], ["how", "what"], ["hello", "hi"], 
     ["how", "what"]]

to:

b = [["hello", "hi"], ["how", "what"]]

What is the pythonic way to do this?

+5
source share
3 answers
>>> a = [["hello", "hi"], ["hello", "hi"], ["how", "what"], ["hello", "hi"], ["how", "what"]]
>>> set(map(tuple, a))
set([('how', 'what'), ('hello', 'hi')])
+14
source

Another approach that is not so good (although it works for unpackable objects, if they are ordered)

>>> from itertools import groupby
>>> a = [["hello", "hi"], ["hello", "hi"], ["how", "what"], ["hello", "hi"], ["how", "what"]]
>>> [k for k, g in groupby(sorted(a))]
[['hello', 'hi'], ['how', 'what']]
+1
source

If you need to keep the original order, and you have Python 2.7 +

>>> from collections import OrderedDict
>>> a = [["hello", "hi"], ["hello", "hi"], ["how", "what"], ["hello", "hi"], ["how", "what"]]
>>> list(OrderedDict.fromkeys(map(tuple, a)))
[('hello', 'hi'), ('how', 'what')]
0
source

More articles:


All Articles