Python remove duplicates from list of lists | Example code

Use the itertools to remove duplicates from a list of lists in Python. The itertools often offers the fastest and most powerful solutions to this kind of problem.

Example remove duplicates from the lists of list in Python

Simple example code. Before removing duplicates from a list you have to sort the list and don’t forget to import the itertools module.

import itertools

k = [[1, 2], [4], [5, 6, 2], [1, 2], [3], [4]]
k.sort()
res = list(k for k, _ in itertools.groupby(k))
print(res)

Output:

Python remove duplicates from list of lists

Python remove duplicates from nested list

Using sorted() + set() : Examples removing duplicate sublist.

list1 = [[1, 2], [4], [5, 6, 2], [1, 2], [3], [4]]

res = res = list(set(tuple(sorted(sub)) for sub in list1))
print(res)

Output: [(1, 2), (3,), (2, 5, 6), (4,)]

Using set() + map() + sorted()

list1 = [[1, 2], [4], [5, 6, 2], [1, 2], [3], [4]]

res = res = list(set(map(lambda i: tuple(sorted(i)), list1)))
  
print(res)

Output: [(1, 2), (3,), (2, 5, 6), (4,)]

Do comment if you have any doubts and suggestions on this list tutorial.

Note: IDE: PyCharm 2021.1.3 (Community Edition)

Windows 10

Python 3.7

All Python Examples are in Python 3, so Maybe its different from python 2 or upgraded versions.


Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.