REMOVE DUPLICATES IN PYTHON

REMOVE DUPLICATES IN PYTHON

When a collection have repeated image(s), the collection is said to have duplicated. Duplicates can occur in python list, dictionary, tuple but never in set.

Remoing Duplicates from a collection that supports iteratio.

Almost all collections (list, tuple, dictionary) support iteration. The easiest way to remove duplicate from a collection is by using set(). This is because set does not supposed duplicate items.

Example

# create a list

items = [1, 2, 3, 4, 5, 3, 6, 2, 9, 8, 1, 7]

# convert items to set using the built in set function.

set(items)

# convert back to list

list(items)

print(items)

# output

[1, 2, 3, 4, 5, 7, 6, 5 9, 8]

You might have noticed that the order of the items in the list have changed, this is because python set is not ordered hence the order of items are not preserved.

Remove duplicates while preserving the order of items

You can remove duplicates while still maintaining the order of item in your list, this can be done using a for loop.

Example

# create a list with duplicats

items = [1, 2, 3, 4, 5, 3, 6, 2, 9, 8, 1, 7]

# room for our duplicate free list

result = []

# loop

for item in items:

    if item not in result:

        result = result.append(item)

                                

# now you can print result

print(result)

# output

[1, 2, 3, 4, 5, 6, 7, 8, 9]