Instance of python eliminating duplicate values of a sequence and leaving the order unchanged

  • 2021-01-25 07:47:22
  • OfStack

python eliminates duplicates in a sequence and preserves the original order

1. If you only eliminate duplicate elements, you can simply construct a set


$ python
Python 3.5.2 (default, Nov 23 2017, 16:37:01) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> a = [1 , 3, 5, 1, 8, 1, 5]
>>> set(a)
{8, 1, 3, 5}
>>> 

Use collections or generators: Values must be of type hashable


$ python
Python 3.5.2 (default, Nov 23 2017, 16:37:01) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> def dupe(items):
... seen = set()
... for item in items:
... if item not in seen:
... yield item
... seen.add(item)
... 
>>> a = [1 , 3, 5, 1, 8, 1, 5]
>>> list(dupe(a))
[1, 3, 5, 8]
>>>

3. Eliminate elements that cannot be hashed: such as dictionary types


Python 3.5.2 (default, Nov 23 2017, 16:37:01) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> def rem(items, key=None):
... seen = set()
... for item in items:
... va = item if key is None else key(item)
... if va not in seen:
... yield item
... seen.add(va)
... 
>>> a = [ {'x':1, 'y':2}, {'x':1, 'y':3}, {'x':1, 'y':2}, {'x':2, 'y':4}]>>> list(rem(a, key=lambda d: (d['x'],d['y'])))
[{'y': 2, 'x': 1}, {'y': 3, 'x': 1}, {'y': 4, 'x': 2}]
>>> list(rem(a, key=lambda d: d['x']))
[{'y': 2, 'x': 1}, {'y': 4, 'x': 2}]

>>>>>> #lambda is an anonymous function:
... fuc = lambda : 'haha'
>>> print (f())
>>> print (fuc())
haha
>>> 


Related articles: