Detailed Explanation of Python Metaclass and Iterator Generator Case

1.__getattr__ and __getattribute__ magic functions

__getattr__ calls the getattr magic function when the class calls a nonexistent property, and the value item passed in is the nonexistent value of your call.
__getattribute__ is an unconditional priority, so it is best not to use __getattribute__ unless it is a special case.

class User(object):
    def __init__(self, name, info):
        self.name = name
        self.info = info

    def __getattr__(self, item):
        return self.info[item]

ls = User(" Li 4",{"gender":"male"})
print(ls.gender)    

2. Attribute descriptors

Introduction to attribute descriptors
Attribute descriptors are a powerful generic protocol. It is the calling principle of properties, methods, static, methods, class, methods, and super ().

Attribute descriptor protocol
Attribute descriptor is a class that implements a specific protocol. As long as it implements any one of __get__, __set__ and __delete___3 methods, this class is a descriptor. It can realize one way to use the same access logic for multiple attributes. Generally speaking, it is to create an instance as a class attribute of another class.

Attention

If an object defines both __get__ and __set__ methods, it is called a data descriptor (data descriptor).

Objects that define only __get__ methods are called non-data descriptors (non-data descriptor).

Creating descriptors using class methods

· Defines an IntField class as a descriptor class

Create an instance of the IntField class as a property of another User class

class IntField(object):
    def __set__(self, instance, value):
        print("__set__")

    def __get__(self, instance, owner):
        print("__get__")

    def __delete__(self, instance):
        print("__delete__")


class User(object):
    age = IntField()


ls = User()
ls.age         
ls.age = 30     
del ls.age   

Creating descriptors using attribute types

In addition to using a class as an attribute descriptor, property (), which we learned earlier, makes it easy to create available descriptors for any attribute. The syntax for creating property () is property (fget=None, fset=None, fdel=None, doc=None)

Descriptor lookup order

• When it is a data descriptor, __get__ takes precedence over __dict__
• When it is a non-data descriptor, __dict__ takes precedence over __get__

Metaclass

Introduction to Metaclass

Metaclass is actually the class that creates the class

The implementation is as follows:

• Define the function create_class to create the class
Create an User class if the parameter passed to create_class is user

type () creates a metaclass

The first parameter: name represents the class name and string type
The second parameter: bases indicates the inherited object (parent class), tuple type, and comma is used for single element
The third parameter: attr indicates attribute. Here, you can fill in class attribute, class mode and static method in dictionary format. key is attribute name and value is attribute value

def __init__(self, name):
    self.name = name
    print("i am __init__")
    
    
User = type("User", (), {"age":18 , "__init__":__init__})
obj = User("amy")       
print(obj.name)      

metaclass Properties

If __metalass__ = xxx is defined in a class, Python creates the class as a metaclass and controls class creation behavior
For example, the following code, without changing the description of class attributes, specifies that attribute names are accessed in uppercase.

class MyClass(object):
    name = "ls"
mc = MyClass()
print(mc.name)

Python iterator

Iterator refers to a tool for iterating values. Iteration refers to a repeated process, and each repeat is based on the previous result
Iteration provides a general index-independent iterative value selection method

Iterable object

Objects that can be traversed by for loop are iterable objects.
str, list, tuple, dict, set, etc. are all iterative objects.
generator, including generator and generator function with yield.

Determine whether it can be iterated

In addition to judging whether an object is an iterable object by looking at the built-in __iter__ method, we can also use isinstance () to determine whether an object is an Iterable object
isinstance ()- > Used to determine whether an object is of the corresponding type, similar to type ().

from collections import Iterable,Iterator
print(isinstance('abc',Iterable))   # True
print(isinstance([1,2,3,4],Iterable))   # True
print(isinstance(123,Iterable))     # False

Iterator object

Objects with built-in __next__ () method, which can be executed without relying on index values
Objects with a built-in __iter__ () method, executing the iterator's __iter__ () method still results in the iterator itself
It should be noted that an iterable object is not necessarily an iterator

iter()

An object that can be called by the next () function and continuously return the next value is called an iterator: Iterator.
Then we can use the iter () method to turn an iterable object into an iterator.

li = [1,2,3,4]
lis = iter(li)
print(type(lis))    # <class 'list_iterator'>

Note:
The iterator cannot take a value by subscript, but uses __next__ () or next (). However, as long as it exceeds the range, it will directly report an error StopIteration.

print(lis[0])    #  Report an error  not subscriptable
print(lis.__next__())
print(lis.__next__())
print(lis.__next__())
print(lis.__next__())
print(next(lis))
print(next(lis))
print(next(lis))
print(next(lis))

next () can only be called postponingly, not forward.

Difference between iterable objects and iterators
What can be used for for loops are iterative types
EN () acting on ES184are all iterator types
• list, dict, str, etc. are iterative but not iterators because the next () function cannot call them. They can be turned into iterators through the iter () function
The essence of for loop of python is realized by constantly calling next () function

Generator

Generator definition
In Python, the mechanism of one-side loop and one-side calculation is called generator: generator.
Why do you need a generator
All the data in the list is in memory, which will consume a lot of memory if there is massive data.
For example, we only need to access the first few elements, but most of the memory occupied by the latter elements will be wasted.
Then the generator is in the process of the loop according to the algorithm to continuously calculate the subsequent elements, so there is no need to create the entire complete list, thus saving a lot of space.
In a word, when we want to use huge data and want it to take up less space, we use generators.

How to Create a Builder

Generator expression
The generator expression comes from a combination of iteration and list parsing. The generator is similar to list parsing, but it uses () instead of [].

g = (x for x in range(5))
print(g)       # generator object
print(next(g))
print(next(g))
print(next(g))
print(next(g))
print(next(g))
#  Excess error reporting 
print(next(g))
for i in g:
    print(i)

Generator function
When a function contains the yield keyword, the function is no longer a normal function, but an generator. Calling a function creates a generator object. It works by repeatedly calling the next () or __next__ () methods until an exception is caught.
For example:
To realize Fibonacci sequence, except the first and second numbers, any one number can be obtained by adding the first two numbers:
1, 1, 2, 3, 5, 8, 12, 21, 34 …

def createNums():
    print("-----func start-----")
    a,b = 0,1
    for i in range(5):
        # print(b)
        print("--1--")
        yield b
        print("--2--")
        a,b = b,a+b
        print("--3--")
    print("-----func end-----")
    
g = createNums()
print(next(g))  
print(next(g))  
print(next(g))
print(next(g))
print(next(g))

Note:

yield returns a value, and remember the location of this return value. The next time you encounter an next () call, the code starts to execute from the next statement of yield. Unlike return, return also returns 1 value, but ends the function directly.

Iterators and generators

The generator can do everything the iterator can do

And because the generator automatically creates the iter () and next () methods, the generator is concise and efficient.

Read large files

File 300G, special file, 1 line separator {}

def readlines(f,newline):
    buf = ""
    while True:
        while newline in buf:
            pos = buf.index(newline)
            yield buf[:pos]
            buf = buf[pos + len(newline):]
        chunk = f.read(4096*10)
        if not chunk:
            yield buf
            break
        buf += chunk
with open('demo.txt') as f:
    for line in readlines(f,"{|}"):
        print(line)