An Example of python Cyclic Timing Interrupt Executing a Certain Program

Problem description

Recently, I am writing a crawler. Because the access frequency of a single account is too high, it will be blocked. Therefore, I need to switch my account cyclically after the crawler executes 1 period of time interval

So I wonder if there is a timing interrupt set like a single chip microcomputer, and then define an interrupt entry, so as to execute an interrupt every 1 period of time

Of course, you can't use sleep, so the whole process stops here instead of crawling data

Solution

Use threading Timer, also similar to MCU, reset the timer in the interrupt program, set the interrupt, python example code is as follows

import threading
import time
def change_user():
  print(' This is an interruption , Switch accounts ')
  t = threading.Timer(3, change_user)
  t.start()
# Every passing 3 Second switching 1 Sub-account number 
t = threading.Timer(3, change_user)
t.start()
while True:
  print(' I'm crawling the data ')
  time.sleep(1)

The output looks like this:

 I'm crawling the data 
 I'm crawling the data 
 I'm crawling the data 
 This is an interruption , Switch accounts 
 I'm crawling the data 
 I'm crawling the data 
 I'm crawling the data 
 This is an interruption , Switch accounts 
 I'm crawling the data 

Now the problem is solved!

Think again about sleep alone for this example, and the code is as follows

import threading
import time
def change_user():
  while True:
    print(' This is an interruption , Switch accounts ')
    time.sleep(3)
def spider():
  while True:
    print(' I'm crawling the data ')
    time.sleep(1)
t1 = threading.Thread(target=change_user)
t2 = threading.Thread(target=spider)
t2.start()
t1.start()
t2.join()
t1.join()

Because when two threads execute sleep again, the GIL lock will be released, and the other thread will grab the GIL lock, and it is also possible to switch accounts regularly, but this is inconvenient for practical application, so it is recommended to use the first method