Dividing Log Records by Time under python Multi Process

python multi-process log record by time division, for your reference, the specific content is as follows

Principle: Custom log handler inherits TimedRotatingFileHandler and overrides computeRollover and doRollover functions. The purpose of rewriting computeRollover is to divide the log by the whole minute/hour/day, such as dividing by day, from 2018-04-100 00:00 to 2018-04-11 00:00, which is a semi-closed and semi-open interval, and it is not the original intention: starting from the log creation time or the current time to this time tomorrow.

The code is as follows:

#!/usr/bin/env python
# encoding: utf-8

""" Custom log processing class """


import os
import time
from logging.handlers import TimedRotatingFileHandler


class MyLoggingHandler(TimedRotatingFileHandler):

  def __init__(self, filename, when='h', interval=1, backupCount=0, encoding=None, delay=False, utc=False, atTime=None):
    TimedRotatingFileHandler.__init__(self, filename, when=when, interval=interval, backupCount=backupCount, encoding=encoding, delay=delay, utc=utc, atTime=atTime)

  def computeRollover(self, currentTime):
    #  Round the time 
    t_str = time.strftime(self.suffix, time.localtime(currentTime))
    t = time.mktime(time.strptime(t_str, self.suffix))
    return TimedRotatingFileHandler.computeRollover(self, t)

  def doRollover(self):
    """
    do a rollover; in this case, a date/time stamp is appended to the filename
    when the rollover happens. However, you want the file to be named for the
    start of the interval, not the current time. If there is a backup count,
    then we have to get a list of matching filenames, sort them and remove
    the one with the oldest suffix.
    """
    if self.stream:
      self.stream.close()
      self.stream = None
    # get the time that this sequence started at and make it a TimeTuple
    currentTime = int(time.time())
    dstNow = time.localtime(currentTime)[-1]
    t = self.rolloverAt - self.interval
    if self.utc:
      timeTuple = time.gmtime(t)
    else:
      timeTuple = time.localtime(t)
      dstThen = timeTuple[-1]
      if dstNow != dstThen:
        if dstNow:
          addend = 3600
        else:
          addend = -3600
        timeTuple = time.localtime(t + addend)
    dfn = self.rotation_filename(self.baseFilename + "." +
                   time.strftime(self.suffix, timeTuple))
    #  Modified content -- Begin 
    #  Under multiple processes, if you find that dfn Already exists, it means that other processes have cut the log file according to time, just reopen the new log file and write the current log; 
    #  If dfn If it does not exist, rename the current log file and open a new one 
    if not os.path.exists(dfn):
      try:
        self.rotate(self.baseFilename, dfn)
      except FileNotFoundError:
        #  There will be an exception: the log file was not found, because other processes renamed the log file, just ignore it, and the current log will not be lost 
        pass
    #  Modified content -- End 
    #  The original content is as follows: 
    """
    if os.path.exists(dfn):
      os.remove(dfn)
    self.rotate(self.baseFilename, dfn)
    """

    if self.backupCount > 0:
      for s in self.getFilesToDelete():
        os.remove(s)
    if not self.delay:
      self.stream = self._open()
    newRolloverAt = self.computeRollover(currentTime)
    while newRolloverAt <= currentTime:
      newRolloverAt = newRolloverAt + self.interval
    # If DST changes and midnight or weekly rollover, adjust for this.
    if (self.when == 'MIDNIGHT' or self.when.startswith('W')) and not self.utc:
      dstAtRollover = time.localtime(newRolloverAt)[-1]
      if dstNow != dstAtRollover:
        if not dstNow: # DST kicks in before next rollover, so we need to deduct an hour
          addend = -3600
        else:      # DST bows out before next rollover, so we need to add an hour
          addend = 3600
        newRolloverAt += addend
    self.rolloverAt = newRolloverAt

Description

If there is anything wrong with the first revision, please point out it, and I would be grateful.