Python How to Limit the Size of Output Log

There are many ways to limit the size of the output log, the most elegant of which is to use the rotate mechanism directly, which exists widely in various programming languages, and Python is no exception. Second, you can also use mount to mount a file as the log storage location, because the file size is limited, so the log size is also limited. You can also use ulimit.

RotatingFileHandler

RotatingFileHandler is one of logging. handler, and in python docs, the constructor example of this class is used as follows

class logging.handlers.RotatingFileHandler(filename, mode= ' a', maxBytes=0, backupCount=0, encoding=None, delay=False)

Parameter

filename Output file name, if you use Python 3.6 or later version of Python, filename can be an Path object, Path object allows you to operate the file path more gracefully, which is a new feature of Python.

mode There are "w" and "a". If it is w, the existing log will be overwritten every time the log is output, and the output will start from the first line. If it is a, the previous log will be retained and output after the previous log.

maxBytes Maximum log size in bytes

backupCount If the log size reaches maxBytes, the old log is distinguished from filename by adding a suffix such as. 1 or. 2, and the output begins at line 1 of the new filename file. If 1, there is a backup of up to 1 old log.

encoding Character encoding, commonly used is utf-8

import logging
from logging.handlers import RotatingFileHandler
log = logging.getLogger()
handler = RotatingFileHandler("test", "a", 4096, 2, "utf-8")
log.addHandler(handler)
# More advanced logging Characteristic 
log.setLevel(logging.INFO) # Set the log level for filtering 
formatter = logging.Formatter("%(asctime)s %(levelname)s: %(message)")
log.setFormatter(formatter) # Format log output 

Mount

First create a file and format it as an ext4 file system

dd if=/dev/zero of=./logfs bs=1M count=4
mkfs.ext4 ./logfs

mkdir log
sudo mount -t ext4 ./logfs ./log
python test.py 2>./log/test.log

It should be noted that this method may have permission problems. If the python program has insufficient permissions, you need to change the permissions of the mounted log directory and use sudo chmod 777-R log

Additional:

python control log level output, can be fully limited flow, can control console output, can be used for online configuration, simple log record

# coding=utf-8
import logging
import datetime
import os
CONSOLE = True
LEVEL = "DEBUG"
# log/2020-09/2020-09-27_APPLICATION_NAME.log
APPLICATION_NAME = "APPLICATION_NAME"
#  Generate only GBK Log file of 
class RecordLog:
 def __init__(self):
  """
  1 Controls whether logs are printed in the console 
  2 That controls the level of log output 
  """
  #  Set whether the console prints or not 
  self._console = CONSOLE
  #  Grade dictionary 
  self._level_dict = {'DEBUG':1,'INFO':2,'WARNING':3,'ERROR':4,'CRITICAL':5}
  #  Set the current supported log level 
  self._current_level_num = self._level_dict[LEVEL]
 def get_log_file_name(self,):
  """
   Create log folders, create or call log files , Log storage 
  :return: log/log_folder( Month )/log_file (Daily log ) 
  """
  now = str(datetime.datetime.now())
  log_folder = now[:7]
  log_file = now[:10] + "_" + APPLICATION_NAME + ".log"
  if os.path.exists("log/" + log_folder):
   pass
  else:
   os.makedirs("log/" + log_folder)
  return "log/" + log_folder + "/" + log_file
 def log_content(self, content=' Start recording ',level="DEBUG"):
  """
   Record a log 
  :param content:  Content of the Record 
  :param level: DEBUG, INFO, WARNING, ERROR, CRITICAL
  :return:
  """
  LOG_FILE_NAME = self.get_log_file_name()
  now1 = datetime.datetime.now()
  now_date = now1.strftime("%Y-%m-%d-%H:%M:%S")
  text = " %s %s -- %s" % (level, now_date, content)
  if level in ['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']:
   level_num = self._level_dict[level]
   #  Console output 
   if self._console:
    print(text)
   logger = logging.getLogger()
   fh = logging.FileHandler(filename=LOG_FILE_NAME, encoding="utf-8", mode="a")
   # formatter = logging.Formatter("%(asctime)s - %(name)s-%(levelname)s %(message)s")
   # fh.setFormatter(formatter)
   logger.addHandler(fh)
   #  Setting log levels 
   if "DEBUG" == level and level_num >= self._current_level_num:
    logger.setLevel(logging.DEBUG)
   elif "INFO" == level and level_num >= self._current_level_num:
    logger.setLevel(logging.INFO)
   elif "WARNING" == level and level_num >= self._current_level_num:
    logger.setLevel(logging.WARNING)
   elif "ERROR" == level and level_num >= self._current_level_num:
    logger.setLevel(logging.ERROR)
   else:
    logger.setLevel(logging.CRITICAL)
   logger.info(text)
  else:
   raise Exception(' Log grade parameter transfer error:  %s ;  The correct format is: DEBUG, INFO, WARNING, ERROR, CRITICAL'%level)
if __name__ == '__main__':
 #
 log = RecordLog()
 log.log_content(level='DEBUG')