python Implementation logging Dynamic Change Output Log File Name

python is a very easy-to-use scripting language, and log output is even simpler. logging module can output logs to the console by simply setting configuration and attributes, and log information can be written into files by setting file names in basicConfig (), which is simply too simple to be simple.

Recently, I encountered a logging problem in my project. The service program 1 written by python runs directly, Handle 1 task continuously, The key information of each task needs to be output to a file, Easy for maintenance personnel to view, But for a simple and practical logging, Log write file is very simple, because the service program runs continuously, 1 straight to a file record log information some inappropriate, common sense developers all know, long-term log output will lead to the log file is too large, but how in the service run, modify the output file of the log, to the date of the day as the log file name.

Code writing environment: python 3.4. 3

1.logging.basicConfig()

The first thing that comes to mind is to change the logging. basicConfig (filename = logfilename) parameter for the purpose of changing the log file name. Write the following code:

log_fmt = '%(asctime)s %(filename)s[line:%(lineno)d] %(levelname)s %(message)s'
for i in range(1,4):
 filename = str.format('mylog%d.txt' % i)
 logging.basicConfig(format=log_fmt, level=logging.DEBUG, filename=filename)
 logging.debug('This is debug message')
 logging.info('This is info message')
 logging.warning('This is warning message')

The running result did not achieve the expected effect, only the log file mylog1.txt was created, mylog2.txt and mylog3.txt were not created, and the output contents of three consecutive times were written into mylog1.txt. Explain that the logging. basicConfig () setting property is global, and it will no longer take effect after the first setting. Check the official documents, and it is true.

logging.basicConfig(**kwargs)

Does basic configuration for the logging system by creating a StreamHandler with a default Formatter and adding it to the root logger. The functions debug(), info(), warning(), error() and critical() will call basicConfig() automatically if no handlers are defined for the root logger.

This function does nothing if the root logger already has handlers configured for it.

This road is impassable, so we have to use other methods.

2. Handler object

logging supports adding several different types of handler objects, and outputs logs to different targets such as console (logging. StreamHandler) and file (logging. FileHandler).

See document logging. handlers for details of logs supported by logging

By adding a number of handler objects, but at the same time in the console, files at the same time output different levels of log information.

#  Default configuration logging Write to local file 
logging.basicConfig(level=logging.DEBUG,
  format='%(asctime)s %(filename)s[line:%(lineno)d] %(levelname)s %(message)s',
  datefmt='%a, %d %b %Y %H:%M:%S',
  filename='myapp2.log',
  filemode='w')
# Definition 1 A StreamHandler , will INFO Log information level or higher is printed to standard error and added to the current log processing object. 
console = logging.StreamHandler()
console.setLevel(logging.INFO)
formatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s')
console.setFormatter(formatter)
logging.getLogger('').addHandler(console)
logging.debug('This is debug message')
logging.info('This is info message')
logging.warning('This is warning message')

For simplicity and effectiveness, write files using the logging. basicConfig () setting, and add a stream processing (StreamHandler) object that outputs to the console, console, to output logs at the same time. Of course, you can do the same by setting the console output log by default, then creating the file object (logging. FileHandler) and adding the processing collection.

logging. getLogger ('') gets the default root node named 'root'

At the same time, logging provides the method of addHandler (), and there will naturally be a method of managing handler.

Extending the idea of Handler before, we can realize the dynamic management of handler and change log files. Every time you need to change the output file path, use handler management to empty the original logging. FileHandler object and recreate an logging. FileHandler object with a new file name.

#  Default configuration logging Write to local file 
logging.basicConfig(level=logging.DEBUG,
  format='%(asctime)s %(filename)s[line:%(lineno)d] %(levelname)s %(message)s',
  datefmt='%a, %d %b %Y %H:%M:%S',
  filename='myapp2.log',
  filemode='w')
# Definition 1 A StreamHandler , will INFO Log information level or higher is printed to standard error and added to the current log processing object. 
console = logging.StreamHandler()
console.setLevel(logging.INFO)
formatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s')
console.setFormatter(formatter)
logging.getLogger('').addHandler(console)
logging.debug('This is debug message')
logging.info('This is info message')
logging.warning('This is warning message')

Use the for loop to perform three processes, creating log files named mylog1. txt, mylog2. tx, mylog3. txt, and writing the same contents. The execution result does produce files with different names, and the log contents are written correctly.

So far, the function of dynamically changing the log name of the output file has been realized. As for outputting the file name by log, you only need to change the file name parameter that created logging. FileHandler () according to the idea of the above code.

Simple implementation scheme

Browse the contents of the official document logging. handlers1. python has encapsulated a simpler implementation scheme considering the conventional usage scenarios of logs. TimedRotatingFileHandler can realize the basic management of output log files with simple configuration, which is flexible and easy to use. The code is as follows:

import logging, logging.handlers
import time
'''
TimedRotatingFileHandler Constructor declaration 
class logging.handlers.TimedRotatingFileHandler(filename, when='h', interval=1, backupCount=0, encoding=None, delay=False, utc=False, atTime=None)
filename  Log file name prefix 
when  Log name change time unit 
 'S' Seconds
 'M' Minutes
 'H' Hours
 'D' Days
 'W0'-'W6' Weekday (0=Monday)
 'midnight' Roll over at midnight
interval  Interval time refers to waiting N A when Automatically rebuild files after units of time 
backupCount  Keep the maximum number of log files, exceed the limit, delete the first file created; Default value 0 Indicates no restrictions. 
delay  Delay file creation until the 1 Secondary call emit() Method to create a log file 
atTime  At the specified time ( datetime.time Format to create a log file. 
'''
def test_TimedRotatingFileHandler():
 #  Define the log output format 
 fmt_str = '%(asctime)s[level-%(levelname)s][%(name)s]:%(message)s'
 #  Initialization 
 logging.basicConfig()
 #  Create TimedRotatingFileHandler Processing object 
 #  Interval 5(S) Create a new one named myLog%Y%m%d_%H%M%S.log The file of, and 1 Direct occupation myLog Files. 
 fileshandle = logging.handlers.TimedRotatingFileHandler('myLog', when='S', interval=5, backupCount=3)
 #  Set the log file suffix with the current time as the log file suffix name. 
 fileshandle.suffix = "%Y%m%d_%H%M%S.log"
 #  Setting Log Output Levels and Formatting 
 fileshandle.setLevel(logging.DEBUG)
 formatter = logging.Formatter(fmt_str)
 fileshandle.setFormatter(formatter)
 #  Add to the Log Processing Object Collection 
 logging.getLogger('').addHandler(fileshandle)
if __name__ == '__main__':
 test_TimedRotatingFileHandler()
 #  Test in 200s Create files within multiple log files 
 for i in range(0, 100):
 logging.debug("logging.debug")
 logging.info("logging.info")
 logging.warning("logging.warning")
 logging.error("logging.error")
 time.sleep(2)

Added: logging. config. fileConfig configuration log using Python

The logging. config. fileConfig configuration log of Python is realized by parsing conf configuration file. The file logglogging. conf is configured as follows:

[loggers]
keys=root,fileLogger,rotatingFileLogger
 
[handlers]
keys=consoleHandler,fileHandler,rotatingFileHandler
 
[formatters]
keys=simpleFormatter
 
[logger_root]
level=DEBUG
handlers=consoleHandler
 
[logger_fileLogger]
level=DEBUG
#  The logger Configured in the handler
handlers=fileHandler
# logger  Name of 
qualname=fileLogger
propagate=0
 
[logger_rotatingFileLogger]
level=DEBUG
#  Configured like this, rotatingFileLogger Is configured at the same time in consoleHandler,rotatingFileHandler
# consoleHandler  Responsible for outputting logs to the console 
# rotatingFileHandler  Responsible for saving log output to a file 
handlers=consoleHandler,rotatingFileHandler
qualname=rotatingFileLogger
propagate=0
 
[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=simpleFormatter
args=(sys.stdout,)
 
[handler_fileHandler]
class=FileHandler
level=DEBUG
formatter=simpleFormatter
args=('logs/logging.log', 'a')
 
[handler_rotatingFileHandler]
class=handlers.RotatingFileHandler
level=WARNING
formatter=simpleFormatter
args=("logs/rotating_logging.log", "a", 1*1024*1024, 5)
 
[formatter_simpleFormatter]
#format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
format=%(asctime)s - %(module)s - %(thread)d - %(levelname)s : %(message)s
datefmt=%Y-%m-%d %H:%M:%S

The above configuration file mainly includes the following parts:

loggers Configure logger information. Must include an logger named root. When using the parameterless function logging. getLogger (), the default return root is logger. Other custom logger can be called through logging. getLogger ("fileLogger")

handlers Defines and declares handlers information. Commonly used handlers include StreamHandler (only exporting logs to the kong console), FileHandler (saving log information output to a file), RotaRotatingFileHandler (saving log output to a file and setting the size and number of log files for a single log wenj file)

formatter : Format the log

logger_xxx Configure logger declared in loggers one by one, and correspond to 11

handler_xxx Configure handler declared in handlers one by one, and correspond to 11

formatter_xxx Configuring the declared formatterjinx

Code Sample

logging.config.fileConfig( " logging.conf " )
 
#  Output log to console , Gets the root Corresponding logger
console_logger = logging.getLogger()
 
#  Output log to a single file 
file_logger = logging.getLogger(name="fileLogger")
 
# rotatingFileLogger Median forehead consoleHandler Output to the console, rotatingHandler Output log to file 
rotating_logger = logging.getLogger(name="rotatingFileLogger")

Friendly reminder

After the above configuration, the logger corresponding to du can be obtained through logging. getLogger () where log output is needed in the project, and then logger. info ("xxx") jinx can be used for log output.

To configure logs in this way, 1 must call the logging. config. fileConfig ("logging. conf") function in the entry function of the project, because in the logging. conf file, the relative address of the log file is configured in handler, and if called in other code files, the log file will appear in an unexpected location in yixi due to the relative address.