Implementation method of filtering log output in python using Filter

Here's the thing, I wrote an tornado service, In the process, I used logging to record some contents. Since 1 didn't carefully observe tornado's own log management at first, Therefore, I use debug to record ordinary logs and error to record problematic logs. However, when the service runs, it is discovered that the access log level of tornado is info, that is, 20, and debug is 10. Therefore, if I define the log level as debug, it will definitely be output to the log file by default.

But I don't care about accessing logs right now, and since my service may have access all the time, it has a big impact when I search for log information.

What shall I do?

There are several ways

Modify the level when initializing the log

One is to modify the log level defined when I initialize, and set it to be higher than info, so that the log of info will not be recorded again

But this method requires me to modify the previous debug method to a higher level than info, that is, greater than 20. There are a lot of modifications, and I am used to using debug to record, so the cost of modifying is a bit high.

Modify tornado itself

You can modify web. py under tornado in site-packages

def log_request(self, handler):
  """Writes a completed HTTP request to the logs.

  By default writes to the python root logger. To change
  this behavior either subclass Application and override this method,
  or pass a function in the application settings dictionary as
  ``log_function``.
  """
  if "log_function" in self.settings:
    self.settings["log_function"](handler)
    return
  if handler.get_status() < 400:
    log_method = access_log.info
  elif handler.get_status() < 500:
    log_method = access_log.warning
  else:
    log_method = access_log.error
  request_time = 1000.0 * handler.request.request_time()
  log_method("%d %s %.2fms", handler.get_status(),
        handler._request_summary(), request_time)

Among them log_method = access_log.info You can modify it, access_log Defined in log. py,

access_log = logging.getLogger( " tornado.access " )

Here you can define the level of access_log, and then modify the implementation of log_request. It is complicated to think about it, and it is a stupid method to directly modify the library files in site-packes, and many inexplicable problems will occur in future migration.

Using logging. Filter to set filtering rules

In fact, logging has already had a corresponding solution. There is one Filterer class in logging library, and Handler and Logger classes in logging library are inherited from Filter class

There are three methods in the Filter class, addFilter(filter) , removeFilter(filter) 和 filter(record) Method, mainly using addFilter and filter methods.

The addFilter method requires an filter object, where I define a new class and override the filter method,

Filter out logs named tornado. access and log level 20.

class NoParsingFilter(logging.Filter):
  def filter(self, record):
    if record.name == 'tornado.access' and record.levelno == 20:
      return False
    return True

So I add this filter after initializing the logging object

logobj = logging.getLogger('server')
logobj.addFilter(NoParsingFilter())

In this way, after adding a filter log, it will be recorded in its own way at will. record is also a class LogRecord of logging, and the commonly used attributes are as follows name, level, pathname, lineno,msg, args, exc_info

name is the name passed in when the logger object is initialized

level is the level

Which file is pathname outputting this line of logs

lineno is the line number

msg is the log itself

ps: Let's look at the use of filter filter in python

# No. 1 1 The parameters are 1 Return bool Value of 1 General function or lambda Function, first 2 The parameters are 1 Iterable objects 
# Final return 1 Iterable objects, which can be passed through list Obtain 
def is_positive(item):
  return item>0
values = [1,-2,3,-4]
print(filter(is_poditive,values))
a = list(filter(is_positive,values))
print(a)
print(values)
#output
<filter object at 0x000002398A1AB4A8>
[1, 3]
[1, -2, 3, -4]
b = list(filter(lambda item:item>0,values))
print(b)
 #output 
[1,3]

Summarize