Detailed explanation of Python for Encoder of special types in JSON

When Python processes JSON data, dumps function is often used. When there are special types in JSON data, it is often a headache, because such an error is often reported.

Custom encoding class

#!/usr/bin/env python
# -*- coding:utf-8 -*-
# Author: wxnacy(wxnacy@gmail.com)

import json
from datetime import datetime

USER_DATA = dict(
  id = 1, name = 'wxnacy', ts = datetime.now()
)
print(json.dumps(USER_DATA))

Traceback (most recent call last):
 File "/Users/wxnacy/PycharmProjects/study/python/office_module/json_demo/dumps.py", line 74, in <module>
  dumps_encoder()
 File "/Users/wxnacy/PycharmProjects/study/python/office_module/json_demo/dumps.py", line 68, in dumps_encoder
  print(json.dumps(USER_DATA))
 File "/Users/wxnacy/.pyenv/versions/3.6.0/Python.framework/Versions/3.6/lib/python3.6/json/__init__.py", line 231, in dumps
  return _default_encoder.encode(obj)
 File "/Users/wxnacy/.pyenv/versions/3.6.0/Python.framework/Versions/3.6/lib/python3.6/json/encoder.py", line 199, in encode
  chunks = self.iterencode(o, _one_shot=True)
 File "/Users/wxnacy/.pyenv/versions/3.6.0/Python.framework/Versions/3.6/lib/python3.6/json/encoder.py", line 257, in iterencode
  return _iterencode(o, 0)
 File "/Users/wxnacy/.pyenv/versions/3.6.0/Python.framework/Versions/3.6/lib/python3.6/json/encoder.py", line 180, in default
  o.__class__.__name__)
TypeError: Object of type 'datetime' is not JSON serializable

The reason is that dumps function doesn't know how to handle datetime object. By default, json module uses json. JSONEncoder class for encoding. At this time, we need to customize the encoding class under 1.

#!/usr/bin/env python
# -*- coding:utf-8 -*-
# Author: wxnacy(wxnacy@gmail.com)

class CustomEncoder(json.JSONEncoder):
  def default(self, x):
    if isinstance(x, datetime):
      return int(x.timestamp())
    return super().default(self, x)

Defines the encoding class CustomEncoder and overrides the instance's default function to deal with special types, while the remaining types continue to use the parsing of the parent class.

#!/usr/bin/env python
# -*- coding:utf-8 -*-
# Author: wxnacy(wxnacy@gmail.com)

import json
from datetime import datetime

class CustomEncoder(json.JSONEncoder):
  def default(self, x):
    if isinstance(x, datetime):
      return int(x.timestamp())
    return super().default(self, x)

USER_DATA = dict(
  id = 1, name = 'wxnacy', ts = datetime.now()
)
print(json.dumps(USER_DATA, cls=CustomEncoder))
# {"id": 1, "name": "wxnacy", "ts": 1562938926}

Finally, you can integrate and pass the class into the dumps function using the cls parameter.

Objects can be transcoded using the encode function of the CustomEncoder instance

#!/usr/bin/env python
# -*- coding:utf-8 -*-
# Author: wxnacy(wxnacy@gmail.com)
print(CustomEncoder().encode(datetime.now()))
# 1562939035

In the parent source code, all the coding logic is in the encode function, default is only responsible for throwing TypeError exception, this is the source of the article to start reporting errors.

#!/usr/bin/env python
# -*- coding:utf-8 -*-
# Author: wxnacy(wxnacy@gmail.com)

def default(self, o):
  """Implement this method in a subclass such that it returns
  a serializable object for ``o``, or calls the base implementation
  (to raise a ``TypeError``).

  For example, to support arbitrary iterators, you could
  implement default like this::

    def default(self, o):
      try:
        iterable = iter(o)
      except TypeError:
        pass
      else:
        return list(iterable)
      # Let the base class default method raise the TypeError
      return JSONEncoder.default(self, o)

  """
  raise TypeError(f'Object of type {o.__class__.__name__} '
          f'is not JSON serializable')

def encode(self, o):
  """Return a JSON string representation of a Python data structure.

  >>> from json.encoder import JSONEncoder
  >>> JSONEncoder().encode({"foo": ["bar", "baz"]})
  '{"foo": ["bar", "baz"]}'

  """
  # This is for extremely simple cases and benchmarks.
  if isinstance(o, str):
    if self.ensure_ascii:
      return encode_basestring_ascii(o)
    else:
      return encode_basestring(o)
  # This doesn't pass the iterator directly to ''.join() because the
  # exceptions aren't as detailed. The list call should be roughly
  # equivalent to the PySequence_Fast that ''.join() would do.
  chunks = self.iterencode(o, _one_shot=True)
  if not isinstance(chunks, (list, tuple)):
    chunks = list(chunks)
  return ''.join(chunks)

Single dispatch decorator processing object

If CustomEncoder handles many kinds of objects, it is necessary to write multiple if elif else to distinguish them. This is not impossible, but it is not elegant enough, and it is not enough for pythonic

Different treatments are made according to different types of objects. There happens to be a decorator that can do this, which is the single dispatch function functools. singledispatch

#!/usr/bin/env python
# -*- coding:utf-8 -*-
# Author: wxnacy(wxnacy@gmail.com)

from datetime import datetime
from datetime import date
from functools import singledispatch

class CustomEncoder(json.JSONEncoder):
  def default(self, x):
    try:
      return encode(x)
    except TypeError:
      return super().default(self, x)

@singledispatch       # 1
def encode(x):
  raise TypeError('Unencode type')

@encode.register(datetime) # 2
def _(x):
  return int(x.timestamp())

@encode.register(date)
def _(x):
  return x.isoformat()

print(json.dumps(dict(dt = datetime.now(), d = date.today()), cls=CustomEncoder))
# {"dt": 1562940781, "d": "2019-07-12"}

1 decorates the encode function with @ singledispatch, which is the default type he handles. At the same time, add a decorator constructor variable to him.
2 ` @ encode. register () is a decorator constructor that takes the type of object to be processed as an argument. Functions decorated with it don't need names, just _ ` instead.

Finally, json can also be used on the command line

$ echo '{"json": "obj"}' | python -m json.tool
{
  "json": "obj"
}

Reference link

json