Summary of python Methods for Converting Strings to json

Recently, I encountered a small problem in my work. If I want to convert string data into dict type, I thought of using json function in the first time. But there are some problems in it

1. Convert through json:

In [1]: import json
In [2]: mes = '{"InsId": 2, "name": "lege-happy", "CreationTime": "2019-04-23T03:18:02Z"}'
In [3]: mes_to_dict = json.loads(mes)
In [4]: print type(mes_to_dict)
<type 'dict'>

The above conversion is no problem, but the format for adding mes is mes = '{"InsId": 2, "name": "lege-happy", "CreationTime": "2019-04-23T03:18:02Z"}' What happens when you use json to convert?

In [5]: import json

In [6]: mes = "{'InsId': 1, 'name': 'lege-error', 'CreationTime': '2019-04-24T03:18:02Z'}"

In [7]: mes_to_dict = json.loads(mes)
---------------------------------------------------------------------------
ValueError                Traceback (most recent call last)
<ipython-input-7-77264851f35b> in <module>()
----> 1 mes_to_dict = json.loads(mes)

/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.pyc in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
  336       parse_int is None and parse_float is None and
  337       parse_constant is None and object_pairs_hook is None and not kw):
--> 338     return _default_decoder.decode(s)
  339   if cls is None:
  340     cls = JSONDecoder

/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.pyc in decode(self, s, _w)
  364
  365     """
--> 366     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  367     end = _w(s, end).end()
  368     if end != len(s):

/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.pyc in raw_decode(self, s, idx)
  380     """
  381     try:
--> 382       obj, end = self.scan_once(s, idx)
  383     except StopIteration:
  384       raise ValueError("No JSON object could be decoded")

ValueError: Expecting property name: line 1 column 2 (char 1)

Therefore, there is one potential limitation to using json for conversion:

Because json syntax stipulates that strings in arrays or objects must use double quotation marks instead of single quotation marks (there is a paragraph on the official website describing "A string is a sequence of zero more Unicode characters, wrapped in double quotes, using backslash escapes"), the above conversion is wrong:

Convert through eval:

In [8]: mes = '{"InsId": 2, "name": "lege-happy", "CreationTime": "2019-04-23T03:18:02Z"}'

In [9]: mes_dict = eval(mes)

In [10]: print type(mes_dict)
<type 'dict'>

In [11]:

In [11]: mes = mes = "{'InsId': 1, 'name': 'lege-error', 'CreationTime': '2019-04-24T03:18:02Z'}"

In [12]: mes_dict = eval(mes)

In [13]: print type(mes_dict)
<type 'dict'>

The above example shows that there is no problem of using json conversion when converting, but we should pay attention to the security problems when using eval, such as:

The input of string type is calculated directly. For example, she will calculate the result directly from the '1 +1' calculation string.

In [14]: value = eval(raw_input('please input a value string:'))
please input a value string:2 + 2

In [15]: value
Out[15]: 4

From the above point of view, eval is very powerful, that is, it can be used as a calculator for type conversion between string and list, tuple and dict! What's more, you can deal with all the strings she can parse, without scruple about the possible consequences! Therefore, behind the strength of eval, there is a huge potential safety hazard! ! !

For example, the user maliciously enters the following string:

open(r'D://filename.txt', 'r').read()
__import__('os').system('dir')
__import__('os').system('rm -rf /etc/*')

Then eval will display the directory structure of your computer, read files, delete files and so on. If it is a more serious operation such as grid, she will do it correctly! ! ! Obviously, this does not meet our needs!
Through literal_eval conversion:

In [20]: import ast
In [21]: mes = '{"InsId": 2, "name": "lege-happy", "CreationTime": "2019-04-23T03:18:02Z"}'
In [22]: mes_dict = ast.literal_eval(mes)
In [23]: print type(mes_dict)
<type 'dict'>
In [24]:
In [24]:
In [24]: mes = mes = "{'InsId': 1, 'name': 'lege-error', 'CreationTime': '2019-04-24T03:18:02Z'}"
In [25]: mes_dict = ast.literal_eval(mes)
In [26]: print type(mes_dict)
<type 'dict'>

Converting with ast.literal_eval has neither the problem of converting with json nor the security problem of converting with eval, so ast.literal_eval is recommended.

Let's take a look at how the official document describes literal_eval:

def literal_eval(node_or_string):
  """
  Safely evaluate an expression node or a string containing a Python
  expression. The string or node provided may only consist of the following
  Python literal structures: strings, numbers, tuples, lists, dicts, booleans,
  and None.
  """

It means that the ast module helps Python applications to deal with abstract syntax parsing. The literal_eval () function under this module will judge whether the content to be calculated is a legal python type after calculation, and if so, it will be operated, otherwise it will not be operated.

For example, the above calculation operation and dangerous operation, if replaced by ast.literal_eval()， Will refuse to execute.

Therefore, it is recommended that when converting dict, it is best to use it when converting strings for safety reasons ast.literal_eval() Function!

Summarize