Use Python to get all kinds of information about the Linux system
- 2020-04-02 13:51:16
- OfStack
In this article, we'll explore using Python programming language tools to retrieve information about Linux systems. Leave you.
Which Python version ?
When I refer to Python, I am referring to CPython 2(2.7 to be exact). I will explicitly remind you that the same code does not work on CPython 3 (3.3) and provide an alternate code that explains the differences. Make sure you have CPython installed, type python or python3 return on the terminal, and you should be able to see python prompt (prompt) on the terminal.
Note that all programs have #! On their first line. /usr/bin/env/python, that is, we want the python interpreter to execute these scripts. So, if you want your script to be executable, use chmod +x your-script.py, so you can execute it using./your-script.py (you'll see this in this article).
Explore platform module
The platform module is in the standard library, and it has a lot of functions that run and we get a lot of system information. Let's run the Python interpreter to explore some of these functions, starting with the platform.uname() function:
>>> import platform
>>> platform.uname()
('Linux', 'fedora.echorand', '3.7.4-204.fc18.x86_64', '#1 SMP Wed Jan 23 16:44:29 UTC 2013', 'x86_64')
If you already know the uname command on Linux, you will recognize this function as an interface to the command. On Python 2, it returns a tuple containing the system type (or kernel version), hostname, version, release, machine hardware, and processor information. You can use subscripts to access individual properties, like this:
>>> platform.uname()[0]
'Linux'
On Python 3, this function returns a named tuple:
>>> platform.uname()
uname_result(system='Linux', node='fedora.echorand',
release='3.7.4-204.fc18.x86_64', version='#1 SMP Wed Jan 23 16:44:29
UTC 2013', machine='x86_64', processor='x86_64')
Because the result is a named tuple, it's easy to specify a specific attribute by name, rather than having to remember the subscript, like this:
>>> platform.uname().system
'Linux'
The platform module also has some direct interfaces to the above attributes, like this:
>>> platform.system()
'Linux'
>>> platform.release()
'3.7.4-204.fc18.x86_64'
The linux_distribution() function returns details about your Linux distribution. For example, on Fedora 18, this command returns the following information:
>>> platform.linux_distribution()
('Fedora', '18', 'Spherical Cow')
This return contains the release name, version, and code tuple. The published version of a particular supported version of Python can be obtained by the value shown in _supported_dists.
>>> platform._supported_dists
('SuSE', 'debian', 'fedora', 'redhat', 'centos', 'mandrake',
'mandriva', 'rocks', 'slackware', 'yellowdog', 'gentoo',
'UnitedLinux', 'turbolinux')
If your Linux distribution is not one of them (or a derivative of one of them). Then you are likely to call the above function without seeing any useful information.
The last function of the platform module, we'll look at the architecture() function. When you call this function without arguments, it returns a tuple containing schema bits and python executable formats, like this:
>>> platform.architecture()
('64bit', 'ELF')
On a 32-bit system, you will see:
>>> platform.architecture()
('32bit', 'ELF')
If you specify any other executable on the system, you will get similar results, like this:
>>> platform.architecture(executable='/usr/bin/ls')
('64bit', 'ELF')
Encourage other functions other than these to explore the platform module to find out what version of Python you are running. If you want to know how this module gets this information, you can take a closer look at the Lib/platform.py file in the PYthon source directory.
OS and sys modules can also get some system properties, such as native byte order. Next, we'll go beyond the Python standard library module to explore ways to make it possible to access information on Linux systems through the proc and sysfs file systems. Note that access to information through the file system will vary from hardware architecture to hardware architecture. So keep in mind as you read this article or write scripts that you can try to get information from these files.
Get CPU information
The /proc/cpuinfo file contains information about your system's processor unit. For example, here's what the python version of the Linux command cat /proc/cpuinfo does:
#! /usr/bin/env python
""" print out the /proc/cpuinfo
file
"""
from __future__ import print_function
with open('/proc/cpuinfo') as f:
for line in f:
print(line.rstrip('n'))
When you execute this program using Python 2 or Python 3, you will see all /proc/cpuinfo on the screen (in the above program, the rstrip() method is used to remove the newline at the end of each line).
The pattern of using the startwith() string method to display your processor unit is listed in the code below.
#! /usr/bin/env python
""" Print the model of your
processing units
"""
from __future__ import print_function
with open('/proc/cpuinfo') as f:
for line in f:
# Ignore the blank line separating the information between
# details about two processing units
if line.strip():
if line.rstrip('n').startswith('model name'):
model_name = line.rstrip('n').split(':')[1]
print(model_name)
When you run this program, you should see the pattern name of each of your processor units. For example, here is what I see on my computer.
Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz
Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz
Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz
Intel(R) Core(TM) i7-3520M CPU @ 2.90GHz
So far, we've had two ways to figure out the architecture of the system we're using. Technically true, the two methods actually report the kernel architecture of your system running, so if your computer is 64-bit but running a 32-bit kernel, then the above method will still show up as a 32-bit architecture. You can find the true architecture of your computer by looking for the lm flag from the flags listed in /proc/cpuinfo. The lm flag represents the long pattern and is only displayed for 64-bit architectures. The following program will show you how to do this:
#! /usr/bin/env python
""" Find the real bit architecture
"""
from __future__ import print_function
with open('/proc/cpuinfo') as f:
for line in f:
# Ignore the blank line separating the information between
# details about two processing units
if line.strip():
if line.rstrip('n').startswith('flags')
or line.rstrip('n').startswith('Features'):
if 'lm' in line.rstrip('n').split():
print('64-bit')
else:
print('32-bit')
As we have seen, it is possible to read the /proc/cpuinfo file and use simple text processing techniques to get the data we are looking for. To make better use of this data for other programs, a better idea is to make the contents of /proc/cpuinfo a standard data structure, such as a dictionary. The note is simple: if you look at the contents of the file, you will see that there are several key-value pairs for each processor unit (in the previous example, we printed the model name of each processor, that is, the model name is the keyword). Information about different processor units can be separated by blank lines. It is easy to construct a dictionary data structure that contains the keywords for each processor unit. For each keyword, the value for the processor unit is in the /proc/cpuinfo file. The following code will show you how to do this.
#!/usr/bin/env/ python
"""
/proc/cpuinfo as a Python dict
"""
from __future__ import print_function
from collections import OrderedDict
import pprint
def cpuinfo():
''' Return the information in /proc/cpuinfo
as a dictionary in the following format:
cpu_info['proc0']={...}
cpu_info['proc1']={...}
'''
cpuinfo=OrderedDict()
procinfo=OrderedDict()
nprocs = 0
with open('/proc/cpuinfo') as f:
for line in f:
if not line.strip():
# end of one processor
cpuinfo['proc%s' % nprocs] = procinfo
nprocs=nprocs+1
# Reset
procinfo=OrderedDict()
else:
if len(line.split(':')) == 2:
procinfo[line.split(':')[0].strip()] = line.split(':')[1].strip()
else:
procinfo[line.split(':')[0].strip()] = ''
return cpuinfo
if __name__=='__main__':
cpuinfo = cpuinfo()
for processor in cpuinfo.keys():
print(cpuinfo[processor]['model name'])
This code USES OrderedDict(ordered dictionary) instead of a regular dictionary, and can be stored in an ordered file using keys. So, data from the first processor unit is followed by data from the second processor unit, and so on. You can use a filter to filter the information you are looking for (as demonstrated in the if/s block == 's block). The above program prints the model name of each processor unit after each execution (as indicated by the cpuinfo[processor]['model name'] statement)
The same code at the page code block index 12
Get memory information
Similar to /proc/cpuinfo, file /proc/meminfo contains information about your computer's main memory. The following program creates a dictionary that is populated with the contents of the file.
#!/usr/bin/env python
from __future__ import print_function
from collections import OrderedDict
def meminfo():
''' Return the information in /proc/meminfo
as a dictionary '''
meminfo=OrderedDict()
with open('/proc/meminfo') as f:
for line in f:
meminfo[line.split(':')[0]] = line.split(':')[1].strip()
return meminfo
if __name__=='__main__':
#print(meminfo())
meminfo = meminfo()
print('Total memory: {0}'.format(meminfo['MemTotal']))
print('Free memory: {0}'.format(meminfo['MemFree']))
As before, with its keywords you can access any of the specified information you are query-expressed in the if/s/s == s/s block. When you execute this program, you should see output similar to the following:
Total memory: 7897012 kB
Free memory: 249508 kB
Network statistics
Next, we'll explore the networking devices of our computer systems. We will get the network interface of the system and the information to send and receive data through the system when it restarts. The /proc/net/dev file makes this information available. If you check the contents of the file, you will notice that the first one or two lines contain header information, etc. The first column of the file is the network interface name, and the second and third columns show the number of bytes received and sent (for example, total number of bytes sent, number of packets, errors, etc.). What we are interested in here is that the different network devices extract the total sending and receiving data. The following code shows how to extract this information from the /proc/net/dev file.
#!/usr/bin/env python
from __future__ import print_function
from collections import namedtuple
def netdevs():
''' RX and TX bytes for each of the network devices '''
with open('/proc/net/dev') as f:
net_dump = f.readlines()
device_data={}
data = namedtuple('data',['rx','tx'])
for line in net_dump[2:]:
line = line.split(':')
if line[0].strip() != 'lo':
device_data[line[0].strip()] = data(float(line[1].split()[0])/(1024.0*1024.0),
float(line[1].split()[8])/(1024.0*1024.0))
return device_data
if __name__=='__main__':
netdevs = netdevs()
for dev in netdevs.keys():
print('{0}: {1} MiB {2} MiB'.format(dev, netdevs[dev].rx, netdevs[dev].tx))
When you run the above program, the following output displays the total amount of data received and sent by network devices in megabytes since your recent reboot.
em1: 0.0 MiB 0.0 MiB
wlan0: 2651.40951061 MiB 183.173976898 MiB
You can use persistent data storage mechanisms to connect and write your own data usage monitor.
Process information
The /proc directory contains all running process directories. The directory names are the same as the process identifiers. So, if you walk through the directories in the /proc directory that use Numbers as their names, you'll get a list of all the processes that are currently running. In the code below, the process_list() function returns a list of identifiers for all currently running processes. When you execute the program, the length of the list is the total number of processes running on the system.
#!/usr/bin/env python
"""
List of all process IDs currently active
"""
from __future__ import print_function
import os
def process_list():
pids = []
for subdir in os.listdir('/proc'):
if subdir.isdigit():
pids.append(subdir)
return pids
if __name__=='__main__':
pids = process_list()
print('Total number of running processes:: {0}'.format(len(pids)))
The above program will display output similar to the following when executed:
Total number of running processes:: 229Each process directory contains some other files and directories, such as the invocation of a process command, the Shared library it is using, and others.
Piece of equipment
The next program lists all the block devices by reading the sysfs virtual file system. Block devices in your system can be found in the /sys/block directory. So there might be directories like /sys/block/sda, /sys/block/ SDB, etc. To get all of these devices, we used regular expressions to scan the /sys/block directory and extract the block devices of interest.
#!/usr/bin/env python
"""
Read block device data from sysfs
"""
from __future__ import print_function
import glob
import re
import os
# Add any other device pattern to read from
dev_pattern = ['sd.*','mmcblk*']
def size(device):
nr_sectors = open(device+'/size').read().rstrip('n')
sect_size = open(device+'/queue/hw_sector_size').read().rstrip('n')
# The sect_size is in bytes, so we convert it to GiB and then send it back
return (float(nr_sectors)*float(sect_size))/(1024.0*1024.0*1024.0)
def detect_devs():
for device in glob.glob('/sys/block/*'):
for pattern in dev_pattern:
if re.compile(pattern).match(os.path.basename(device)):
print('Device:: {0}, Size:: {1} GiB'.format(device, size(device)))
if __name__=='__main__':
detect_devs()
If you run the program, you will see the following similar output:
Device:: /sys/block/sda, Size:: 465.761741638 GiB
Device:: /sys/block/mmcblk0, Size:: 3.70703125 GiB
When I run the program, there is an SD memory card plugged into the computer, so you can see that the program detects it. You can also extend the program to identify other block devices (such as virtual hard drives).
Create a command line utility
Command line tools are ubiquitous in Linux [@lesus note: someone once said: Linux is a mess without the command line]. , which allows you to specify command-line arguments to customize the default behavior of your program. The argparse module provides a similar interface to the Linux command-line utility. The following code shows how the program gets all the users on the system and the login shell that prints them (using the PWD standard library module) :
#!/usr/bin/env python
"""
Print all the users and their login shells
"""
from __future__ import print_function
import pwd
# Get the users from /etc/passwd
def getusers():
users = pwd.getpwall()
for user in users:
print('{0}:{1}'.format(user.pw_name, user.pw_shell))
if __name__=='__main__':
getusers()
When the program runs, it prints all the users on the system and their login shell names.
Now, users of your desired application can choose whether or not they want to see system users (like daemon, apache). We extended the previous code to implement this feature for the first time using the argparse module, as shown below.
#!/usr/bin/env python
"""
Utility to play around with users and passwords on a Linux system
"""
from __future__ import print_function
import pwd
import argparse
import os
def read_login_defs():
uid_min = None
uid_max = None
if os.path.exists('/etc/login.defs'):
with open('/etc/login.defs') as f:
login_data = f.readlines()
for line in login_data:
if line.startswith('UID_MIN'):
uid_min = int(line.split()[1].strip())
if line.startswith('UID_MAX'):
uid_max = int(line.split()[1].strip())
return uid_min, uid_max
# Get the users from /etc/passwd
def getusers(no_system=False):
uid_min, uid_max = read_login_defs()
if uid_min is None:
uid_min = 1000
if uid_max is None:
uid_max = 60000
users = pwd.getpwall()
for user in users:
if no_system:
if user.pw_uid >= uid_min and user.pw_uid <= uid_max:
print('{0}:{1}'.format(user.pw_name, user.pw_shell))
else:
print('{0}:{1}'.format(user.pw_name, user.pw_shell))
if __name__=='__main__':
parser = argparse.ArgumentParser(description='User/Password Utility')
parser.add_argument('--no-system', action='store_true',dest='no_system',
default = False, help='Specify to omit system users')
args = parser.parse_args()
getusers(args.no_system)
Use the --help option to execute the program above and you'll see friendly help messages: options and what they do.
$ ./getusers.py --help
usage: getusers.py [-h] [--no-system]
User/Password Utility
optional arguments:
-h, --help show this help message and exit
--no-system Specify to omit system users
An example used by the above program is shown below:
$ ./getusers.py --no-system
gene:/bin/bash
The program whines when you pass in an illegal parameter
$ ./getusers.py --param
usage: getusers.py [-h] [--no-system]
getusers.py: error: unrecognized arguments: --param
In the program above, we simply learned how to use the argparse module. The parser = argparse.argumentparser (description="User/Password Utility") statement creates a ArgumentParser object with an optional description of what the program does,
Then, we add parameters. We want the program to be able to recognize the following statement, add_argument().
Parser.add_argument ('--no-system', action='store_true', dest='no_system', default = False, help='Specify to omit system users') The first method's argument is when the system calls the program, the program USES the name of the argument to be supplied, and the next argument acton = store_true indicates that it is a Boolean choice. That is, it really or really affects some behavior of the program. Dest is a customizable parameter whose value can be supplied to the program. If this value is not provided by the user, this value defaults to false. The last parameter program displays the help information. Finally, the argument is parsed through the args=parser.parse_args() method. Once the parse method is done, the value of the user option can be fetched through the corresponding syntax parameter option_dest, which is the destination variable you specify when you configure the parameter. The getusers(args.no_system) statement will call back the getusers() method using the value of the user-supplied parameter.
The following program shows how to specify options for non-boolean types. This program is a rewrite of the sixth program, with an option to specify which network device you are interested in.
#!/usr/bin/env python
from __future__ import print_function
from collections import namedtuple
import argparse
def netdevs(iface=None):
''' RX and TX bytes for each of the network devices '''
with open('/proc/net/dev') as f:
net_dump = f.readlines()
device_data={}
data = namedtuple('data',['rx','tx'])
for line in net_dump[2:]:
line = line.split(':')
if not iface:
if line[0].strip() != 'lo':
device_data[line[0].strip()] = data(float(line[1].split()[0])/(1024.0*1024.0),
float(line[1].split()[8])/(1024.0*1024.0))
else:
if line[0].strip() == iface:
device_data[line[0].strip()] = data(float(line[1].split()[0])/(1024.0*1024.0),
float(line[1].split()[8])/(1024.0*1024.0))
return device_data
if __name__=='__main__':
parser = argparse.ArgumentParser(description='Network Interface Usage Monitor')
parser.add_argument('-i','--interface', dest='iface',
help='Network interface')
args = parser.parse_args()
netdevs = netdevs(iface = args.iface)
for dev in netdevs.keys():
print('{0}: {1} MiB {2} MiB'.format(dev, netdevs[dev].rx, netdevs[dev].tx))
When you execute a program without any arguments, the program behaves exactly as it did in the previous version. Then, you can also specify network devices of interest. Such as:
$ ./net_devs_2.py
em1: 0.0 MiB 0.0 MiB
wlan0: 146.099492073 MiB 12.9737148285 MiB
virbr1: 0.0 MiB 0.0 MiB
virbr1-nic: 0.0 MiB 0.0 MiB
$ ./net_devs_2.py --help
usage: net_devs_2.py [-h] [-i IFACE]
Network Interface Usage Monitor
optional arguments:
-h, --help show this help message and exit
-i IFACE, --interface IFACE
Network interface
$ ./net_devs_2.py -i wlan0
wlan0: 146.100307465 MiB 12.9777050018 MiB
System-wide availability of scripts
With the help of this article, you've probably been able to write one or more useful scripts that, like any other Linux command, you want to use every day. The easiest way is to set the script to be executable, and then set a BASH alias for the script. You can also remove the.py extension and place the script in a standard location such as /usr/local/sbin.
Other useful standard library modules
In addition to the standard library modules already mentioned in this article, there are many other useful standard modules: subprocess, ConfigParser, readline, and curses.
What's next?
At this stage, depending on your own experience with Python, you can explore Linux internals in one of the following ways. If you've ever had to write a lot of shell scripts/command pipelines to explore the internals of Linux, try Python. If you want an easier way to write utility scripts that perform many tasks, try Python. Finally, if you're already using Python to write programs for other purposes on Linux, try exploring Linux internals in Python.