In Linux openssl and opensslv. h cannot find the solution to the problem

preface

It is well known that scrapy is a well-known crawler framework in Python. During the installation of scrapy, we encountered the problem that some file of openssl could not be found, and analyzed and recorded it.

1. scrapy and installation process

Scrapy is a well-known crawler framework in python. The author installed Centos 7 system and found the following problems:

>> pip install scrapy

Since there is a lot of process information in the installation process, only the key pieces of information are listed here:

 running egg_info 
 writing requirements to src/cryptography.egg-info/requires.txt 
 writing src/cryptography.egg-info/PKG-INFO 
 writing top-level names to src/cryptography.egg-info/top_level.txt 
 writing dependency_links to src/cryptography.egg-info/dependency_links.txt 
 writing entry points to src/cryptography.egg-info/entry_points.txt 
 reading manifest file 'src/cryptography.egg-info/SOURCES.txt' 
 reading manifest template 'MANIFEST.in' 
 no previously-included directories found matching 'docs/_build' 
 warning: no previously-included files matching '*' found under directory 'vectors' 
 writing manifest file 'src/cryptography.egg-info/SOURCES.txt' 
 running build_ext 
 generating cffi module 'build/temp.linux-x86_64-2.7/_padding.c' 
 creating build/temp.linux-x86_64-2.7 
 generating cffi module 'build/temp.linux-x86_64-2.7/_constant_time.c' 
 generating cffi module 'build/temp.linux-x86_64-2.7/_openssl.c' 
 building '_openssl' extension 
 creating build/temp.linux-x86_64-2.7/build 
 creating build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7 
 gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c build/temp.linux-x86_64-2.7/_openssl.c -o build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7/_openssl.o 
 build/temp.linux-x86_64-2.7/_openssl.c:434:30: fatal error: openssl/opensslv.h: No such file or directory 
 #include <openssl/opensslv.h> 
     ^ 
 compilation terminated. 
 error: command 'gcc' failed with exit status 1 
 
 ---------------------------------------- 
Command "/usr/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-hRMlG0/cryptography/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-ReCoWo-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-hRMlG0/cryptography/ 
[root@AY131203102210033c39Z ~]# yum install openssl build/temp.linux-x86_64-2.7/_openssl.c:434:30: fatal error: openssl/opensslv.h: No such file or directory 
^C 
 
Exiting on user cancel. 

The installation failed because the openssl.c file was identified. No corresponding file appears to have been found

2. Problem analysis

First, it is suspected that openssl is not installed, so check openssl first:

>> yum info openssl

Loaded plugins: fastestmirror 
Loading mirror speeds from cached hostfile 
 * base: mirrors.aliyun.com 
 * epel: mirrors.aliyun.com 
 * extras: mirrors.aliyun.com 
 * updates: mirrors.aliyun.com 
Installed Packages 
Name : openssl 
Arch : x86_64 
Epoch : 1 
Version : 1.0.1e 
Release : 60.el7_3.1 
Size : 1.5 M 
Repo : installed 
Summary : Utilities from the general purpose cryptography library with TLS implementation 
URL  : http://www.openssl.org/ 
License : OpenSSL 
Description : The OpenSSL toolkit provides support for secure communications between 
  : machines. OpenSSL includes a certificate management tool and shared 
  : libraries which provide various cryptographic algorithms and 
  : protocols. 

Based on the information, we can know that openssl has been installed by 1 company. How can openssl.c still be missing?

After some reflection, it turns out that a basic rule is that openssl already has a binary executable installed, whereas scrapy here requires the source file of openssl, such as ES47en.h. The development version of ES49en. h, which contains the associated installation source code files, should be installed here.

3. Problem solving

Once the problem has been identified, the next step is to install the es55EN-ES56en installation package:

>> yum install openssl-devel

After the installation is complete, reinstall scrapy and the installation will be successful

conclusion

By extension, there are similar problems in Linux system. During the process of installing a specific installation package, it depends on some third party development package, and some files cannot be found. Generally, it is necessary to install the development version of the dependent package. This rule should be universal.