Process and Common Errors of Installing Scrapy Crawler Framework in Python3 Environment

Windows

• Installing lxml

The best way to install is through the wheel file, http://www. lfd. uci. edu/~ gohlke/pythonlibs/, and find the relevant files of lxml from this website. If it is Python version 3.5, WIndows 64-bit system, then find lxml‑3.7.2‑cp35‑cp35m‑win_amd64.whl This file is downloaded and then installed through pip.

After downloading, run the following command to install:

pip3 install wheel
pip3 install lxml‑3.7.2‑cp35‑cp35m‑win_amd64.whl

You can complete the installation of lxml, and replace the file name with other files.

• Install zope. interface

Download the corresponding version of wheel file from the official website https://pypi.python.org/pypi/zope. interface # downloads, and then install pip. For example, Python version 3.5, Windows 64-bit system, download zope.interface-4.3.3-cp35-cp35m-win_amd64.whl .

Then install the wheel file with the following command:

pip3 install zope.interface-4.3.3-cp35-cp35m-win_amd64.whl

Other versions can replace the file name.

• Installing pyOpenSSL

Download the wheel file from the official website, https://pypi.python.org/pypi/pyOpenSSL # downloads. For example, the name of the latest version is pyOpenSSL-16.2.0-py2.py3-none-any.whl , download and install.

pip3 install pyOpenSSL-16.2.0-py2.py3-none-any.whl

• Installing Twisted

By the same token, http://www. lfd. uci. edu/~ gohlke/pythonlibs/# twisted download the wheel file and install it with pip.

For example, Python version 3.5, Windows 64-bit system, download

Twisted‑17.1.0‑cp35‑cp35m‑win_amd64.whl And then pip is installed.

pip3 install Twisted‑17.1.0‑cp35‑cp35m‑win_amd64.whl

• Installing pywin32

Download the corresponding version of the installation package from the official website https://sourceforge.net/projects/pywin32/files/pywin32/Build% 2020/.

• Installing Scrapy

Finally, install Scrapy, and still use pip. The command is as follows:

pip3 install Scrapy


ps: Take a look at Python 3.6 Installation scrapy Error Python. h: There is no file or directory

When installing scrapy, an error is reported. In fact, this error is an indirect one, which is caused by its dependence.

If you are Python2. X

sudo apt-get install python-dev

If it is below python 3.6

sudo apt-get install python3-dev

If it is Python 3.6

sudo apt-get install python3.6-dev

The last one is a good pervert. I am updating my Python3-dev 1 and reporting an error. Finally, I guess that there is Python3.6-dev... I am tired.

Summarize