关于Python安装Scrapy库的常见报错解决

由于Scrapy该库在Windows下会存在兼容问题，下面介绍的是在Linux系统进行安装。

1、关于pip3命令的报错

报错代码：

error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    
    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.
    
    See /usr/share/doc/python3.11/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

解决方法：在原有命令后面追加参数 --break-system-packages

因为scrapy可以直接当做命令使用，所以我们也可以在终端输入不存在的命令，它会提示你安装

有时候pip3不行，我们也可以换pipx，或者apt install python3-文件名来进行安装

pipx install scrapy

apt install python3-scrapy

2、执行scrapy报错（Python3下的OpenSSL模块出错）

报错代码：

Traceback (most recent call last):
  File "/usr/bin/scrapy", line 33, in <module>
    sys.exit(load_entry_point('Scrapy==2.10.0', 'console_scripts', 'scrapy')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/bin/scrapy", line 25, in importlib_load_entry_point
    return next(matches).load()
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/metadata/__init__.py", line 202, in load
    module = import_module(match.group('module'))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1128, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
   ...
 AttributeError: module 'lib' has no attribute 'SSL_CTX_set_ecdh_auto'

这个是Python3下的OpenSSL模块出错，我们需要卸载它并重装

解决方法：

pip3 uninstall pyopenssl
pip3 install pyopenssl

尝试了一些命令发现不行

3、卸载pyopenssl时报错

报错代码：

Found existing installation: pyOpenSSL 21.0.0
Not uninstalling pyopenssl at /usr/lib/python3/dist-packages, outside environment /usr
Can't uninstall 'pyOpenSSL'. No files were found to uninstall.

Requirement already satisfied: pyopenssl in /usr/lib/python3/dist-packages (21.0.0)

解决办法：

找到该文件夹所在路径，直接将pyOpenSSL-21.0.0目录删掉，再重新安装

该文件夹在 /usr/lib/python3/dist-packages/ 下面

删掉整个文件夹

重新使用命令安装pyopenssl

pip3 install pyopenssl --break-system-packages

安装成功后输入scrapy命令

如下图，则代表该库已安装成功

我们也可以进入ipython3进行验证：

ipython是一个python的交互式shell，比默认的python shell好用很多，支持变量自动补全，自动缩进，支持bash shell命令，内置了许多很有用的功能和函数

只要可以成功导入该库，没有报错，证明已经安装成功