Linux, Python, Windows

Scrapy (web crawling) installation [Windows / Kali Linx]

For Python 2.7 / 3.3+

Windows

Install scrapy using pip

 

 

Kali Linux

Since I have both of the Python 2.7 (python) and 3.5.3+ (python3) installed, in order to install the scrapy on python3:

Install the pip3 for python3, instead of using pip for old python

 

Try install scrapy using pip3

 

However, the dependency python-cryptography of the python3 on kali is missing, result in the following errors

 

Quick check the install requirement for pip3 using

 

In the requirement.txt generated

 

Install the required library

 

Upon successful installation, the following text should be observed

 

Quick test scrapy (symbolic link should be created automatically)