Scrapy (web crawling) installation [Windows / Kali Linx]

For Python 2.7 / 3.3+


Install scrapy using pip



Kali Linux

Since I have both of the Python 2.7 (python) and 3.5.3+ (python3) installed, in order to install the scrapy on python3:

Install the pip3 for python3, instead of using pip for old python


Try install scrapy using pip3


However, the dependency python-cryptography of the python3 on kali is missing, result in the following errors


Quick check the install requirement for pip3 using


In the requirement.txt generated


Install the required library


Upon successful installation, the following text should be observed


Quick test scrapy (symbolic link should be created automatically)