# 1.1.Scrapy简介

Scrapy是一个开源框架，用一种**快速**、**简单**、**可扩展**的方式从网站中提取你需要的数据。

## Scrapy特点

* 快速且强大：你仅仅需要编写用于提取数据的规则，Scrapy会为你完成其余工作
* 扩展容易：通过设计扩展，无需接触核心即可轻松插入新功能
* 便携：基于python语言，各平台兼容

## 构建并运行你的网络爬虫

```bash
$ pip install scrapy 
$ cat > myspider.py <<EOF
import scrapy

class BlogSpider(scrapy.Spider):
    name = 'blogspider'
    start_urls = ['https://blog.scrapinghub.com']

    def parse(self, response):
        for title in response.css('.post-header>h2'):
            yield {'title': title.css('a ::text').get()}

        for next_page in response.css('a.next-posts-link'):
            yield response.follow(next_page, self.parse)
EOF
$ scrapy runspider myspider.py
```

## 部署网络爬虫

Scrapy支持多种部署部署方式，你可根据你的实际情况决定选用哪一种方式

* 部署爬虫到[Scrapy Cloud](https://scrapinghub.com/scrapy-cloud/)

  ```bash
  $ pip install shub
  $ shub login
  Insert your Scrapinghub API Key: <API_KEY>

  # Deploy the spider to Scrapy Cloud
  $ shub deploy

  # Schedule the spider for execution
  $ shub schedule blogspider 
  Spider blogspider scheduled, watch it running here:
  https://app.scrapinghub.com/p/26731/job/1/8

  # Retrieve the scraped data
  $ shub items 26731/1/8
  {"title": "Improved Frontera: Web Crawling at Scale with Python 3 Support"}
  {"title": "How to Crawl the Web Politely with Scrapy"}
  ...
  ```
* 或者使用 [Scrapyd](https://github.com/scrapy/scrapyd) 在你自己的服务器中托管爬虫
