定时爬虫抓当日免费应用:Scrapy + Tkinter + LaunchControl

花了个周末学了下Scrapy,正好一直想买mindnode,于是顺手做了个爬虫,抓取爱范儿每天的限免应用信息。

Thinking

大概思路就是使用LaunchControl每天定时(比如早上9点50,这时正好刚到公司不久)跑一下爬虫脚本,如果找到感兴趣的应用在限免,就使用Tkinter弹出提示。当然,也可以直接用Scrapy做定时任务,以后再说。

Coding

Scrapy + Tkinter

``` bash

-- coding: utf-8 --

import scrapy
import Tkinter
from scrapy.shell import inspect_response
import json

设置感兴趣的app名称

I_want_apps = set([‘mindnode pro’, ‘u.memory’])

class XianmianSpider(scrapy.Spider):
user_agent = ‘Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36’
name = “xianmian”
allowed_domains = [“app.so”]
start_urls = (
http://app.so/api/v1.1/appso/discount/?platform=web&limit=10‘,
)

def parse(self, response):

    jsonresponse = json.loads(response.body_as_unicode())

    apps = jsonresponse['objects']

    appTitles = {item['display_name'].lower() for item in apps}

    self.logger.info('today\' apps are: ' + str(appTitles))

    the_apps = appTitles & I_want_apps
    if the_apps:
        self.showMsg('found the apps: {}'.format(list(the_apps)))

def showMsg(self, msg):
    import Tkinter
    root = Tkinter.Tk()
    root.title('福利到!')
    label = Tkinter.Label(root, text=msg)
    label.pack()
    center_window(root, 300, 240)
    root.maxsize(600, 400)
    root.minsize(300, 240)
    Tkinter.mainloop()

def get_screen_size(window):
return window.winfo_screenwidth(),window.winfo_screenheight()

def get_window_size(window):
return window.winfo_reqwidth(),window.winfo_reqheight()

def center_window(root, width, height):
screenwidth = root.winfo_screenwidth()
screenheight = root.winfo_screenheight()
size = ‘%dx%d+%d+%d’ % (width, height, (screenwidth - width)/2, (screenheight - height)/2)
print(size)
root.geometry(size)

```

LaunchControl

title

LaunchControl用起来比较直观,当然,也可以直接用mac自带的launchctl,具体可参考launchctl使用说明

坚持原创技术分享,您的支持将鼓励我继续创作!