Posted on | In , |
Words count in article: 318 | Reading time ≈ 1

Serialization is the process of converting a data structure or object into a sequence of bits so that it can be stored in a file or memory buffer, or transmitted across a network connection link to be reconstructed later in the same or another computer environment.

Design an algorithm to serialize and deserialize a binary tree. There is no restriction on how your serialization/deserialization algorithm should work. You just need to ensure that a binary tree can be serialized to a string and this string can be deserialized to the original tree structure.

Read more »

Posted on | In , , |
Words count in article: 253 | Reading time ≈ 1 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
#!coding=utf-8
import time
from selenium import webdriver
import pickle
 
class BaiduSpider(object):
    def __init__(self,username,password):
        self.username = username
        self.password = password
        self.driver = webdriver.Chrome()
        self.driver.get(url='http://www.baidu.com')
        self.set_cookie()
        self.is_login()
    def is_login(self):
        '''判断当前是否登陆'''
        self.driver.refresh()
        html = self.driver.page_source
        if html.find(self.username) == -1: #利用用户名判断是否登陆
            # 没登录 ,则手动登录
            self.login()
        else:
            #已经登录  尝试访问搜索记录，可以正常访问
            self.driver.get(url='http://i.baidu.com/my/history')
            time.sleep(30)  # 延时看效果
 
    def login(self):
        '''登陆'''
        time.sleep(60) #等待手动登录
        self.driver.refresh()
        self.save_cookie()
 
    def save_cookie(self):
        '''保存cookie'''
        # 将cookie序列化保存下来
        pickle.dump(self.driver.get_cookies(), open("cookies.pkl", "wb"))
 
    def set_cookie(self):
        '''往浏览器添加cookie'''
        '''利用pickle序列化后的cookie'''
        try:
            cookies = pickle.load(open("cookies.pkl", "rb"))
            for cookie in cookies:
                cookie_dict = {
                    "domain": ".baidu.com",  # 火狐浏览器不用填写，谷歌要需要
                    'name': cookie.get('name'),
                    'value': cookie.get('value'),
                    "expires": "",
                    'path': '/',
                    'httpOnly': False,
                    'HostOnly': False,
                    'Secure': False}
                self.driver.add_cookie(cookie_dict)
        except Exception as e:
            print(e)
 
 
if __name__ == '__main__':
    BaiduSpider('usename','!!!!') # 你的百度账号，密码

Posted on | In , , |
Words count in article: 3.1k | Reading time ≈ 12

多进程

Python标准库原本有threading和multiprocessing模块编写相应的多线程/多进程代码。但从Python3.2开始，标准库为我们提供了concurrent.futures模块，它提供了ThreadPoolExecutor和ProcessPoolExecutor两个类，实现了对threading和multiprocessing的更高级的抽象，对编写线程池/进程池提供了直接的支持。多进程我们介绍futures的ProcessPoolExecutor

ProcessPoolExecutor类是Executor类的子类，实例化ProcessPoolExecutor类以创建进程池，在实例化的过程中应指定同时运行的最大进程数

1
2
3
4
5
6
7
8
9
10
11
from concurrent.futures import  ProcessPoolExecutor
pool = ProcessPoolExecutor(max_workers=4) # 运行最大进程数4
#进程池的操作...
pool.shutdown(wait=True) # 关闭进程池，默认等待所有进程的完成。
print('Deep') # 有shutdown的情况下所有进程完成后才会运行下面的print，没有的话会马上运行

'''
创建进程也可用with,这时会自带shutdown功能
with ProcessPoolExecutor(4) as pool:
    #进程池的操作...
'''

该类有两种方法对进程池提交任务建立进程(函数及一组参数构成一个任务)，分别是submit()和map()，如果单纯想多开进程别无他想，用哪个都行，但submit()会有更灵活的用法

Read more »

Posted on | In , , |
Words count in article: 1.9k | Reading time ≈ 6

模拟登录，cookie的使用

某些网站，登录和没登录，用户的权限是不一样的，帐号登录之后才能获取更多的信息。更有甚者一上来就是登录界面，不登录就不给你进去。爬取目标不用登录固然是好，但需要时也没办法啊，这时如果还想爬取信息，就必须让爬虫学会登录。

Read more »

Posted on | In , |
Words count in article: 597 | Reading time ≈ 3

Given n balloons, indexed from 0 to n-1. Each balloon is painted with a number on it represented by array nums. You are asked to burst all the balloons. If the you burst balloon i you will get nums[left] nums[i] nums[right] coins. Here left and right are adjacent indices of i. After the burst, the left and right then becomes adjacent.

Read more »