V2EX = way to explore
V2EX 是一个关于分享和探索的地方
Sign Up Now
For Existing Member  Sign In
httpbin - 协议调试工具
httpstatuses - 协议状态码查询
httpie - cURL-like tool for humans
Fiddler
hahahalololo
V2EX  ›  HTTP

求教同样的 url 参数头部 cookies, scrapy 请求返回验证码, requests 直接请求就正常,代码如下

  •  
  •   hahahalololo · Jan 5, 2024 · 2062 views
    This topic created in 846 days ago, the information mentioned may be changed or developed.
    url='xxxxxxxxxxxxxxx'
    
    yield scrapy.Request(
        url=url,
        cookies=self.cookies,
        headers=self.headers,
        meta={
        'download_timeout': 60  
        },
        callback=self.save,
    )
    
    rs = requests.get(url, headers=self.headers, cookies=self.cookies, timeout=60)
    

    这可能是什么原因造成的呢?找了一圈也没发现原因。COOKIES_ENABLED 设置成 True 的

    2 replies    2024-01-09 18:09:16 +08:00
    laihan4396a
        1
    laihan4396a  
       Jan 5, 2024
    抓个包,或者设置个代理,看下请求的区别?
    hahahalololo
        2
    hahahalololo  
    OP
       Jan 9, 2024
    @laihan4396a 3q ,抓包后发现是 cookes 的问题,scrapy 框架对 cookies 自动处理了,少了许多
    About   ·   Help   ·   Advertise   ·   Blog   ·   API   ·   FAQ   ·   Solana   ·   2709 Online   Highest 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 39ms · UTC 14:57 · PVG 22:57 · LAX 07:57 · JFK 10:57
    ♥ Do have faith in what you're doing.