V2EX = way to explore
V2EX 是一个关于分享和探索的地方
现在注册
已注册用户请  登录
DavidA
V2EX  ›  程序员

为什么我 get 请求这个地址,服务器会返回 403 呢,它是怎么判断的?

  •  
  •   DavidA · 268 天前 · 2510 次点击
    这是一个创建于 268 天前的主题,其中的信息可能已经有所发展或是发生改变。

    我如果在同源网页浏览器上 fetch 是可以正常返回结果的,但是我放到 jetbrains 客户端的 http 请求上执行就返回 403forbidden 了

    下面是我的请求内容:

    GET https://www.nodeseek.com/api/attendance/board?page=30
    authority: www.nodeseek.com
    accept: */*
    accept-language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7,zh-TW;q=0.6,yue-HK;q=0.5,yue-CN;q=0.4,yue;q=0.3
    cookie: 打码处理
    referer: https://www.nodeseek.com/board
    sec-ch-ua: "Chromium";v="122", "Not(A:Brand";v="24", "Google Chrome";v="122"
    sec-ch-ua-mobile: ?0
    sec-ch-ua-platform: "Windows"
    sec-fetch-dest: empty
    sec-fetch-mode: cors
    sec-fetch-site: same-origin
    user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36
    

    下面是返回的网页内容:

    Please enable cookies.

    Sorry, you have been blocked

    You are unable to access nodeseek.com Why have I been blocked? This website is using a security service to protect itself from online attacks. The action you just performed triggered the security solution. There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

    What can I do to resolve this? You can email the site owner to let them know you were blocked. Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

    11 条回复    2024-03-02 20:09:50 +08:00
    seers
        1
    seers  
       268 天前
    很多特征可以检测,例如浏览器指纹
    yulgang
        2
    yulgang  
       268 天前
    如:referer 、ua
    xguanren
        3
    xguanren  
       268 天前   ❤️ 1
    直接浏览器抓包 复制 curl 然后放到 postman 当中发包.看看是否能正常返回
    puzzle9
        4
    puzzle9  
       268 天前
    啊 感谢你让我了解到了这个论坛
    这玩意用的 cloudflare 用接口抓数据暂时先别考虑了
    如果你实现了 大佬 请受小弟一拜
    我看他们 tg 有个官方群组 要帖子数据可以从哪里拿
    要不 整个无头浏览器
    jinliming2
        5
    jinliming2  
       268 天前 via iPhone
    还有些网站会看 TLS 指纹
    xguanren
        6
    xguanren  
       268 天前
    ```
    xguanren
        7
    xguanren  
       268 天前
    ```
    curl 'https://www.nodeseek.com/page-2' \
    -H 'authority: www.nodeseek.com' \
    -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7' \
    -H 'accept-language: zh-CN,zh;q=0.9,en;q=0.8' \
    -H 'cache-control: no-cache' \
    -H 'pragma: no-cache' \
    -H 'referer: https://www.nodeseek.com/' \
    -H 'sec-ch-ua: "Chromium";v="122", "Not(A:Brand";v="24", "Google Chrome";v="122"' \
    -H 'sec-ch-ua-mobile: ?0' \
    -H 'sec-ch-ua-platform: "Windows"' \
    -H 'sec-fetch-dest: document' \
    -H 'sec-fetch-mode: navigate' \
    -H 'sec-fetch-site: same-origin' \
    -H 'sec-fetch-user: ?1' \
    -H 'upgrade-insecure-requests: 1' \
    -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36'
    ```
    如果是帖子的话 这个包就行了
    JasonEWNL
        8
    JasonEWNL  
       268 天前   ❤️ 2
    CF 防守的判断原理基本就同 @jinliming2 所言检查 TLS 指纹以区别浏览器和其他库。

    解决方法比如 curl 则用特供版 https://github.com/lwthiker/curl-impersonate 🔗

    测试直接执行:

    ```bash
    curl_chrome116 https://www.nodeseek.com/api/attendance/board?page=30
    ```

    可以得到 JSON 结果,因而确为前述原理,没有常规请求头检查。
    DavidA
        9
    DavidA  
    OP
       267 天前
    @JasonEWNL 666 ,学习了
    bug123
        10
    bug123  
       267 天前
    cf 判断没那么简单的,比如可以根据你这个网站常用的用户行为来判断,行为异常就直接弹质询框,反正正常用户也不会受影响
    Hydsiun
        11
    Hydsiun  
       267 天前
    @JasonEWNL 6666
    关于   ·   帮助文档   ·   博客   ·   API   ·   FAQ   ·   实用小工具   ·   6064 人在线   最高记录 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 25ms · UTC 02:44 · PVG 10:44 · LAX 18:44 · JFK 21:44
    Developed with CodeLauncher
    ♥ Do have faith in what you're doing.