我如果在同源网页浏览器上 fetch 是可以正常返回结果的,但是我放到 jetbrains 客户端的 http 请求上执行就返回 403forbidden 了
下面是我的请求内容:
GET https://www.nodeseek.com/api/attendance/board?page=30
authority: www.nodeseek.com
accept: */*
accept-language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7,zh-TW;q=0.6,yue-HK;q=0.5,yue-CN;q=0.4,yue;q=0.3
cookie: 打码处理
referer: https://www.nodeseek.com/board
sec-ch-ua: "Chromium";v="122", "Not(A:Brand";v="24", "Google Chrome";v="122"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
sec-fetch-dest: empty
sec-fetch-mode: cors
sec-fetch-site: same-origin
user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36
下面是返回的网页内容:
Please enable cookies.
Sorry, you have been blocked
You are unable to access nodeseek.com Why have I been blocked? This website is using a security service to protect itself from online attacks. The action you just performed triggered the security solution. There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.
What can I do to resolve this? You can email the site owner to let them know you were blocked. Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.
1
seers 268 天前
很多特征可以检测,例如浏览器指纹
|
2
yulgang 268 天前
如:referer 、ua
|
3
xguanren 268 天前 1
直接浏览器抓包 复制 curl 然后放到 postman 当中发包.看看是否能正常返回
|
4
puzzle9 268 天前
啊 感谢你让我了解到了这个论坛
这玩意用的 cloudflare 用接口抓数据暂时先别考虑了 如果你实现了 大佬 请受小弟一拜 我看他们 tg 有个官方群组 要帖子数据可以从哪里拿 要不 整个无头浏览器 |
5
jinliming2 268 天前 via iPhone
还有些网站会看 TLS 指纹
|
6
xguanren 268 天前
```
|
7
xguanren 268 天前
```
curl 'https://www.nodeseek.com/page-2' \ -H 'authority: www.nodeseek.com' \ -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7' \ -H 'accept-language: zh-CN,zh;q=0.9,en;q=0.8' \ -H 'cache-control: no-cache' \ -H 'pragma: no-cache' \ -H 'referer: https://www.nodeseek.com/' \ -H 'sec-ch-ua: "Chromium";v="122", "Not(A:Brand";v="24", "Google Chrome";v="122"' \ -H 'sec-ch-ua-mobile: ?0' \ -H 'sec-ch-ua-platform: "Windows"' \ -H 'sec-fetch-dest: document' \ -H 'sec-fetch-mode: navigate' \ -H 'sec-fetch-site: same-origin' \ -H 'sec-fetch-user: ?1' \ -H 'upgrade-insecure-requests: 1' \ -H 'user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36' ``` 如果是帖子的话 这个包就行了 |
8
JasonEWNL 268 天前 2
CF 防守的判断原理基本就同 @jinliming2 所言检查 TLS 指纹以区别浏览器和其他库。
解决方法比如 curl 则用特供版 https://github.com/lwthiker/curl-impersonate 🔗 测试直接执行: ```bash curl_chrome116 https://www.nodeseek.com/api/attendance/board?page=30 ``` 可以得到 JSON 结果,因而确为前述原理,没有常规请求头检查。 |
10
bug123 267 天前
cf 判断没那么简单的,比如可以根据你这个网站常用的用户行为来判断,行为异常就直接弹质询框,反正正常用户也不会受影响
|