Method for detecting whether python crawler proxy ip camouflage is successful
- 2021-07-16 02:45:45
- OfStack
Sometimes our crawler adds an agent, but we don't know whether the program has obtained ip, especially in dynamic forwarding mode, so we need to detect it. The following is a way to detect whether the agent disguises successfully. Here, we recommend using the code example provided by Yiniuyun.
Python¶
requests
#! -*- encoding:utf-8 -*-
import requests
import random
# Target page to access
targetUrl = "http://httpbin.org/ip"
# Destination to access HTTPS Page
# targetUrl = "https://httpbin.org/ip"
# Proxy server
proxyHost = "t.16yun.cn"
proxyPort = "31111"
# Proxy tunnel authentication information
proxyUser = "16ZKBRLB"
proxyPass = "234076"
proxyMeta = "http://%(user)s:%(pass)s@%(host)s:%(port)s" % {
"host" : proxyHost,
"port" : proxyPort,
"user" : proxyUser,
"pass" : proxyPass,
}
# Settings http And https Visits are all made with HTTP Agent
proxies = {
"http" : proxyMeta,
"https" : proxyMeta,
}
# Settings IP Switching head
tunnel = random.randint(1,10000)
headers = {"Proxy-Tunnel": str(tunnel)}
resp = requests.get(targetUrl, proxies=proxies, headers=headers)
print resp.status_code
print resp.text
To check whether the agent has been obtained, visit the website http://httpbin.org/ip directly after configuring the agent, and then visit www.ip138.com after obtaining ip to know whether ip has been obtained.