Method for detecting whether python crawler proxy ip camouflage is successful

  • 2021-07-16 02:45:45
  • OfStack

Sometimes our crawler adds an agent, but we don't know whether the program has obtained ip, especially in dynamic forwarding mode, so we need to detect it. The following is a way to detect whether the agent disguises successfully. Here, we recommend using the code example provided by Yiniuyun.


Python¶
requests
 #! -*- encoding:utf-8 -*-
 import requests
 import random
 
 #  Target page to access 
 targetUrl = "http://httpbin.org/ip"
 #  Destination to access HTTPS Page 
 # targetUrl = "https://httpbin.org/ip"
 #  Proxy server 
 proxyHost = "t.16yun.cn"
 proxyPort = "31111"
 #  Proxy tunnel authentication information 
 proxyUser = "16ZKBRLB"
 proxyPass = "234076"
 proxyMeta = "http://%(user)s:%(pass)s@%(host)s:%(port)s" % {
 "host" : proxyHost,
 "port" : proxyPort,
 "user" : proxyUser,
 "pass" : proxyPass,
 }
 #  Settings  http And https Visits are all made with HTTP Agent 
 proxies = {
 "http" : proxyMeta,
 "https" : proxyMeta,
 }
 #  Settings IP Switching head 
 tunnel = random.randint(1,10000)
 headers = {"Proxy-Tunnel": str(tunnel)}
 resp = requests.get(targetUrl, proxies=proxies, headers=headers)
 print resp.status_code
 print resp.text

To check whether the agent has been obtained, visit the website http://httpbin.org/ip directly after configuring the agent, and then visit www.ip138.com after obtaining ip to know whether ip has been obtained.


Related articles: