An example of the use of CURL PycURL in Python

  • 2020-04-02 13:41:07
  • OfStack

There is a common (and very useful) command on Linux, curl, which is supported by the well-known libcurl library. Libcurl is powerful and a very efficient library. In addition to providing its own C API, libcurl has bindings for up to 40 programming languages, and PycURL here is the Python Binding for libcurl.
When it comes to performance, libcurl is a good choice for making GET/POST Requests to web pages in Python, generally faster than liburl or liburl2, and probably more efficient than Requests. This is especially effective when using PycURL for multiple concurrent requests. Personally, the only downside is that, since it's called directly into the libcurl C library, the function interface of PycURL or something like that, it's probably not quite as Pythonic, and the learning curve for writing code is a little bit higher than liburl.
Here's a simple example:

#! /usr/bin/env python
# -*- coding: utf-8 -*-

'''
Created on Dec 15, 2013

@author: Jay
'''

import sys
import pycurl
import time

class Test:
    def __init__(self):
        self.contents = ''

    def body_callback(self, buf):
        self.contents = self.contents + buf

sys.stderr.write("Testing %sn" % pycurl.version)

start_time = time.time()

url = 'http://www.dianping.com/shanghai'
t = Test()
c = pycurl.Curl()
c.setopt(c.URL, url)
c.setopt(c.WRITEFUNCTION, t.body_callback)
c.perform()
end_time = time.time()
duration = end_time - start_time
print c.getinfo(pycurl.HTTP_CODE), c.getinfo(pycurl.EFFECTIVE_URL)
c.close()

print 'pycurl takes %s seconds to get %s ' % (duration, url)

print 'lenth of the content is %d' % len(t.contents)
#print(t.contents)

Related articles: