Python downloads data from FTP to save the instance

  • 2020-04-02 13:12:06
  • OfStack

(link: what's the weather like in #) data can be downloaded at ftp://ftp3.ncdc.noaa.gov/pub/data/noaa, the online to see this data is very happy, open the FTP found a problem, ah, ah, so many file I ordered save as, one by one to point to what time ah, thunderbolt there should be a batch download, but I didn't find that estimate is banned from my browser the thunderbolt, simply their written in python is an implementation of download, online early and found very simple


#!/usr/bin/python
#-*- coding: utf-8 -*-
from ftplib import FTP
def ftpconnect():
    ftp_server = 'ftp3.ncdc.noaa.gov'
    username = ''
    password = ''
    ftp=FTP()
    ftp.set_debuglevel(2) # Turn on debug level 2 , showing details 
    ftp.connect(ftp_server,21) # The connection 
    ftp.login(username,password) # Login, if anonymous login with an empty string can be replaced 
    return ftp

def downloadfile():  
    ftp = ftpconnect()    
    #print ftp.getwelcome() # According to ftp Server welcome message 
    datapath = "/pub/data/noaa/"
    year=1911
    while year<=1930:
        path=datapath+str(year)
        li = ftp.nlst(path)
        for eachFile in li:
            localpaths = eachFile.split("/")
            localpath = localpaths[len(localpaths)-1]
            localpath='weatherdata/'+str(year)+'--'+localpath# Put the dates first to facilitate sorting 
            bufsize = 1024 # Set the buffer block size       
            fp = open(localpath,'wb') # Open the file locally in write mode 
            ftp.retrbinary('RETR ' + eachFile,fp.write,bufsize) # Receive the file on the server and write to the local file 
        year=year+1
    ftp.set_debuglevel(0) # Close the debugging 
    fp.close()
    ftp.quit() # exit ftp The server 

if __name__=="__main__":
    downloadfile()


Related articles: