Of Stack

Flask and MongoDB are used in Python to build a simple picture server

  • 2020-04-02 14:33:25
  • OfStack

1. Preliminary preparation

Once pymongo is installed by PIP or easy_install, you can tune mongodb in Python.
Then install a flask to act as a web server.

Of course, mongo also has to be installed. For Ubuntu users, especially those who use Server 12.04, it is a little more complicated to install the latest version 


sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10
echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | sudo tee /etc/apt/sources.list.d/mongodb.list
sudo apt-get update
sudo apt-get install mongodb-10gen

If, like me, you feel that letting the suffix on the file name tell you what the user has uploaded is all a case of cheating yourself with a yam and a cucumber, then you'd better have a Pillow library 



pip install Pillow

Or (better for Windows users) 



easy_install Pillow

2, positive

2.1 Flask file upload

It's hard to make fun of the fact that the Flask example is split in two. Here's the easiest one, no matter what the file is 


import flask
app = flask.Flask(__name__)
app.debug = True
@app.route('/upload', methods=['POST'])
def upload():
  f = flask.request.files['uploaded_file']
  print f.read()
  return flask.redirect('/')
@app.route('/')
def index():
  return '''
  <!doctype html>
  <html>
  <body>
  <form action='/upload' method='post' enctype='multipart/form-data'>
     <input type='file' name='uploaded_file'>
     <input type='submit' value='Upload'>
  </form>
  '''
if __name__ == '__main__':
  app.run(port=7777)

Note: in the upload function, the flask. Request-files [KEY] is used to get the uploaded file object. The KEY is the name value of input in the page form

Because you're outputting content in the background, it's best to test it in a plain text file.

2.2 save to mongodb

If not, the fastest basic storage solution is all you need 


import pymongo
import bson.binary
from cStringIO import StringIO
app = flask.Flask(__name__)
app.debug = True
db = pymongo.MongoClient('localhost', 27017).test
def save_file(f):
  content = StringIO(f.read())
  db.files.save(dict(
    content= bson.binary.Binary(content.getvalue()),
  ))
@app.route('/upload', methods=['POST'])
def upload():
  f = flask.request.files['uploaded_file']
  save_file(f)
  return flask.redirect('/')

Insert the content into one   Bson. Binary. Binary   Object and drop it into mongodb.

Now try uploading another file through the mongo shell   Db. Files. Find ().

But the content   This field is almost impossible to tell with the naked eye, and even for plain text files, mongo displays as Base64 encoded.

2.3 provide file access

Given the ID of the file stored in the database (as part of the URI), the file contents are returned to the browser as follows 


def save_file(f):
   content = StringIO(f.read())
   c = dict(content=bson.binary.Binary(content.getvalue()))
   db.files.save(c)
   return c['_id']
@app.route('/f/<fid>')
def serve_file(fid):
  f = db.files.find_one(bson.objectid.ObjectId(fid))
  return f['content']
@app.route('/upload', methods=['POST'])
def upload():
  f = flask.request.files['uploaded_file']
  fid = save_file(f)
  return flask.redirect( '/f/' + str(fid))

After uploading the file,   The upload   This allows you to preview the text file's contents, if it's not so fussy that line breaks and contiguous Spaces are consumed by the browser.

2.4 when the file cannot be found

There are two situations, one, the database ID format is not correct, in which case pymongo will throw an exception   Bson. Errors. InvalidId; Second, there is no object (!) , pymongo will return   None.
So let's do that for simplicity 


@app.route('/f/<fid>')
def serve_file(fid):
  import bson.errors
  try:
    f = db.files.find_one(bson.objectid.ObjectId(fid))
    if f is None:
      raise bson.errors.InvalidId()
    return f['content']
  except bson.errors.InvalidId:
    flask.abort(404)

2.5 correct MIME

From now on, upload files must be strictly checked. Text files, dogs and scissors cannot be uploaded.
Before judging the picture file, we said to use Pillow for real 


from PIL import Image
allow_formats = set(['jpeg', 'png', 'gif'])
def save_file(f):
  content = StringIO(f.read())
  try:
    mime = Image.open(content).format.lower()
    if mime not in allow_formats:
      raise IOError()
  except IOError:
    flask.abort(400)
  c = dict(content=bson.binary.Binary(content.getvalue()))
  db.files.save(c)
  return c['_id']

Then try to upload the text file must be empty, the transfer of the image file can be normal.
To solve this problem, you have to store MIME in the database. Also, transfer the mimetype correctly when the file is given 


def save_file(f):
  content = StringIO(f.read())
  try:
    mime = Image.open(content).format.lower()
    if mime not in allow_formats:
      raise IOError()
  except IOError:
    flask.abort(400)
  c = dict(content=bson.binary.Binary(content.getvalue()), mime=mime)
  db.files.save(c)
  return c['_id']
@app.route('/f/<fid>')
def serve_file(fid):
  try:
    f = db.files.find_one(bson.objectid.ObjectId(fid))
    if f is None:
      raise bson.errors.InvalidId()
    return flask.Response(f['content'], mimetype='image/' + f['mime'])
  except bson.errors.InvalidId:
    flask.abort(404)

Of course, the original stored object doesn't have the mime property, so it's best to go to the mongo shell first   Db. Files. Drop ()   Clear out the old data.

2.6 NOT MODIFIED is given according to the upload time
With HTTP 304 NOT MODIFIED, you can squeeze as much out of the browser cache and save bandwidth as possible. This requires three operations

1) record the last upload time of the file
2) when the browser requests the file, insert a timestamp string into the header of the request
3) when the browser requests the file, try to get this timestamp from the request header. If it is consistent with the timestamp of the file, it will be 304 directly

Embodied as code is 


import datetime
def save_file(f):
  content = StringIO(f.read())
  try:
    mime = Image.open(content).format.lower()
    if mime not in allow_formats:
      raise IOError()
  except IOError:
    flask.abort(400)
  c = dict(
    content=bson.binary.Binary(content.getvalue()),
    mime=mime,
     time=datetime.datetime.utcnow(),
  )
  db.files.save(c)
  return c['_id']
@app.route('/f/<fid>')
def serve_file(fid):
  try:
    f = db.files.find_one(bson.objectid.ObjectId(fid))
    if f is None:
      raise bson.errors.InvalidId()
    if flask.request.headers.get('If-Modified-Since') == f['time'].ctime():
      return flask.Response(status=304)
    resp = flask.Response(f['content'], mimetype='image/' + f['mime'])
    resp.headers['Last-Modified'] = f['time'].ctime()
    return resp
  except bson.errors.InvalidId:
    flask.abort(404)

Then, you have to get a script that timestamps the images already in the database.
By the way, in fact, NoSQL DB in this environment does not reflect any advantages, used almost the same as RDB.

2.7 utilize sha-1 row weight

Unlike the coke in your fridge, most of the time you don't want a bunch of identical images in your database. Images, along with data like EXIFF, should be unique in your database, and a slightly stronger hash technique is appropriate.

The easiest way to do this is to build one   SHA - 1   Unique index, so that the database prevents the same thing from being put in.

Create a unique index in the MongoDB table, execute (in the Mongo console) 



db.files.ensureIndex({sha1: 1}, {unique: true})

If there are multiple records in your library, directing to a fault. It looks very harmonious harmless index operation was informed that the database has a repeat of null values (in fact, the current existing entry in the database no this attribute). Unlike normal RDB, directing a null, or there is no attribute values is also a kind of the same attribute values, so the ghost attributes can lead to unique index cannot be established.

There are three solutions:

1) delete all the current data (must be the test database in such an irresponsible way!)
2) build a sparse index, which does not require ghost attributes to be unique, but multiple null values determine that they are redundant (regardless of existing data)
3) write a script to run through the database once, pull out all the stored data, recalculate sha-1, and save it again
Let's assume that the problem is now solved and the index is done, then the Python code is all that's left. 


import hashlib
def save_file(f):
  content = StringIO(f.read())
  try:
    mime = Image.open(content).format.lower()
    if mime not in allow_formats:
      raise IOError()
  except IOError:
    flask.abort(400)
  sha1 = hashlib.sha1(content.getvalue()).hexdigest()
  c = dict(
    content=bson.binary.Binary(content.getvalue()),
    mime=mime,
    time=datetime.datetime.utcnow(),
    sha1=sha1,
  )
  try:
    db.files.save(c)
  except pymongo.errors.DuplicateKeyError:
    pass
  return c['_id']

This is fine for uploading files. However, following the above logic, if you upload an existing file, return   [c] '_id'   Will be a nonexistent data ID. Fix this problem, preferably return   Sha1, in addition, when accessing the file, modify it accordingly to use the file sha-1 instead of the ID.
Last modified results and this complete source code as follows: 


import hashlib
import datetime
import flask
import pymongo
import bson.binary
import bson.objectid
import bson.errors
from cStringIO import StringIO
from PIL import Image
app = flask.Flask(__name__)
app.debug = True
db = pymongo.MongoClient('localhost', 27017).test
allow_formats = set(['jpeg', 'png', 'gif'])
def save_file(f):
  content = StringIO(f.read())
  try:
    mime = Image.open(content).format.lower()
    if mime not in allow_formats:
      raise IOError()
  except IOError:
    flask.abort(400)
  sha1 = hashlib.sha1(content.getvalue()).hexdigest()
  c = dict(
    content=bson.binary.Binary(content.getvalue()),
    mime=mime,
    time=datetime.datetime.utcnow(),
    sha1=sha1,
  )
  try:
    db.files.save(c)
  except pymongo.errors.DuplicateKeyError:
    pass
  return sha1
@app.route('/f/<sha1>')
def serve_file(sha1):
  try:
    f = db.files.find_one({'sha1': sha1})
    if f is None:
      raise bson.errors.InvalidId()
    if flask.request.headers.get('If-Modified-Since') == f['time'].ctime():
      return flask.Response(status=304)
    resp = flask.Response(f['content'], mimetype='image/' + f['mime'])
    resp.headers['Last-Modified'] = f['time'].ctime()
    return resp
  except bson.errors.InvalidId:
    flask.abort(404)
@app.route('/upload', methods=['POST'])
def upload():
  f = flask.request.files['uploaded_file']
  sha1 = save_file(f)
  return flask.redirect('/f/' + str(sha1))
@app.route('/')
def index():
  return '''
  <!doctype html>
  <html>
  <body>
  <form action='/upload' method='post' enctype='multipart/form-data'>
     <input type='file' name='uploaded_file'>
     <input type='submit' value='Upload'>
  </form>
  '''
if __name__ == '__main__':
  app.run(port=7777)


3, REF

Developing RESTful Web APIs with Python, Flask and MongoDB

(link: http://www.slideshare.net/nicolaiarocci/developing-restful-web-apis-with-python-flask-and-mongodb)

(link: https://github.com/nicolaiarocci/eve)

Related articles: