TensorFlow tutorial Softmax logistic regression recognition of handwritten numerals MNIST data sets

Logistic regression model based on MNIST data set to do 10 classification tasks

Softmax Regression without hidden layer can only infer which number is directly from the pixels of the image, without the process of feature abstraction. Multi-layer neural network can combine high-order features, such as horizontal lines, vertical lines, circles, etc., and then combine these high-order features or components into numbers to achieve accurate matching and classification.

import tensorflow as tf
import numpy as np
import input_data
print('Download and Extract MNIST dataset')
mnist = input_data.read_data_sets('data/', one_hot=True) # one_hot=True It means that the encoding format is 01 Code 
print("tpye of 'mnist' is %s" % (type(mnist)))
print("number of train data is %d" % (mnist.train.num_examples))
print("number of test data is %d" % (mnist.test.num_examples))
trainimg = mnist.train.images
trainlabel = mnist.train.labels
testimg = mnist.test.images
testlabel = mnist.test.labels
print("MNIST loaded")

"""
print("type of 'trainimg' is %s"    % (type(trainimg)))
print("type of 'trainlabel' is %s"  % (type(trainlabel)))
print("type of 'testimg' is %s"     % (type(testimg)))
print("type of 'testlabel' is %s"   % (type(testlabel)))
print("------------------------------------------------")
print("shape of 'trainimg' is %s"   % (trainimg.shape,))
print("shape of 'trainlabel' is %s" % (trainlabel.shape,))
print("shape of 'testimg' is %s"    % (testimg.shape,))
print("shape of 'testlabel' is %s"  % (testlabel.shape,))

"""
x = tf.placeholder(tf.float32, [None, 784])
y = tf.placeholder(tf.float32, [None, 10]) # None is for infinite
w = tf.Variable(tf.zeros([784, 10])) #  For convenience, use it directly 0 Initialization, can be Gaussian initialization 
b = tf.Variable(tf.zeros([10])) # 10 Classified tasks, 10 Species label So you only need to initialize 10 A b
pred = tf.nn.softmax(tf.matmul(x, w) + b) #  Predicted value of forward propagation 
cost = tf.reduce_mean(-tf.reduce_sum(y*tf.log(pred), reduction_indices=[1])) #  Cross entropy loss function 
optm = tf.train.GradientDescentOptimizer(0.01).minimize(cost)
corr = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1)) # tf.equal() Compare the index and reality of predicted values label Is the index of 1 Sample, 1 Sample return True No 1 Sample return False
accr = tf.reduce_mean(tf.cast(corr, tf.float32))
init = tf.global_variables_initializer() #  Global parameter initializer 
training_epochs = 100 #  Iteration of all samples 100 Times 
batch_size = 100 #  Every time 1 Sub-iterative selection 100 Samples 
display_step = 5
# SESSION
sess = tf.Session() #  Definition 1 A Session
sess.run(init) #  In sess Li run1 Next initialization operation 
# MINI-BATCH LEARNING
for epoch in range(training_epochs): #  Every 1 A epoch Carry on a cycle 
    avg_cost = 0. #  The initial loss value is defined as 0
    num_batch = int(mnist.train.num_examples/batch_size)
    for i in range(num_batch): #  Every 1 A batch Make a choice 
        batch_xs, batch_ys = mnist.train.next_batch(batch_size) #  Pass next_batch() You can 1 A 1 A batch Get the data, 
        sess.run(optm, feed_dict={x: batch_xs, y: batch_ys}) # run1 Gradient descent is used to solve the problem, through placeholder Put x , y Pass in 
        avg_cost += sess.run(cost, feed_dict={x: batch_xs, y:batch_ys})/num_batch
    # DISPLAY
    if epoch % display_step == 0: # display_step Previously defined as 5 , every time here 5 A epoch Print 1 Under 
        train_acc = sess.run(accr, feed_dict={x: batch_xs, y:batch_ys})
        test_acc = sess.run(accr, feed_dict={x: mnist.test.images, y: mnist.test.labels})
        print("Epoch: %03d/%03d cost: %.9f TRAIN ACCURACY: %.3f TEST ACCURACY: %.3f"
              % (epoch, training_epochs, avg_cost, train_acc, test_acc))
print("DONE")

Iterate 100 times and run the model once. Finally, the accuracy rate can reach 92.2% on the test set. Although it is not bad, it is not practical. The main application scenario of handwritten digit recognition is to recognize bank checks. If the accuracy is not high enough, it may cause serious consequences.

Epoch: 095/100 loss: 0.283259882 train_acc: 0.940 test_acc: 0.922

Insert some knowledge points about the usage of some functions in tensorflow

sess = tf.InteractiveSession()
arr = np.array([[31, 23,  4, 24, 27, 34],
                [18,  3, 25,  0,  6, 35],
                [28, 14, 33, 22, 30,  8],
                [13, 30, 21, 19,  7,  9],
                [16,  1, 26, 32,  2, 29],
                [17, 12,  5, 11, 10, 15]])

 In tensorflow To print in .eval()
tf.rank(arr).eval() #  Print matrix arr Dimensions of 
tf.shape(arr).eval() #  Print matrix arr The size of 
tf.argmax(arr, 0).eval() #  Print the index of the maximum value, parameter 0 To index by column, 1 Indexing by row 

The above is the TensorFlow tutorial Softmax logistic regression recognition handwritten digits MNIST data set details, more about Softmax logistic regression MNIST data set handwriting recognition information please pay attention to other related articles on this site!