TensorFlow neural network to create MNIST data set of multilayer perceptron

A complete Softmax Regression was implemented using TensorFlow, and the accuracy rate was about 92% on MNIST data sum.

Portal: TensorFlow Tutorial Softmax Logistic Regression Recognition of Handwritten Numerals MNIST Datasets

Now, a neural network model (multilayer perceptron) with one hidden layer is established.

import tensorflow as tf
import numpy as np
import input_data
mnist = input_data.read_data_sets('data/', one_hot=True)
n_hidden_1 = 256
n_input    = 784
n_classes  = 10
# INPUTS AND OUTPUTS
x = tf.placeholder(tf.float32, [None, n_input]) #  Use placeholder Preemptive place, the number of samples is uncertain None
y = tf.placeholder(tf.float32, [None, n_classes]) #  Use placeholder Preemptive place, the number of samples is uncertain None
# NETWORK PARAMETERS
weights = {
    'w1': tf.Variable(tf.random_normal([n_input, n_hidden_1], stddev=0.1)),
    'out': tf.Variable(tf.zeros([n_hidden_1, n_classes]))
}
biases = {
    'b1': tf.Variable(tf.zeros([n_hidden_1])),
    'out': tf.Variable(tf.zeros([n_classes]))
}
print("NETWORK READY")

def multilayer_perceptron(_X, _weights, _biases): #  Forward propagation, l1 , l2 Every 1 Add after layer relu Activation function 
    layer_1 = tf.nn.relu(tf.add(tf.matmul(_X, _weights['w1']), _biases['b1'])) #  Hidden layer 
    return (tf.matmul(layer_1, _weights['out']) + _biases['out']) #  Returns the results of the output layer and obtains 10 Score value of each category 

pred = multilayer_perceptron(x, weights, biases) #  Predicted value of forward propagation 
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y)) #  Cross entropy loss function, parameters are predicted values respectively pred And actual label Value y , reduce_mean For average loss
optm = tf.train.GradientDescentOptimizer(0.01).minimize(cost) #  Gradient descent optimizer 
corr = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1)) # tf.equal() Compare the index of predicted values with the actual ones label Is the index of 1 Sample, 1 Sample return True No 1 Sample return False
accr = tf.reduce_mean(tf.cast(corr, tf.float32)) #  Will pred I.e. True Or False Convert to 1 Or 0, And calculate the average value of all the judgment results 
init = tf.global_variables_initializer()
print("FUNCTIONS READY")

#  After the above neural network structure is defined, the following is defined 1 Some hyper-parameters 
training_epochs = 100 #  Iteration of all samples 100 Times 
batch_size = 100 #  Every time 1 Sub-iterative selection 100 Samples 
display_step = 5
# LAUNCH THE GRAPH
sess = tf.Session() #  Definition 1 A Session
sess.run(init) #  In sess Li run1 Next initialization operation 
# OPTIMIZE
for epoch in range(training_epochs):
    avg_cost = 0.
    total_batch = int(mnist.train.num_examples/batch_size)
    # Loop over all batches
    for i in range(total_batch):
        batch_xs, batch_ys = mnist.train.next_batch(batch_size) #  One by one batch To fetch data of 
        sess.run(optm, feed_dict={x: batch_xs, y: batch_ys})
        avg_cost += sess.run(cost, feed_dict={x: batch_xs, y: batch_ys})/total_batch
    # Display logs per epoch step
    if epoch % display_step == 0:
        train_acc = sess.run(accr, feed_dict={x: batch_xs, y: batch_ys})
        test_acc = sess.run(accr, feed_dict={x: mnist.test.images, y: mnist.test.labels})
        print("Epoch: %03d/%03d cost: %.9f TRAIN ACCURACY: %.3f TEST ACCURACY: %.3f"
              % (epoch, training_epochs, avg_cost, train_acc, test_acc))
print("DONE")

Iterate 100 times to see the effect, and the program runs as follows:

Epoch: 095/100 cost: 0.076462782 TRAIN ACCURACY: 0.990 TEST ACCURACY: 0.970

Finally, the accuracy rate on the test set reaches 97%, and the accuracy rate will increase with the increase of iteration times. Compared with the previous Softmax, our error rate has been reduced from 8% to 3% after 100 training iterations, which can be said to be a leap improvement in the scene with high accuracy for identifying bank bills. This promotion is achieved only by adding one hidden layer, which shows how remarkable the effect of multi-layer neural network is.

Softmax Regression without hidden layer can only infer which number is directly from the pixels of the image, without the process of feature abstraction. Multi-layer neural network can combine high-order features, such as horizontal lines, vertical lines, circles, etc., and then combine these high-order features or components into numbers to achieve accurate matching and classification.

However, the use of fully connected neural networks is limited. Even if we use deep networks, many hidden nodes and a large number of iterations, it is difficult to achieve an accuracy rate of over 99% on MNIST data sets.

The above is the TensorFlow neural network to create multilayer perceptron MNIST data set details, more about TensorFlow to create multilayer perceptron MNIST data set, please pay attention to other related articles on this site!