Introduction to python Neural Network TensorFlow Common Basic Operation Tutorial

Directory other 1 mainstream deep learning tool framework comparison variables: create, initialize, save and load-add neural layer-loss-create-initialize-initialize by another variable

To apply deep learning to new problems faster and more conveniently, choosing a deep learning tool is an essential step.

TensorFlow is a computing framework officially opened by Google on November 9, 2015.

TensorFlow computing framework can well support various algorithms of deep learning.

TensorFlow is well compatible with the different needs of academic research and industrial production.

On the one hand, the flexibility of TensorFlow enables researchers to use it to quickly realize new model design;

On the other hand, the strong distributed support of TensorFlow is also very important for the model training of industry on massive data sets.

As Google's open source deep learning framework, TensorFlow includes Google's exploration of artificial intelligence and successful commercial applications in the past 10 years.

In addition to TensorFlow, there are currently 1 mainstream deep learning open source tools available. Each tool has its own characteristics, and you can choose your own deep learning tools according to your own needs and preferences. For example, when I started to learn deep learning, I came into contact with Caffe, and then after TensorFlow opened source, I learned some features of TensorFlow, and I still feel that I prefer the style of TensorFlow. Of course, it is a good thing to use more than one deep learning tool.

Other mainstream deep learning tools

Caffe: http://caffe. berkeleyvision. org/(Convolutional Architecture for Fast Feature Embedding) BVLC

MXNet: http://mxnet.readthedocs.io/en/latest/(Baidu DMLC (Distributed Machine Learning Community) is referred to as "Shenmeng")

Torch: http://torch.ch/(Facebook Google DeepMind Twitter FAIR)

Theano: http://deeplearning.net/software/theano/(the LISA group at the University of Montreal (Montreal))

TensorFlow: https://www.tensorflow.org/(Google)

CNTK (Microsoft Deep Learning Toolkit)

DeepLearning4J: http://deeplearning4j.org/

deepmat: https://github.com/kyunghyuncho/deepmat

Neon: http://neon.nervanasys.com/docs/latest/index.html

Eblearn: http://eblearn.sourceforge.net/

PyLearn: http://deeplearning.net/software/pylearn2/

chainer: https://github.com/pfnet/chainer

Bahrampour S, Ramakrishnan N, Schott L, et al. Comparative Study of Deep Learning Software Frameworks[J]. Computer Science, 2016.

Frame contrast

This article makes a rigorous comparison of five popular open source deep learning frameworks caffe, Neon, TensorFlow, Theano and Torch.

The authors open source their comparative Benchmarks code: https://github.com/DL-Benchmarks/DL-Benchmarks

The article compares:

Scalability (extensibility), hardware utilization (hardware utilization) and speed (speed)

The evaluation tests are all deployed on a stand-alone machine and tested for multithreaded CPU, GPU (Nvidia Titan X) Velocity evaluation criteria include gradient calculation time (gradient computation time) and forward propagation time (forward time). For convolutional neural networks, the author also The different convolution algorithms supported by these depth frameworks and their corresponding performance are tested

The following conclusions are obtained through experiments

Theano and Torch are the most extensible deep learning frameworks According to the test performance on CPU, Torch is the best, followed by Theano On GPU, Torch is the best for large-scale convolution and fully connected network, followed by Neon Theano won the battle in deploying and training LSTM network. caffe is the standard deep learning framework that is the easiest to test and evaluate performance Finally, TensorFlow is somewhat similar to Theano, which is a flexible framework, but its performance is not as good as the above frameworks at present

However, after all, this article is a thing of the past. At that time, TensorFlow could only use cuDNN v. 2, but now I have installed v 5.1, and TensorFlow has released version 1.0. Now, the performance of each tool needs a new evaluation to explain the problem.

Variables: Create, initialize, save and load

When training the model, variables are used to store and update parameters. The variable containing tensor (Tensor) is stored in the buffer area of memory. They need to be explicitly initialized when modeling, and they must be stored to disk after training the model. The values of these variables can be loaded after model training and analysis.

This document describes the following two TensorFlow classes. Click the following link to view the complete API documentation:
Refer to TensorFlow Chinese Community

-Adding a neural layer

The input parameters are inputs , in_size , out_size , and activation_function

#  Add layer 
def add_layer(inputs, in_size, out_size, activation_function=None):
            weights = tf.Variable(tf.random_normal([in_size, out_size]), name='weights')
            biases = tf.Variable(tf.zeros([1, out_size]) + 0.1, name='biases')
            y = tf.matmul(inputs, weights) + biases
        if activation_function is None:
            outputs = y
        else:
            outputs = activation_function(y)
        return outputs

- loss

loss = tf.reduce_mean(tf.reduce_sum(tf.square(ys - prediction), reduction_indices=[1]))

loss function cross_entropy cross entropy of classification problem
loss = tf.reduce_mean(-tf.reduce_sum(ys * tf.log(prediction), reduction_indices=[1]))

-Create

When you create a variable, you pass a tensor as the initial value to the constructor Variable (). TensorFlow provides 1 series of operators to initialize a tensor, with the initial value being constant or random.
Note that all these operators require you to specify the shape of the tensor. That shape automatically becomes the shape of the variable. The shape of a variable is usually fixed, but the TensorFlow provides an advanced mechanism to readjust the number of columns and columns.

# Create two variables.
weights = tf.Variable(tf.random_normal([784, 200], stddev=0.35), name="weights")
biases = tf.Variable(tf.zeros([200]), name="biases")

-Initialize

The initialization of variables must be explicitly completed before other operations of the model run. The easiest way is to add an operation that initializes all variables and run that operation first before using the model.
Use tf.global_variables_initializer() Add 1 action to initialize the variable. Remember to run that operation after the model is completely built and loaded.

# 7. Initialization variable 
init = tf.global_variables_initializer()
# tf.global_variables_initializer() Is to initialize all variables in parallel 
#  Sometimes you need to use another 1 Initializes the current variable with the initialization value of , This requires attention 
#  Class with the values of other variables 1 When a new variable is created, use the initialized_value() Property. 
#  You can take the initialized value directly as the initial value of the new variable, or treat it as the initial value of the new variable tensor Calculate 1 A value is assigned to the new variable. 
# w1 = tf.Variable(tf.random_normal([784, 200], stddev=0.35), name="w1")
# w2 = tf.Variable(w1.initialized_value(), name="w2")

# 8. Startup diagram  (graph)
sess = tf.Session()
sess.run(init)

-Initialized by another variable

Sometimes you need to initialize the current variable with the initialization value of another variable. Due to tf.global_variables_initializer() Is to initialize all variables in parallel, so be careful when there is such a requirement. When initializing a new variable with the value of another variable, use the initialized_value() Property. You can take the initialized value directly as the initial value of the new variable, or calculate it as tensor to get a value and give it to the new variable.

w1 = tf.Variable(tf.random_normal([784, 200], stddev=0.35), name="w1")
w2 = tf.Variable(w1.initialized_value(), name="w2")

The above is python neural network TensorFlow introduction commonly used basic operation tutorial details, more information about TensorFlow basic operation introduction please pay attention to other related articles on this site!