Extension of Pytorch Operation of tensor

I won't talk too much, let's just look at the code ~

b = torch.zeros((3, 2, 6, 6))
a = torch.zeros((3, 2, 1, 1))
a.expand_as(b).size()
Out[32]: torch.Size([3, 2, 6, 6])
a = torch.zeros((3, 2, 2, 1))
a.expand_as(b).size()
Traceback (most recent call last):
 File "/home/lart/.conda/envs/pt/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3267, in run_code
  exec(code_obj, self.user_global_ns, self.user_ns)
 File "<ipython-input-34-972575f79e92>", line 1, in <module>
  a.expand_as(b).size()
RuntimeError: The expanded size of the tensor (6) must match the existing size (2) at non-singleton dimension 2. Target sizes: [3, 2, 6, 6]. Tensor sizes: [3, 2, 2, 1]
a = torch.zeros((3, 2, 1, 2))
a.expand_as(b).size()
Traceback (most recent call last):
 File "/home/lart/.conda/envs/pt/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3267, in run_code
  exec(code_obj, self.user_global_ns, self.user_ns)
 File "<ipython-input-36-972575f79e92>", line 1, in <module>
  a.expand_as(b).size()
RuntimeError: The expanded size of the tensor (6) must match the existing size (2) at non-singleton dimension 3. Target sizes: [3, 2, 6, 6]. Tensor sizes: [3, 2, 1, 2]
a = torch.zeros((3, 2, 2, 2))
a.expand_as(b).size()
Traceback (most recent call last):
 File "/home/lart/.conda/envs/pt/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3267, in run_code
  exec(code_obj, self.user_global_ns, self.user_ns)
 File "<ipython-input-38-972575f79e92>", line 1, in <module>
  a.expand_as(b).size()
RuntimeError: The expanded size of the tensor (6) must match the existing size (2) at non-singleton dimension 3. Target sizes: [3, 2, 6, 6]. Tensor sizes: [3, 2, 2, 2]
a = torch.zeros((3, 2, 6, 2))
a.expand_as(b).size()
Traceback (most recent call last):
 File "/home/lart/.conda/envs/pt/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3267, in run_code
  exec(code_obj, self.user_global_ns, self.user_ns)
 File "<ipython-input-40-972575f79e92>", line 1, in <module>
  a.expand_as(b).size()
RuntimeError: The expanded size of the tensor (6) must match the existing size (2) at non-singleton dimension 3. Target sizes: [3, 2, 6, 6]. Tensor sizes: [3, 2, 6, 2]
a = torch.zeros((3, 2, 6, 1))
a.expand_as(b).size()
Out[44]: torch.Size([3, 2, 6, 6])
a = torch.zeros((3, 2, 1, 6))
a.expand_as(b).size()
Out[46]: torch.Size([3, 2, 6, 6])

tensor. expand_as is used here to extend tensor to the target shape, most commonly in the direction of H and W.

Assuming that the target shapes are N, C, H and W, tensor. size () = n, c, h and w are required (assuming that N and C are unchanged here):

1. h=w=1

2, h=1, w! = 1

3. h! = 1, w=1

Additional: tensorflow extends and compresses the tensor dimension with expand_dims and squeeze

When using tensorflow for text mining, it often involves dimension expansion and compression.

For example, after the embedding operation of text is completed, if you want to perform convolution operation, you need to expand the dimension of the vector of embedded, and expand [batch_size, embedding_dims] to [batch_size, embedding_dims, 1], which can be realized by tf.expand_dims (input,-1), and conversely, squeeze (input,-1) or tf. squeeze (input).

tf.expand_dims()

tf.squeeze()

tf.expand_dims()

tf.expand_dims(input, axis=None, name=None, dim=None)

Add 1 dimension at axis.

Given the tensor input, this operation inserts the dimension of 1 at the dimension index axis of the input shape. The dimension index axis starts from zero; If you specify a negative number for the axis, count backward from the last.

This is useful if you want to add a batch dimension to a single element. For example, if you have a single 1 shape [height, width, channels], you can use expand_dims (image, 0) to make it a 1 image, which will make the shape [1, height, width, channel].

Example

# 't' is a tensor of shape [2]
shape(expand_dims(t, 0)) ==> [1, 2]
shape(expand_dims(t, 1)) ==> [2, 1]
shape(expand_dims(t, -1)) ==> [2, 1]
# 't2' is a tensor of shape [2, 3, 5]
shape(expand_dims(t2, 0)) ==> [1, 2, 3, 5]
shape(expand_dims(t2, 2)) ==> [2, 3, 1, 5]
shape(expand_dims(t2, 3)) ==> [2, 3, 5, 1]

tf.squeeze()

tf.squeeze(input, axis=None, name=None, squeeze_dims=None)

Go directly to the example

# 't' is a tensor of shape [1, 2, 1, 3, 1, 1]
 shape(squeeze(t)) ==> [2, 3]
# 't' is a tensor of shape [1, 2, 1, 3, 1, 1]
 shape(squeeze(t, [2, 4])) ==> [1, 2, 3, 1]