前向传播（张量）- 实战

目录

* 手写数字识别流程 <https://www.cnblogs.com/nickchen121/p/10849484.html#手写数字识别流程>
* 前向传播（张量）- 实战
<https://www.cnblogs.com/nickchen121/p/10849484.html#前向传播张量--实战>
手写数字识别流程

* MNIST手写数字集7000*10张图片
* 60k张图片训练，10k张图片测试
* 每张图片是28*28，如果是彩色图片是28*28*3
* 0-255表示图片的灰度值，0表示纯白，255表示纯黑
* 打平28*28的矩阵，得到28*28=784的向量
* 对于b张图片得到[b,784];然后对于b张图片可以给定编码
* 把上述的普通编码给定成独热编码，但是独热编码都是概率值，并且概率值相加为1，类似于softmax回归
* 套用线性回归公式
* X[b,784] W[784,10] b[10] 得到 [b,10]
* 高维图片实现非常复杂，一个线性模型无法完成，因此可以添加非线性因子
* f(X@W+b)，使用激活函数让其非线性化，引出relu函数
* 用了激活函数，模型还是太简单
* 使用工厂
* H1 =relu(X@W1+b1)
* H2 = relu(h1@W2+b2)
* Out = relu(h2@W3+b3)
* 第一步，把[1,784]变成[1,512]变成[1,256]变成[1,10]
* 得到[1,10]后将结果进行独热编码
* 使用欧氏距离或者使用mse进行误差度量
* [1,784]通过三层网络输出一个[1,10]
前向传播（张量）- 实战
import tensorflow as tf from tensorflow import keras from tensorflow.keras
import datasets import os # do not print irrelevant information #
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2' # x: [60k,28,28] # y: [60k] (x, y), _
= datasets.mnist.load_data() Downloading data from
https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
11493376/11490434 [==============================] - 1s 0us/step # transform
Tensor # x: [0~255] ==》 [0~1.] x = tf.convert_to_tensor(x, dtype=tf.float32) /
255. y = tf.convert_to_tensor(y, dtype=tf.int32) f'x.shape: {x.shape}, y.shape:
{y.shape}, x.dtype: {x.dtype}, y.dtype: {y.dtype}' "x.shape: (60000, 28, 28),
y.shape: (60000,), x.dtype: <dtype: 'float32'>, y.dtype: <dtype: 'int32'>"
f'min_x: {tf.reduce_min(x)}, max_x: {tf.reduce_max(x)}' 'min_x: 0.0, max_x: 1.0'
f'min_y: {tf.reduce_min(y)}, max_y: {tf.reduce_max(y)}' 'min_y: 0, max_y: 9' #
batch of 128 train_db = tf.data.Dataset.from_tensor_slices((x, y)).batch(128)
train_iter = iter(train_db) sample = next(train_iter) f'batch:
{sample[0].shape,sample[1].shape}' 'batch: (TensorShape([128, 28, 28]),
TensorShape([128]))' # [b,784] ==> [b,256] ==> [b,128] ==> [b,10] #
[dim_in,dim_out],[dim_out] w1 = tf.Variable(tf.random.truncated_normal([784,
256], stddev=0.1)) b1 = tf.Variable(tf.zeros([256])) w2 =
tf.Variable(tf.random.truncated_normal([256, 128], stddev=0.1)) b2 =
tf.Variable(tf.zeros([128])) w3 = tf.Variable(tf.random.truncated_normal([128,
10], stddev=0.1)) b3 = tf.Variable(tf.zeros([10])) # learning rate lr = 1e-3
for epoch in range(10): # iterate db for 10 # tranin every train_db for step,
(x, y) in enumerate(train_db): # x: [128,28,28] # y: [128] # [b,28,28] ==>
[b,28*28] x = tf.reshape(x, [-1, 28*28]) with tf.GradientTape() as tape: # only
data types of tf.variable are logged # x: [b,28*28] # h1 = x@w1 + b1 #
[b,784]@[784,256]+[256] ==> [b,256] + [256] ==> [b,256] + [b,256] h1 = x @ w1 +
tf.broadcast_to(b1, [x.shape[0], 256]) h1 = tf.nn.relu(h1) # [b,256] ==>
[b,128] # h2 = x@w2 + b2 # b2 can broadcast automatic h2 = h1 @ w2 + b2 h2 =
tf.nn.relu(h2) # [b,128] ==> [b,10] out = h2 @ w3 + b3 # compute loss # out:
[b,10] # y:[b] ==> [b,10] y_onehot = tf.one_hot(y, depth=10) # mse =
mean(sum(y-out)^2) # [b,10] loss = tf.square(y_onehot - out) # mean:scalar loss
= tf.reduce_mean(loss) # compute gradients grads = tape.gradient(loss, [w1, b1,
w2, b2, w3, b3]) # w1 = w1 - lr * w1_grad # w1 = w1 - lr * grads[0] # not in
situ update # in situ update w1.assign_sub(lr * grads[0]) b1.assign_sub(lr *
grads[1]) w2.assign_sub(lr * grads[2]) b2.assign_sub(lr * grads[3])
w3.assign_sub(lr * grads[4]) b3.assign_sub(lr * grads[5]) if step % 100 == 0:
print(f'epoch:{epoch}, step: {step}, loss:{float(loss)}') epoch:0, step: 0,
loss:0.5366693735122681 epoch:0, step: 100, loss:0.23276552557945251 epoch:0,
step: 200, loss:0.19647717475891113 epoch:0, step: 300,
loss:0.17389704287052155 epoch:0, step: 400, loss:0.1731622964143753 epoch:1,
step: 0, loss:0.16157487034797668 epoch:1, step: 100, loss:0.16654588282108307
epoch:1, step: 200, loss:0.15311869978904724 epoch:1, step: 300,
loss:0.14135733246803284 epoch:1, step: 400, loss:0.14423415064811707 epoch:2,
step: 0, loss:0.13703864812850952 epoch:2, step: 100, loss:0.14255204796791077
epoch:2, step: 200, loss:0.1302051544189453 epoch:2, step: 300,
loss:0.12224273383617401 epoch:2, step: 400, loss:0.12742099165916443 epoch:3,
step: 0, loss:0.1219201311469078 epoch:3, step: 100, loss:0.12757658958435059
epoch:3, step: 200, loss:0.11587800830602646 epoch:3, step: 300,
loss:0.10984969139099121 epoch:3, step: 400, loss:0.11641304194927216 epoch:4,
step: 0, loss:0.11171815544366837 epoch:4, step: 100, loss:0.11717887222766876
epoch:4, step: 200, loss:0.10604140907526016 epoch:4, step: 300,
loss:0.10111508518457413 epoch:4, step: 400, loss:0.10865814983844757 epoch:5,
step: 0, loss:0.10434548556804657 epoch:5, step: 100, loss:0.10952303558588028
epoch:5, step: 200, loss:0.09875871241092682 epoch:5, step: 300,
loss:0.09467941522598267 epoch:5, step: 400, loss:0.10282392799854279 epoch:6,
step: 0, loss:0.09874211996793747 epoch:6, step: 100, loss:0.10355912148952484
epoch:6, step: 200, loss:0.09315416216850281 epoch:6, step: 300,
loss:0.08971598744392395 epoch:6, step: 400, loss:0.0982089415192604 epoch:7,
step: 0, loss:0.09428335726261139 epoch:7, step: 100, loss:0.09877124428749084
epoch:7, step: 200, loss:0.08866965025663376 epoch:7, step: 300,
loss:0.08573523908853531 epoch:7, step: 400, loss:0.09440126270055771 epoch:8,
step: 0, loss:0.09056715667247772 epoch:8, step: 100, loss:0.09483197331428528
epoch:8, step: 200, loss:0.0849832147359848 epoch:8, step: 300,
loss:0.08246967941522598 epoch:8, step: 400, loss:0.09117519855499268 epoch:9,
step: 0, loss:0.08741479367017746 epoch:9, step: 100, loss:0.09150294959545135
epoch:9, step: 200, loss:0.08185736835002899 epoch:9, step: 300,
loss:0.07972464710474014 epoch:9, step: 400, loss:0.08842341601848602

• « 上一篇：bugku-本地包含(文件包含漏洞)
• » 下一篇：Unity 与C#服务器 实现Socket的UDP通信（多客户端）