吴恩达神经网络与深度学习——深度神经网络 - 好文

吴恩达神经网络与深度学习——深度神经网络

* 深度神经网络 <https://blog.csdn.net/cherry1307/article/details/83503218#_1>
* 符号 <https://blog.csdn.net/cherry1307/article/details/83503218#_3>
* 前向传播 <https://blog.csdn.net/cherry1307/article/details/83503218#_16>
* 矩阵维度 <https://blog.csdn.net/cherry1307/article/details/83503218#_53>
* m个样本 <https://blog.csdn.net/cherry1307/article/details/83503218#m_102>
* 为什么使用深层表示 <https://blog.csdn.net/cherry1307/article/details/83503218#_144>
* 搭建深层神经网络块 <https://blog.csdn.net/cherry1307/article/details/83503218#_150>
* 正向传播和反向传播 <https://blog.csdn.net/cherry1307/article/details/83503218#_163>
* 前向和反向传播 <https://blog.csdn.net/cherry1307/article/details/83503218#_173>
* 前向传播 <https://blog.csdn.net/cherry1307/article/details/83503218#_174>
* 反向传播 <https://blog.csdn.net/cherry1307/article/details/83503218#_184>
* 参数和超参数 <https://blog.csdn.net/cherry1307/article/details/83503218#_201>
* 和大脑的关系 <https://blog.csdn.net/cherry1307/article/details/83503218#_213>

<>深度神经网络

<>符号

l：层数 l = 4 n^[l]:每一次的单元数 n^[1] = 5 n^[2] = 5 n^[3] = 3 n^[4] = 1
a^[l]:每一次的激活函数 a^[l] = g^[l](z^[l]) w^[l]:每一次的权值 b^[l]:每一次的偏置
<>前向传播

x z^[1] = w^[1]x + b^[1] a^[1] = g^[1](z^[1]) z^[2] = w^[2]a^[1] + b^[2] a^[2]
= g^[2](z^[2]) z^[3] = w^[3]a^[2] + b^[3] a^[3] = g^[3](z^[3]) z^[4] =
w^[4]a^[3] + b^[4] a^[4] = g^[4](z^[4]) for l =1 to 4 z^[l] = w^[l]a^[l-1] +
b^[l] a^[l] = g^[l](z^[l]) #m个样本向量化 Z^[1] = W^[1]A^[0] + b^[1] # X=A^[0] A^[1]
= g^[1](Z^[1]) Z^[2] = W^[2]A^[1]+ b^[2] A^[2] = g^[2](z^[2]) Z^[3] =
W^[3]A^[2] + b^[3] A^[3] = g^[3](Z^[3]) Z^[4] = W^[4]A^[3] + b^[4] A^[4] =
g^[4](Z^[4]) for l = 1 to 4 Z^[l] = w^[l]A^[l-1] + b^[l] A^[l] = g^[l](Z^[l])
<>矩阵维度

n^[0] = 2 n^[1] = 3 n^[2] = 5 n^[3] = 4 n^[4] = 2 n^[4] = 1 z^[1] = w^[1] x +
b^[1] (3,1) (3,2) (2,1) (3,1) (n^[1],1) (n^[1],n^[0]) (n^[0],1) (n^[1],1) a^[1]
= g^[1](z^[1]) (3,1) (3,1) (n^[1],1) (n^[1],1) z^[2] = w^[2] a^[1] + b^[1]
(5,1) (5,3) (3,1) (5,1) (n^[2],1) (n^[2],n^[1]) (n^[1],1) (n^[2],1) a^[2] =
g^[2](z^[2]) (5,1) (5,1) (n^[2],1) (n^[2],1) z^[3] = w^[3] a^[2] + b^[3] (4,1)
(4,5) (5,1) (4,1) (n^[3],1) (n^[3],n^[2]) (n^[2],1) (n^[3],1) a^[3] =
g^[3](z^[3]) (4,1) (4,1) (n^[3],1) (n^[3],1) z^[4] = w^[4] a^[3] + b^[4] (2,1)
(2,4) (4,1) (2,1) (n^[4],1) (n^[4],n^[3]) (n^[3],1) (n^[4],1) a^[4] =
g^[4](z^[4]) (2,1) (2,1) (n^[4],1) (n^[4],1) z^[5] = w^[5] a^[4] + b^[5] (1,1)
(1,2) (2,1) (1,1) (n^[5],1) (n^[5],n^[4]) (n^[4],1) (n^[5],1) a^[5] =
g^[5](z^[5]) (1,1) (1,1) (n^[5],1) (n^[5],1) for l = 1 to 5 z^[l] = w^[l]
a^[l-1] + b^[l] (n^[l],1) (n^[l],n^[l-1]) (n^[l-1],1) (n^[l],1) a^[l] =
g^[l](z^[l]) (n^[l],1) (n^[l],1)
<>m个样本
Z^[1] = W^[1] X + b^[1] (3,m) (3,2) (2,m) (3,1) (n^[1],m) (n^[1],n^[0])
(n^[0],m) (n^[1],1) A^[1] = g^[1](Z^[1]) (3,m) (3,m) (n^[1],m) (n^[1],m) Z^[2]
= W^[2] A^[1] + b^[1] (5,m) (5,3) (3,m) (5,1) (n^[2],m) (n^[2],n^[1]) (n^[1],m)
(n^[2],1) A^[2] = g^[2](Z^[2]) (5,m) (5,m) (n^[2],m) (n^[2],m) Z^[3] = W^[3]
A^[2] + b^[3] (4,m) (4,5) (5,m) (4,1) (n^[3],m) (n^[3],n^[2]) (n^[2],m)
(n^[3],1) A^[3] = g^[3](Z^[3]) (4,m) (4,m) (n^[3],m) (n^[3],m) Z^[4] = W^[4]
A^[3] + b^[4] (2,m) (2,4) (4,m) (2,1) (n^[4],m) (n^[4],n^[3]) (n^[3],m)
(n^[4],1) A^[4] = g^[4](Z^[4]) (2,m) (2,m) (n^[4],m) (n^[4],m) Z^[5] = W^[5]
A^[4] + b^[5] (1,m) (1,2) (2,m) (1,1) (n^[5],m) (n^[5],n^[4]) (n^[4],m)
(n^[5],1) A^[5] = g^[5](Z^[5]) (1,m) (1,m) (n^[5],m) (n^[5],m) for l = 1 to 4
Z^[l] = w^[l] A^[l-1] + b^[l] (n^[l],m) (n^[l],n^[l-1]) (n^[l-1],m) (n^[l],1)
A^[l] = g^[l](Z^[l]) (n^[l],m) (n^[l],m)
<>为什么使用深层表示
深度神经网络有很多的隐层，较早的前几层能学习一些低层次的简单特征，后几层就能将简单的特征结合起来去探测更加复杂的东西
<>搭建深层神经网络块

第l层参数： w^[l],b^[l] 前向传播：输入 a^[l-1] 输出 a^[l] 存储 z^[l] z^[l] = w^[l]a^[l-1] +
b^[l] a^[l] = g^[l](z^[l]) 反向传播：输入 da^[l] 输出da^[l-1] dw^[l] db^[l] 前向传播存储的z^[l]

<>正向传播和反向传播

从a^[0]开始，也就是x，经过一系列正向传播计算得到yhat,之后再用输出值计算实现反向传播，得到所有的导数项，w，b也在每一层更新
编程细节:将z^[l],w^[l],b^[l]存储
<>前向和反向传播

<>前向传播
前向传播：输入 a^[l-1] 输出 a^[l] 存储 z^[l] z^[l] = w^[l]a^[l-1] + b^[l] a^[l] =
g^[l](z^[l]) 向量化： Z^[l] = W^[l]A^[l-1] + b^[l] A^[l] = g^[l](Z^[l])
<>反向传播
反向传播：输入 da^[l] 输出da^[l-1] dw^[l] db^[l] dz^[l] = da^[l]*g'^[l](z^[l]) dw^[l]
= dz^[l]a^[l-1] db^[l] = dz^[l] da^[l-1] = w^[l]Tdz^[l] dz^[l] =
w^[l+1]Tdz^[l+1]*g'^[l](z^[l]) 向量化： dZ^[l] = dA^[l]*g'^[l](Z^[l]) dW^[l]
=(1/m)dZ^[l]A^[l-1]T db^[l] = (1/m)np.sum(dZ^[l],axis = 1,keepdims = True)
dA^[l-1] = W^[l]TdZ^[l] dZ^[l] = W^[l+1]TdZ^[l+1]*g'^[l](Z^[l])

<>参数和超参数
参数： w^[1],b^[1],w^[2],b^[2]... 超参数：学习率：alpha 循环下降法的迭代次数：iteration 隐藏层数：l
隐藏单元数：n^[1],n^[2]... 激活函数：sigmoid ,relu, tanh
<>和大脑的关系

热门工具换一换