Download as PDF, PPTX
![Discriminator
1 0
0 1
w1
w2
w4
w3
1
b
Bias
σ
x1
x2
x3
x4
Prediction
D(x) = σ(x1w1+x2w2+x3w3+x4w4 + b)
∂E
∂wi
=
∂E
∂D
⋅
∂D
∂wi
Loss function (error) from images
E = − ln(D(x))
=
−1
D(x)
⋅ σ(
4
∑
j=1
xjwj + b)[1 − σ(
4
∑
j=1
xjwj + b)]xi
=
−1
D(x)
⋅ D(x)[1 − D(x)]xi
Derivatives
∂E
∂b
=
∂E
∂D
⋅
∂D
∂b
= − [1 − D(x)]
= − [1 − D(x)]xi](https://image.slidesharecdn.com/gans-201013014510/75/Generative-Adversarial-Networks-GANs-45-2048.jpg)
![Discriminator
0.25 1
0.5 0.75
w1
w2
w4
w3
1
b
Bias
σ
x1
x2
x3
x4
∂E
∂wi
=
∂E
∂D
⋅
∂D
∂wi
Loss function (error) from noise
E = − ln(1 − D(x))
=
1
1 − D(x)
⋅ σ(
4
∑
j=1
xjwj + b)[1 − σ(
4
∑
j=1
xjwj + b)]xi
=
1
1 − D(x)
⋅ D(x)[1 − D(x)]xi
Derivatives
∂E
∂b
=
∂E
∂D
⋅
∂D
∂b
= D(x)
Prediction
D(x) = σ(x1w1+x2w2+x3w3+x4w4 + b)
= D(x)xi](https://image.slidesharecdn.com/gans-201013014510/75/Generative-Adversarial-Networks-GANs-46-2048.jpg)
![Generator Predictions
G(z) = (G1, G2, G3, G4)
∂E
∂wi
=
∂E
∂D
⋅
∂D
∂G
⋅
∂G
∂z
Loss function (error)
E = − ln(D(G(z))
Derivatives
=
−1
D(G(z))
⋅ σ(
4
∑
j=1
Giwi + b)[1 − σ(
4
∑
j=1
Giwi + b)]G(z) ⋅ σ(wiz + bi)[1 − σ(wiz + bi]z
=
−1
D(G(z))
⋅ D(G(z))[1 − D(G(z))] ⋅ Gi(1 − Gi)z
= (σ(v1z+c1), σ(v2z+c2), σ(v3z+c3), σ(v4z+c4))
D(G(z)) = σ(G1w1 + G2w2 + G3w3 + G4w4 + b)
= − [1 − D(G(z))] ⋅ Gi(1 − Gi)z
Biases
1
z
v1
v2
v4
v3
c1
c2
c4
c3
σ
σ
σ
σ
G1 G2
G3 G4
∂E
∂b
= − [1 − D(G(z))] ⋅ Gi(1 − Gi)](https://image.slidesharecdn.com/gans-201013014510/75/Generative-Adversarial-Networks-GANs-48-2048.jpg)
![Discriminator
1 0
0 1
w1
w2
w4
w3
1
b
Bias
σ
x1
x2
x3
x4
Prediction
D(x) = σ(x1w1+x2w2+x3w3+x4w4 + b)
∂E
∂wi
=
∂E
∂D
⋅
∂D
∂wi
Loss function (error) from images
E = − ln(D(x))
=
−1
D(x)
⋅ σ(
4
∑
j=1
xjwj + b)[1 − σ(
4
∑
j=1
xjwj + b)]xi
=
−1
D(x)
⋅ D(x)[1 − D(x)]xi
Derivatives
∂E
∂b
=
∂E
∂D
⋅
∂D
∂b
= − [1 − D(x)]
= − [1 − D(x)]xi](https://crownmelresort.com/image.slidesharecdn.com/gans-201013014510/75/Generative-Adversarial-Networks-GANs-45-2048.jpg)
![Discriminator
0.25 1
0.5 0.75
w1
w2
w4
w3
1
b
Bias
σ
x1
x2
x3
x4
∂E
∂wi
=
∂E
∂D
⋅
∂D
∂wi
Loss function (error) from noise
E = − ln(1 − D(x))
=
1
1 − D(x)
⋅ σ(
4
∑
j=1
xjwj + b)[1 − σ(
4
∑
j=1
xjwj + b)]xi
=
1
1 − D(x)
⋅ D(x)[1 − D(x)]xi
Derivatives
∂E
∂b
=
∂E
∂D
⋅
∂D
∂b
= D(x)
Prediction
D(x) = σ(x1w1+x2w2+x3w3+x4w4 + b)
= D(x)xi](https://crownmelresort.com/image.slidesharecdn.com/gans-201013014510/75/Generative-Adversarial-Networks-GANs-46-2048.jpg)
![Generator Predictions
G(z) = (G1, G2, G3, G4)
∂E
∂wi
=
∂E
∂D
⋅
∂D
∂G
⋅
∂G
∂z
Loss function (error)
E = − ln(D(G(z))
Derivatives
=
−1
D(G(z))
⋅ σ(
4
∑
j=1
Giwi + b)[1 − σ(
4
∑
j=1
Giwi + b)]G(z) ⋅ σ(wiz + bi)[1 − σ(wiz + bi]z
=
−1
D(G(z))
⋅ D(G(z))[1 − D(G(z))] ⋅ Gi(1 − Gi)z
= (σ(v1z+c1), σ(v2z+c2), σ(v3z+c3), σ(v4z+c4))
D(G(z)) = σ(G1w1 + G2w2 + G3w3 + G4w4 + b)
= − [1 − D(G(z))] ⋅ Gi(1 − Gi)z
Biases
1
z
v1
v2
v4
v3
c1
c2
c4
c3
σ
σ
σ
σ
G1 G2
G3 G4
∂E
∂b
= − [1 − D(G(z))] ⋅ Gi(1 − Gi)](https://crownmelresort.com/image.slidesharecdn.com/gans-201013014510/75/Generative-Adversarial-Networks-GANs-48-2048.jpg)
More Related Content
Similar to Generative Adversarial Networks (GANs)
![[Redis Released]- FalkorDB - Redis + Graph Agentic Memory’s Secret Sauce](https://cdn.slidesharecdn.com/ss_thumbnails/redisreleased-falkordbslidedeck-1125-251115194922-e1c0046b-thumbnail.jpg?width=640&height=640&fit=bounds)
Generative Adversarial Networks (GANs)
- 1. A friendly introductionto Generative Adversarial Networks Luis Serrano
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.
- 12.
- 13. Slanted Land Slanted people2x2 screens 1-layer Neural networks
- 14.
- 15.
- 16.
- 17. 0.25 0 1 0.75 0.251 0.5 0.75 0.75 0.5 0.75 0 1 1 0 0.75 0.75 0 0 0.75 1 0.25 0.25 0.75 1 0 0 1 0.75 0 0.25 0.75 Faces Noise 10
- 18.
- 19. Building the discriminator FacesNoise 1 0 0 1 0.25 1 0.5 0.75 Big Big small small any any any any
- 20. Building the discriminator FacesNoise 1 0 0 1 0.25 1 0.5 0.75 + + + + -- - - 1*1 + 0*(-1) + 0*(-1) + 1*1 = 2 0.25*1 + 1*(-1) + 0.5*(-1) + 0.75*1 = -0.5Threshold = 1 More than 1: face. Less than 1: no face
- 21. Discriminator 1 0 0 1 +1 -1 +1 -1 1 -1 Bias 1*1 +0*(-1) +0*(-1) +1*1 -1 1σ() = 0.73 σ(x) = 1 1 + e−x 1 1 0.73
- 22. Discriminator 1Bias 0.25*1 +1*(-1) +0.5*(-1) +0.75*1 -1 -0.5σ( ) 0.25 1 0.50.75 = 0.37 +1 -1 +1 -1 -1
- 23.
- 24. Building the generator FacesNoise 1 0 0 1 0.25 1 0.5 0.75 Big Big small small any any any any
- 25. Generator 0.85 0.15 0.15 0.85 Bias 1 -1 +1 -1 -1 +1 +1 +1-1 z 0 1 0.7 +1*0.7 + 1 +1*0.7 + 1 -1*0.7 - 1 -1*0.7 - 1 1.7 -1.7 -1.7 1.7 σ( ) σ( ) σ( ) σ( ) = 0.85 = 0.15 = 0.85 = 0.15
- 26. The training process: Errorfunctions
- 27. Log-loss error function Label:1 Prediction: 0.1 Error: large Label: 1 Prediction: 0.9 Error: small −ln(0.1) = 2.3 −ln(0.9) = 0.1 Error = -ln(prediction)
- 28. Log-loss error function Label:0 Prediction: 0.1 Error: small Label: 0 Prediction: 0.9 Error: large −ln(0.1) = 2.3 −ln(0.9) = 0.1 Error = -ln(1 - prediction)
- 29. Summary If we wanta prediction to be 1: Log-loss = -ln(prediction) y = − ln(x) High error Low error
- 30. Summary If we wanta prediction to be 0: Log-loss = -ln(1-prediction) y = − ln(1 − x) High error Low error
- 31.
- 32.
- 33.
- 34. Training the generatorand the discriminator
- 35. σ 1 Bias z σ σ σ σ 1 z 0 1 0.25 1 0.50.75 0.68 Want 0 Error = -ln(1-0.68) Want 1 Error = -ln(0.68) Generated image Bias Generator Discriminator −ln (D(G(z))) −ln (1 − D(G(z)))
- 36. σ 1 Bias 0.44 Error = -ln(0.44) 10 0 1 Real image Want 1 Discriminator 0 z σ σ σ σBias 1 z 1 Generator −ln (D(x))
- 37.
- 38.
- 39. σ 1 Bias z σ σ σ σBias 1 z 0 1 1 0 01 Real image Generator Discriminator
- 40. After many ofthese iterations (epochs)
- 41.
- 42.
- 43.
- 45. Discriminator 1 0 0 1 w1 w2 w4 w3 1 b Bias σ x1 x2 x3 x4 Prediction D(x)= σ(x1w1+x2w2+x3w3+x4w4 + b) ∂E ∂wi = ∂E ∂D ⋅ ∂D ∂wi Loss function (error) from images E = − ln(D(x)) = −1 D(x) ⋅ σ( 4 ∑ j=1 xjwj + b)[1 − σ( 4 ∑ j=1 xjwj + b)]xi = −1 D(x) ⋅ D(x)[1 − D(x)]xi Derivatives ∂E ∂b = ∂E ∂D ⋅ ∂D ∂b = − [1 − D(x)] = − [1 − D(x)]xi
- 46. Discriminator 0.25 1 0.5 0.75 w1 w2 w4 w3 1 b Bias σ x1 x2 x3 x4 ∂E ∂wi = ∂E ∂D ⋅ ∂D ∂wi Lossfunction (error) from noise E = − ln(1 − D(x)) = 1 1 − D(x) ⋅ σ( 4 ∑ j=1 xjwj + b)[1 − σ( 4 ∑ j=1 xjwj + b)]xi = 1 1 − D(x) ⋅ D(x)[1 − D(x)]xi Derivatives ∂E ∂b = ∂E ∂D ⋅ ∂D ∂b = D(x) Prediction D(x) = σ(x1w1+x2w2+x3w3+x4w4 + b) = D(x)xi
- 48. Generator Predictions G(z) =(G1, G2, G3, G4) ∂E ∂wi = ∂E ∂D ⋅ ∂D ∂G ⋅ ∂G ∂z Loss function (error) E = − ln(D(G(z)) Derivatives = −1 D(G(z)) ⋅ σ( 4 ∑ j=1 Giwi + b)[1 − σ( 4 ∑ j=1 Giwi + b)]G(z) ⋅ σ(wiz + bi)[1 − σ(wiz + bi]z = −1 D(G(z)) ⋅ D(G(z))[1 − D(G(z))] ⋅ Gi(1 − Gi)z = (σ(v1z+c1), σ(v2z+c2), σ(v3z+c3), σ(v4z+c4)) D(G(z)) = σ(G1w1 + G2w2 + G3w3 + G4w4 + b) = − [1 − D(G(z))] ⋅ Gi(1 − Gi)z Biases 1 z v1 v2 v4 v3 c1 c2 c4 c3 σ σ σ σ G1 G2 G3 G4 ∂E ∂b = − [1 − D(G(z))] ⋅ Gi(1 − Gi)
- 50.
- 52.
- 53. With a littlehelp from my friends… Sahil Juneja @sjuneja90 Diego Gomez Mosquera https://medium.com/@diegoalejogm Alejandro Perdomo @aperdomoortiz
- 54.
- 55.
- 56. https://www.manning.com/books/grokking-machine-learning Discount code: serranoyt GrokkingMachine Learning By Luis G. Serrano
- 57. Thank you! @luis_likes_math Subscribe, like, share,comment! youtube.com/c/LuisSerrano http://serrano.academy