Download as PDF, PPTX
![A psychological point of view
• Transfer of Learning (学习迁移）in
Educa7on and Psychology
– The study of dependency of human conduct,
learning or performance on prior experience.
– [Thorndike and Woodworth, 1901] explored how individuals would
transfer in one context to another context that share similar
characteristics.
• E.g.
! C++ " Java
! Math/Physics " Computer Science/Economics
2](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-2-2048.jpg)
![Transfer Learning
Different fields
• Transfer learning for
reinforcement learning.
[Taylor and Stone, Transfer
Learning for Reinforcement
Learning Domains: A Survey,
JMLR 2009]
• Transfer learning for
classifica7on, and
regression problems.
[Pan and Yang, A Survey on
Transfer Learning, IEEE TKDE
2010]
6
Focus!](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-6-2048.jpg)
![Instance-based Approaches
Case I (cont.)
22
Correcting Sample Selection Bias / Covariate Shift
[Quionero-Candela, etal, Data Shift in Machine Learning, MIT Press 2009]](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-22-2048.jpg)
![Instance-based Approaches
Correcting sample selection bias (cont.)
• The distribu7on of the selector variable maps
the target onto the source distribu7on
23
! Label instances from the source domain with label 1
! Label instances from the target domain with label 0
! Train a binary classifier
[Zadrozny, ICML-04]](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-23-2048.jpg)
![Instance-based Approaches
Kernel mean matching (KMM)
Maximum Mean Discrepancy (MMD)
[Alex Smola, Arthur Gretton and Kenji Kukumizu, ICML-08 tutorial]
24](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-24-2048.jpg)
![Instance-based Approaches
Direct density ratio estimation
25
[Sugiyama etal., NIPS-07, Kanamori etal., JMLR-09]
KL divergence loss Least squared loss
[Sugiyama etal., NIPS-07] [Kanamori etal., JMLR-09]](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-25-2048.jpg)
![Instance-based Approaches
Case II (cont.)
! TrAdaBoost [Dai etal ICML-07]
– For each boosting iteration,
# Use the same strategy as AdaBoost to
update the weights of target domain data.
# Use a new mechanism to decrease the
weights of misclassified source domain
data.
27](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-27-2048.jpg)
![Feature-based Approaches
Encode application-specific knowledge (cont.)
31
compact sharp blurry hooked realistic boring
1 1 0 0 0 0
0 1 0 0 0 0
0 0 1 0 0 0
( ) sgn( ), [1,1, 1,0,0,0]T
y f x w x w= = ⋅ = −
compact sharp blurry hooked realistic boring
0 0 0 1 0 0
0 0 0 1 1 0
0 0 0 0 0 1
Electronics
Video Game
Training
Prediction](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-31-2048.jpg)
![Feature-based Approaches
Encode application-specific knowledge (cont.)
! How to identify pivot features?
! Term frequency on both domains
! Mutual information between features and labels (source
domain)
! Mutual information on between features and domains
! How to utilize pivots to align features across domains?
! Structural Correspondence Learning (SCL) [Biltzer etal.
EMNLP-06]
! Spectral Feature Alignment (SFA) [Pan etal. WWW-10]
34](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-34-2048.jpg)
![Spectral Feature Alignment (SFA)
Derive new features (cont.)
sharp/hooked compact/realistic blurry/boring
1 1 0
1 0 0
0 0 1
38
( ) sgn( ), [1,1, 1]T
y f x w x w= = ⋅ = −
sharp/hooked compact/realistic blurry/boring
1 0 0
1 1 0
0 0 1
Electronics
Video Game
Training
Prediction](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-38-2048.jpg)
![Feature-based Approaches
Transfer Component Analysis [Pan etal., IJCAI-09, TNN-11]
41
TargetSource
Latent factors
Temperature Signal
properties
Building
structure
Power of APs
Motivation](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-41-2048.jpg)
![Feature-based Approaches
Self-taught Feature Learning (Andrew Ng. et al.)
! Intuition: Useful higher-level features can be learned from
unlabeled data.
! Steps:
1) Learn higher-level features from a lot of unlabeled data.
2) Use the learned higher-level features to represent the data of the
target task.
3) Train models from the new representations of the target task
(supervised)
! How to learn higher-level features
# Sparse Coding [Raina etal., 2007]
# Deep learning [Glorot etal., 2011]
48](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-48-2048.jpg)
![Multi-task Learning
Assumption:
If tasks are related, they may share similar parameter vectors.
For example, [Evgeniou and Pontil, KDD-04]
50
Common part
Specific part for individual task](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-50-2048.jpg)
![Mul7-task Feature Learning
51
[Argyriou etal., NIPS-07]
[Ando and Zhang, JMLR-05]
[Ji etal, KDD-08]](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-51-2048.jpg)
![DASH-N [1]
Finetuning [2,3]
SCNN [4]
Overview
• Overview
supervised unsupervised
single
modality
multiple
modalities
[5]
DLID [6]
DCC [7]
DAN [8]
BA [9]
SHL-MDNN [10]
ST [11]
Are there
labelled
target
data?
Are dimensions of source
and target domains equal?
[12] Ngiam, Jiquan, et al. "Multimodal deep
learning." ICML. 2011.
[13] Srivastava, Nitish, and Ruslan
Salakhutdinov. "Multimodal learning with deep
Boltzmann machines." JMLR. 2014
[14] Sohn, Kihyuk, Wenling Shang, and Honglak
Lee. "Improved multimodal deep learning
with variation of information." NIPS. 2014.
DBN [12]
DBM [13]
MDRNN [14]
[5] Glorot, Xavier, Antoine Bordes, and Yoshua Bengio.
"Domain adaptation for large-scale sentiment
classification: A deep learning approach." ICML. 2011.
[6] Chopra, Sumit, Suhrid Balakrishnan, and
Raghuraman Gopalan. "Dlid: Deep learning for domain
adaptation by interpolating between domains." ICML.
2013.
[7] Tzeng, Eric, et al. "Deep domain confusion:
Maximizing for domain invariance." arXiv
preprint arXiv:1412.3474. 2014.
[8] Long, Mingsheng, and Jianmin Wang. "Learning
transferable features with deep adaptation
networks." arXiv preprint arXiv:1502.02791. 2015.
[9] Ganin, Yaroslav, and Victor Lempitsky.
"Unsupervised Domain Adaptation by
Backpropagation." ICML. 2015.
[10] Huang, Jui-Ting, et al. "Cross-language
knowledge transfer using multilingual deep
neural network with shared hidden
layers." ICASSP. 2013.
[11] Gupta, Saurabh, Judy Hoffman, and
Jitendra Malik. "Cross Modal
Distillation for Supervision Transfer."
arXiv preprint arXiv:1507.00448. 2015.
[1] Nguyen, Hien V., et al. "Joint hierarchical
domain adaptation and feature
learning." PAMI. 2013.
[2] Oquab, Maxime, et al. "Learning and
transferring mid-level image representations
using convolutional neural networks." CVPR
2014.
[3] Yosinski, Jason, et al. "How transferable
are features in deep neural networks?." NIPS
2014.
[4] Tzeng, Eric, et al. "Simultaneous
deep transfer across domains and
tasks." CVPR. 2015.](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-56-2048.jpg)
![Single Modality
• Transferability of layer-wise features
[3]
Conclusion 1: lower layer features are more general and transferrable, and higher
layer features are more specific and non-transferrable.
Conclusion 2: transferring features + fine-tuning always improve generalization. What if we do not have any labelled data to finetune in the target domain?
What happens if the source and target domain are very dissimilar?
ImageNet is not
randomly split, but
into A = {man-made
classes} and
B = {natural classes}](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-59-2048.jpg)
![Single Modality
• Overall training objec7ve
• Domain distance losses
– Maximum Mean Discrepancy [7]
source domain classification lossdomain distance loss
a particular representation, e.g. the representation after 5th
layer](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-61-2048.jpg)
![Single Modality
• Domain distance losses
– MK-MMD (Mul7-kernel variant of MMD) [8]
– Domain classifier [4, 9]
an embedding
A distribution-free metric - maximizes the domain classification error
Learn a more flexible distance metric than MMD by adjusting](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-62-2048.jpg)
![Single Modality
• Other factors to improve transfer
– Which layers should the domain distance loss be considered?
• By learning, pinpoint the layer that minimizes the domain distance
among all specific layers, say the fourth. [7]
• All the specific layers, say the last two layers. [8]
source
domain
input
output
target
domain
inpu
t
domain
distanc
e loss](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-63-2048.jpg)
![Single Modality• Other factors to improve transfer
– When we have some training data in the target
domain?
• soj label supervision [4]: categories without any
labeled target data are s7ll updated to output non-zero
probabili7es
target
doma
in
inpu
t outpu
t
source
domain](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-64-2048.jpg)
![Results
Mean Average Precision (MAP) by applying LR to different layers [13]
Transferring either one of the two domains to the other (joint hidden), outperforms the
domain itself (image_input OR text_input).
DBN [12]
DBM [13]](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-70-2048.jpg)
![References
[1] Nguyen, Hien V., et al. "Joint hierarchical domain adaptation and feature learning." PAMI. 2013.
[2] Oquab, Maxime, et al. "Learning and transferring mid-level image representations using convolutional neural
networks." CVPR. 2014.
[3] Yosinski, Jason, et al. "How transferable are features in deep neural networks?." NIPS. 2014.
[4] Tzeng, Eric, et al. "Simultaneous deep transfer across domains and tasks." CVPR. 2015.
[5] Glorot, Xavier, Antoine Bordes, and Yoshua Bengio. "Domain adaptation for large-scale sentiment classification:
A deep learning approach." ICML. 2011.
[6] Chopra, Sumit, Suhrid Balakrishnan, and Raghuraman Gopalan. "Dlid: Deep learning for domain adaptation by
interpolating between domains." ICML. 2013.
[7] Tzeng, Eric, et al. "Deep domain confusion: Maximizing for domain invariance." arXiv preprint
arXiv:1412.3474. 2014.
[8] Long, Mingsheng, and Jianmin Wang. "Learning transferable features with deep adaptation
networks." arXiv preprint arXiv:1502.02791. 2015.
[9] Ganin, Yaroslav, and Victor Lempitsky. "Unsupervised Domain Adaptation by Backpropagation."
ICML. 2015.
[10] Huang, Jui-Ting, et al. "Cross-language knowledge transfer using multilingual deep neural network with
shared hidden layers." ICASSP. 2013.
[11] Gupta, Saurabh, Judy Hoffman, and Jitendra Malik. "Cross Modal Distillation for Supervision
Transfer." arXiv preprint arXiv:1507.00448. 2015.
[12] Ngiam, Jiquan, et al. "Multimodal deep learning." ICML. 2011.
[13] Srivastava, Nitish, and Ruslan Salakhutdinov. "Multimodal learning with deep Boltzmann machines." JMLR.
2014
[14] Sohn, Kihyuk, Wenling Shang, and Honglak Lee. "Improved multimodal deep learning with variation of
information." NIPS. 2014.](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-71-2048.jpg)
![Relational Transfer Learning
Approaches (cont.)
81
Actor(A) Director(B)
WorkedFor
Movie (M)
Student (B) Professor (A)
AdvisedBy
Paper (T)
Publication Publication
Academic domain (source) Movie domain (target)
MovieMember MovieMember
AdvisedBy (B, A) ˄ Publication (B, T)
=> Publication (A, T)
WorkedFor (A, B) ˄ MovieMember (A, M)
=> MovieMember (B, M)
P1(x, y) ˄ P2 (x, z) => P2 (y, z)
[Mihalkova etal., AAAI-07, Davis and Domingos, ICML-09]](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-81-2048.jpg)
![Cross Domain Ac7vity Recogni7on
[Zheng, Hu, Yang, Ubicomp 2009]
• Challenges:
– A new domain of
ac7vi7es without
labeled data
• Cross-domain ac7vity
recogni7on
– Transfer some available
labeled data from
source ac7vi7es to help
training the recognizer
for the target ac7vi7es.
92
Cleaning
Indoor
Laundry
Dishwashing](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-92-2048.jpg)
![Reference
! [Thorndike and Woodworth, The Influence of Improvement in one
mental function upon the efficiency of the other functions, 1901]
! [Taylor and Stone, Transfer Learning for Reinforcement Learning
Domains: A Survey, JMLR 2009]
! [Pan and Yang, A Survey on Transfer Learning, IEEE TKDE 2009]
! [Quionero-Candela, etal, Data Shift in Machine Learning, MIT Press
2009]
! [Biltzer etal.. Domain Adapta7on with Structural Correspondence
Learning, EMNLP 2006]
! [Pan etal., Cross-Domain Sentiment Classification via Spectral Feature
Alignment, WWW 2010]
! [Pan etal., Transfer Learning via Dimensionality Reduction, AAAI
2008]
126](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-126-2048.jpg)
![Reference (cont.)
! [Pan etal., Domain Adaptation via Transfer Component Analysis,
IJCAI 2009]
! [Evgeniou and Pontil, Regularized Multi-Task Learning, KDD 2004]
! [Zhang and Yeung, A Convex Formulation for Learning Task
Relationships in Multi-Task Learning, UAI 2010]
! [Agarwal etal, Learning Multiple Tasks using Manifold
Regularization, NIPS 2010]
! [Argyriou etal., Multi-Task Feature Learning, NIPS 2007]
! [Ando and Zhang, A Framework for Learning Predictive Structures
from Multiple Tasks and Unlabeled Data, JMLR 2005]
! [Ji etal, Extracting Shared Subspace for Multi-label Classification,
KDD 2008]
127](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-127-2048.jpg)
![Reference (cont.)
! [Raina etal., Self-taught Learning: Transfer Learning from Unlabeled
Data, ICML 2007]
! [Dai etal., Boosting for Transfer Learning, ICML 2007]
! [Glorot etal., Domain Adaptation for Large-Scale Sentiment
Classification: A Deep Learning Approach, ICML 2011]
! [Davis and Domingos, Deep Transfer vis Second-order Markov Logic,
ICML 2009]
! [Mihalkova etal., Mapping and Revising Markov Logic Networks for
Transfer Learning, AAAI 2007]
! [Li etal., Cross-Domain Co-Extraction of Sentiment and Topic
Lexicons, ACL 2012]
128](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-128-2048.jpg)
![Reference (cont.)
! [Sugiyama etal., Direct Importance Estimation with Model Selection
and Its Application to Covariate Shift Adaptation, NIPS 2007]
! [Kanamori etal., A Least-squares Approach to Direct Importance
Estimation, JMLR 2009]
! [Cris7anini etal., On Kernel Target Alignment, NIPS 2002]
! [Huang etal., Correcting Sample Selection Bias by Unlabeled Data,
NIPS 2006]
! [Zadrozny, Learning and Evaluating Classifiers under Sample
Selection Bias, ICML 2004]
129](https://image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-129-2048.jpg)
![A psychological point of view
• Transfer of Learning (学习迁移）in
Educa7on and Psychology
– The study of dependency of human conduct,
learning or performance on prior experience.
– [Thorndike and Woodworth, 1901] explored how individuals would
transfer in one context to another context that share similar
characteristics.
• E.g.
! C++ " Java
! Math/Physics " Computer Science/Economics
2](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-2-2048.jpg)
![Transfer Learning
Different fields
• Transfer learning for
reinforcement learning.
[Taylor and Stone, Transfer
Learning for Reinforcement
Learning Domains: A Survey,
JMLR 2009]
• Transfer learning for
classifica7on, and
regression problems.
[Pan and Yang, A Survey on
Transfer Learning, IEEE TKDE
2010]
6
Focus!](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-6-2048.jpg)
![Instance-based Approaches
Case I (cont.)
22
Correcting Sample Selection Bias / Covariate Shift
[Quionero-Candela, etal, Data Shift in Machine Learning, MIT Press 2009]](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-22-2048.jpg)
![Instance-based Approaches
Correcting sample selection bias (cont.)
• The distribu7on of the selector variable maps
the target onto the source distribu7on
23
! Label instances from the source domain with label 1
! Label instances from the target domain with label 0
! Train a binary classifier
[Zadrozny, ICML-04]](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-23-2048.jpg)
![Instance-based Approaches
Kernel mean matching (KMM)
Maximum Mean Discrepancy (MMD)
[Alex Smola, Arthur Gretton and Kenji Kukumizu, ICML-08 tutorial]
24](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-24-2048.jpg)
![Instance-based Approaches
Direct density ratio estimation
25
[Sugiyama etal., NIPS-07, Kanamori etal., JMLR-09]
KL divergence loss Least squared loss
[Sugiyama etal., NIPS-07] [Kanamori etal., JMLR-09]](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-25-2048.jpg)
![Instance-based Approaches
Case II (cont.)
! TrAdaBoost [Dai etal ICML-07]
– For each boosting iteration,
# Use the same strategy as AdaBoost to
update the weights of target domain data.
# Use a new mechanism to decrease the
weights of misclassified source domain
data.
27](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-27-2048.jpg)
![Feature-based Approaches
Encode application-specific knowledge (cont.)
31
compact sharp blurry hooked realistic boring
1 1 0 0 0 0
0 1 0 0 0 0
0 0 1 0 0 0
( ) sgn( ), [1,1, 1,0,0,0]T
y f x w x w= = ⋅ = −
compact sharp blurry hooked realistic boring
0 0 0 1 0 0
0 0 0 1 1 0
0 0 0 0 0 1
Electronics
Video Game
Training
Prediction](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-31-2048.jpg)
![Feature-based Approaches
Encode application-specific knowledge (cont.)
! How to identify pivot features?
! Term frequency on both domains
! Mutual information between features and labels (source
domain)
! Mutual information on between features and domains
! How to utilize pivots to align features across domains?
! Structural Correspondence Learning (SCL) [Biltzer etal.
EMNLP-06]
! Spectral Feature Alignment (SFA) [Pan etal. WWW-10]
34](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-34-2048.jpg)
![Spectral Feature Alignment (SFA)
Derive new features (cont.)
sharp/hooked compact/realistic blurry/boring
1 1 0
1 0 0
0 0 1
38
( ) sgn( ), [1,1, 1]T
y f x w x w= = ⋅ = −
sharp/hooked compact/realistic blurry/boring
1 0 0
1 1 0
0 0 1
Electronics
Video Game
Training
Prediction](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-38-2048.jpg)
![Feature-based Approaches
Transfer Component Analysis [Pan etal., IJCAI-09, TNN-11]
41
TargetSource
Latent factors
Temperature Signal
properties
Building
structure
Power of APs
Motivation](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-41-2048.jpg)
![Feature-based Approaches
Self-taught Feature Learning (Andrew Ng. et al.)
! Intuition: Useful higher-level features can be learned from
unlabeled data.
! Steps:
1) Learn higher-level features from a lot of unlabeled data.
2) Use the learned higher-level features to represent the data of the
target task.
3) Train models from the new representations of the target task
(supervised)
! How to learn higher-level features
# Sparse Coding [Raina etal., 2007]
# Deep learning [Glorot etal., 2011]
48](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-48-2048.jpg)
![Multi-task Learning
Assumption:
If tasks are related, they may share similar parameter vectors.
For example, [Evgeniou and Pontil, KDD-04]
50
Common part
Specific part for individual task](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-50-2048.jpg)
![Mul7-task Feature Learning
51
[Argyriou etal., NIPS-07]
[Ando and Zhang, JMLR-05]
[Ji etal, KDD-08]](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-51-2048.jpg)
![DASH-N [1]
Finetuning [2,3]
SCNN [4]
Overview
• Overview
supervised unsupervised
single
modality
multiple
modalities
[5]
DLID [6]
DCC [7]
DAN [8]
BA [9]
SHL-MDNN [10]
ST [11]
Are there
labelled
target
data?
Are dimensions of source
and target domains equal?
[12] Ngiam, Jiquan, et al. "Multimodal deep
learning." ICML. 2011.
[13] Srivastava, Nitish, and Ruslan
Salakhutdinov. "Multimodal learning with deep
Boltzmann machines." JMLR. 2014
[14] Sohn, Kihyuk, Wenling Shang, and Honglak
Lee. "Improved multimodal deep learning
with variation of information." NIPS. 2014.
DBN [12]
DBM [13]
MDRNN [14]
[5] Glorot, Xavier, Antoine Bordes, and Yoshua Bengio.
"Domain adaptation for large-scale sentiment
classification: A deep learning approach." ICML. 2011.
[6] Chopra, Sumit, Suhrid Balakrishnan, and
Raghuraman Gopalan. "Dlid: Deep learning for domain
adaptation by interpolating between domains." ICML.
2013.
[7] Tzeng, Eric, et al. "Deep domain confusion:
Maximizing for domain invariance." arXiv
preprint arXiv:1412.3474. 2014.
[8] Long, Mingsheng, and Jianmin Wang. "Learning
transferable features with deep adaptation
networks." arXiv preprint arXiv:1502.02791. 2015.
[9] Ganin, Yaroslav, and Victor Lempitsky.
"Unsupervised Domain Adaptation by
Backpropagation." ICML. 2015.
[10] Huang, Jui-Ting, et al. "Cross-language
knowledge transfer using multilingual deep
neural network with shared hidden
layers." ICASSP. 2013.
[11] Gupta, Saurabh, Judy Hoffman, and
Jitendra Malik. "Cross Modal
Distillation for Supervision Transfer."
arXiv preprint arXiv:1507.00448. 2015.
[1] Nguyen, Hien V., et al. "Joint hierarchical
domain adaptation and feature
learning." PAMI. 2013.
[2] Oquab, Maxime, et al. "Learning and
transferring mid-level image representations
using convolutional neural networks." CVPR
2014.
[3] Yosinski, Jason, et al. "How transferable
are features in deep neural networks?." NIPS
2014.
[4] Tzeng, Eric, et al. "Simultaneous
deep transfer across domains and
tasks." CVPR. 2015.](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-56-2048.jpg)
![Single Modality
• Transferability of layer-wise features
[3]
Conclusion 1: lower layer features are more general and transferrable, and higher
layer features are more specific and non-transferrable.
Conclusion 2: transferring features + fine-tuning always improve generalization. What if we do not have any labelled data to finetune in the target domain?
What happens if the source and target domain are very dissimilar?
ImageNet is not
randomly split, but
into A = {man-made
classes} and
B = {natural classes}](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-59-2048.jpg)
![Single Modality
• Overall training objec7ve
• Domain distance losses
– Maximum Mean Discrepancy [7]
source domain classification lossdomain distance loss
a particular representation, e.g. the representation after 5th
layer](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-61-2048.jpg)
![Single Modality
• Domain distance losses
– MK-MMD (Mul7-kernel variant of MMD) [8]
– Domain classifier [4, 9]
an embedding
A distribution-free metric - maximizes the domain classification error
Learn a more flexible distance metric than MMD by adjusting](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-62-2048.jpg)
![Single Modality
• Other factors to improve transfer
– Which layers should the domain distance loss be considered?
• By learning, pinpoint the layer that minimizes the domain distance
among all specific layers, say the fourth. [7]
• All the specific layers, say the last two layers. [8]
source
domain
input
output
target
domain
inpu
t
domain
distanc
e loss](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-63-2048.jpg)
![Single Modality• Other factors to improve transfer
– When we have some training data in the target
domain?
• soj label supervision [4]: categories without any
labeled target data are s7ll updated to output non-zero
probabili7es
target
doma
in
inpu
t outpu
t
source
domain](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-64-2048.jpg)
![Results
Mean Average Precision (MAP) by applying LR to different layers [13]
Transferring either one of the two domains to the other (joint hidden), outperforms the
domain itself (image_input OR text_input).
DBN [12]
DBM [13]](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-70-2048.jpg)
![References
[1] Nguyen, Hien V., et al. "Joint hierarchical domain adaptation and feature learning." PAMI. 2013.
[2] Oquab, Maxime, et al. "Learning and transferring mid-level image representations using convolutional neural
networks." CVPR. 2014.
[3] Yosinski, Jason, et al. "How transferable are features in deep neural networks?." NIPS. 2014.
[4] Tzeng, Eric, et al. "Simultaneous deep transfer across domains and tasks." CVPR. 2015.
[5] Glorot, Xavier, Antoine Bordes, and Yoshua Bengio. "Domain adaptation for large-scale sentiment classification:
A deep learning approach." ICML. 2011.
[6] Chopra, Sumit, Suhrid Balakrishnan, and Raghuraman Gopalan. "Dlid: Deep learning for domain adaptation by
interpolating between domains." ICML. 2013.
[7] Tzeng, Eric, et al. "Deep domain confusion: Maximizing for domain invariance." arXiv preprint
arXiv:1412.3474. 2014.
[8] Long, Mingsheng, and Jianmin Wang. "Learning transferable features with deep adaptation
networks." arXiv preprint arXiv:1502.02791. 2015.
[9] Ganin, Yaroslav, and Victor Lempitsky. "Unsupervised Domain Adaptation by Backpropagation."
ICML. 2015.
[10] Huang, Jui-Ting, et al. "Cross-language knowledge transfer using multilingual deep neural network with
shared hidden layers." ICASSP. 2013.
[11] Gupta, Saurabh, Judy Hoffman, and Jitendra Malik. "Cross Modal Distillation for Supervision
Transfer." arXiv preprint arXiv:1507.00448. 2015.
[12] Ngiam, Jiquan, et al. "Multimodal deep learning." ICML. 2011.
[13] Srivastava, Nitish, and Ruslan Salakhutdinov. "Multimodal learning with deep Boltzmann machines." JMLR.
2014
[14] Sohn, Kihyuk, Wenling Shang, and Honglak Lee. "Improved multimodal deep learning with variation of
information." NIPS. 2014.](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-71-2048.jpg)
![Relational Transfer Learning
Approaches (cont.)
81
Actor(A) Director(B)
WorkedFor
Movie (M)
Student (B) Professor (A)
AdvisedBy
Paper (T)
Publication Publication
Academic domain (source) Movie domain (target)
MovieMember MovieMember
AdvisedBy (B, A) ˄ Publication (B, T)
=> Publication (A, T)
WorkedFor (A, B) ˄ MovieMember (A, M)
=> MovieMember (B, M)
P1(x, y) ˄ P2 (x, z) => P2 (y, z)
[Mihalkova etal., AAAI-07, Davis and Domingos, ICML-09]](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-81-2048.jpg)
![Cross Domain Ac7vity Recogni7on
[Zheng, Hu, Yang, Ubicomp 2009]
• Challenges:
– A new domain of
ac7vi7es without
labeled data
• Cross-domain ac7vity
recogni7on
– Transfer some available
labeled data from
source ac7vi7es to help
training the recognizer
for the target ac7vi7es.
92
Cleaning
Indoor
Laundry
Dishwashing](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-92-2048.jpg)
![Reference
! [Thorndike and Woodworth, The Influence of Improvement in one
mental function upon the efficiency of the other functions, 1901]
! [Taylor and Stone, Transfer Learning for Reinforcement Learning
Domains: A Survey, JMLR 2009]
! [Pan and Yang, A Survey on Transfer Learning, IEEE TKDE 2009]
! [Quionero-Candela, etal, Data Shift in Machine Learning, MIT Press
2009]
! [Biltzer etal.. Domain Adapta7on with Structural Correspondence
Learning, EMNLP 2006]
! [Pan etal., Cross-Domain Sentiment Classification via Spectral Feature
Alignment, WWW 2010]
! [Pan etal., Transfer Learning via Dimensionality Reduction, AAAI
2008]
126](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-126-2048.jpg)
![Reference (cont.)
! [Pan etal., Domain Adaptation via Transfer Component Analysis,
IJCAI 2009]
! [Evgeniou and Pontil, Regularized Multi-Task Learning, KDD 2004]
! [Zhang and Yeung, A Convex Formulation for Learning Task
Relationships in Multi-Task Learning, UAI 2010]
! [Agarwal etal, Learning Multiple Tasks using Manifold
Regularization, NIPS 2010]
! [Argyriou etal., Multi-Task Feature Learning, NIPS 2007]
! [Ando and Zhang, A Framework for Learning Predictive Structures
from Multiple Tasks and Unlabeled Data, JMLR 2005]
! [Ji etal, Extracting Shared Subspace for Multi-label Classification,
KDD 2008]
127](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-127-2048.jpg)
![Reference (cont.)
! [Raina etal., Self-taught Learning: Transfer Learning from Unlabeled
Data, ICML 2007]
! [Dai etal., Boosting for Transfer Learning, ICML 2007]
! [Glorot etal., Domain Adaptation for Large-Scale Sentiment
Classification: A Deep Learning Approach, ICML 2011]
! [Davis and Domingos, Deep Transfer vis Second-order Markov Logic,
ICML 2009]
! [Mihalkova etal., Mapping and Revising Markov Logic Networks for
Transfer Learning, AAAI 2007]
! [Li etal., Cross-Domain Co-Extraction of Sentiment and Topic
Lexicons, ACL 2012]
128](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-128-2048.jpg)
![Reference (cont.)
! [Sugiyama etal., Direct Importance Estimation with Model Selection
and Its Application to Covariate Shift Adaptation, NIPS 2007]
! [Kanamori etal., A Least-squares Approach to Direct Importance
Estimation, JMLR 2009]
! [Cris7anini etal., On Kernel Target Alignment, NIPS 2002]
! [Huang etal., Correcting Sample Selection Bias by Unlabeled Data,
NIPS 2006]
! [Zadrozny, Learning and Evaluating Classifiers under Sample
Selection Bias, ICML 2004]
129](https://crownmelresort.com/image.slidesharecdn.com/random-160520003633/75/Transfer-Learning-An-overview-129-2048.jpg)
Uploaded byjins0618
Transfer Learning: An overview
Transfer learning aims to improve learning in a target domain by leveraging knowledge from a related source domain. It is useful when the target domain has limited labeled data. There are several approaches, including instance-based approaches that reweight or resample source instances, and feature-based approaches that learn a transformation to align features across domains. Spectral feature alignment is one technique that builds a graph of correlations between pivot features shared across domains and domain-specific features, then applies spectral clustering to derive new shared features.
In this document
Powered by AI
Introduction to transfer learning by Qiang Yang, HKUST, thanking collaborators.
Discusses transfer of learning in education and psychology, referencing Thorndike's research variations.
Defines transfer learning as applying knowledge from previous domains to new tasks based on similarities.
Highlights time-consuming model training, suggesting reusing existing knowledge for efficiency.
Illustrates the process of transfer learning using the example of electronics over differing time periods.
Describes various fields of transfer learning including reinforcement learning and classification tasks.
Presents a motivating example related to indoor WiFi localization and training-testing device comparisons.
Contrasts two time periods with shifts in device functionalities and implications for learning.
Facilitates understanding through an example of and training/testing on sentiment classification accuracy.
Addresses the discrepancies in feature representations between electronics and video games.
Explains the assumption that training and testing data must originate from the same domain.
Parallels deep learning, reinforcement learning, and transfer learning approaches across timelines.
Explores the occurrence of training and testing data from varying domains and features.
Categorizes transfer learning approaches into instance-based, feature-based, parameter/model based, and relational.
Introduces and discusses instance-based transfer learning approaches focusing on overlapping features.
Details sample selection bias corrections and outlines various techniques like kernel mean matching.
Further elaborates on kernel density estimation and the reuse of labeled data in target domains.
Focuses on learning transformations when features between source and target domains overlap.
Details the method for feature alignment across domains using bipartite graphs and spectral clustering.
Explains transfer component analysis methods for aligning data distributions between two domains.
Overview of self-taught learning and multi-task feature learning concepts in transfer learning.
Discusses deep learning's role in transfer learning and its advantages on benchmark datasets.
Explores the mechanisms of using deep learning for feature transfer across domains.
Introduces frameworks for handling multi-modal transfers, emphasizing paired and classification losses.
Analyzes the MIR-Flickr dataset for image classification and how it benefits from transfer learning.
Discusses query classification and bridging classifiers for adapting queries in online systems.
Looks at cross-domain activity recognition and how similarities between activities can aid learning.
Highlights the significance of integrating social media data with physical domain knowledge.
Explores the use of transfer learning in collaborative filtering methodologies.
Presents source-free transfer learning strategies for dealing with unlabeled training data.
Introduces transitive transfer learning utilizing intermediate domains for distant source-target relationships.
Describes the combination of textual and visual data for enhanced transfer learning.
Elaborates on using sparse coding to enhance learning speed and task adaptability.
Summarizes critical references and essential acknowledgments in the field of transfer learning.
More Related Content
Similar to Transfer Learning: An overview
![[ppt]](https://cdn.slidesharecdn.com/ss_thumbnails/ppt990-thumbnail.jpg?width=640&height=640&fit=bounds)
![[Redis Released]- FalkorDB - Redis + Graph Agentic Memory’s Secret Sauce](https://cdn.slidesharecdn.com/ss_thumbnails/redisreleased-falkordbslidedeck-1125-251115194922-e1c0046b-thumbnail.jpg?width=640&height=640&fit=bounds)
Transfer Learning: An overview
- 1. 迁移学习理论与应用 Transfer Learning: An Overview 杨强，香港科大 Qiang Yang, HKUST Thanks: SinnoJialin Pan, NTU, Singapore Ying Wei, HKUST, Hong Kong Ben Tan, HKUST, Hong Kong
- 2. A psychological point of view • Transfer ofLearning (学习迁移）in Educa7on and Psychology – The study of dependency of human conduct, learning or performance on prior experience. – [Thorndike and Woodworth, 1901] explored how individuals would transfer in one context to another context that share similar characteristics. • E.g. ! C++ " Java ! Math/Physics " Computer Science/Economics 2
- 3. Transfer Learning In the machine learning community • Theability of a system to recognize and apply knowledge and skills learned in previous domains/ tasks to novel tasks/domains, which share some commonality. • Given a target domain/task, how to transfer knowledge to new domains/tasks (target)? • Key: – Representation Learning, Change of Representation 3
- 4. Why Transfer? ! Build every model from scratch? # Time consuming and expensive # Expense: • Data Collec7on/Labeling • Privacy • Time to train ! Reuse common knowledge extracted from exis7ng systems? # More prac7cal 4
- 5. Why Transfer Learning? 5 Source DomainData Target Domain Data Predictive Models Labeled Training Unlabeled data/a few labeled data for adaptation Transfer Learning Algorithms Target Domain Data Testing Electronics Time Period A Device A DVDDevice B Time Period B
- 6. Transfer Learning Different fields • Transfer learning for reinforcement learning. [Taylor and Stone, Transfer Learning for Reinforcement Learning Domains: A Survey, JMLR 2009] • Transfer learning for classifica7on, and regression problems. [Pan and Yang, A Survey on Transfer Learning, IEEE TKDE 2010] 6 Focus!
- 7.
- 8. Indoor WiFi Localization(cont.) 8 Training Training Test Device A Test Device B ~ 1.5 meters ~10 meters Device A Device A S=(-37dbm, .., -77dbm), L=(1, 3) S=(-41dbm, .., -83dbm), L=(1, 4) … S=(-49dbm, .., -34dbm), L=(9, 10) S=(-61dbm, .., -28dbm), L=(15,22) S=(-37dbm, .., -77dbm) S=(-41dbm, .., -83dbm) … S=(-49dbm, .., -34dbm) S=(-61dbm, .., -28dbm) S=(-37dbm, .., -77dbm) S=(-41dbm, .., -83dbm) … S=(-49dbm, .., -34dbm) S=(-61dbm, .., -28dbm) S=(-33dbm, .., -82dbm), L=(1, 3) … S=(-57dbm, .., -63dbm), L=(10, 23) Localization model Localization model Drop! Average Error Distance
- 9. Difference between Domains 9 TimePeriod A Time Period B Device B Device A
- 10.
- 11. Sentiment Classification (cont.) 11 Training TrainingTest Electronics Test ~ 84.6% ~72.65% Sentiment Classifier Sentiment Classifier Drop! Electronics Classification Accuracy ElectronicsDVD
- 12. Difference in Representation 12 ElectronicsVideo Games (1) Compact; easy to operate; very good picture quality; looks sharp! (2) A very good game! It is action packed and full of excitement. I am very much hooked on this game. (3) I purchased this unit from Circuit City and I was very excited about the quality of the picture. It is really nice and sharp. (4) Very realistic shooting action and good plots. We played this and were hooked. (5) It is also quite blurry in very dark settings. I will never buy HP again. (6) The game is so boring. I am extremely unhappy and will probably never buy UbiSoft again.
- 13. A Major Assump7on in Tradi7onal Machine Learning ! Training and future(test) data come from the same domain, which implies # Represented in the same feature spaces. # Follow the same data distribution. 13
- 14.
- 15.
- 16. Different Scenarios • Training and tes7ng data may come from different domains: # Different differentfeature spaces/ marginal distributions: # Different conditional distributions or different label spaces: 16
- 17. Transfer Learning Approaches 17 Instance-based Approaches Feature-based Approaches Parameter/Model- based Approaches Relational Approaches
- 18. Instance-based Transfer Learning Approaches Sourceand target domains have a lot of overlapping features 18 General Assumption
- 19. Instance-based Transfer Learning Approaches Case I: Unlabeled Target Case II: Some Labels in Target 19 ProblemSetting Assumption Assumption Problem Setting
- 20.
- 21. Instance-based Approaches Case I(cont.) Assumption: 21
- 22. Instance-based Approaches Case I(cont.) 22 Correcting Sample Selection Bias / Covariate Shift [Quionero-Candela, etal, Data Shift in Machine Learning, MIT Press 2009]
- 23. Instance-based Approaches Correcting sampleselection bias (cont.) • The distribu7on of the selector variable maps the target onto the source distribu7on 23 ! Label instances from the source domain with label 1 ! Label instances from the target domain with label 0 ! Train a binary classifier [Zadrozny, ICML-04]
- 24. Instance-based Approaches Kernel meanmatching (KMM) Maximum Mean Discrepancy (MMD) [Alex Smola, Arthur Gretton and Kenji Kukumizu, ICML-08 tutorial] 24
- 25. Instance-based Approaches Direct densityratio estimation 25 [Sugiyama etal., NIPS-07, Kanamori etal., JMLR-09] KL divergence loss Least squared loss [Sugiyama etal., NIPS-07] [Kanamori etal., JMLR-09]
- 26. Instance-based Approaches Case II • Intui7on: Part of the labeled data in the source domain can be reused in the target domain after re-weighting 26
- 27. Instance-based Approaches Case II(cont.) ! TrAdaBoost [Dai etal ICML-07] – For each boosting iteration, # Use the same strategy as AdaBoost to update the weights of target domain data. # Use a new mechanism to decrease the weights of misclassified source domain data. 27
- 28. Feature-based Transfer Learning Approaches Whensource and target domains only have some overlapping features. (lots of features only have support in either the source or the target domain) 28
- 29. Feature-based Transfer Learning Approaches(cont.) How to learn ? ! Solution 1: Encode application-specific knowledge to learn the transformation. ! Solution 2: General approaches to learning the transformation. 29
- 30. Feature-based Approaches Encode application-specificknowledge 30 Electronics Video Games (1) Compact; easy to operate; very good picture quality; looks sharp! (2) A very good game! It is action packed and full of excitement. I am very much hooked on this game. (3) I purchased this unit from Circuit City and I was very excited about the quality of the picture. It is really nice and sharp. (4) Very realistic shooting action and good plots. We played this and were hooked. (5) It is also quite blurry in very dark settings. I will never_buy HP again. (6) The game is so boring. I am extremely unhappy and will probably never_buy UbiSoft again.
- 31. Feature-based Approaches Encode application-specificknowledge (cont.) 31 compact sharp blurry hooked realistic boring 1 1 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 ( ) sgn( ), [1,1, 1,0,0,0]T y f x w x w= = ⋅ = − compact sharp blurry hooked realistic boring 0 0 0 1 0 0 0 0 0 1 1 0 0 0 0 0 0 1 Electronics Video Game Training Prediction
- 32. Feature-based Approaches Encode application-specificknowledge (cont.) 32 Electronics Video Games (1) Compact; easy to operate; very good picture quality; looks sharp! (2) A very good game! It is action packed and full of excitement. I am very much hooked on this game. (3) I purchased this unit from Circuit City and I was very excited about the quality of the picture. It is really nice and sharp. (4) Very realistic shooting action and good plots. We played this and were hooked. (5) It is also quite blurry in very dark settings. I will never_buy HP again. (6) The game is so boring. I am extremely unhappy and will probably never_buy UbiSoft again.
- 33. Feature-based Approaches Encode application-specificknowledge (cont.) ! Three different types of features ! Source domain (Electronics) specific features, e.g., compact, sharp, blurry ! Target domain (Video Game) specific features, e.g., hooked, realistic, boring ! Domain independent features (pivot features), e.g., good, excited, nice, never_buy 33
- 34. Feature-based Approaches Encode application-specificknowledge (cont.) ! How to identify pivot features? ! Term frequency on both domains ! Mutual information between features and labels (source domain) ! Mutual information on between features and domains ! How to utilize pivots to align features across domains? ! Structural Correspondence Learning (SCL) [Biltzer etal. EMNLP-06] ! Spectral Feature Alignment (SFA) [Pan etal. WWW-10] 34
- 35. Feature-based Approaches Spectral FeatureAlignment (SFA) ! Intuition # Use a bipartite graph to model the correlations between pivot features and other features # Discover new shared features by applying spectral clustering techniques on the graph 35
- 36. ! If two domain-specificwords have connections to more common pivot words in the graph, they tend to be aligned or clustered together with a higher probability. ! If two pivot words have connections to more common domain-specific words in the graph, they tend to be aligned together with a higher probability. Spectral Feature Alignment (SFA) High level idea 36 exciting good never_buy sharp boring blurry hooked compact realistic Pivot features Domain-specific features 7 6 8 3 6 2 4 5 Electronics Video Game
- 37. exciting good never_buy sharp boring blurry hooked compact realistic Pivot features Domain-specific features 7 6 8 3 6 2 4 5 Electronics VideoGame boring realistic hooked blurry sharp compact Electronics Video Game Electronics Electronics Video Game Video Game Derive new features Spectral Clustering 37
- 38. Spectral Feature Alignment(SFA) Derive new features (cont.) sharp/hooked compact/realistic blurry/boring 1 1 0 1 0 0 0 0 1 38 ( ) sgn( ), [1,1, 1]T y f x w x w= = ⋅ = − sharp/hooked compact/realistic blurry/boring 1 0 0 1 1 0 0 0 1 Electronics Video Game Training Prediction
- 39. Spectral Feature Alignment(SFA) 1. Identify P pivot features 2. Construct a bipartite graph between the pivot and remaining features. 3. Apply spectral clustering on the graph to derive new features 4. Train classifiers on the source using augmented features (original features + new features) 39
- 40. Feature-based Approaches Develop generalapproaches 40 Time Period A Time Period B Device B Device A
- 41. Feature-based Approaches Transfer ComponentAnalysis [Pan etal., IJCAI-09, TNN-11] 41 TargetSource Latent factors Temperature Signal properties Building structure Power of APs Motivation
- 42. Transfer Component Analysis(cont.) 42 TargetSource Latent factors Temperature Signal properties Building structure Power of APs Causes the data distributions between two domains to be different
- 43. Transfer Component Analysis(cont.) 43 TargetSource Signal properties Noisy component Building structure Principal components
- 44. Transfer Component Analysis(cont.) Learning by only minimizing the distance between distribu7ons 44
- 45. Transfer Component Analysis(cont.) Main idea: the learned should map the source and target domain data to the latent space spanned by the factors which can reduce domain difference and preserve original data structure. 45 High level optimization problem
- 46. Transfer Component Analysis(cont.) 46 Recall: Maximum Mean Discrepancy (MMD)
- 47. Transfer Component Analysis(cont.) 47 An illustrative example Latent features learned by PCA and TCA PCAOriginal feature space TCA
- 48. Feature-based Approaches Self-taught FeatureLearning (Andrew Ng. et al.) ! Intuition: Useful higher-level features can be learned from unlabeled data. ! Steps: 1) Learn higher-level features from a lot of unlabeled data. 2) Use the learned higher-level features to represent the data of the target task. 3) Train models from the new representations of the target task (supervised) ! How to learn higher-level features # Sparse Coding [Raina etal., 2007] # Deep learning [Glorot etal., 2011] 48
- 49. Feature-based Approaches Mul7-task Feature Learning ! Assumption: Iftasks are related, they should share some good common features. ! Goal: Learn a low-dimensional representation shared across related tasks. 49 General Multi-task Learning Setting
- 50. Multi-task Learning Assumption: If tasksare related, they may share similar parameter vectors. For example, [Evgeniou and Pontil, KDD-04] 50 Common part Specific part for individual task
- 51. Mul7-task Feature Learning 51 [Argyriou etal., NIPS-07] [Andoand Zhang, JMLR-05] [Ji etal, KDD-08]
- 52.
- 53. Transfer Learning with Deep LearningTransfer Learning Perspec7ve: Why need Deep Learning? • Deep neural networks learn nonlinear representa7ons – that are hierarchical; – that disentangle different explanatory factors of varia7on behind data samples; – that manifest invariant factors underlying different popula7ons. Deep Learning Perspec7ve: Why need Transfer Learning? • Transfer Learning alleviates – the incapability of learning on a dataset which may not be large enough to train an en7re deep neural network from scratch
- 54. Benchmark Dataset: Office • Descrip7on: leverage source images to improve classifica7on of target images 3 domains 31categories backpackbike amazon 2,817images webcam 957 images dslr 795 images object images in Amazon low-resolution images taken from a web camera high-resolution images taken from a digital SLR camera
- 55. Results Unsupervised domain adaptationAmazon→Webcam over time 2011 2012 2013 2014 2015 Multi-classaccuracy 10 20 30 40 50 60 70 TCA GFK CNN DD C DLID DAN BA TL without DL TL with DL DL without TL With Deep Learning, Transfer Learning improves. Applying Transfer Learning techniques outperforms directly applying Deep Learning models trained on the source.
- 56. DASH-N [1] Finetuning [2,3] SCNN[4] Overview • Overview supervised unsupervised single modality multiple modalities [5] DLID [6] DCC [7] DAN [8] BA [9] SHL-MDNN [10] ST [11] Are there labelled target data? Are dimensions of source and target domains equal? [12] Ngiam, Jiquan, et al. "Multimodal deep learning." ICML. 2011. [13] Srivastava, Nitish, and Ruslan Salakhutdinov. "Multimodal learning with deep Boltzmann machines." JMLR. 2014 [14] Sohn, Kihyuk, Wenling Shang, and Honglak Lee. "Improved multimodal deep learning with variation of information." NIPS. 2014. DBN [12] DBM [13] MDRNN [14] [5] Glorot, Xavier, Antoine Bordes, and Yoshua Bengio. "Domain adaptation for large-scale sentiment classification: A deep learning approach." ICML. 2011. [6] Chopra, Sumit, Suhrid Balakrishnan, and Raghuraman Gopalan. "Dlid: Deep learning for domain adaptation by interpolating between domains." ICML. 2013. [7] Tzeng, Eric, et al. "Deep domain confusion: Maximizing for domain invariance." arXiv preprint arXiv:1412.3474. 2014. [8] Long, Mingsheng, and Jianmin Wang. "Learning transferable features with deep adaptation networks." arXiv preprint arXiv:1502.02791. 2015. [9] Ganin, Yaroslav, and Victor Lempitsky. "Unsupervised Domain Adaptation by Backpropagation." ICML. 2015. [10] Huang, Jui-Ting, et al. "Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers." ICASSP. 2013. [11] Gupta, Saurabh, Judy Hoffman, and Jitendra Malik. "Cross Modal Distillation for Supervision Transfer." arXiv preprint arXiv:1507.00448. 2015. [1] Nguyen, Hien V., et al. "Joint hierarchical domain adaptation and feature learning." PAMI. 2013. [2] Oquab, Maxime, et al. "Learning and transferring mid-level image representations using convolutional neural networks." CVPR 2014. [3] Yosinski, Jason, et al. "How transferable are features in deep neural networks?." NIPS 2014. [4] Tzeng, Eric, et al. "Simultaneous deep transfer across domains and tasks." CVPR. 2015.
- 57.
- 58. Single Modality • Transferability oflayer-wise features ImageNet 1000 class A: 500 class B: 500 class source domain target domain random split baseA: train all layers with A baseB: train all layers with B BnB: initialize the first n layers with baseB and fix, randomly initialize the other layers and train with B BnB+: initialize the first n layers with baseB, randomly initialize the other layers, and train all layers with B AnB: initialize the first n layers with baseA and fix, randomly initialize the other layers and train with B AnB+: initialize the first n layers with baseA, randomly initialize the other layers, and train all layers with B
- 59. Single Modality • Transferability oflayer-wise features [3] Conclusion 1: lower layer features are more general and transferrable, and higher layer features are more specific and non-transferrable. Conclusion 2: transferring features + fine-tuning always improve generalization. What if we do not have any labelled data to finetune in the target domain? What happens if the source and target domain are very dissimilar? ImageNet is not randomly split, but into A = {man-made classes} and B = {natural classes}
- 60. Single Modality • General framework of unsupervised transfer source domai n input outpu t target domai n input outpu t domain distanc e loss For lowerlevel features (more general & transferrable), the source transfers to the target directly. For higher level features (more domain specific & not transferrable), the source transfers to the target by minimizing domain distances. shared weights If some labelled target data are available, it would be better.
- 61. Single Modality • Overall training objec7ve • Domain distance losses – Maximum Mean Discrepancy [7] sourcedomain classification lossdomain distance loss a particular representation, e.g. the representation after 5th layer
- 62. Single Modality • Domain distance losses – MK-MMD (Mul7-kernel variant of MMD) [8] – Domain classifier [4, 9] an embedding A distribution-free metric - maximizes the domain classification error Learn a more flexible distance metric than MMD by adjusting
- 63. Single Modality • Other factors to improve transfer – Which layers should the domain distance loss be considered? • By learning, pinpoint the layer that minimizes the domain distance among all specific layers, say the fourth. [7] • All the specific layers, say the last two layers. [8] source domain input output target domain inpu t domain distanc e loss
- 64.
- 65. Mul7ple Modali7es • The source domain and target domain could have different feature spaces, i.e., dimensionality. – Mul7media on the web • Images • Text documents • Audio • Video – Recommender systems • Douban • Taobao • Xiami Music – Robo7cs • Vision • Audio • Sensors How to deal with multi-modal transfer with Deep Learning?
- 66. Mul7ple Modali7es • Key The catis sitting on a sofa with ears cocking. shared concept cat ears kitteneyes
- 67. Mul7ple Modali7es • General framework of unsupervised transfer source domai n input target domai n input commo n Paired loss reconstruction layer reconstructionlayer Reconstruction errors: Paired loss: the similarity of a pair of source and target instances is preserved in the common space. Paired loss: similarity
- 68.
- 69. MIR-Flickr Dataset • 1 millionimages with user-generated tags • 25,000 images are labelled with 24 categories • 10,000 for training, 5,000 for validation, 10,000 for testing categories baby, female, portrait, people plant life, river, water clouds, sea, sky, transport, water animals, dog, food domain 1: images domain 2: text
- 70. Results Mean Average Precision(MAP) by applying LR to different layers [13] Transferring either one of the two domains to the other (joint hidden), outperforms the domain itself (image_input OR text_input). DBN [12] DBM [13]
- 71. References [1] Nguyen, HienV., et al. "Joint hierarchical domain adaptation and feature learning." PAMI. 2013. [2] Oquab, Maxime, et al. "Learning and transferring mid-level image representations using convolutional neural networks." CVPR. 2014. [3] Yosinski, Jason, et al. "How transferable are features in deep neural networks?." NIPS. 2014. [4] Tzeng, Eric, et al. "Simultaneous deep transfer across domains and tasks." CVPR. 2015. [5] Glorot, Xavier, Antoine Bordes, and Yoshua Bengio. "Domain adaptation for large-scale sentiment classification: A deep learning approach." ICML. 2011. [6] Chopra, Sumit, Suhrid Balakrishnan, and Raghuraman Gopalan. "Dlid: Deep learning for domain adaptation by interpolating between domains." ICML. 2013. [7] Tzeng, Eric, et al. "Deep domain confusion: Maximizing for domain invariance." arXiv preprint arXiv:1412.3474. 2014. [8] Long, Mingsheng, and Jianmin Wang. "Learning transferable features with deep adaptation networks." arXiv preprint arXiv:1502.02791. 2015. [9] Ganin, Yaroslav, and Victor Lempitsky. "Unsupervised Domain Adaptation by Backpropagation." ICML. 2015. [10] Huang, Jui-Ting, et al. "Cross-language knowledge transfer using multilingual deep neural network with shared hidden layers." ICASSP. 2013. [11] Gupta, Saurabh, Judy Hoffman, and Jitendra Malik. "Cross Modal Distillation for Supervision Transfer." arXiv preprint arXiv:1507.00448. 2015. [12] Ngiam, Jiquan, et al. "Multimodal deep learning." ICML. 2011. [13] Srivastava, Nitish, and Ruslan Salakhutdinov. "Multimodal learning with deep Boltzmann machines." JMLR. 2014 [14] Sohn, Kihyuk, Wenling Shang, and Honglak Lee. "Improved multimodal deep learning with variation of information." NIPS. 2014.
- 72.
- 73.
- 75.
- 76. Oquab, Bottou, Laptev,Sivic: Learning and Transferring Mid-Level Image Representations using Convolutional Neural Networks. CVPR 2014.
- 77. Transfer Learning in Convolu7onal Neural Networks • Source Domain: ImageNet – 1000 classes, 1.2 million images • Target Domain: Pascal VOC 2007 object classifica7on – 20 classes, about 5000 images • PRE-1000C: the proposed method
- 78. DeCAF: A Deep Convolu7onal Ac7va7on Feature for Generic Visual Recogni7on • Jeff Donahue, Yangqing Jia, Oriol Vinyals, Judy Hoffman, Ning Zhang, Eric Tzeng, Trevor Darrell. ICML2014 • Ques7ons: – How to transfer features to tasks with different labels – Do features extracted from the CNN generalize to other datasets? – How does performance vary with network depth? • Algorithm: – A deep convolu7onal model is first trained in a fully supervised seqng using a state-of-the-art method Krizhevsky et al. (2012 ). – extract various features from this network, and evaluate the efficacy of these features on generic vision tasks. 78
- 79.
- 80.
- 81. Relational Transfer Learning Approaches(cont.) 81 Actor(A) Director(B) WorkedFor Movie (M) Student (B) Professor (A) AdvisedBy Paper (T) Publication Publication Academic domain (source) Movie domain (target) MovieMember MovieMember AdvisedBy (B, A) ˄ Publication (B, T) => Publication (A, T) WorkedFor (A, B) ˄ MovieMember (A, M) => MovieMember (B, M) P1(x, y) ˄ P2 (x, z) => P2 (y, z) [Mihalkova etal., AAAI-07, Davis and Domingos, ICML-09]
- 82.
- 83. 83 Query Classification andOnline Advertisement • ACM KDDCUP 05 Winner • SIGIR 06 • ACM Transactions on Information Systems Journal 2006 – Joint work with Dou Shen, Jiantao Sun and Zheng Chen
- 84. 84 84 QC asMachine Learning Inspired by the KDDCUP’05 competition – Classify a query into a ranked list of categories – Queries are collected from real search engines – Target categories are organized in a tree with each node being a category
- 85. 85 Target-transfer Learning inQC • Classifier, once trained, stays constant – Target Classes Before • Sports, Politics (European, US, China) – Target Classes Now • Sports (Olympics, Football, NBA), Stock Market (Asian, Dow, Nasdaq), History (Chinese, World) How to allow target to change? • Application: – advertisements come and go, – but our query"target mapping needs not be retrained! • We call this the target-transfer learning problem
- 86. 86 86 Solutions: QueryEnrichment + Staged Classification Target Categories Queries Solution: Bridging classifier Construction of Synonym- based Classifiers Construction of Statistical Classifier Query Search Engine Labels of Returned Pages Text of Returned Pages Classified results Classified results Finial Results Phase II: the testing phase Phase I: the training phase
- 87. 87 87 $ Categoryinformation Full Step 1: Query enrichment • Textual information Title Snippet Category
- 88. 88 88 Step 2:Bridging Classifier • Wish to avoid: – When target is changed, training needs to repeat! • Solution: – Connect the target taxonomy and queries by taking an intermediate taxonomy as a bridge
- 89. 89 89 Bridging Classifier(Cont.) $ How to connect? Prior prob. of I jC The relation between and I jC T iC The relation between and I jC q The relation between and T iC q
- 90. 90 90 Category Selectionfor Intermediate Taxonomy – Category Selection for Reducing Complexity • Total Probability (TP) • Mutual Information
- 91. 91 Result of BridgingClassifiers – Using bridging classifier allows the target classes to change freely • no the need to retrain the classifier! $ Performance of the Bridging Classifier with Different Granularity of Intermediate Taxonomy
- 92. Cross Domain Ac7vity Recogni7on [Zheng, Hu, Yang, Ubicomp 2009] • Challenges: – A new domain of ac7vi7es without labeled data • Cross-domain ac7vity recogni7on – Transfer some available labeled data from source ac7vi7es to help training the recognizer for the target ac7vi7es. 92 Cleaning Indoor Laundry Dishwashing
- 93.
- 94. Calcula7ng Ac7vity Similari7es ! How similar are two ac7vi7es? ◦ Use Web search results ◦ TFIDF: Tradi7onal IR similarity metrics (cosine similarity) ◦ Example " Mined similarity between the ac7vity “sweeping” and “vacuuming”, “making the bed”, “gardening” Calculated Similarity with the activity "Sweeping" Similarity with the activity "Sweeping " 94
- 95. Cross-Domain AR: Performance Mean Accuracy with Cross Domain Transfer # Activities (Source Domain) #Activities (Target Domain) Baseline (Random Guess) MIT Dataset (Cleaning to Laundry) 58.9% 13 8 12.5% MIT Dataset (Cleaning to Dishwashing) 53.2% 13 7 14.3% Intel Research Lab Dataset 63.2% 5 6 16.7% 95 ! Ac7vi7es in the source domain and the target domain are generated from ten random trials, mean accuracies are reported.
- 96.
- 97.
- 98.
- 99.
- 100.
- 101.
- 102. Transfer Learning in Collabora7ve Filtering • Source (Dense): Encode cluster-level ra7ng pazerns • Target (Sparse): Map users/items to the encoded prototypes 102 AB C III II I A B III II I a e b f c d 2 6 4 5 1 3 c d a b e 1 3 6 2 4 7 a b c d e f a b c d e 1 6 5 4 3 2 7 1 6 5 4 3 2 BOOKS (Target-Sparse) MOVIES (Auxiliary-Dense) Cluster-level Rating Pattern Matching 3 2 3 2 3 1 3 2 3 2 3 1 1 3 2 3 ? 3 3 1 1 1 1 2 2 ? 2 2 2 2 2 3 ? 3 3 3 3 3 3 ? 3 3 3 2 2 2 2 1 1 1 ? ? 1 1 1 1 ? 1 ? 1 2 ? 2 2 3 3 3 3 ? 3 ? 3 3 3 2 2 ? ? 2 2 2 3 2 ? 3 3 3 2 3 2 1 1 3 1 3 1 ? 2 3 ? 3 1 2 2 2 3 2 3 3 ? ? 2 3 2 1 1 ? 3 1 1 2 3 1 ? ? 1 2 3 3 2 ? 3 ? 2 3 2 3 ? 3 ? 1 3 1 ? ? 3 ? 2 3 3 2 Permuterows&cols 5 ReducetoGroups 3 3 3 ? ? 3
- 103.
- 104.
- 105.
- 106.
- 107.
- 108.
- 109. Compila7on: Learning a projec7on matrix W to amp the target instance to latent space vs. vs. vs. vs. vs. V Projection matrix Target Domain Labeled& Unlabeled Data q m W d m Learned Projection matrix Our regression model Loss on labeled data Loss on unlabeled data For each target instance, we first aggregate its predic=on in the base label space, and then project it onto the latent space
- 110.
- 111. Transi7ve Transfer Learning with intermediate domains Qiang Yang Hong KongUniversity of Science and Technology http://www.cse.ust.hk/~qyang
- 112.
- 113.
- 114. Previous work and TTL % Tradi7onal machine learning & training and test data should be from the same problem domain. % Transfer learning & training and test data should be from similar problem domains. % Transi7ve transfer learning & training and test data could be from distant problem domains. ML: Same domain TL: Similar domains TTL: Distant domains
- 115. Text-to-Image Classifica7on Source and target domains have few overlaps Text-to-image Classification with co- occurrencedata as intermediate domain accelerator-to-gyroscope activity recognition with data from intelligent devices as intermediate domains
- 116. TTL: single intermediate domain Intermediate domain selec7on, then propagate knowledge ! Crawl a lot of images with annota7ons from Internet ! Use domain distance, such as A-distance, to iden7fy domain ! Transi7ve transfer through shared hidden factors in row by matrix tri- factoriza7on Matrix tri-factoriza7on for clustering/classifica7on
- 117.
- 118.
- 119.
- 120. Designing Objec7ve Func7on of TTL Transitive Transfer Learningwith intermediate data Intermediate domain weighting/selection The weights for the intermediate domains are learned from data. The intermediate data help find a better hidden layer. Predictive Model Learning Feature Engineering
- 121. TTL with supervised auto- encoder Source Feature Engineering Predictive Model Learning SharedTarget Intermediates ! The NUS-WISEdata ! 45 text-to-image tasks ! Each task is composed of 1200 text documents, 600 images, and 1600 co- occurred text-image pairs. In each task, 1600*45 co-occurred text-image pairs will be used for knowledge transfer.
- 122.
- 123. Reinforcement Transfer Learning via Sparse Coding • Slow learningspeed remains a fundamental problem for reinforcement learning in complex environments. • Main problem: the numbers of states and actions in the source and target domains are different. – Existing works: hand-coded inter-task mapping between state- action pairs • Tool: new transfer learning based on sparse coding Ammar, Tuyls, Taylor, Driessens, Weiss: Reinforcement Learning Transfer via Sparse Coding. AAMAS, 2012.
- 124. Reinforcement Learning Transfer via Sparse CodingA u th o r s m e a s u r e d t h e performance as the number of steps during an episode to control the pole in an upright posi7on on a given fixed amount of samples.
- 125. • Given State-Ac7on-State Triplets in the source task, learn dic7onary as • Using the coefficient matrix in the first step, we can learn the dic7onary in the target task as • Then for each triplet in the target task, - sparse projec7on is used to find its coefficients • As a result, the inter-task mapping can be learned! Reinforcement Transfer Learning via Sparse Coding
- 126. Reference ! [Thorndike andWoodworth, The Influence of Improvement in one mental function upon the efficiency of the other functions, 1901] ! [Taylor and Stone, Transfer Learning for Reinforcement Learning Domains: A Survey, JMLR 2009] ! [Pan and Yang, A Survey on Transfer Learning, IEEE TKDE 2009] ! [Quionero-Candela, etal, Data Shift in Machine Learning, MIT Press 2009] ! [Biltzer etal.. Domain Adapta7on with Structural Correspondence Learning, EMNLP 2006] ! [Pan etal., Cross-Domain Sentiment Classification via Spectral Feature Alignment, WWW 2010] ! [Pan etal., Transfer Learning via Dimensionality Reduction, AAAI 2008] 126
- 127. Reference (cont.) ! [Pan etal.,Domain Adaptation via Transfer Component Analysis, IJCAI 2009] ! [Evgeniou and Pontil, Regularized Multi-Task Learning, KDD 2004] ! [Zhang and Yeung, A Convex Formulation for Learning Task Relationships in Multi-Task Learning, UAI 2010] ! [Agarwal etal, Learning Multiple Tasks using Manifold Regularization, NIPS 2010] ! [Argyriou etal., Multi-Task Feature Learning, NIPS 2007] ! [Ando and Zhang, A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data, JMLR 2005] ! [Ji etal, Extracting Shared Subspace for Multi-label Classification, KDD 2008] 127
- 128. Reference (cont.) ! [Raina etal.,Self-taught Learning: Transfer Learning from Unlabeled Data, ICML 2007] ! [Dai etal., Boosting for Transfer Learning, ICML 2007] ! [Glorot etal., Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach, ICML 2011] ! [Davis and Domingos, Deep Transfer vis Second-order Markov Logic, ICML 2009] ! [Mihalkova etal., Mapping and Revising Markov Logic Networks for Transfer Learning, AAAI 2007] ! [Li etal., Cross-Domain Co-Extraction of Sentiment and Topic Lexicons, ACL 2012] 128
- 129. Reference (cont.) ! [Sugiyama etal.,Direct Importance Estimation with Model Selection and Its Application to Covariate Shift Adaptation, NIPS 2007] ! [Kanamori etal., A Least-squares Approach to Direct Importance Estimation, JMLR 2009] ! [Cris7anini etal., On Kernel Target Alignment, NIPS 2002] ! [Huang etal., Correcting Sample Selection Bias by Unlabeled Data, NIPS 2006] ! [Zadrozny, Learning and Evaluating Classifiers under Sample Selection Bias, ICML 2004] 129
- 130. Transfer Learning in Convolu7onal Neural Networks • Convolutional neuralnetworks (CNN): outstanding image-classification. • Learning CNNs requires a very large number of annotated image samples – Millions of parameters, to many that prevents application of CNNs to problems with limited training data. • Key Idea: – the internal layers of the CNN can act as a generic extractor of mid-level image representation – Model-based Transfer Learning
- 131.