11 Training Deep ANN by Transfer Learning
.pdf
School
University of Wollongong**We aren't endorsed by this school
Course
CSCI 316
Subject
Information Systems
Date
Jan 1, 2025
Pages
6
Uploaded by PresidentSnailPerson1210
Transfer Learning: An IntroductionCSCI316: Big Data Mining Techniques and Implementation
Deep Learning is Data Hungry•High-quality models are trained using large labeled datasets–Vision domain:ImageNetcontains over 14 million labeled imagesWhere do small companies get such large datasets?2
A Prevailing Solution: Transfer LearningCompany XLimited Training DataHighly-trained Model+High-quality ModelStudent AStudent BStudent CRecommended by Google, Microsoft, and Facebook DL frameworksTeacherStudentTransfer and re-use pre-trained model3
Deep Learning 101Photo credit: Google4
Transfer Learning: DetailsOutputInput𝑁Layers𝑁 − 1LayersIn general, first 𝐾layers can be directly transferred(𝐾 = 𝑁 − 1)Insight: high-quality features can be re-usedTeacherTeacher-specific classification layerStudentOutputStudent-specific classification layerInput5
Transfer Learning: An Example•Face recognition: recognize faces of 65 peopleTeacher (VGG-Face)900 images/person2,622 peopleStudentTransfer 15 out of 16 layers10 images/person65 peopleClassification AccuracyWithout Transfer LearningWith Transfer Learning1%93.47%Company X6