Local: D3−2040
Périodes de cours: Lundi 15:30−16:20, Mardi 15:30−17:20

Objectifs du cours

Apprentissage supervisé par réseaux de neurones : classification et régression avec réseaux à propagation avant et prédiction de cibles. Réseaux de neurones classiques : perceptron multi-couches et régression logistique. Réseaux à convolution et architectures profondes (“deep learning”) modernes : VGG, InceptionNet, ResNet, UNet, etc. Applications à l’imagerie : reconnaissance, segmentation, localisation, transfert de style, etc. Réseaux de neurones récurrents et applications à l’analyse de texte et d’images. Modèles génératifs adversaires et réseaux de neurones non-supervisés : auto-encodeurs et auto-encodeurs variationnels. Bonnes pratiques : transfert d’entrainement, augmentation de données, normalisation, méthodes d’entrainement modernes, visualisation. Concepts avancés: modèles d’attention, autoML, compression, convolution dilatées.

Examen final

• Samedi 23 Avril 2022, de 9h à 12h au D7-2023
• Récapitulatif, du chapite 1 au chapitre 10 au complet (pas seulement ce qui a été présenté dans la conclusion)
• Vous aurez droit à deux feuilles de notes recto-version manuscrites
• L’examen évaluera vos connaissances théoriques quant à la matière du cours. Donc, pas de code ni pseudo-code à l’examen.
• Amusez-vous !

Méthode pédagogique

Le plan de cours complet est disponible ici: plan de cours

Travaux pratiques

Ressources

Il n’est absolument pas nécessaire de consulter des ressources additionnelles aux diapositives présentées dans le cours en assistant aux séances. Par contre, si vous souhaitez consulter les livres sur lequel le cours est basé:

[21] Deep Learning - Ian J. Goodfellow, Yoshua Bengio and Aaron Courville, MIT Press, 2016
[9] Pattern recognition and machine learning - Christopher M. Bishop, Springer, 2006

De plus, le livre suivant est très complet et pertinent pour les gens ayant un background moins mathématique:

[51] Mathematics for Machine Learning - M. Deisenroth, A. Faisal, and C. Ong, Cambridge University Press, 2020

Ces livres sont disponibles à la bibliothèque du Frère Théode et sur commande à la coop de l’Université de Sherbrooke. [21] et [9] sont aussi disponibles en ligne.

Références

[1] BADRINARAYANAN, VIJAY AND KENDALL, ALEX AND CIPOLLA, ROBERTO : Segnet : A deep convolutional encoderdecoder architecture for image segmentation. IEEE transactions on pattern analysis and machine intelligence, 39(12):2481–2495, 2017

[2] BAHDANAU, DZMITRY AND CHO, KYUNGHYUN AND BENGIO, YOSHUA : Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv :1409.0473, 2014.

[3] BROCK, ANDREW AND DONAHUE, JEFF AND SIMONYAN, KAREN : Large scale GAN training for high fidelity natural image synthesis. arXiv preprint arXiv :1809.11096, 2018.

[4] BROWN, TOM B AND MANN, BENJAMIN AND RYDER, NICK AND SUBBIAH, MELANIE AND KAPLAN, JARED AND DHARIWAL, PRAFULLA AND NEELAKANTAN, ARVIND AND SHYAM, PRANAV AND SASTRY, GIRISH AND ASKELL, AMANDA AND OTHERS : Language models are few-shot learners. arXiv preprint arXiv :2005.14165, 2020.

[5] ÇIÇEK, ÖZGÜN AND ABDULKADIR, AHMED AND LIENKAMP, SOEREN S AND BROX, THOMAS AND RONNEBERGER, OLAF : 3D U-Net : learning dense volumetric segmentation from sparse annotation. In International conference on medical image computing and computer-assisted intervention, pages 424–432. Springer, 2016.

[6] CHEN, LIANG-CHIEH AND PAPANDREOU, GEORGE AND KOKKINOS, IASONAS AND MURPHY, KEVIN AND YUILLE, ALAN L : Deeplab : Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence, 40(4):834–848, 2017.

[7] CHENG, JIANPENG AND DONG, LI AND LAPATA, MIRELLA : Long short-term memory-networks for machine reading. arXiv preprint arXiv :1601.06733, 2016.

[8] CHO, KYUNGHYUN AND VAN MERRIËNBOER, BART AND GULCEHRE, CAGLAR AND BAHDANAU, DZMITRY AND BOUGARES, FETHI AND SCHWENK, HOLGER AND BENGIO, YOSHUA : Learning phrase representations using RNN encoderdecoder for statistical machine translation. arXiv preprint arXiv :1406.1078, 2014.

[9] CHRISTOPHER M. BISHOP : Pattern Recognition and Machine Learning (Information Science and Statistics). SpringerVerlag, 2006.

[10] CIREGAN, DAN AND MEIER, UELI AND SCHMIDHUBER, JÜRGEN : Multi-column deep neural networks for image classification. In 2012 IEEE conference on computer vision and pattern recognition, pages 3642–3649. IEEE, 2012.

[11] DEVLIN, JACOB AND CHANG, MING-WEI AND LEE, KENTON AND TOUTANOVA, KRISTINA : Bert : Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv :1810.04805, 2018.

[12] DOERSCH, CARL : Tutorial on variational autoencoders. arXiv preprint arXiv :1606.05908, 2016.

[13] GIRSHICK, ROSS : Fast r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 1440–1448, 2015.

[14] GIRSHICK, ROSS AND DONAHUE, JEFF AND DARRELL, TREVOR AND MALIK, JITENDRA : Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 580–587, 2014.

[15] GOODFELLOW, IAN AND POUGET-ABADIE, JEAN AND MIRZA, MEHDI AND XU, BING AND WARDE-FARLEY, DAVID AND OZAIR, SHERJIL AND COURVILLE, AARON AND BENGIO, YOSHUA : Generative adversarial nets. Advances in neural information processing systems, 27, 2014.

[16] HE, KAIMING AND GKIOXARI, GEORGIA AND DOLLÁR, PIOTR AND GIRSHICK, ROSS : Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017.

[17] HE, KAIMING AND ZHANG, XIANGYU AND REN, SHAOQING AND SUN, JIAN : Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.

[18] HOCHREITER, SEPP AND SCHMIDHUBER, JÜRGEN : Long short-term memory. Neural computation, 9(8):1735–1780, 1997.

[19] HOWARD, ANDREW G AND ZHU, MENGLONG AND CHEN, BO AND KALENICHENKO, DMITRY AND WANG, WEIJUN AND WEYAND, TOBIAS AND ANDREETTO, MARCO AND ADAM, HARTWIG : Mobilenets : Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv :1704.04861, 2017.

[20] HUANG, GAO AND LIU, ZHUANG AND VAN DER MAATEN, LAURENS AND WEINBERGER, KILIAN Q : Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4700–4708, 2017.

[21] IAN GOODFELLOW, YOSHUA BENGIO ET AARON COURVILLE : Deep Learning. MIT Press, 2016.

[22] IOFFE, SERGEY AND SZEGEDY, CHRISTIAN : Batch normalization : Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pages 448–456. PMLR, 2015.

[23] ISENSEE, FABIAN AND JAEGER, PAUL F AND KOHL, SIMON AA AND PETERSEN, JENS AND MAIER-HEIN, KLAUS H : nnU-Net : a self-configuring method for deep learning-based biomedical image segmentation. Nature methods, 18(2):203–211, 2021.

[24] KARRAS, TERO AND LAINE, SAMULI AND AILA, TIMO : A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4401–4410, 2019.

[25] KINGMA, DIEDERIK P AND WELLING, MAX : Auto-encoding variational bayes. arXiv preprint arXiv :1312.6114, 2013.

[26] KRIZHEVSKY, ALEX AND SUTSKEVER, ILYA AND HINTON, GEOFFREY E : Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25:1097–1105, 2012.

[27] LECUN, YANN AND BOTTOU, LÉON AND BENGIO, YOSHUA AND HAFFNER, PATRICK : Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.

[28] LIU, WEI AND ANGUELOV, DRAGOMIR AND ERHAN, DUMITRU AND SZEGEDY, CHRISTIAN AND REED, SCOTT AND FU, CHENG-YANG AND BERG, ALEXANDER C : Ssd : Single shot multibox detector. In European conference on computer vision, pages 21–37. Springer, 2016.

[29] LONG, JONATHAN AND SHELHAMER, EVAN AND DARRELL, TREVOR : Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3431–3440, 2015.

[30] LUONG, MINH-THANG AND PHAM, HIEU AND MANNING, CHRISTOPHER D : Effective approaches to attention-based neural machine translation. arXiv preprint arXiv :1508.04025, 2015.

[31] MIKOLOV, TOMAS AND CHEN, KAI AND CORRADO, GREG AND DEAN, JEFFREY : Efficient estimation of word representations in vector space. arXiv preprint arXiv :1301.3781, 2013.

[32] MILLETARI, FAUSTO AND NAVAB, NASSIR AND AHMADI, SEYED-AHMAD : V-net : Fully convolutional neural networks for volumetric medical image segmentation. In 2016 fourth international conference on 3D vision (3DV), pages 565–571. IEEE, 2016.

[33] MIRZA, MEHDI AND OSINDERO, SIMON : Conditional generative adversarial nets. arXiv preprint arXiv :1411.1784, 2014.

[34] PASZKE, ADAM AND CHAURASIA, ABHISHEK AND KIM, SANGPIL AND CULURCIELLO, EUGENIO : Enet : A deep neural network architecture for real-time semantic segmentation. arXiv preprint arXiv :1606.02147, 2016.

[35] PLAUT, ELAD : From principal subspaces to principal components with linear autoencoders. arXiv preprint arXiv :1804.10253, 2018.

[36] RADFORD, ALEC AND METZ, LUKE AND CHINTALA, SOUMITH : Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv :1511.06434, 2015.

[37] RADFORD, ALEC AND NARASIMHAN, KARTHIK AND SALIMA

[38] RADFORD, ALEC AND WU, JEFFREY AND CHILD, REWON AND LUAN, DAVID AND AMODEI, DARIO AND SUTSKEVER, ILYA AND OTHERS : Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.

[39] REDMON, JOSEPH AND DIVVALA, SANTOSH AND GIRSHICK, ROSS AND FARHADI, ALI : You only look once : Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 779–788, 2016.

[40] REDMON, JOSEPH AND FARHADI, ALI : YOLO9000 : better, faster, stronger. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7263–7271, 2017.

[41] REN, SHAOQING AND HE, KAIMING AND GIRSHICK, ROSS AND SUN, JIAN : Faster r-cnn : Towards real-time object detection with region proposal networks. Advances in neural information processing systems, 28:91–99, 2015.

[42] RONNEBERGER, OLAF AND FISCHER, PHILIPP AND BROX, THOMAS : U-net : Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234–241. Springer, 2015.

[43] SIMONYAN, KAREN AND ZISSERMAN, ANDREW : Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv :1409.1556, 2014.

[44] SOHN, KIHYUK AND LEE, HONGLAK AND YAN, XINCHEN : Learning structured output representation using deep conditional generative models. Advances in neural information processing systems, 28:3483–3491, 2015.

[45] SRIVASTAVA, NITISH AND HINTON, GEOFFREY AND KRIZHEVSKY, ALEX AND SUTSKEVER, ILYA AND SALAKHUTDINOV, RUSLAN : Dropout : a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1):1929–1958, 2014.

[46] SZEGEDY, CHRISTIAN AND LIU, WEI AND JIA, YANGQING AND SERMANET, PIERRE AND REED, SCOTT AND ANGUELOV, DRAGOMIR AND ERHAN, DUMITRU AND VANHOUCKE, VINCENT AND RABINOVICH, ANDREW : Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1–9, 2015.

[47] VASWANI, ASHISH AND SHAZEER, NOAM AND PARMAR, NIKI AND USZKOREIT, JAKOB AND JONES, LLION AND GOMEZ, AIDAN N AND KAISER, ŁUKASZ AND POLOSUKHIN, ILLIA : Attention is all you need. In Advances in neural information processing systems, pages 5998–6008, 2017.

[48] WANG, YI AND LUO, ZHIMING AND JODOIN, PIERRE-MARC : Interactive deep learning method for segmenting moving objects. Pattern Recognition Letters, 96:66–75, 2017.

[49] XU, KELVIN AND BA, JIMMY AND KIROS, RYAN AND CHO, KYUNGHYUN AND COURVILLE, AARON AND SALAKHUDINOV, RUSLAN AND ZEMEL, RICH AND BENGIO, YOSHUA : Show, attend and tell : Neural image caption generation with visual attention. In International conference on machine learning, pages 2048–2057. PMLR, 2015.

[50] ZEILER, MATTHEW D AND FERGUS, ROB : Visualizing and understanding convolutional networks. In European conference on computer vision, pages 818–833. Springer, 2014.