Introduction to Deep Learning with Python (2026-05-26 and 2026-05-27)

--- title: Introduction to Deep Learning with Python (2026-05-26 and 2026-05-27) tags: [carpentrieslab, teaching, notes] description: The collaborative notes for the course --- # Introduction to Deep Learning with Python (2026-05-26 and 2026-05-27)  <details><summary>Table of Contents</summary> [TOC] </details> --- ## :calendar: Day 1 - 26 May 2026 <details><summary>🎉 Welcome!</summary> Please, fill the [pre-workshop survey][pre-survey] and, if you have not already done so, ensure you have [the required software installed][setup-instructions]. | Start time | Lesson | | ---------- | ------ | | 09:30 | :wrench: Installation troubleshooting | | 10:00 | Introduction | | 11:20 | :coffee: Morning break | | 11:30 | Classification by a neural network using Keras | | 13:00 | :bread: Lunch break | | 14:00 | Classification by a neural network using Keras (continued) | | 14:20 | Monitor the training process | | 15:20 | :coffee: Afternoon break | | 15:30 | Monitor the training process (continued) | | 17:00 | Wrap-up | #### :female-teacher::male-teacher: Staff Teaching: [Devaraj Gopinathan][ucl-profile-dg], [Dimitra Salmanidou][ucl-profile-ds], [Miguel Xochicale][ucl-profile-mx] If you are stuck or have questions you can speak up. #### :link: Links :gear: UCL Unified AI: https://kubeflow.arc-unified-ai.condenser.arc.ucl.ac.uk/ :books: Course notes: [Introduction to Deep Learning][course-notes-idl] 📜 Slides: [Slides on onedrive][slides] :card_file_box: Lesson datasets: [Weather prediction v3][dataset-weather] ([description][dataset-weather-description]) | [Dollar street 10 v2][dataset-dollar] ([description][dataset-dollar-description]) :question: Class exercises: [menti][menti] :+1:/:-1: [End of day feedback][eod-feedback] </details> --- ## :calendar: Day 2 - 27 May 2026 <details><summary>🎉 Welcome again!</summary> | Start time | Lesson | | ---------- | ------ | | 09:30 | :wrench: Troubleshooting | | 10:00 | Monitor the training process (continued) | | 10:50 | Advanced layer types | | 11:20 | :coffee: Morning break | | 11:30 | Advanced layer types (continued) | | 13:00 | :bread: Lunch break | | 14:00 | Advanced layer types (continued) | | 14:20 | Transfer learning | | 15:20 | :coffee: Afternoon break | | 15:30 | Outlook | | 17:00 | [Post-workshop survey][post-survey] | #### :female-teacher::male-teacher: Staff Teaching: [Devaraj Gopinathan][ucl-profile-dg], [Miguel Xochicale][ucl-profile-mx], [Dimitra Salmanidou][ucl-profile-ds] If you are stuck or have questions you can speak up. #### :link: Links :gear: UCL Unified AI: https://kubeflow.arc-unified-ai.condenser.arc.ucl.ac.uk/ :books: Course notes: [Introduction to Deep Learning][course-notes-idl] 📜 Slides: [Slides on onedrive][slides] :card_file_box: Lesson datasets: [Weather prediction v3][dataset-weather] ([description][dataset-weather-description]) | [Dollar street 10 v2][dataset-dollar] ([description][dataset-dollar-description]) :question: Class exercises: [menti][menti] :ballot_box_with_ballot: [Post-workshop survey][post-survey] </details> --- ## UCL Advanced Research Computing (ARC) <details><summary>Support for your research ⚙️ 💻 🎓</summary> :bulb: [UCL Research Programming Hub (Slack)][ucl-rph-slack] Ask programming questions to the UCL research community, anything from "how to I get started on X" to "what would people recommend for solving Y". You can join using your UCL email. :speaking_head_in_silhouette: [UCL ARC Drop-in sessions](https://www.ucl.ac.uk/advanced-research-computing/community-events/drop-sessions) ([in-person](https://www.ucl.ac.uk/advanced-research-computing/person-drop-sessions)) ([online](https://www.ucl.ac.uk/advanced-research-computing/online-drop-sessions)) UCL ARC and the Research Data Management team (Library Services) run regular drop-in sessions for all research staff and research students, providing digital research support, troubleshooting and advice. :email: [UCL ARC upcoming workshops][ucl-arc-events] Join us at our upcoming workshops! :email: [UCL ARC mailing list][ucl-arc-mailing_list] Subscribe to the Digital Research Community Newsletter, and announcements relating to the topics of interest you select below. </details> --- ## Questions :::info  <details><summary> To edit this document</summary> ### :pencil: To edit this document: Click on the pencil symbol in the top left of the window to go into editing view and start typing! Here you can post any questions you have while we are going through this document. Please, use a new bullet point for each question and sub-bullet points for their answers. For example writing like this: ``` - Example question - [name=student_a] Example answer - [name=TA_1] Example answer ``` produces the following result: - Example question - [name=student_a] Example answer - [name=TA_1] Example answer Code snippets follow this format: ```python import numpy as np ``` :::  </details> ### **Write new questions below :point_down:** - [name=Devaraj] UCl Advanced Research Computing, natural language processing - [name=Sisi Chen] Faculty of Medical Sciences - [name= Richard Armstrong-Wood] Faculty of Medical Sciences, UCL - [name=Karen Stepanyan] - Department of Information Studies, UCL - [name=Vincent Nowak] - Bartlett Faculty of the Built Environment, UCL Feedback: The workshop was really well prepared. I especially liked the online documentation and the fact that the lessons followed it very closely. I know how much work goes into preparing material like that, so I really appreciated it. Thank you as well for your patience throughout the workshop. For quite some time, I have been struggling to find a good entry point into deep learning, so this introduction helped me a lot. As I am usually working during the week, it is not easy for me to free up two full days. In this case, I took two days of holiday, but I am not sure how often I will be able to do that. I still need to think more about to what extent I will use deep learning techniques within my teaching. I feel that the workshop covered the fundamentals very well, and I would also have liked to learn more about transfer learning and the adaptation of deep learning techniques to specific applications. Personally, I would have been very interested in topics such as image segmentation, classification, and training custom models without necessarily building everything from scratch. ## Notes <details><summary>Code snippets</summary> - Download weather prediction data ``` data = pd.read_csv("https://zenodo.org/record/5071376/files/weather_prediction_dataset_light.csv?download=1") ``` - Some code snippets for classification (penguins) ```python # predict using trained model y_pred = model.predict(X_test) # prediction = pd.DataFrame(y_pred, columns=target.columns) prediction predicted_species = prediction.idxmax(axis = 'columns') predicted_species # plot confusion matrix from sklearn.metrics import confusion_matrix true_species = y_test.idxmax(axis = 'columns') matrix = confusion_matrix(true_species, predicted_species) print(matrix) # confusion matrix confusion_df = pd.DataFrame(matrix, index = y_test.columns.values, columns = y_test.columns.values) confusion_df.index.name = 'true label' confusion_df.columns_name = 'predicted_label' confusion_df.head() sns.heatmap(confusion_df, annot=True,cmap = 'Blues') ``` - Download dollar dataset ```python # Download dataset !mkdir -p /home/jovyan/data/dataset_dollarstreet !wget -nc -P /home/jovyan/data/dataset_dollarstreet https://zenodo.org/records/10970014/files/test_images.npy !wget -nc -P /home/jovyan/data/dataset_dollarstreet https://zenodo.org/records/10970014/files/test_labels.npy !wget -nc -P /home/jovyan/data/dataset_dollarstreet https://zenodo.org/records/10970014/files/train_images.npy !wget -nc -P /home/jovyan/data/dataset_dollarstreet https://zenodo.org/records/10970014/files/train_labels.npy ``` ``` # Setup path and variables DATA_FOLDER = pathlib.Path('/home/jovyan/data/dataset_dollarstreet') # change to location where you stored the data train_images = np.load(DATA_FOLDER / 'train_images.npy') val_images = np.load(DATA_FOLDER / 'test_images.npy') train_labels = np.load(DATA_FOLDER / 'train_labels.npy') val_labels = np.load(DATA_FOLDER / 'test_labels.npy') ``` - Plot image class distribution ```python # plot class distribution unique_classes, counts = np.unique(train_labels, return_counts = True) class_names = [label_to_categories[i] for i in unique_classes] plt.figure(figsize = (12,5)) plt.bar(class_names, counts) plt.xticks(rotation = 50) plt.xlabel('Category') plt.ylabel('Number of images') plt.title('Training class distribution') ``` - dropout neural network ```python def create_nn_with_dropout(): inputs = keras.Input(shape=train_images.shape[1:]) x = keras.layers.Conv2D(50, (3, 3), activation='relu')(inputs) x = keras.layers.MaxPooling2D((2, 2))(x) x = keras.layers.Dropout(0.8)(x) # This is new! x = keras.layers.Conv2D(50, (3, 3), activation='relu')(x) x = keras.layers.MaxPooling2D((2, 2))(x) x = keras.layers.Dropout(0.8)(x) # This is new! x = keras.layers.Conv2D(50, (3, 3), activation='relu')(x) x = keras.layers.MaxPooling2D((2, 2))(x) x = keras.layers.Dropout(0.8)(x) # This is new! x = keras.layers.Flatten()(x) x = keras.layers.Dense(50, activation='relu')(x) outputs = keras.layers.Dense(10)(x) model = keras.Model(inputs=inputs, outputs=outputs, name="dropout_model") return model model_dropout = create_nn_with_dropout() model_dropout.summary() ``` </details> <details><summary> Questions/Answers - links</summary> - What is the difference across keras layers eg. `keras.layers.Dense` - In TensorFlow, the Dense layer represents a fully connected layer where each neuron is fully conected to neurons in adjacent layers. This is one of the most basic layer types where "every neuron receives a weighted sum of all the inputs, applies an activation function, and then passes the result to the next layer" [Source](https://www.geeksforgeeks.org/deep-learning/what-is-fully-connected-layer-in-deep-learning/). The layers we build are highly connected to the data. Although not limited to these use cases some examples are given below: | Layer | Use cases | | ------------- | ------- | -------- | | Dense | core keras layer, 1D- 2D vectors | | Convolutional | spatial data, image classification, cases where spatial hierachy plays an important role | | Recurrent | sequential data, time series,text, cases where current input depends on previous input in the sequence | | Normalisation | scaling, normalising the data, cases where gradients need to be stabilised | - What are the different types of loss functions and example use cases? - There are a few choices of loss functions that can be used for each problem, sometimes it depends on the nature of the prediction e.g. cross-entropy loss function are usually used for classification vs mean squared error for regression, identifying the right loss function may depend on the nature of the research question and data, it is part of the optimisation process. - some useful resources for loss functions and use cases may be found in this paper ([Loss Functions in Deep Learning: A Comprehensive Review](https://arxiv.org/html/2504.04242v1)) and these web articles [1](https://medium.com/codex/quick-overview-types-of-loss-functions-and-their-use-cases-2868fc649d79) [2](https://builtin.com/machine-learning/common-loss-functions) - How to tune no. of layers/neurons per layer? - These are called hyperparameters. You could use other packages that tune the hyperparameters. *e.g.* https://optuna.org/ for [`learning_rate`](https://optuna.org/#code_examples) - How to get weight from `keras` model? `model.layers[1].get_weights()` - Some links for good learing material: - [scikit-learn user guide](https://scikit-learn.org/stable/user_guide.html) - [cross validation (CV)](https://scikit-learn.org/stable/modules/cross_validation.html) - [prevent data leakage](https://scikit-learn.org/stable/common_pitfalls.html#data-leakage) - [inria scikit-learn mooc](https://inria.github.io/scikit-learn-mooc/) - [nested cross validation (CV)](https://www.analyticsvidhya.com/blog/2021/03/a-step-by-step-guide-to-nested-cross-validation/) - [batch-norm-explained visually explained](https://towardsdatascience.com/batch-norm-explained-visually-how-it-works-and-why-neural-networks-need-it-b18919692739/) - [image kernels](https://setosa.io/ev/image-kernels/) - [The convolutional neural network cheat sheet ](https://stanford.edu/~shervine/teaching/cs-230/cheatsheet-convolutional-neural-networks/) </details> --- <details><summary>Setup Instructions</summary> Follow instructions at [the workshop overview site][setup-instructions] </details> --- <details><summary>Glossary</summary> - HackMD = collaborative notes = this document - text = characters = string = anything inside double quotes " or single quotes ' - digit = number - alphabetic characters = abcdefghijklmnopqrstuvwxyz - alphanumeric = alphabetic or numeric (abcdefghijklmnopqrstuvwxyz0123456789) - docstring = documentation = help text on how to use a function - arguments = inputs = parameters = the things in brackets after a function name, e.g. `max(x)`: the function `max` takes `x` as input, so `x` is the argument or parameter of `max` - int = integer = whole number (e.g. 5) - float = floating point number = decimal (e.g. 5.0 or 4.3) - brackets = parentheses = () - tilde = ~ (on a UK keyboard it is near the Enter key) - operand = things that an operator acts on - type = sort of object - for example an integer number or string of characters - dunder = double underscore, used to indicate special methods in Python for example `__add__` </details> ---  [setup-instructions]: https://carpentries-lab.github.io/deep-learning-intro/index.html [pre-survey]: https://forms.cloud.microsoft/e/gx0dr2d4TL [post-survey]: https://forms.cloud.microsoft/e/4JEnCqtZ92 [menti]: https://www.menti.com/alz3gsojzre8 [slides]: https://liveuclac-my.sharepoint.com/:p:/g/personal/ccaemxo_ucl_ac_uk/IQC5BZHhYH9qS6oSBAQhjzYpAXT3aErPxBe9zgQrkoa2CNU?e=FHzsFk [inkpath]: https://webapp.inkpath.co.uk [eod-feedback]: TBC [course-notes-idl]: https://carpentries-lab.github.io/deep-learning-intro/index.html [dataset-weather]: https://zenodo.org/record/5071376/files/weather_prediction_dataset_light.csv?download=1 [dataset-weather-description]: https://zenodo.org/record/5071376 [dataset-dollar]: https://zenodo.org/api/records/10970014/files-archive [dataset-dollar-description]: https://zenodo.org/records/10970014 [ucl-rph-slack]: https://www.ucl.ac.uk/advanced-research-computing/community-events/ucl-research-programming-hub [ucl-arc-drop-ins]: https://www.ucl.ac.uk/advanced-research-computing/community-events/drop-sessions [ucl-arc-events]: https://ucl.ac.uk/arc/edu-events [ucl-arc-mailing_list]: https://r1.dotdigital-pages.com/p/6T1L-JUR/ucl-arc-subscribers [ucl-profile-dg]: https://profiles.ucl.ac.uk/61652-devaraj-gopinathan [ucl-profile-ds]: https://profiles.ucl.ac.uk/63143-dimitra-salmanidou [ucl-profile-mx]: https://profiles.ucl.ac.uk/91071-miguel-xochicale/