11 views
# Collaborative Lesson Development Training :::info **Dates**: 21-24 October, 2024 **Time**: 19:00-23:00 ([follow link for your local time](https://www.timeanddate.com/worldclock/fixedtime.html?msg=Collaborative+Lesson+Development+Training&iso=20241021T19&p1=%3A&ah=4)) **Zoom link**: TBC **Code of Conduct**: https://docs.carpentries.org/topic_folders/policies/code-of-conduct.html **Curriculum**: https://carpentries.github.io/lesson-development-training/ **Trainers:** - Erin Becker - Sarah Stevens ::: [TOC] ## About This CodiMD We will use this CodiMD to take notes, share links, exercises, etc with participants throughout the training. Participants are encouraged to take shared notes on this page. The Trainers will show you how to use CodiMD at the beginning of the workshop. ## Attending * Name / pronouns (optional) / affiliation / email address * Sarah Stevens (she/her/hers) / University of Wisconsin-Madison / sarah.stevens@wisc.edu * Erin Becker (she/her) / The Carpentries / ebecker@carpentries.org * Erwin Lares (he) / University of Wisconsin-Madison / erwin.lares@wisc.edu * Michelle Busch (she/hers) / University of Kansas / m.h.busch@ku.edu * Trisha Adamus (she/her) / University of Wisconsin-Madison / adamus@wisc.edu * Maggi Kraft (she/her)/ Idaho State University/ maggikraft4@gmail.com * Ella Belfer (she/her), Catalyst Cooperative, ella.belfer@catalyst.coop * Dazhong Xia (he/him), Catalyst Cooperative, dazhong.xia@catalyst.coop * Jean-Yves Sgro (he/him/his) Biochemistry Dept and Biotechnology Center, University of Wisconsin-Madison / jsgro@wisc.edu - my Biochem [pages/blog](https://bcrf.biochem.wisc.edu/) * * ## Notes **Breaks:** 10 min breaks at about 1.25 hrs (20:15 UTC) in and ~2.75 (21:50) hrs in today ### Introduction **Objectives**: After completing this episode, participants should be able to... - Explain how trainers and participants will interact throughout the workshop. - Summarise the main skills that will be taught in this workshop. **Questions**: - What is covered in this training? - Who are the trainers? - Who is participating? #### Code of Conduct [The Carpentries Code of Conduct](https://carpentries.github.io/lesson-development-training/instructor/CODE_OF_CONDUCT.html) #### CodiMD - Edit/Split/View Modes - Write in Markdown - click the '?' button in the top bar for a quick Markdown reference sheet. - Indicate your name with [name=yourname] #### Our First Exercise (10 minutes) What was the best lesson you ever followed (were taught in a class, read through online, read in a book)? Try to differentiate between **what was good about the performance** of the teacher/trainer and **what was good about the content** of the lesson itself. Take a few minutes to write down some notes about your answer. - UW-Madison: - Sarah Stevens: When I think of a good lesson, I always seem to think of my experience with Carpentries Instructor Training. Greg Wilson taught it and is an excellent performer as an instructor, he responds well to the active discussion in the room which I think was helpful about it. For content, the materials have a lot of examples, exercises, and are based on education research which seemed very effective. - Erwin Lares: I can't recall a specific lesson, but lessons in general that are (1) well-structured, have a context, and build on the previous material and (2) lessons where speakers take their time going through the material, allow participants to digest and apply what they are learning. - Jean-Yves - Followed a lesson on YouTube about ["Hugo"](https://gohugo.io/documentation/) a Jekyll-like method **without any dependencies** to create a "static web site" *i.e.* held completely within a single directory. The lessons spanned many episodes, were long enough to have a comprehensive understanding of the material, but short enough to not be overwhelming. The instructor provided clear examples of what was the effect of the various commands or functions. The instructor had planned lessons with progressive difficulties and always reminded the previous learnings before a new lesson. REF: First tutorial: [Giraffe Academy](https://youtu.be/qtIqKaDlqXo) - Trisha Adamus: I really enjoyed Programming For Everybody by Chuck Severance. Until I took this course, I felt very nervous about coding. My previous expereinces had been taught to a CS audience, not a general audience. The material was desinged to meet people where they were at, and Chuch made it seem harmless. This experience was part of the reason I wanted to get into teaching computational skills and the Carpentries. - Catalyst Cooperative: - Ella Belfer - A spatial data analysis course that was incredibly systematic about structure - we started learning about points, then moved to lines, then polygons and then finally rasters. The mix of theory and practical exercises was well balanced, letting us move through difficult material without getting lost in the weeds or without assuming anyone had prior coding experience. The lessons really built on one another in a way that was very informative (even though the teacher himself was somewhat difficult!). - Dazhong Xia - My first-year college math course was very clearly structured and built on itself in small steps. When we looked back we had somehow gone from "how do we know that -1 * -1 = 1" to proving properties of derivatives and integrals. - I think one key piece here was that the content was well-sequenced, so that no individual step was too big, but that many small steps still added up towards being able to do something big. - Another piece would be that the actual activity in the class involved the students contributing a lot to the proofs, figuring things out together. We almost mob-programmed the whole set of proofs which, I think, helped us learn the sort of ineffable bits about how other students / the instructor thought about things. - Kansas Uni: - Michelle Hope Busch - took a graduate course in spatial analysis in R. very consistent lessons with powerpoint introductions to the topics/analyses we would learn about and then hands on R analyses on how to apply those topics/analyses. it felt very practical and useful and I can still go back to reference the code when I forget how to run things / what analyses are appropriate. lessons built on each other. - Maggi Kraft - A graduate course in geophysics. The professor worked through the material slowly and catered the course to the students interests. He wrote everything on the board rather than using slides which helped slow the course pace down. As he was writing/drawing he included the students in the discussion, and opened the room for questions. There was never a need to cover a certain amount of material and he would adjust the schedule and course based on where the class was. #### Training Structure ![Collaborative Lesson Development Training is taught over three days (six half days), with an extended break between the first two days ("part 1") and the last day ("part 2"). The training in part 1 focusses on good practices in lesson design and development, and the concepts and skills needed to build a lesson website with The Carpentries lesson infrastructure. Part 2 of the training begins with a reflective discussion of lesson deisgn and iterative development, before shifting focus to the skills required for effective collaboration on open source projects. ](https://carpentries.github.io/lesson-development-training/fig/cldt-structure.svg) We will focus on three main areas: * Designing a lesson * Building a lesson website * Collaborative effectively **Key Points**: - This training aims to teach you a process for designing a lesson and the skills to develop it as an open source website, in collaboration with others. ### Lesson Design **Objectives**: After completing this episode, participants should be able to... - Explain the lesson design process we will be adopting for this course. - Summarise the lessons that participants will be working on. **Questions**: - What are the recommended steps to take when developing a new lesson? - What lesson do you want to develop during and after this workshop? #### Discussion (10 minutes) Share your answers to the following questions in the shared notes, then discuss them with the Trainers, your collaborators, and the other participants. 1. What is the topic of the lesson that you plan to develop based on this training? 2. Have you created training material on this topic before? 3. What is motivating you to create this lesson? - UW-Madison: 1. What is the topic of the lesson that you plan to develop based on this training? * Intermediate Research Software Development in R + How do develop a package in R + Testing in R + with github actions - Continuous integration + CI-CD with gitlab runners + Shiny/Quarto? 2. Have you created training material on this topic before? * No, only some individual pieces, but we have taught the python version before. 3. What is motivating you to create this lesson? - Requests for the communtiy to have more R - We've built a lot of support for the novice learning but then need more the next steps for learning intermediate skills - Catalyst Cooperative: 1. topic? Foundational Data and Software Skills for Open Energy Research (in Python) * data analysis & ingestion in a programmatic environment (and when to use this tool) * "core" skills that are already taught elsewhere (like git, SQL) * techniques for analyzing non-spreadsheet data * collaboration tools + practices * testing techniques for data processes * maybe the data engineering tasks of data pipeline design? * maybe strategies for *finding* accessible data in the first place? 2. experience? Not really - we've a few demo notebooks but nothing with focused pedagogical intent 3. motivation? helping energy researchers address common technical obstacles to reproducible, open, fast, etc. analysis (to then hasten the energy transition) - Kansas Uni: - 1. Timeseries manipulation and analysis - cleaning / manipulation (subsetting, filtering, selecting, pivot wide to long and vice versa, etc.) - base R and tidyr // what would be most useful going forward for the audience? - statistics (full, annual, monthly, seasonal, breakpoints) - visualization / plotting (boxplots, skew, patterns) - spatial component (top of watershed vs watershed output)?? - 2. Neither one of us has developed material on this topic - 3. Develop a lesson for undergrad - early graduate student level (reducing barriers to coding) and gaining skills in pedagogy #### Iterative Development ![Diagram of the life cycle of a lesson in The Carpentries ecosystem. A lesson is proposed at the beginning of the pre-alpha stage. It enters alpha when it is taught for the first time. In beta, it is taught by other instructors. A full release of the lesson is made when it is stable. Pilot workshops take place during the alpha and beta phases.](https://carpentries.github.io/lesson-development-training/fig/life_cycle.png) - `pre-alpha`: first draft of lesson is being created - `alpha`: lesson is being taught by authors - `beta`: lesson is ready to be taught by other instructors - `stable`: lesson has been tested by other instructors and improved based on feedback. Major changes and updates are relatively infrequent. **Key Points**: - We will learn to develop lessons based on the (slightly adapted) Nicholl’s backward lesson design process. - There can be many reasons to create a new lesson. - This training will give you a process to follow to ensure your lesson is effective. --- ### Identifying Your Target Audience **Objectives**: After completing this episode, participants should be able to... - Describe the importance of aligning lesson design with the intended audience. - Compose a list of prior knowledge required to follow a lesson. **Questions**: - Why is it so important to think about the target audience early in the process? - How can you ensure that your lesson reaches the right audience? #### Lesson Design Notes We will now create documents to capture your decisions, drafted content etc as you begin designing your new lesson. Use the template below to create this document. Make a copy of [the Lesson Design Notes template](https://codimd.carpentries.org/HPwUE3FnTeSQJ9-_5EfU7Q?view#). 1. You can type your desired URL, e.g. https://codimd.carpentries.org/my-cool-lesson-title and (unless you are very unlucky) get a blank document to paste into. Already content on this page? Please choose a different URL! 2. You can click the '+New' button in the top bar of the CodiMD window 3. (Not recommended) You can paste the rendered version (copied from the view mode) into a GoogleDoc/similar if you prefer. But we recommend sticking with Markdown! This will save you time and pain when putting content into your new lesson website laer in the week. The content you draft there will be transferred into a new lesson website later in this first part of the training. - AIMS: https://codimd.carpentries.org/timeseriesR - UW-Madison: https://codimd.carpentries.org/int-software-devel-in-R - Catalyst Co-Op: https://codimd.carpentries.org/catalyst-coop-open-energy-data-for-all# #### Exercise: thinking about target audience (15 minutes total for both parts) ##### Part 1 (all, 5 minutes) Think about a member of the target audience for your lesson, and answer the following questions in the context of your lesson topic: 1. What is their background? 2. What do they already know how to do? 3. What do they want to do with the skills they will learn from your lesson? 4. What problem will your lesson help them solve? ##### Part 2a (for groups collaborating on the same lesson, 10 minutes) Share your answers with your collaborators. How do they compare? If you have identified different audiences, are they compatible? Or would your time be better spent focussing on one particular audience for this lesson? Take notes on your discussion in your Lesson Design Notes document. It can be particularly helpful to note down any decisions made e.g. potential target audiences that were explicitly discounted, and your reasons for doing so. #### Exercise: Defining Prerequisite Knowledge (10 minutes) Write a list of the skills/knowledge your learners will be required to have before they can follow your lesson. Add this list to the 'Target Audience' section of your Lesson Design Notes document. If you are struggling with this exercise because your lesson audience is novices, think about skills like touch typing, using a web browser, or interacting with a command line or graphical interface. These are skills commonly overlooked by experts and competent practitioners. **Key Points**: - We recommend an iterative lesson design process that begins with identifying the target audience, before defining learning outcomes, then creating assessments, writing explanatory content, and evaluating the lesson in a workshop. - Thinking about the target audience early in the design process helps to ensure that your lesson is built around the needs and motivations of real people. - Use the description of your target audience to help attract people with the appropriate interests and prior knowledge to your lesson. --- ### Defining Lesson Objectives/Outcomes **Objectives**: After completing this episode, participants should be able to... - Explain the importance of defining specific, measurable, attainable, relevant and time-bound objectives for a lesson. - Evaluate a written lesson objective according to these criteria. **Questions**: - How can describing the things you intend to teach aid the process of writing a lesson? - How can you be specific and realistic about what you will teach in your lesson? - What are some of the risks associated with unrealistic or undefined expectations of a lesson? #### What does an objective look like? ![An example learning objective, "import data into an indexed DataFrame with read_csv", with emphasis placed on the action verb ("import") and the specificity ("indexed") of the objective.](https://carpentries.github.io/lesson-development-training/fig/objective.svg) BLoom Taxonomy: https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/ Computing: https://ccecc.acm.org/files/publications/Blooms-for-Computing-Verb-Table-for-printing-Computing-verbs-bold-Letter-size.pdf #### SMART Objectives Should Be... * **Specific**: they should clearly describe a particular skill or ability the learner should have. * **Measurable**: it should be possible to observe and ascertain when the learner has learned the skill/abilities described in the objectives. * **Attainable**: the learner should realistically be able to acquire the skills or abilities in the time available in a workshop/by following the text of the lesson. * **Relevant**: they should be relevant to the overall topic or domain of the lesson as a whole. * **Time-bound**: they should include some timeframe within which the goal will be reached. For learning objectives, this is built into the approach described above. #### Exercise: evaluating learning objectives (15 minutes) Look at the example learning objectives below. Fill in the table for each objective, checking off the cells if you think an objective meets the criteria or leaving it unchecked if not. You should assume each objective is for a lesson to be taught in a two-day workshop. Note down any observations you make as you move through the list. If you have time, try to imagine the titles of lessons that would have these objectives. This part of the exercise should take 10 minutes. _At the end of this lesson, learners will be able to:_ 1. create formatted page content with Markdown. 2. program with Rust. 3. fully understand GitHub Actions. | Objective | | | --------- | ------------ | ----------- | ---------------------------------------------------------- | ----------------- | | 1 | Yes | Maybe? | Maybe needs to be a little more specific to be measureable | Yes | | 2 | Yes | No | No | Maybe? Maybe not? | | 3 | No | No - Maybe? | | No-No maybe? | Tool for evaluating learning objectives: https://web.cs.manchester.ac.uk/iloadvisor/ #### Exercise: defining objectives for your lesson (20 minutes) Write learning objectives for your lesson - what do you want learners to be able to do at the end of the workshop? When writing these lesson-level objectives, try to follow the SMART framework: make them specific, measurable, attainable, relevant, and time-bound. Take notes about the your discussion in your Lesson Design Notes document. A record of the decisions you made and your reasons for choosing these objectives can be very helpful for you and others to understand the design and scope of the lesson later. #### Exercise: reviewing lesson objectives (10 minutes) Swap objectives written in the previous exercise with a partner (you can also explain or show them what you wrote about your target audience, but this not essential) and review them with the following questions in mind: - Are the objectives clear? - Do they use "action" verbs? - Could you directly observe whether a learner had reached this objective? Now run the objectives through [this Lesson Objective Advisor tool from the University of Manchester's Faculty of Science and Engineering](https://web.cs.manchester.ac.uk/iloadvisor/). Do the results match your assessment? - Where do the skills described in these objectives sit on the scale? - (optional) Are these objectives realistic, given the target audience of the lesson? **Key Points**: - Defining objectives for a lesson can help to focus your content on the most important outcomes, and outline the scope of the project. - Following the SMART framework can help make your learning objectives as useful as possible. - Leaving objectives unrealistic or undefined increases the risk of a lesson losing focus or spending time on activities that do not help learners gain the most important skills. --- ## Welcome back!!! **Breaks:** 10 min breaks at about 1.25 hrs (20:15 UTC) in and ~2.75 (21:50) hrs in today ### Sign-in: * Sarah Stevens (she/her/hers) * Maggi Kraft (she/her) * Michelle Busch (she/her) * Ella Belfer (she/her) * Erwin Lares (he) * Dazhong Xia (he/his) * Trisha Adamus (she/her) * Erin Becker (she/her) ### Example Data and Narrative **Objectives**: After completing this episode, participants should be able to... - Find candidate datasets to use in a lesson. - Evaluate the suitability of a dataset to be used in a lesson. - Choose examples that will prepare learners for formative assessments in the lesson. - Develop a story **Questions**: - Why should a lesson tell a story? - What considerations are there when choosing an example dataset for a lesson? - Where can I find openly-licensed, published data to use in a lesson? #### Dataset considerations - Ethical use (see prompts below) - License - CC0 Recommended - Complexity - _Is it easy to understand?_ _Is it sufficiently authentic?_ - Number and types of variables #### Questions about Ethical Use of Datasets - Does the data contain personally identifiable information? - Was the data collected without permission from the groups or individuals included? - Will the data be upsetting to learners in the workshop? [CARE Principles for Indigenous Data Governance](https://doi.org/10.5334/dsj-2020-043) - Collective Benefit, Authority to Control, Responsibility, and Ethics #### Example Datasets from Existing Lessons * The [Ecology Data Carpentry lessons'](https://datacarpentry.org/lessons/#ecology-workshop) dataset comes from the [Portal Project Teaching Database](https://figshare.com/articles/dataset/Portal_Project_Teaching_Database/1314459). * The [Social Sciences Data Carpentry lessons'](https://datacarpentry.org/lessons/#social-science-curriculum) dataset is the [teaching version](https://figshare.com/articles/dataset/SAFI_Survey_Results/6262019) of the full Studying African Farmer-Led Irrigation (SAFI) dataset. * The Software Carpentry Python [novice](https://swcarpentry.github.io/python-novice-inflammation/index.html) and [intermediate](https://carpentries-incubator.github.io/python-intermediate-development/index.html) (in incubation) lessons use an artifically-generated [Patient inflammation dataset](https://swcarpentry.github.io/python-novice-inflammation/data/python-novice-inflammation-data.zip). * The [Earth and Environmental Sciences Intermediate Python lesson](https://carpentries-incubator.github.io/python-intermediate-development-earth-sciences/index.html) (in incubation) uses a [river catchment dataset](https://github.com/carpentries-incubator/python-intermediate-rivercatchment/tree/main/data) from the [Lowland Catchment Research (LOCAR) Datasets](https://catalogue.ceh.ac.uk/documents/db9f6ef9-9512-4f39-aca3-3c55f51a7487). * Data Carpentry's [Astronomical Data Science with Python](https://datacarpentry.org/astronomy-python/) lesson uses two astronomical datasets, from the Gaia satellite and the Pan-STARRS photometric survey, to reproduce part of an analysis described in a published article #### Examples of Public Repositories - [Dryad](https://datadryad.org/) - [The Data Curation Network’s datasets](https://datacurationnetwork.org/datasets/) - [The Offical Portal for European Data](https://data.europa.eu/) - [DataONE](https://www.dataone.org/) - [The Official Portal for Argentina Data](https://www.datos.gob.ar/) - In Spanish - [LANFRICA](https://lanfrica.com/) #### Exercise: Choosing a Dataset or Narrative (15 minutes) Referring to the advice given above, find an appropriate dataset or a narrative for your lesson. Identify one or more potential candidates and note down the advantages and disadvantages of each one. As a reminder, here are some aspects we suggest that you consider: * For datasets: * size * complexity * "messiness"/noise * relevance to target audience * availability * license * ethics * For narratives: * authenticity * relevance to target audience * complexity * possibility to teach useful things first/early Takes notes in your Lesson Design Notes document about your discussion and the decisions made. It may be particularly helpful to record: * Which datasets and narratives did you consider? * How and why did you choose between them? * What implications do you think your choice of dataset and/or narrative will have for the design and further implementation of your lesson? **Key Points**: - Using a narrative throughout a lesson helps reduce learner cognitive load - Choosing a lesson includes considering data license and ethical considerations. - Openly-licensed datasets can be found in subject area repositories or general data repositories. --- ### Episodes **Questions**: - How can the objectives for a lesson be used to break its content into sections? - How should objectives be written for a smaller part of a whole lesson? **Objectives**: After completing this episode, participants should be able to... - identify appropriate parts of their lesson to break into individual sections - define learning objectives for a section of a lesson. #### Exercise: Defining Episodes for a Lesson (25 minutes) With your team, in the shared notes document for your lesson: 1. Based on the lesson-level objectives and your knowledge of the lesson topic, divide the lesson up into logical blocks (episodes), that should each take approximately 20-60 minutes to teach. Think of these logical blocks as topics that you need to cover within your lesson but do not go too deep into defining learning objectives for individual episodes - we will cover that soon. 2. Next, assign responsibility for one of these episodes to each collaborator in your team. They will focus on this episode for the rest of this training, and you will teach these episodes in a trial run between parts 1 and 2 of this training. Some questions you might ask yourself to help break down your lesson-level learning objectives include: * What new knowledge and skills will learners need to acquire to be ready to fulfil the overall objectives for your lesson? * What order should these concepts and skills be introduced in? Are some dependent on others? * If some of these concepts and skills are complex, can they be broken down even further? If the length of the list of episodes you create is smaller than the number of collaborators you have on your team, try assigning two people to a single episode, with each taking responsibility for a subset of the objectives defined for that episode. If you plan more episodes than you have people in your team, do not assign more than one episode to each collaborator for now, but **we strongly recommend that you assign yourselves consecutive episodes at the beginning of your lesson**. #### Exercise: define objectives for your episode (30 minutes) 1. Using the same approach as you did for your whole-lesson objectives, define a set of SMART objectives for your chosen episode. (15 minutes) 1. Compare your list with those created by your collaborators on the lesson: - are there any gaps in these objectives, i.e. anything that should be covered in these episodes but is not captured in the objectives? - are there any overlaps, i.e. anything that looks like it will be covered more than once? 1. As a group, discuss how you will address any problems identified in the previous step, and edit your objectives accordingly. **Key Points**: - Learning objectives for a lesson can help you split up its content into chunks - Objectives can be defined for episodes within a lesson, to guide individual steps along the pathway of the lesson. --- ### Designing Exercises **Objectives**: After completing this episode, participants should be able to... - Describe the importance of regular assessment while a lesson is being taught. - Choose the format for an exercise based on the outcome it is intended to measure. **Questions**: - How can you measure learners' progress towards your lesson objectives? - Why are exercises so important in a lesson? - What are some different types of exercises, and when should they be used? - Why should we create assessments before we have written the explanatory content of our lesson? #### Types of Formative Assessment - reflection - concept maps and diagrams - checking in - think, pair, share - exercises #### Formative Assessments in this Training (5 minutes) Think back through the parts of this training you have followed so far. Identify two examples of formative assessment that the Trainers have employed so far. As an extra challenge, try to decide whether these assessments were used to assess progress towards a particular learning objective and, if so, what the relevant objective might have been. [Learning objectives](https://carpentries.github.io/lesson-development-training/instructor/index.html#learning-objectives) - group exercise of whether an object followed SMART format (yes, no, maybe table) - objective: define SMART learning objectives - quick check (yes and no) on the difference between summative and formative assessments = objective: create exercises for formative assessment. - Group work - evaluate leaning objectives * Write your own lesson objective, and assess a peer's using the SMART principles. -- measuring progress towards "define SMART learning objectives". - peer work - think outloud - reporting out - think-pair-share? - writing the target audience down, what their prereqs were, etc: this was a problem-solving exercise that also turned into a group discussion - with the aim of supporting the "target audience" objective * Review of objectives on day one with non group member - checking in, objective defining SMART learning objectives #### Recommended reading: More Example Assessment Types * [Exercise Types Chapter from Teaching Tech Together](https://teachtogether.tech/en/index.html#s:exercises) * [Edutopia’s 56 Examples of Formative Assessment](https://www.edutopia.org/groups/assessment/250941) * ["21 Ways to Check for Student Understanding"](https://www.opencolleges.edu.au/informed/features/21-ways-to-check-for-student-understanding/) * [H5P Examples and Downloads for Interactive Content](https://h5p.org/content-types-and-applications) #### Exercise: Exercise Types and When to Use Them (15 minutes) The Trainers will assign you to pairs or small groups, and give each group an exercise type to work on. Each group should assign a notetaker, to summarise your discussion at the end of the exercise. ##### For lessons predominantly teaching coding skills The Trainers will assign your group a type of exercise to focus on. Read about your given exercise type [in the *Exercise Types* chapter of *Teaching Tech Together*](https://teachtogether.tech/en/index.html#s:exercises) by following the relevant link below. * [multiple choice questions](http://teachtogether.tech/en/index.html#a-multiple-choice-question) * [fill-in-the-blanks](https://teachtogether.tech/en/index.html#fill-in-the-blanks) * [faded examples](https://teachtogether.tech/en/index.html#faded-examples) * [Parsons problems](https://teachtogether.tech/en/index.html#parsons-problem) * [minimal fix](https://teachtogether.tech/en/index.html#minimal-fix) (A [Spanish version of _Teaching Tech Together_](https://teachtogether.tech/es/index.html#s:exercises) is also available.) Then, considering the exercise type in general, as opposed to the specific example given in the text, discuss the following questions together: * What kind of skills would an exercise of this type assess? Try to identify some action verbs like those we used to write lesson objectives earlier in the workshop. * Would this type of exercise be suited to a novice audience? Or is it a better fit for intermediate or advanced learners? * Would this kind of exercise work well in an in-person workshop setting? Would it be better suited to self-directed learning and/or a virtual workshop? #### Exercise: Exercise Types and When to Use Them (Continued, 15 minutes) Share the major points of your discussion in the collaborative notes document. ### minimal fix * What kind of skills would an exercise of this type assess? * coding * debugging * diagnosing (and other "analyze" skills) * solving (and other "apply" skills) * may be useful for non-coding skills that still have a right/wrong answer and a logical process of getting there * Would this type of exercise be suited to a novice audience? Or is it a better fit for intermediate or advanced learners? * Definitely can be used in a novice setting, but can be used in much more advanced settings as well * Could be intimidating for true novices, but we should encourage them! * For novices, this would be "learn how to debug" as well as "figure out what the bug is" * Would this kind of exercise work well in an in-person workshop setting? Would it be better suited to self-directed learning and/or a virtual workshop * I think would work well in both, in-person people will have to hold their tongues in a way that remote/self-directed doesn't have to worry about * Self-directed, you'd need some way of replicating the instructor's hints in case you don't figure it out * ### Parsons Problems * What kind of skills would an exercise of this type assess? Try to identify some action verbs like those we used to write lesson objectives earlier in the workshop. - adapts existing code - compare coding example to working code - selecting what lines are needed to complete the exercise - Exemplify order of operations - Implement good syntax * Would this type of exercise be suited to a novice audience? Or is it a better fit for intermediate or advanced learners? - ideal for novice audience - gives most of the answer for the students, provides some of the code needed and gives students a place to start from (although could still be intimidating). - also useful for intermediate / advance learners with more advanced exercises. maybe useful as a de-bugging exercise (given a full script that doesn't work and have to identify why) * Would this kind of exercise work well in an in-person workshop setting? Would it be better suited to self-directed learning and/or a virtual workshop? - all of the above: teaches code review process - could use sticky notes to indicate when accomplished (in person), share code in zoom chat (virtual), allows learner to test versions of the code themselves before looking at the answer (self-directed, all) ### faded examples * What kind of skills would an exercise of this type assess? Try to identify some action verbs like those we used to write lesson objectives earlier in the workshop. * apply * create * analyze * debug * helps to teach the process of coding * Would this type of exercise be suited to a novice audience? Or is it a better fit for intermediate or advanced learners? * better suited for novices-intermediate * great for scaffolding, prevents students from getting overwhelmed. * outline processes, if it's well set up. break down the order of things. * this is a labor-intensive exercise to create. * Would this kind of exercise work well in an in-person workshop setting? Would it be better suited to self-directed learning and/or a virtual workshop? * Either, though possibly easier online. Can get overwhelming in person, online you can hide the many steps and cycle through them more simply. * Can use [call-out blocks](https://quarto.org/docs/authoring/callouts.html) to implement this in Quarto, giving novices additional scaffolding #### Exercise: Assessing Progress Towards an Objective (30 minutes) Using one of the exercise formats you have learned about so far, design an exercise that will require learners to perform one of the actions described in the objectives you wrote earlier, and that assesses their ability to do so. Draft the exercise in your shared lesson design notes document. These should be assessments of the lower-level objectives defined for individual episodes in the lesson, as opposed to the lesson-level objectives you wrote first. Trainees working as a team can choose whether to work together on discussing and designing a single exercise to assess a single objective, or to divide their efforts and each focus on an exercise for their own episode. If you choose to take the latter approach and finish with time to spare, spend the remainder reviewing and providing feedback on one another's assessments. **Day 3 Sign In** * Ella Belfer (she/her) * Trisha Adamus (she/her) * Jean-Yves Sgro (he/him/his; il/lui/son) * Sarah Stevens(she/her/hers) * Dazhong Xia (he/him) * Maggi Kraft (she/her) * Michelle Busch (she / her) * Erwin Lares **Key Points**: - Assessments are a way to determine whether the objectives you defined for the lesson have been reached. - Exercises help learners commit what they've learned to long-term memory. - Some types of exercises are better for particular audiences and to address certain objectives. - **Formative assessment** happens *during teaching* and provides feedback both to an instructor and a learner - about progress and whether learning of new concepts occurred but also about any misunderstandings and misconceptions which can hinder further learning. --- --- ### How to Write a Lesson **Objectives**: After completing this episode, participants should be able to... - Estimate the time required to teach a lesson. - Summarise the content of a lesson as a set of questions and key points. - Connect the examples and exercises in a lesson with its learning objectives. **Questions**: - Why do we write explanatory content last? - How can I avoid demotivating learners? - How can I prioritise what to keep and what to cut when a lesson is too long? #### Lesson Time Management (10 minutes) (5 minutes) In the shared notes document, note down your answers to these questions: - From a design perspective, at what point is a lesson too long? - If the lesson focus covers too many non essentials or extraneous concepts. - when you reach the limits of your learners' attention - when it has achieved the learning objectives & continues going - If there are too many concepts and the learners are experiencing cognitive overload - clear lack of interest / frustration expressed from the learners - not enough breaks - What factors influence and constrain the length of a lesson? - installation of software usually takes unexpected time out of lesson time - lack of prior knowledge from the learners expected in the pre-requisites - Who your audience or learners are and how much time they might have to attend - How abstract/applicable the topic is. How many intermediate steps are needed before the topic can reach its culmination - How many exercises you include - How excited people are to ask about tangential questions, and how skilled you are at productively redirecting them. - How prepared you are to cut/adapt the lesson plan to respond to time pressure - the size of the group attending the lesson - how many different types of computers are there (macs, pcs, linux) - external technical problems - e.g., I don't have admin permissions to install something locally. - How might you prioritise what to keep if you have to cut lesson content down? - thinking about what is essential for the most basic piece of lesson objectives (i.e., core parts of `ggplot2` to make a graph) vs what is extra / would be more fun for learners (i.e., customizing plots: changing shapes, colors, adding lines, etc.) - material that could be handed out as a cheat/sheet or extra to read through if they are interested - what things are core skills that allow learners to self-study the other parts of the lesson? - is there a cluster of lesson objectives that is important, but relatively independent from the other lesson objectives? - Keep essential steps the learners will need to reuse their code (ie: saving code/script) - keep it if you see a straight relationship to the lesson/episode objectives - worth not only cutting exercises, as these can break up lectures in a helpful and engaging way. (5 minutes) In the remaining time, your Trainers will lead a discussion based on the responses. #### Exercise: Examples Before Exercises (20 minutes) Looking back at one of the exercises you designed before: what examples could you include in your narrative to teach learners the skills they will need to apply to complete the formative assessments you have designed? Outline one of these examples in your design notes document ##### Examples: **In the Software Carpentry Plotting and Programming with Python Lesson:** Exercise to load and inspect CSV file for Americas -> In the lesson, the Instructor demonstrates how to load the data table for another continent (Oceania) and explores the values with a few different functions. This shows learners how to call the function to load the CSV into a data frame, and demonstrates what success looks like for this task. **In the Python Interactive Data Visualization Lesson in the Incubator:** Exercise to find the correct widget (a slider) for an action and modify the script to use it -> In the lesson the instructor introduces a cheatsheet and documentation for interactive widgets and then creates a dropdown widget for the application. The slider widget required in the exercise has not been demonstrated but the preceding example shows all of the necessary steps to add a widget, and provides the supporting information that learners can consult to discover how to implement the specific tool. #### Length Considerations - What is essential to include? - What can be left out if needed? - Are there checkpoints where the lesson could end if needed? - Can important concepts be moved up earlier to ensure they are covered? [Template for notes on pilot workshops](https://codimd.carpentries.org/lesson-pilot-observation-notes-template#) #### Review Your Text for Demotivations - dismissive language - e.g. ‘simply’, ‘just’ - use of stereotypes - check learner profiles for stereotypes too - expert awareness gaps, i.e. places where you may be assuming the learners know more than they actually do - fluid representations, i.e. using different terms with the same meaning interchangeably - unexplained or unnecessary jargon/terminology - unexplained assumptions - sudden jumps in difficulty/complexity #### Review Your Text for Accessibliity - Avoiding regional/cultural references and idioms that will not translate across borders/cultures - Reference: https://www.bbc.com/worklife/article/20161028-native-english-speakers-are-the-worlds-worst-communicators - Avoiding contractions i.e. don’t, can’t, won’t etc. - Checking that all figures/images have well written alternative text, including writing altnerative text for data visualizations. - Checking the header hierarchy - no h1 headers in the lesson body, no skipped levels - Using descriptive link text - no “click here” or “this page”, etc. - Checking the text and foreground contrast for images [Writing alt text for data viz](https://medium.com/nightingale/writing-alt-text-for-data-visualization-2a218ef43f81) #### Exercise: Completing episode metadata (10 minutes) In your shared notes document, add key points and questions to your episode. To check the formatting requirements, see the Introduction Episode example in your lesson or [the Workbench Documentation](https://carpentries.github.io/sandpaper-docs/episodes.html#questions-objectives-keypoints) **Key Points**: - The objectives and assessments provide a good outline for an episode and then the text fills in the gaps to support anyone learning or teaching from the lesson. - It is important to review your lesson for demotivating language, cognitive load, and accessibility. - To reduce cognitive load and ensure there is enough time for for the materials, consider which lesson objectives are not needed and remove related content and assessments. --- ### The Carpentries Workbench **Objectives**: After completing this episode, participants should be able to... - Identify the key tools used in The Carpentries lesson infrastructure. - Complete the fundamental setup steps for a new lesson repository. - Edit Markdown files using the GitHub web interface. **Questions**: - How is a lesson site set up and published with GitHub? - What are the essential configuration steps for a new lesson? - How do you create and modify the pages in a lesson? #### Creating a lesson repository Templates for new lesson repositories: * [For lessons using R Markdown](https://github.com/carpentries/workbench-template-rmd/generate) * recommended for lessons that will contain example R code blocks/programs * [For lessons using Markdown](https://github.com/carpentries/workbench-template-md/generate) - Click *Include all branches* option - Give repository a name #### Improving the `README.md` (5 minutes) Take a few minutes to update it with some basic information about the project: - the lesson title - a short description of the lesson - a list of the names of the authors, optionally linked to their GitHub profile **Key Points**: - Lesson sites are built from source repositories with GitHub Pages. - A new lesson repository can be created from a template maintained by The Carpentries, and configured by adjusting the `config.yaml` file. - The main pages of a lesson website are created from Markdown or RMarkdown files in the `episodes` folder. --- [Minute Cards](https://forms.gle/jtU52ZAB1Ei1mb8VA) Welcome Back to Day 4!! ### Adding Lesson Content **Objectives**: After completing this episode, participants should be able to... - Add lesson episodes as individual pages of a lesson website. - Use _fenced divs_ to create different structural elements within a page to format objectives, questions, keypoints, exercises and their solutions in a lesson website. **Questions**: - How do you create and modify the pages in a lesson? - How should different structural elements be presented in a lesson website? #### Exercise: practice editing Markdown in GitHub Add the objectives you defined for your lesson as a bullet list in the `index.md` file of your lesson repository. [Sandpaper Docs - guide to rendering locally with RStudio](https://https://carpentries.github.io/sandpaper-docs/) #### Exercise: practice creating episodes (10 minutes) Repeat the steps you just saw, to create another new episode file and add it to the lesson site navigation. If you know what another episode in your lesson will be about, create the page for that. Otherwise, feel free to use any values you like for the file name and episode title. Also try out the fenced difs for keypoints, questions, and objectives. [List of callout types available in Workbench](https://carpentries.github.io/sandpaper-docs/instructor/component-guide.html) #### Exercise: Formatting Exercises in a Lesson Site (10 minutes) Using the approach demonstrated above, format the exercise you designed previously as an exercise in your lesson site. ### How we Operate **Objectives**: After completing this episode, participants should be able to... - Describe the role that feedback plays in the life cycle of a lesson. - Connect with other members of the community. **Questions**: - What are the important milestones in the development of a new lesson? - How can The Carpentries lesson development community help me complete my lesson? #### Lesson Life Cycle Revisited ![The life cycle of a lesson, annotated to indicate the platforms provided for lesson projects at each stage of the cycle.](https://carpentries.github.io/lesson-development-training/fig/life_cycle_annotated.png) - `pre-alpha`: first draft of lesson is being created - `alpha`: lesson is being taught by authors - `beta`: lesson is ready to be taught by other instructors - `stable`: lesson has been tested by other instructors and improved based on feedback. Major changes and updates are relatively infrequent. #### Pilot workshops - _alpha pilot_: a workshop taught by the lesson authors, often one of the first few times the lesson has been taught. - _beta pilot_: a workshop taught by instructors who have not had previous (major) involvement in developing the lesson. Questions that can be answered in a pilot workshop: - _How much time does it take to teach each section of the lesson?_ - _How much time is required for each exercise?_ - _What technical issues were encountered during the lesson?_ - _What questions did learners ask during the workshop?_ - _Which parts of the lesson were confusing for learners?_ - _Which exercises could be improved to provide more information to the instructors?_ [More guidance for organising/teaching pilot workshops](https://docs.carpentries.org/topic_folders/lesson_development/lesson_pilots.html) [Pilot workshop template notes doc](https://codimd.carpentries.org/lesson-pilot-observation-notes-template#) [Carpentries Slack Workspace](https://slack-invite.carpentries.org/) Relevant channels - lesson-dev - workbench [TopicBox list - incubator developers](https://carpentries.topicbox.com/groups/incubator-developers) [Guide to translation of Carpentries lessons](https://hackmd.io/@joelnitta/SkCSC6ZNT) #### Exercise: join relevant channels (5 minutes) Use this time to explore the options listed above and join/subscribe to any communication channels that you find interesting. **Key Points**: - Teaching a lesson for the first time is an essential intermediate step in the lesson development process. - The Carpentries lesson developer community shares their experience on multiple communication channels. --- ### Preparing to Teach **Objectives**: After completing this episode, participants should be able to... - Summarise lesson content as a teaching plan. - Add Setup Instructions and Instructor Notes to the lesson site. - Create a feedback collection plan. **Questions**: - What can I do to prepare to teach my lesson for the first time? - How should I communicate lesson setup instructions to learners? - What information should be recorded for instructors teaching a lesson? - How should information be collected as part of the feedback process? ### Teaching Plan - welcome, introductions and motivation for the lesson - to introduce yourself, aims and objectives of the lesson and how it will help learners - setup - to check if everyone is ready to proceed with the lesson - segments of teaching and exercises from the lesson and any other planned activities (e.g. group discussions, wrap-up and feedback) - checkpoints - places where you have planned to stop to check-in with learners - slides, figures and other visual aids needed to deliver teaching and other parts of the teaching plan - resources/references/recommended reading #### Exercise: Prepare a teaching plan (15 minutes) Create a bullet point list or brief notes describing what you will say and do when teaching the episodes you have been focussing on during this training. [Post workshop survey template](https://docs.google.com/forms/d/1OGCQBotD2nOJkc7KpFZLhFfb3EBcxEDwHz_3p48qz3U/template/preview) #### Exercise: Homework The final part of this training will focus on the skills needed to collaborate effectively. Before that there will be a break, during which we would like you to complete the following three tasks: 1. Teach one episode of your lesson (probably the one you have been working on in these two days). See the [Lesson Trial Runs](https://carpentries.github.io/lesson-development-training/trial-runs.html) page for full details. 2. After your trial run has concluded (immediately after, or when you have reviewed any feedback you collected from learners), note down your answers to the following questions: - What worked? - What did not? - What will you do differently next time? - What will you change in your material you taught? We will refer to these notes when we reconvene for the last episode of this training. 3. Based on your experience teaching the material and the feedback you received from your learners and helpers, make a list of issues you have identified with the material you prepared, e.g.: - examples that did not work as expected, - improvements that could be made to exercises, - parts that learners found particularly challenging, - unexpected questions or misconceptions that came up during the trial run. We will return to these notes during the final training session, so please make sure you save them somewhere you will be able to find them again easily when the time comes. #### What questions do you have? (15 minutes) The homework from this workshop includes a trial run of one of the episodes you have been developing in your lesson, to a real audience. After reading [the information provided about the trial run task](../learners/trial-runs.md) and thinking about this task, what questions do you have about how you should approach teaching that trial run? Is there anything you are unsure of? What resources might help you prepare for that experience? **Key Points**: - Spending time on preparing your teaching and feedback collection will make you and your participants get the most out of your workshop pilot. - Creating clear setup instructions as part of your lesson and circulating them ahead of the pilot is time well-invested and will give you more time when teaching the lesson. - Instructor Notes are teaching tips that you should include with your lesson to help you (a few months down the line) and other instructors, who have the relevant topic knowledge but have not been involved in the lesson design and development, deliver your lesson more successfully. --- #### Feedback (5 min) The Trainer(s) will ask for feedback on the training so far. Take some time to provide this feedback, before moving onto the second part of this task. ##### One Up, One Down - :raised_hands: Michelle - Appreciated the time we spent in GitHub, really useful to go through the screensharing of the commits (PRs mentioned) - :hammer_and_wrench: Maggi - Wish we had finished the lesson before the end of the course but that is a lot of time - :raised_hands: Sarah - I appreciated getting to try the process and see it from the learner perspective. Appreciate this group! - :hammer_and_wrench: Jean-Yves - Found that some sections that are not as technical are a bit long - evaluation, splitting episodes, some of the sections were not as interesting to me. More interested in the hands on - :raised_hands: Erwin - Loved that we spent time discussing a bit of pedagogy and could be different approaches that I'm thinking about - :hammer_and_wrench: Ella - General having the breakout times was super useful, the last few rounds of 10 min to be not quite enough time to get into it. Felt like we were more talking about things we needed to but not working on them - :raised_hands: Dazhong - Really helpful to have one project as a throughline throughout, working on the lesson and having each person work on an episode. Good way to make the whole giant process feel a lot more mangable and glues all the piece together. **Key Points**: - We have learned about the importance of exercises and data in a lesson, the ecosystem The Carpentries provides for lesson development, and how to prepare to teach a lesson for the first time. - Before the next part of the training, you should teach a part of your new lesson to a real audience and reflect on the experience. **------->** [Post workshop survey ](https://docs.google.com/forms/d/e/1FAIpQLSeHT27hkuq-D9jFlqVHSPVIr0w4f-hY8tIdHh-X7149WJQTSA/viewform?usp=sf_link) **<-------**