Collaborative Lesson Development Training

Collaborative Lesson Development Training Dates: 13 - 16 May, 2024 Time: 17:00 - 21:00 UTC (https://www.timeanddate.com/worldclock/fixedtime.html?msg=Collaborative+Lesson+Development+Training%3A+Part+1&iso=20240513T17&p1=%3A&ah=4 for your local time) Zoom link: https://carpentries.zoom.us/j/82947379898?pwd=rASGd6xUsna66clSjksAFfYaeoW1Da.1 Code of Conduct: https://docs.carpentries.org/topic_folders/policies/code-of-conduct.html Curriculum: https://carpentries.github.io/lesson-development-training/ Trainers: Erin Becker Toby Hodges Sarah Stevens Schedule (All timings listed in UTC) This schedule is tentative, and exact timings may vary during the event. However, regular breaks will be taken regardless of how the schedule for the specific sections may change. Day 1 Time (UTC) Session Title 17:00 - 17:45 Introduction 17:45 - 17:55 Break 17:55 - 18:20 Lesson Design 18:20 - 19:05 Identifying Your Target Audience 19:05 - 19:20 Break 19:20 - 20:10 Defining Lesson Objectives/Outcomes (1) 20:10 - 20:20 Break 20:20 - 21:00 Defining Lesson Objectives/Outcomes (2) 21:00 End Day 2 Time (UTC) Session Title 17:00 - 17:30 Episodes 17:30 - 17:45 Break 17:45 - 18:40 The Carpentries Workbench (1) 18:40 - 18:55 Break 18:35 - 19:00 The Carpentries Workbench (2) 19:00 - 19:15 Break 19:15 - 19:50 Defining Episode Objectives 21:00 End Day 3 Time (UTC) Session Title 17:00 - 17:55 Designing Assessments (1) 17:55 - 18:10 Break 18:10 - 18:45 Designing Assessments (2) 18:45 - 19:00 Break 19:00 - 19:35 Implementing Exercises 19:35 - 19:50 Example Data and Narrative 21:00 End Day 4 Time (UTC) Session Title 17:00 - 17:50 How To Write A Lesson (1) 17:50 - 18:00 Break 18:00 - 18:20 How To Write A Lesson (2) 18:20 - 18:45 How We Operate 18:45 - 19:00 Break 19:00 - 20:05 Preparing to Teach 20:05 - 20:20 Break 20:20 - 21:00 Wrap-up 21:00 End About This CodiMD We will use this CodiMD to take notes, share links, exercises, etc with participants throughout the training. Participants are encouraged to take shared notes on this page. The Trainers will show you how to use CodiMD at the beginning of the workshop. Attending Name (pronouns (optional)) / affiliation / email address Toby Hodges (he/him/his) / The Carpentries / tobyhodges@carpentries.org Erin Becker (she/her/hers) / The Carpentries / ebecker@carpentries.org Sarah Stevens (she/her/hers) / University of Wisconsin-Madison / sarah.stevens@wisc.edu Saira Haider (she/her/hers) / US Geological Survey / shaider@usgs.gov April Clyburne-Sherin (she/her/hers) / UPenn / aprilcs@seas.upenn.edu Kaija Gahm (she/her) / UCLA / kgahm@g.ucla.edu Hao Ye (he/him) / UPenn - Community for Rigor / haoye@seas.upenn.edu Madison Langseth (she/her) / U.S. Geological Survey / mlangseth@usgs.gov Kevin Rusch (he/him) / UPenn (C4R) & UMass Chan / kwrusch@seas.upenn.edu Ani Govjian (she/her) / UPenn (C4R) / agovjian@seas.upenn.edu Peter Laurin (he/him/his) / UCLA / peterlaurin@ucla.edu Karen Word (she/her) / UPenn - Community for RIgor / wordk@seas.upenn.edu Xochitl “so-chill” Ortiz Ross (she/they) / UCLA / xortizross@g.ucla.edu Eric Martinez (he/him/his) / U.S. Geological Survey / emartinez@usgs.gov Notes - Day 4 Notes from previous days Notes from yesterday’s material have been move to an archive document Feedback from Day 3 (paraphrased and summarized) Thank you everyone again for your detailed feedback! Things you learned that you found useful Markdown sandpaper syntax and learning the workbench platform Introducing a topic and then going into breakout rooms for disucssion is helpful Different kinds of exercises for assessments The discussion of image/dataset licensing x2 Time to create exercises and then putting them in our lesson webpages - liked this pattern Improvement feedback Felt quick today Confusing when the two people introduce an exercise slightly differently, maybe separate person to introduce exercise isn’t needed More time working on our lessons, discussions today not as helpful Still struggling with CodiMD x2 Would have liked to discuss multiple question types in individual groups Picking up where we left off Exercise: Choosing a Dataset or Narrative (14 minutes for today, maybe more tomorrow) Referring to the advice given above, find an appropriate dataset or a narrative for your lesson. Identify one or more potential candidates and note down the advantages and disadvantages of each one. As a reminder, here are some aspects we suggest that you consider: For datasets: size complexity “messiness”/noise relevance to target audience availability license ethics For narratives: authenticity relevance to target audience complexity possibility to teach useful things first/early Takes notes in your Lesson Design Notes document about your discussion and the decisions made. It may be particularly helpful to record: Which datasets and narratives did you consider? How and why did you choose between them? What implications do you think your choice of dataset and/or narrative will have for the design and further implementation of your lesson? C4R descriptive data: Palmer Penguins want some other dataset with panel, longitudinal, treatments – allows for interesting decisions on grouping and choices for plots want to introduce some errors / outliers https://mavenanalytics.io/data-playground options: https://portal.conp.ca/dataset?id=projects/Longitudinal_structural_MRI_and_behavioural_data_for_mice_prenatally_exposed_to_maternal_immune_activation_either_early_or_late_in_gestation https://github.com/conpdatasets/MB-Heterogenous-task-representative-data How to Write a Lesson Objectives: After completing this episode, participants should be able to… Estimate the time required to teach a lesson. Summarise the content of a lesson as a set of questions and key points. Connect the examples and exercises in a lesson with its learning objectives. Questions: Why do we write explanatory content last? How can I avoid demotivating learners? How can I prioritise what to keep and what to cut when a lesson is too long? Lesson Time Management (10 minutes) (5 minutes) In the shared notes document, note down your answers to these questions: From a design perspective, at what point is a lesson too long? What factors influence and constrain the length of a lesson? How might you prioritise what to keep if you have to cut lesson content down? (5 minutes) In the remaining time, your Trainers will lead a discussion based on the responses. April: Too long: When it is difficult for a student to summarize the lesson easily. When there are too many objectives. When the students have been doing the “same thing” for a long time. Constraints: Cognitive load - the lesson should end before the students “need” a break. Prioritise: Feel it out with students - you can see them start to fidget or glaze over. Run it for a colleague - see if they can help you identify the key things and drop things you are attached to but aren’t necessary. Ani: Too Long:In terms of visual design, when the text gets unweidly and looks more like a stream of text without a break. If there are breaks, variations, pauses for activities, the overall length (visually) is not a problem. As far as demands on student attention, 15-20 min is a useful starting place for most new content with short breaks. Stringing a few of these 15-20min chunks together is great, but probably not exceeding 2 hours for most intensive content. Length constraints: Schedules, attention span, difficulty of topic. Prioritization: Most often, I think it’s best to offer foundations for learners to build on, but you can also start with some intro and then invest heavily in “the most important” item. Eric: Too long: Running over on allocated time Feedback during alpha/beta testing Constraints: Relevance to lesson learning objectives Available time Cognitive burnout Priorities: Focusing on what is both necessary and sufficient to achieve desired learning objectives Hao: rule of thumb: a lesson is too long if it cannot be summarized in a single sentence (maybe 2) criteria for too long: physical length (scrolling maybe) time duration (more than 50 minutes) concept amount (more than 1 focused idea) factors that influence length: desire to include related content attempting to speak to different audiences attempting to defend against potential criticism or fill gaps instead of focusing on the learning objectives technical or modality constraints of the lesson platform or delivery prioritize does this support instruction of the learning objective may need further feedback on appropriateness or splitting of LO across lessons does this make me the author / instructor feel smart (but neutral or negative impacts on LO, or could be replaced with something else that better supports LO) Kaija: Lesson too long: A lesson is too long if learners stop being able to follow it, if it’s too much cognitive load, etc. This could happen if there are too many individual topics, or if there’s too much detail about any one topic. Factors influencing: The time in which to teach it/teaching environment. Learners’ attention spans, willingness to attend. Prioritize: Think back to the overarching lesson-level learning objectives and make sure they’re being met. Try out the lesson and ask learners which portions were most or least helpful. Both formal and informal version control–keeping a “question bank” or “exercise bank” so we’re not too scared to remove exercises (can always go back and retrieve them later!) Karen: Length of a lesson can be constrained by time to teach in a classroom, word count for written text, etc. No one section should take more than an hour to get through in a classroom, or probably about 20 minutes if working independently? I aim for minimal text, but could use a rule here. Factors that influence appropriate length are difficulty/complexity and value/relevance. Difficult things in which value is not immediately apparent have to be a LOT shorter. Retaining attention is hard. Prioritization should be very centered around learning objectives. If those are well constructed it should be clear what’s optional. If too much has to be included in order to reach those objectives, the objectives need to be re-scoped to match the audience better. Kevin: Too Long: User struggles to remember all of the lesson’s contents. Users finds themselves looking at the time. Factors: Cognitive load. Active versus passive content. Pace of the content. How to prioritize: Which aspects of the material are least intuitive? Which aspects of the material are more technical? Which aspects of the material are required for later content? Madison: If you think about having some sort of break every 1 hour, each episode should be able to be taught within an hour. Carpentries lessons are meant to be short term committments from learners. If you can’t fit all of the content into about 4-8 hours, it is probably too long. Especially, because you often need to teach multiple lessons within a workshop. We tend to run workshops that are 12-16 hours in total (spread out over multiple days, sometimes one, 4-hour session per week for 3-4 weeks). Factors influencing the length of a lesson: how much content can someone reasonably fit into their brain in the timeframe alloted? how much time to teach do you have Prioritizing: Is the concept really essential to meeting the lesson objectives Peter: A lesson might be too long if too many learners feel like it is not worth continuing, they are learning too little for how much time they invest in learning the skill. The main strategy to constrain a lesson is setting very strict and reasonable goals before sitting down to write the lesson. Make sure it says everything you think needs to be said and no more. I would proritize skills I consider absolutely essential, or those that are needed to learn content in future lessons. Otherwise, consider getting rid. People also have a great ability to build on a good foundation and learn new content by themselves. Saira: When might it be too long: We are aiming for a 16 hour lesson, so anything over that would be too long; in our practice teaching sessions, if we find we having to cut out exercises or breaks to get through the lesson, then it’s too long Factors that constrain: The background knowledge of the audience will greatly influence how long the lesson takes; the number of exercises and their complexity; any material that is cumulative can take longer, because you don’t have room for someone to ‘sort of’ get something (for example: setting up a git profile) - in this case everyone has to 100% be successful How to prioritize: Go back to your learning objectives Xochitl: Too long when: there are too many lesson objectives, too many exercises, the page length is really long. Could come up with an average time per episode and see how long the lesson would end up being. Length constraints: Complexity of the material or number of exercises would contrain how much can get covered. Cognitive load. Priotization: Choose which LOs are essential, which are useful, and which are a bonus Which objectives build off of previous ones vs. which stand alone survey what the learners are interested in/what they already know about what will be most useful AND is easy to master vs dropping what takes a long time to master and may be only sometimes useful Length Considerations What is essential to include? What can be left out if needed? Are there checkpoints where the lesson could end if needed? Can important concepts be moved up earlier to ensure they are covered? Template for notes on pilot workshops <-- we will discuss pilot workshops more at the end of today 5 Ways to Handle Extraneous Overload - Renkl 2014 Eliminate extraneous material Insert signals emphasizing the essential material Eliminate redundant printed text Place printed text next to corresponding parts of graphics Eliminate the need to hold essential material in working memory for long periods of time Review Your Text for Demotivations dismissive language - e.g. ‘simply’, ‘just’ use of stereotypes - check learner profiles for stereotypes too expert awareness gaps, i.e. places where you may be assuming the learners know more than they actually do fluid representations, i.e. using different terms with the same meaning interchangeably unexplained or unnecessary jargon/terminology unexplained assumptions sudden jumps in difficulty/complexity Review Your Text for Accessibliity Avoiding regional/cultural references and idioms that will not translate across borders/cultures Avoiding contractions i.e. don’t, can’t, won’t etc. Checking that all figures/images have well written alternative text, including writing altnerative text for data visualizations. Checking the header hierarchy - no h1 headers in the lesson body, no skipped levels Using descriptive link text - no “click here” or “this page”, etc. Checking the text and foreground contrast for images Optional Exercise: Alternative Text for Images (5 minutes) Which of the following is a good alt-text option for the image below? {alt=“Line graph of increasing carbon dioxide at the Mauna Loa Observatory from 1958 to present”} Graph of data Graph with increasing lines Line graph of increasing carbon dioxide in ppm at the Mauna Loa Observatory from 1958 to present Line graph of increasing carbon dioxide in ppm at the Mauna Loa Observatory, Hawaii, United States, from 1959 to present including values from each year. Red line shows variation in each year and black line is average for each year. 1959 = 315.90 ppm, 1960 = 316.91, 1961 = 317.64 … Data/Image provided by NOAA Global Monitoring Laboratory, Boulder, Colorado, USA Exercise: Completing episode metadata (10 minutes) Add key points and questions to your episode. To check the formatting requirements, see the Introduction Episode example in your lesson or the Workbench Documentation Keypoints should be learner-accessible (understandable by learners without understanding the jargon in the LOs). Can be written as answers to the episode questions. https://github.com/carpentries-lab/reviews/blob/main/docs/reviewer_guide.md#reviewer-checklist Key Points: The objectives and assessments provide a good outline for an episode and then the text fills in the gaps to support anyone learning or teaching from the lesson. It is important to review your lesson for demotivating language, cognitive load, and accessibility. To reduce cognitive load and ensure there is enough time for for the materials, consider which lesson objectives are not needed and remove related content and assessments. How we Operate Objectives: After completing this episode, participants should be able to… Describe the role that feedback plays in the life cycle of a lesson. Connect with other members of the community. Questions: What are the important milestones in the development of a new lesson? How can The Carpentries lesson development community help me complete my lesson? Lesson Life Cycle pre-alpha: first draft of lesson is being created alpha: lesson is being taught by authors beta: lesson is ready to be taught by other instructors stable: lesson has been tested by other instructors and improved based on feedback. Major changes and updates are relatively infrequent. Pilot workshops alpha pilot: a workshop taught by the lesson authors, often one of the first few times the lesson has been taught. beta pilot: a workshop taught by instructors who have not had previous (major) involvement in developing the lesson. Carpentries Logo Usage Questions that can be answered in a pilot workshop: How much time does it take to teach each section of the lesson? How much time is required for each exercise? What technical issues were encountered during the lesson? What questions did learners ask during the workshop? Which parts of the lesson were confusing for learners? Which exercises could be improved to provide more information to the instructors? More guidance for organising/teaching pilot workshops Exercise: join relevant channels (5 minutes) Use this time to explore the options listed above and join/subscribe to any communication channels that you find interesting. Join The Carpentries Slack TopicBox (email lists) Direct link to the Incubator Developers list Places to find learners (do not need to exactly match your target audience): Carpentries channels Local learners Members of your lab group Key Points: Teaching a lesson for the first time is an essential intermediate step in the lesson development process. The Carpentries lesson developer community shares their experience on multiple communication channels.