Research, Evaluation, & Assessment

Schoolyard Scientists/East Bay Academy for Young Scientists

Based at the Lawrence Hall of Science at UC Berkeley, the East Bay Academy for Young Scientists (EBAYS) serves youth from local East Bay communities where schools are under-resourced and have a large proportion of students of color. EBAYS engages youth in innovative, hands-on science research investigations that examine problems of soil, water, and air quality within their communities. It then develops new knowledge through innovative research that can be widely used in community struggles for environmental and social justice. Since 2013, for example, the Research Group has led various research and evaluation studies to examine the implementation and impact of the EBAYS program, which most recently was presented at the American Education Research Association 2019 Annual Meeting, announcing programs such as:

Clean Energy Literacy and Leadership- Engaging Youth in and After School (PDF, 143KB )
Fostering Environmental Activism Through Community Based Research Investigations (PDF, 123KB )

Currently, with funding from the National Science Foundation (NSF DRK12 #1720585), the Research Group is leading a four-year study, “Schoolyard Scientists in collaboration with EBAYS.” In the study, the Research Group works in close partnership with local students, teachers, school districts, and EBAYS itself to develop, implement, and examine participatory science-learning experiences for youth living in urban communities around the San Francisco Bay Area, particularly the East Bay. The study examines the impact of participatory science research (PSR) activities on youths’ interests in and identification with STEM, and further explores how such experiences support student learning.

BEETLES

Better Environmental Education Teaching, Learning, Expertise & Sharing (BEETLES) is a program affiliated with the Lawrence Hall of Science that offers professional learning techniques and develops curricula that support nature-based, student-centered science learning experiences in various outdoor science programs. The Research Group has served as a research and evaluation partner to BEETLES for several years. From 2013 to 2015, we conducted an evaluation study, funded by the S. D. Bechtel, Jr. Foundation, to examine the implementation of the BEETLES professional learning model and to better understand how various outdoor science programs have used the model to support professional learning among their instructional staff.

BEETLES: Final Evaluation Report (PDF, 3.4MB )

We currently are working on a two-phase research study, funded by the National Science Foundation (AISL 1612512), to examine, first, the ways in which BEETLES supports the design and facilitation of outdoor science learning experiences and, second, the impact of such ventures on students’ dispositions related to science as well as overall environmental literacy.

Activation Lab

The Learning Activation Lab is a national research and design effort to dramatically strengthen learning in the United States and beyond. Our goal is to first explore and then demonstrate how educators can activate children’s and young adults’ interest and curious minds in ways that can ignite their persistent engagement in learning and innovation. Our work spans multiple disciplines and considers the combinations of dispositions, skills, and knowledge that can best position individuals for success in learning within those disciplines.

The Research Group continues to build on the research of the Learning Activation Lab, examining the ways in which learning experiences can activate interest and curiosity among youth. Further, that lab has created a web-based toolkit, called ActApp to support organizations, researchers, and evaluators in using Activation as a way to measure programmatic outcomes and to understand the experience of their learners.

Reflecting on Practice

Reflecting on Practice (RoP) is a professional learning program designed to build participants’ understanding of best practices for teaching in informal learning environments. RoP is devised for institutions to implement it with their own staff and volunteers, at their own pace. The Research Group has served as an evaluation partner to RoP since 2011.

We are currently conducting an evaluation study, funded by the National Science Foundation (NSF AISL 1612515), to examine how best to disseminate RoP via two coaching workshop models: either two– to three-day coaching workshops that take place in person, or “blended” coaching workshops combined with a one-day in-person meeting that subsequently takes place online. The evaluation considers which factors are required for an institution to implement RoP.

FIRST

For Inspiration and Recognition of Science & Technology (FIRST®) is a national and international nonprofit organization that seeks to support learning and positive attitudes toward Science, Technology, Engineering, and Math (STEM) subjects, as well as to enhance 21st-century skills such as teamwork, cooperation, and critical thinking. To achieve these goals, FIRST offers four engineering and technology programs for youth, ranging from K through grade 12: FIRST® LEGO® League Jr., FIRST® LEGO® League, FIRST® Technology Challenge, and FIRST® Robotics Competition.

The Research Group has served as an evaluation partner to FIRST since 2012. Through this partnership, we have examined the implementation and outcomes of numerous FIRST programs and initiatives. Most recently, we completed a three-year study of the FIRST LEGO League Jr. Season Pass Model—an effort to broaden participation in its programs by reducing registration costs for larger, primarily school-based sites. The evaluation examined the implementation and outcomes of the Season Pass Model in institutions in the United States, Canada, and Mexico.

FIRST® LEGO® League Jr. Season Pass Model Evaluation - Final Report (PDF, 1.7MB)

Working Towards Equitable Organizations in Environmental Education

Working Towards Equitable Organizations Workshop Series is a collaboration between the Lawrence Hall of Science and an organization called Youth Outside, which aims to support California-based residential outdoor science organizations as they strive to build capacity to foster equitable, inclusive, and culturally relevant work environments and organizations for educators of color. The Research Group serves as the evaluation partner to examine the design, development, and implementation of the pilot professional learning model. Through this work we conducted a study to examine how outdoor science programs operationalize equity, inclusion, and diversity in the work environment, so as to identify which tools and strategies might better support organizational change. This work has lead to a practitioners brief and has been presented at various conferences, including the American Evaluation Association Annual Meeting, the Culturally Responsive Evaluation and Assessment 5th Annual Conference, and the North American Association for Environmental Education and the Children and Nature Network International Conference.

Examining Equitable and Inclusive Work Environments in Environmental Education (PDF, 3.9MB)

Positioning Youth for Success in Science: Studying the Malleability and Impact of Computational Thinking for Science

In collaboration with the Learning Design Group at the Lawrence Hall of Science, the Research Group is engaging in a three-year project (funded by NSF STEM+C #1838992) that will investigate computational thinking as both an input into and an outcome of science learning. After synthesizing a variety of frameworks and definitions of computational thinking (CT) to define the aspects of CT that best position youth specifically for learning science, we are testing whether this new construct, called computational thinking for science (CT-S), prepares youth from diverse backgrounds for achieving success with their science learning in technology-rich classrooms. We are specifically investigating whether CT-S is valuable above and beyond the previously identified dimensions of Science Learning Activation (that is: fascination, values, competency beliefs, and scientific sensemaking), each of which has been shown to enable success in science learning during the middle school years. We are also investigating the relationship between CT-S and the development of STEM career preferences. The study is situated within the Amplify Science Middle School curriculum, also developed by the Learning Design Group. The project includes measurement development, validity testing, and a one-year longitudinal study to explore how CT-S develops; how such development varies, based on the content domain in which it is emphasized; and which in-school and out-of-school experiences are most associated with increases in CT-S.