[soft piano music]
Heather Britt: Hi, everybody welcome back to Talking Through the Numbers. This is Wilder Research's podcast that we do periodically. Super excited to have you join us today. My name is Heather Britt, I'm the executive director for Wilder Research. And I am joined by two lovely humans and colleagues who are gonna talk with us today about storybanking.
So we have Jessica and Lucy with us. And I'm going to ask them to introduce themselves. So I'm going to start with Jessica first.
Jessica Tokunaga: Hi, Heather my name is Jessica Tokenaga, and I'm a research associate a Wilder Research. My background is in public health, but I work on a variety of projects here at Wilder, ranging from behavioral health to early childhood development which is related to the project we're gonna talk about today. My main role that I had on the PDG Indigenous evaluation was to lead the collection of stories from families through the method storybanking. So I'm excited to talk more about that with you today.
Heather Britt: Great thanks, Jessica. Lucy.
Lucy Arias: Boozhoo Gabaanimad indizhinikaaz. My name is Lucy Arias. My Ojibwe name is Everlasting Wind and for this project I was the PDG Tribal nations lead so I worked with Wilder and the Indigenous grantees for the Preschool Development Grant.
Heather Britt: Great. Thanks, Lucy, thanks, Jessica. So let's just start talking about kind of storybanking at the very basic level. So what is it? How are you using it in evaluation? How does it sort of compare to or how is it different from sort of other kinds of evaluation methods that might be used? Jessica, can you give us a little bit of insight?
Jessica Tokunaga: Yeah. So storybanking. The name of it kind of says it all in a way. So storybanking is basically a method focused on gathering stories and then banking them or storing them and cataloging them in a safe space. When looking into this method, I found that it's used in a wide variety of fields, but primarily used in a lot of social work or early childhood development, family services areas, and the purpose of it is to gather stories from people who are most impacted by different policies, programs, or services. And because it's based on hearing from people directly about their experiences, it really has resonated with a lot of parents and families. So that's why most of the ways in which I found storybanking used were in that sort of social work, family services area. And so we were really drawn to this method partially because of Lucy here, and other states that were also implementing different PDG, which is Preschool Development Grants, throughout the country. And we were inspired by the work of Kansas Our Tomorrows. They were implementing their own PDG evaluation and using storybanking as one of the ways to find out how families are doing. So we adapted part of their approach in their tool to storybanking to fit with our own families here in Minnesota, particularly from Indigenous communities here.
I can go a little bit more into some of the other key points or differences about storybanking. One of the really cool parts about this method is that it is very open-ended. And it's based on really what rises to the top for whatever stories people want to share.
So a lot of times, the questions that you ask in a storybanking process look different than traditional focus group or interview questions. And a lot of those ask directly about a program. You know, what was it like being a part of this program? What would you change about it? Whereas in storybanking, you want to ask more broadly about people's lives and their experiences. And then it's up to the storyteller to decide what is really important for them to share and what really feels like they need to get off their chest, or they want people to know about them or their lives or their families. So that's one of the main differences you could find between storybanking and other different types of qualitative methods and evaluation.
Heather Britt: Thanks, Jessica. Lucy, as as the state’s thinking about using this and as you're gathering information, what what's been your reflection on kind of the benefits of engaging in this in this type of work, in this type of of collection.
Lucy Arias: Well, to start off with, you know, your podcast is called data, right? There's data in there. And unfortunately, when a lot of the data is shared with the American Indian community in particular. It doesn't paint a good picture. It's, you know, there's a lot of disparities in our community. And unfortunately, that's the narrative that's told. That's the narrative that is shared. And so me, knowing my community and working in my community, that's not the only story to tell.
And so when thinking about this approach, and knowing that as a professional and as a community member, really finding more of the the qualitative sort of evaluative methods to really tell a more complete story.
And for us in the American Indian community, stories are powerful. There are how we know our history, and our culture, and our values. It's all done through stories. And so I felt this would be a beneficial method to use for the American Indian community. But the state is also understanding the value of more qualitative approaches, and have broadened the storybank to a present day story collective, which is statewide, all community storybank. But they've titled theirs the story collective. And they're collecting stories. So I think, even in general, we've seen the data for many, many, many years, right? And we're not seeing a lot of movement, and we need to learn more, so we can better serve communities. And storybanking is one avenue of doing that.
Heather Britt: Great. I love to hear kind of the the work that you're doing in this part, and then that this is starting to sort of seep through the rest of the state, across other agencies, and you're thinking about it more collectively.
What are some of the things that you heard through this storybanking through the work around PDG? What are some of the key messages, key themes. What are some of those excerpts?
Jessica Tokunaga: Yeah. Well, building off what Lucy was saying about, we wanted the questions to be different. It bring a different focus, uplift different types of stories from American Indian families. So, for instance, some of the questions that we asked were:
Think about something your child did recently that filled you with joy. What happened? And why did this bring you joy?
Think of a time when you observe something special about your child's development or learning. What did you notice? And why do you think it was important? And also some other questions about how families feel supported by their community, different ways that they engage in Indigenous cultural practices as a family together, and what that meant for them. So the questions are really aimed at thinking about how people are in community, and that home and with each other rather than just thinking programs or services or, like we said, deficit. So we wanted to ask questions that would solicit other types of stories. And to hear, you know, just in general, like, how are children and families doing together?
And what we ended up getting was really beautiful and powerful stories. And you know, sometimes people thought the questions were so broad, and they would say, I think we got a question one time, you know, how is this really evaluation? You're not even asking about the program, you know, directly. But what ended up happening is people shared impact stories, so actual, tangible examples of child development outcomes. And they attributed a lot of it to programs and services and cultural events that were funded by PDG and supported by the state. So we ended up getting a lot of really rich evaluative data along with beautiful stories about families. I'm happy to share a couple right now with some main learnings.
So one of the central themes that we've heard where parents and caregivers expressed a lot of pride and joy when their children are engaging and leading in Indigenous practices. And one parent shared with us: “Being at the Saint Paul, public school powwow with my children opened my child's mind, body, and soul. She was so happy and just excited to be here encouraging me to be the parent I want to be in this society felt amazing. Knowing how I parent runs in my cultural teachings and being told I'm doing it good is the greatest feel feeling. Watching my child use her words and think with their heart is a great thing to see.”
So that was just, you know, one of 65 stories we were able to get where parents just really talked a lot about the joy that they get from doing things with their children. So again, we have a lot of other stories like that. I could share more if we have time. But yeah, that's just a little short story that we heard from people.
Heather Britt: Great. It's such a sort of holistic story, and I love that the stories themselves are leading you also to information about their experience with the program. Without that feeling so, sort of, front and center, allowing that journey to unfold as folks are telling their story.
Lucy, can you talk a little bit more about how the state is sort of taking these data in, how it's shaping how you think about PDG, the Preschool Development Grant, what it's leading you all sort of toward.
Lucy Arias: Well, you know, we did present at AEA this this year, and the first question from from the the participants who attended was: is this evaluation or valuation? And I remember when I first heard it and I was like, what is he talking about? Quite honestly, right. But then I it just sat with me for a couple of moments, and I was like, it's both.
So you know, thinking especially about the story. You know, Jessica shared that this work wasn't necessarily picking away at the projects of what worked, what didn't work of what was funded right? PDG was funding community solutions. PDG was funding community resource hubs. And so a lot of the how the programs chose to deliver their programs was self-identified, based on their understanding of the community and who they were and how they wanted to best serve the community. And so this evaluation just really found value in that approach so funding, you know, agencies who serve families directly.
It's evaluated right? But it's also valuation, right? And so I just wanted to come back to, though it really affirmed, I think, to a lot of programs that they're they're on the right path and serving families. They are serving families past providing them information of the different daycares that are in their neighborhood, or just all the different things that how programs support families. This really affirmed to me that they're on the right path of serving families. And so we found we proved value. I guess you could say.
Heather Britt: Yeah, yeah, I love that sort of both/and as opposed to kind of the it's not one or the other one. It's okay for it to be both of those things right? So really reinforcing for programs where they're headed. Giving you that sort of really rich information about the experience of families and young people as you're making your way.
So talk a little bit about pragmatically implementing this. You know what that feels like, and adjustments you maybe had to make along the way. So what does it feel like in practice when you are out in community collecting and sort of storybanking. What’s that process like? And and where did you have to adjust as you were as you were, sort of unfolding this.
Lucy Arias: Well, I'll start off. I'll say I was really excited. I had seen work by others in the country of their storybanks, and I was so excited. And so I brought it to Wilder. But a really important piece of this is, the grantees chose this. There was a menu of different evaluative approaches and methods that could be utilized and the the Indigenous evaluation for the Preschool Development Grant and it, and they chose storybanking. And I hope they shared my enthusiasm. But I don't think they understood exactly. They got a brief, brief what this would entail, and so I just thought, Oh, my gosh! We'll send out links, and we'll get all kinds of stories. But I learned that it it needs a more personal touch. And I did support Wilder at some of the events. I set goals for myself, of how many families I could engage in in the work, and, you know, really encourage them to share, or, you know, request that they share a story with us. And so yeah, I love the method. I was gung ho, but to realize that the community didn't necessarily know exactly what it entailed and potentially the power. So yeah, that was my angst. And I think I felt it was Wilder, but I think in the end, we found ways to to connect with families and create a space for them to share their stories.
Heather Britt: Thanks, Lucy, so I love that you had enthusiasm for it, that there was also selection by your grantees and sort of engagement, and which one of those sort of techniques and approaches you might use. And then the question of how you pragmatically and practically are doing it. You know, in community and sort of the the lessons along the way. Jessica, talk a little bit more about just like practically making this happen. And some of the things you and the team had to adjust or learn as you were, you know, as you were out and about.
Jessica Tokunaga: Yeah, like Lucy mentioned, you know we started off making it a bit more asynchronous. So more online. We created an online portal here at Wilder for families to engage with and submit stories, and did a lot of outreach with the grantees who were really interested in supporting the method. And I met with them over the course of months. But it was hard to get people to use the website. Which makes a lot of sense. People are busy and websites can sometimes feel a little hard to engage with and put a story into.
So we pivoted about midway through our data collection timeframe to doing more of pop-ups with storybanking. So I had a couple of large crates and wagons full of story forms and coloring books. So kids could have an activity while their parents shared with us and little gifts and tokens of appreciation along with gift cards for people who shared stories. And you know, I just want to say a quick thank you to some of the programs that hosted us: Baby Space, Montessori American Indian Child Care Center, and Interfaith Action. They helped us set up our pop-ups at different program activities. And then we also attended different community events around the metro area, some powwows, some vaccine clinics that were held at schools, and I think a 5K run at one point, we did that. So we went to where families were, and that made a huge difference and these turned into a bit more like intercept interviews. You know, we were able to talk with them about storybanking.
Why, we wanted to collect stories and the power behind them. And we're in settings that were more comfortable for them, and already joyful settings. So it was pretty easy for parents to come up with stories, and a lot of them told me, “Oh, if I had all the time, you know, the next several hours, I could share with you so many stories.” And we would transcribe them as they shared with us, or they would take a paper form and sit and have a cup of coffee and write it on their own and give it back to us. So the pop-ups made a really big difference doing this. And we then had to pivot with some staff time and some resources, and it began with just me, Jessica, doing it, and we built out a team of four other researchers to help do the pop-ups. So over time we learned how to adapt and really make the method work more for parents and communities. So I think the number we're able to collect in just a few months after doing that pivot really showed that we, I think we made the right decisions to do the pop-ups instead.
Heather Britt: Alright. So tried one kind of mode, moved to a different kind of mode. Sounds like it really did offer an opportunity to kind of magnify some of the joy that families were we're showing up with as they came into those spaces. I know when you all presented at the American Evaluation Association meeting, you got you got sort of a question about how do you think about storybanking in the context of other parts of an evaluation. So I'd love to hear from you both about how you're thinking about that in the overall PDG evaluation. So how does storybanking fit into that, and then how might you encourage listeners who are thinking about storybanking as a possible technique to consider it up against other possible approaches that they might think about?
Heather Britt: So maybe, Lucy, I'll start with you about how. How were you thinking about storybanking in that broader context of the PDG evaluation?
Lucy Arias: Well, one thing that came up, I don't know if it was during the storybanking… I can't exactly remember, was but one thing about the Indigenous evaluation, it was strength based. That was one of the principles guiding the work was strength based. So we wanted to learn about strengths enjoys right? And when there were others considering this option, retelling the story can be traumatizing right? It wasn't something you were necessarily thinking about that day. It wasn't something you were thinking of even sharing. And so really being conscious of that, of what is the intent? What is the space? Just like we learned it was best to go to spaces where families were at where they were enjoying themselves, and you know they could sit down and have that moment to write the story right? So the context of where and why you're collecting stories, I think, is important. And that goes along with the prompts as well to be very, very conscious of that, I think, is important. And it was nothing we necessarily had to wrestle with because we were coming at it from a strength strength base, but there is power in stories for learning about other issues in the community as well. But just being very conscious and be being very mindful of the individual because there are so much power and stories.
I had another thought, but maybe Jessica can add on, and then I'll come back. But really just being conscious and being mindful. I will say one of the other evaluators on the program, she said, “Nobody owes us their stories, right?” And so, just knowing that, just valuing the stories that we collected, right? And really honoring them even in that moment, was important.
Jessica Tokunaga: Yeah, in terms of how this fit in with the overall PDG Indigenous evaluation is this was one part to a variety of mixed methods we did, including some quantitative looks, engaging the community around programs they've used, and what was valuable to them. I think we looked at some website data program utilization data. So a lot of more your standard evaluation data we collected, and we also did some key informant interviews, and a couple of other really neat, innovative ideas came out of this as well. And so piecing them all together.
Where the stories for me fit in was sort of explaining why something was important to families if you could have them do a survey? And they say, “Yeah, I went to this event,” or, “Yes, I am. My children go to Baby Space.” What does that really mean, how does that impact how you feel as a parent? You know, what are different things that are coming out of your experience? Raising your children. And so that's sort of where storybanking came in is to provide that other level of detail or understanding about how families are doing in relation to these programs, even though we didn't ask directly about them. And so, you know, I completely agree with what Lucy is saying.
You know, being really mindful about the types of questions you want to ask because stories can be powerful and can bring up a lot of different emotions and memories for people, particularly if you're thinking about families. And so just being very mindful of what you're asking, how you're asking them. And making sure that you know…we had a really good relationship with the programs that we did pop-ups with. They felt comfortable with the questions. And they thought, “Yeah, come and ask this of our families. They will be so excited to share these great stories with you.” And I think if we had maybe different types of questions, the conversation and the type of partnership to build with the program staff, would be maybe a little different. So just the the question is really important, and what you ask and how you ask it is what I would say to people who are considering this method.
Heather Britt: Okay, so sounds like, the context within which you're doing this is really important. There's some considerations about it as a companion to some other methods that you're thinking about the relationships that you're going to be leaning into matter a lot. And then that those prompts, those questions, the things that you're inviting folks to reflect on, taking some care there sounds like it's really, really important.
Lucy, what's next for the state, you know, kind of as you're as you're looking forward into the future. And and as you're thinking about this as one of the companion methods that you'll that you'll likely be be using in other endeavors? How are how are you thinking about this kind of across the Department of Education, or across other agencies? Talk a little bit more about that.
Lucy Arias: Well, one of the the principles of Indigenous evaluation is the evaluation should benefit the community. So this can't be this lovely evaluation report that hits the shelf. Let's be honest. A lot of people read things. It informs them they use it more immediately. But this one has to have a life. There's a burden on my shoulders and my colleague's shoulders to continue to tell the stories, continue to use what we learned through the Indigenous evaluation to really create better systems and a better community for American Indian families, and, more importantly, the children. So how are we serving the children, and how we can use what we learn from storybanking and the other methods? How can we use that information to really strengthen the programs and support better.
So that burdens on my shoulders. Anybody who is involved. I’ll let Wilder off the hook. They did all the work. But really, I knew that entering into this into this work, and I'm taking it on how I can continue to to use what was learned to really benefit the community.
Heather Britt: Great. Thank you. It's great to hear it's going to continue to live, like stories continue to live. That that that will shape the way you all behave, the way you interact, intersect with community, the way the agency sort of behaves internally. That's great to hear.
Jessica, resources for folks who are thinking about storybanking or considering it? Where would you point folks if they are interested in learning more?
Jessica Tokunaga: Well, feel free to reach out to me here at Wilder, or Lucy in the future, if you have more questions about it, or what it was like to implement. Not sure if it's made public, we do have resources that we developed for our presentation at the Evaluation [American Evaluation Association] Annual Conference from this year.
Kansas Our Tomorrows has some webinars and some trainings online on their website. If anyone is interested in getting those materials and also contact the MN StoryCollective as they are building out their storybanking as well, and just a general Google search. There's a lot of different places in which storybanking has been used. And so there's some really great examples and some nonprofit how-tos, things to consider in terms of setting up a storybank, or if it makes sense for you to do in your work. So there's a lot of that out there.
And I also just wanna put in a plug for the Indigenous Evaluation Guidebook that was developed by our colleagues here at Wilder and at Bowman Performance Consulting. It has a lot of examples and principles, and different types of methods and approaches that can be used through Indigenous evaluation and believe we can find that online at Wilder’s website. But that is a great resource and tool in helping people think through different types of methods, including storybanking, for a lot of cultural communities that value more qualitative ways of sharing their experiences.
Heather Britt: Great thanks, Jessica. Lucy, any other resources that you point folks toward who may be asking, here or another agency, where do you send folks?
Lucy Arias: I would uplift our work just like Jessica did so well. Wilder did really good work on this project, and the 101 Guidebook to Indigenous Evaluation. But yeah, this, I think qualitative research is seen as a more, I don't know the right term to say it, it's getting more credibility, I guess you could say, in the research and evaluation world. So really explore it as to be in company with regular sort of evaluative approaches. It brings a richness, I would say.
Heather Britt: Great. Thank you. Thank you. Well, I just wanna say, thanks to both of you, Jessica and Lucy, for spending a little bit of time with us today talking about storybanking, sharing some resources, sharing your experiences. For folks who are listening to the podcast you'll see those resources linked. You can always check out wilderresearch.org for more information.
I appreciate you both being here today. I'm Heather Britt, and this is Talking Through the Numbers. Thank you. Have a great one.
[soft piano music]