Winning hackathon project is now helping millions of people worldwide to read and write

Permission to use this Microsoft Feature Article by Deborah Back, published on June 14, 2018 

For years, Merlyne Graves spent several hours every weekend reading text aloud and recording it for her students who struggled with reading.

Graves, a fourth-grade English language arts teacher at Garfield Preparatory Academy in Washington, D.C., would line up 10 unused iPods she borrowed from the school, then read and record the text her class would work on the following week. Her improvised method helped introduce struggling readers to the text, but it was time-consuming and limited.

That changed last December when Graves got a Surface Pro and tapped into Microsoft’s online educator community. She discovered Microsoft Learning Tools, a set of free features created to help improve reading and writing, especially for people with learning disorders such as dyslexia and dysgraphia. Immersive Reader, the tools’ main feature, allows users to have content read aloud to them, change text size and background color, break words into syllables, increase space between letters, highlight one or more lines of text and highlight parts of speech.

Graves immediately knew she’d found exactly what her students needed. “I thought, ‘This is going to be so awesome for my kids who struggle,’” she said.

After just a few weeks of using Immersive Reader in her classroom, Graves noticed an improvement in her students’ reading. Struggling readers showed better fluency — reading quickly and accurately with proper expression — and stronger readers were able to focus on inferencing and higher-order thinking, she said.

“I’ve noticed a change in a lot of their reading,” she said. “It is definitely making a difference.”

That impact is being felt in classrooms around the U.S. and the world. More than 13 million people worldwide are using Learning Tools, up from 100,000 just a year ago. The tools are being used across applications and platforms in more than 40 languages. Their remarkable growth, from an employee hackathon project less than three years ago to a solution now used by millions of teachers and students worldwide, has surprised even their creators.

“What came out of it was, frankly, more than what we initially anticipated,” said Jeff Petty, Microsoft’s Windows accessibility program lead, who was part of the hackathon team. “We had high hopes, but I don’t think we thought this was going to be as big as it is.”

 

‘We had IP’

One in five children in the U.S. have learning and attention issues such as dyslexia and attention deficit disorder, according to the National Center for Learning Disabilities, and up to 15 percent of Americans are thought to have dyslexia. Petty, who joined Microsoft in 2014, was looking for ways to improve accessibility in Windows and saw an untapped opportunity for reading assistance tools that could be built into Microsoft technologies. He reached out to Microsoft’s Advanced Reading Technologies team and consulted with the group’s researchers about what features would most help struggling readers.

“I just got fixated on the fact that people don’t talk about cognitive impairment a lot. They don’t talk about the fact that it’s hard for people to read and write,” Petty said. “And we had IP. We had brilliant concepts and technology that we weren’t leveraging that we could use to do more and to completely differentiate ourselves.”

Petty soon met Mike Tholfsen, principal product manager for Microsoft Education, and put together a team of more than a dozen people to work on a dyslexia-focused project for Microsoft’s 2015 company-wide hackathon. The team included engineers from Microsoft’s office in Vancouver, B.C., who had developed a prototype for a dyslexic mode in OneNote. The combined team created an extension for OneNote that offered text-formatting tools to make reading, writing, and note-taking easier. The project won the hackathon, edging out more than 3,300 others.

At the time, Tholfsen was the product manager for OneNote Class Notebook, an app that allows teachers to collaborate with students and set up personalized workspaces in a combined digital notebook. He’d been talking with teachers and school leaders around the country and using their feedback to grow the app.

“I’m a huge fan of talking with teachers, meeting with teachers, bringing engineers to conferences to build empathy,” Tholfsen said.

Tholfsen took the same approach to building out the OneNote extension, using feedback from educators to improve the tools’ features. Originally, Immersive Reader was designed only with black and white backgrounds, but after teachers told Tholfsen that some students with Irlen syndrome — a disorder that impacts the brain’s ability to process visual information — preferred reading with backgrounds in specific colors, the design team incorporated the colors into Immersive Reader.

“That’s been one of our most popular features so far,” Tholfsen said.

 

‘How fast can you go?’

After Immersive Reader launched in January 2016, Tholfsen pushed to incorporate the functionality into as many apps, platforms and languages as possible, including OneNote Class Notebooks. His passion for Immersive Reader was furthered by the potential he saw for combining it with OneNote Class Notebooks into a sort of supercharged digital content hub.

“When you have a OneNote Class Notebook and you put content in there, and you distribute it to students and then they can use Immersive Reader in a Class Notebook, for me that’s like utopia,” he said. “And that’s the way a lot of teachers are experiencing it.”

Mike Tholfsen, right, consulted with teachers and school leaders and used their feedback to improve Immersive Reader.

Tholfsen’s efforts got support from Eran Megiddo, the Microsoft corporate vice president who oversees education products. The day of the hackathon win, Tholfsen recalled, Megiddo organized an impromptu celebration with cookies and drinks on Microsoft’s campus and in the midst of the festivities, approached him to talk about next steps.

“He came up and said, ‘What do you need, how fast can you go and how can I help?’” Tholfsen said. “Those are the best things you can hear from a VP.

“He helped support the project and he made space for us to finish it. There are a lot of hackathon projects that win, but sometimes the products don’t ship because there are too many other competing deliverables that the organization needs done first.”

 

‘Built in, not bolted on’

In the months prior to the tools’ launch, Tholfsen began inviting educators to Skype calls where he would demo Learning Tools and get their feedback. Lauren Pittman, a special education teacher at Holly Springs Elementary School in Canton, Georgia, didn’t know what to expect when she joined one of those calls. She’d tried numerous other education technologies but couldn’t find any with a reading assistance component. As Tholfsen talked about the hackathon project, then shared his screen and went through the features Learning Tools offered, Pittman started crying.

“I was blown away because there aren’t a lot of people out there who create education tech that has a lot of accessibility features,” she said. “A lot of the technology is for the general education community, and then you kind of make it work for yourself. And this was completely specialized for a specific group of students who struggle with reading.”

Unlike other reading programs that are typically costly and can require schools to purchase individual licenses for each student, Immersive Reader is free and available across multiple platforms and applications, from Edge to Outlook and Word for Desktop, and Word and OneNote for Mac.

“It’s built in, not bolted on,” Petty said. “If you’re a school, this is available for every student. There’s no additional IT administration, and teachers don’t have to learn something special.”

Pittman now uses Immersive Reader daily in her classroom. The tools are a time-saver for her and give students more independence and control to personalize their learning experience. Children who previously sat and waited for her to read the text to them, or were separated into a group with other struggling readers, can now put on their headphones, access Immersive Reader and work at their own pace without their classmates knowing what they’re doing. They are able to participate in class in a way they previously couldn’t, Pittman said.

“A lot of these students have never been able to work with other students who work at grade level because they couldn’t access the reading like the other students can,” she said. “They’re usually the student that you see kind of sitting back behind everyone else and watching because they’re not able to work with the other students. Now they’re able to have that rich experience of working with others that they hadn’t been able to before.”

“When we see the impact these tools can have, it’s our responsibility and our duty to get them to as many people as possible.”

And since the tools are incorporated across platforms and apps, Tholfsen said, students, don’t feel stigmatized by using them.

“If a student has to use special software that the teacher can’t quite configure, it can be stigmatizing and students can feel like they have less independence,” he said. “The kids love Immersive Reader because they feel like, ‘I’m just using Office now,’ or ‘I’m just using Windows.’ It’s just personalizing the software in a way that works for them.

“And there’s a cycle that happens. They use the technology and they gain some independence, and as they gain independence, they gain a little confidence,” Tholfsen said. “That little wheel starts rolling, and all of a sudden they’re learning to read.”

With Immersive Reader, Pittman said, reading comprehension among her students “has skyrocketed.” She mentions one boy who started the school year reading just 11 words correctly per minute. The student had trouble focusing on a line of text and dreaded having to read aloud to Pittman. After using Immersive Reader, the boy can now read 61 words a minute with few errors, Pittman said, and eagerly looks forward to library day each Thursday.

“Before, he would roll his eyes when it was library day, and now he’s like, ‘Don’t forget it’s Thursday and we’re going to the library. I finished my books and want to check out more,’” she said. “That’s the biggest transformation you see in these students — watching them fall in love with literature, with text.”

Immersive Reader has “completely transformed” her classroom, Pittman said. “If you were to see my classroom today versus what it was three and a half years ago, it is a completely different experience. I’ve been able to be more of a facilitator of my students’ learning rather than being the controller of their learning.”

The tools are being used in unexpected ways, evincing the inclusive design principle that when products are created for inclusivity, they benefit people universally. Megiddo used Immersive Reader’s line focus view and read-aloud function to help him review his daughter’s college entrance essays and ensure they flowed smoothly, and his younger daughter regularly uses the tool to gauge how her writing sounds when read aloud. A 76-year-old friend of Pittman’s grandmother, who never learned to read as a child, uses Immersive Reader to have books read aloud to her and has gained enough skill to read to her great-granddaughter. Law firms are even using the tools to help navigate dense legal documents.

“You have no idea how much this has impacted people’s lives — and not just students, but anybody who has struggled with a reading disability,” Pittman said.

Karrick Johnson, an 8-year old who lives in Renton, Washington, has dyslexia and struggled to learn to read. At school, he avoided reading in class and felt embarrassed and sad about his reading difficulties. Then a friend showed him how to use Immersive Reader. With the click of a key, he was able to focus on a line of text at a time.

 

Immersive Reader was designed to help improve reading and writing, especially for people with learning disorders such as dyslexia and dysgraphia.

“You have no idea how much this has impacted people’s lives — and not just students, but anybody who has struggled with a reading disability.”

His parents soon noticed improvement — Karrick began labeling his drawings with the names of objects and reading signs around town. And in May, he made his mom a Mother’s Day card, thanking her for making him “yummy food” and for helping him learn to read. Newly confident in his abilities, Karrick now has an ambitious goal.

“I am going to read as many books as I can in the world,” he said.

There is also evidence that Immersive Reader can benefit a broad cross-section of readers, not just people with learning disabilities. A 2017 study of students across grade levels in Bellevue, Washington, found that reading comprehension increased 10 percent points on a nationally normed assessment among those who used Immersive Reader, and that the tools helped level the playing field for struggling readers. In recent visits to schools in New York City, Microsoft Education representatives noted that high-achieving students were using Immersive Reader to help them focus.

There are plans to continue expanding Learning Tools to other Microsoft products, and efforts are underway to add more features based on feedback from teachers, students and reading experts. As Megiddo sees it, making Learning Tools and Immersive Reader available as broadly as possible is core to Microsoft’s mission of enabling every person and planet on the organization to achieve more.

“When we see the impact these tools can have,” he said, “it’s our responsibility and our duty to get them to as many people as possible.”