Scoring Panel for Writing Assignments
Product Design Lead
User Experience Researcher
1 project manager
1 product manager
1 researcher
1 designer (me)
Jun 2023 - Aug 2023 (Design)
Project Overview
From Paper to Pixels: Digitalize the scoring experience for 940 School Districts...
Literacy Design Collaborative is an Ed-tech non-profit providing authentic writing assignments for K-12 class in 940 school districts in US.
However currently the teachers need to download and print to all the material, and LDC sees an opportunity here to to make the experience digital.
As the design lead, I designed the whole experience from scratch, and delivered a clickable prototype and demo for LDC to apply for grants.
Project Goal
At the every beginning of the project, we clarified the goals with our clients.
Through analyzing and tracking rubric-based writing tasks’ scoring data, we will contribute to:
Providing a workflow design that assists teachers scoring with rigorous writing task rubrics
Provides teachers with informed data presentation through analysis of pathway milestones.
Deliver targeted instructional design recommendations through predictive trend analysis
Success Metrics
At the end of the project, we got pretty good results from the user testing:
Almost all tasks got completed without much difficulties
Save teachers' scoring time ~50% compared to scoring offline
Teachers are happy with the scoring experience
User Group
Design for public grade school teachers
The first question is, who are we designing for? The users of LDC are public grade school teachers whose teaching involve reading and writing across different disciplines and states.
User Research
Survey, interview, and contextual inquiries: Understanding how teachers scoring with rubrics on paper...
We initiated by thoroughly understanding the market and the needs of our users. This process involved conducting comprehensive primary and secondary research. By the end of the spring semester, we meticulously synthesized our research findings into valuable insights. Subsequently, we engaged in brainstorming sessions to generate and present initial ideas to our client for their consideration and selection.
Literature Review
Competitive Analysis
SWOT Analysis
Customized Case Study
Survey Responses
Interviews
Contextual Inquiries
Focus Group
Contextual Inquiry
Insight synthesis
Insight synthesis: Teachers' needs and pains in different stages
I synthesis the results in this map based on our extensive primary and secondary research, we gained a deep understanding of the workflow involved in scoring a writing assignment. You can see all insights in process book↗.
Ideation
Building on insights from our primary and secondary research, we generated six ideas that directly addressed the identified insights and workflow.
Prioritize project goal
Creative Solutions Within Boundaries: Descope to Scoring...
At the spring presentation, we introduced our ideas to LDC's executive director and Chief Program Officer. We also conducted a 'Rose, Bud, and Thorn' activity to evaluate and rank the ideas. As the design lead, I identified some problems given all the information:
🧾 Pain Point Priority
The scoring process is the primary challenge for teachers, making improvements here the most impactful.
🚫 Client Constraints
LDC's lack of access to student identifiers restricts the feasibility of monitoring, directing focus to scoring.
🕤 Time Efficiency
Within two months, concentrating on a high-fidelity prototype for scoring ensures timely and quality delivery.
Therefore, our client and we decided to rescope the project to:
Providing a workflow design that assists teachers scoring with rigorous writing task rubrics
Provides teachers with informed data presentation through analysis of pathway milestones.
Deliver targeted instructional design recommendations through predictive trend analysis
Then I conclude the HMW question as:
How might we create a streamlined online scoring process for teachers to scoring students' writing assignments using LDC's rubrics?
Co-design workshop with users
Interactive Co-Design Sessions: 5+ Workshops Directly with Teachers...
Given the refined focus on the scoring aspect, it's essential to revisit the users and gather their feedback on their specific needs. I led co-design sessions with the users, and I synthesis their pain points and actions they need to take in each step:
Design
Based on the established structure, I took the lead in guiding the team through the development of wireframes, facilitated design discussions within the team, and oversaw two rounds of user testing.
Low fidelity prototype
User Testing
Maze results
1 Before Scoring: be Prepared with Assignments and Classes
📁 1.1 Create and manage classes
To start with I created a new “my classes” panel in the navigation bar to help teachers create and manage their classes. The pages resembles the layout of existing pages
📝 1.2 Add assignments to classes
The teachers can use quick action to add the assignments to the classes they created
⏫ 1.3 Upload Students' Works
2 During Scoring: Leverage Rubric and Feedback
After the preparation comes the most exciting part: scoring, where I leveraged the power of rubrics and feedback. In this phase, teachers struggle with informative rubrics and time-consuming feedback.
There is so much information needed when scoring, such as students' work, feedback, and rubrics. I mimicked the layout of teachers’ desks, with students' work on the left, rubrics on the right, and feedback in the middle, so for our non-tech-savvy users, this familiar layout would reduce their stress of digitizing.
After the preparation comes the most exciting part: scoring, where I leveraged the power of rubrics and feedback. In this phase, teachers struggle with informative rubrics and time-consuming feedback.
Giving feedback is very time-consuming, and I provided a feedback bank where teachers can use their pre-designed feedback and their previous feedback. This is extremely helpful when they need to give the same feedback to multiple students, saving a lot of time.
3 After Scoring: Reports for different audience
After scoring, teachers need to export the scored work for different audience. I provided options for them so theycan choose what contents they want to include in the reports.
Iterate
I led the team to conduct two rounds of user testing and preference testing using Maze. Check user testing 1 and user testing 2 to see the user testing guide I created. Here are some findings:
🗂️ Track students without identifiers
Due to the limitations of our client LDC not having access to students' identifiers, teachers, when adding students to a class, are required to generate a unique code for each student rather than using their names or student ID.
👎 Before
👍 After
Through user testing, I did these changes related to adding students to a class:
- I introduced the option of creating multiple students simultaneously with default student codes in response to teachers' need to add students collectively.
- I reorganized the options based on teachers' rankings of their preferences for various student-adding scenarios.
- I added descriptions to provide clarity and assist teachers in understanding each available option.
🕒 Display the feedback
According to the research, giving feedback as the most time-consuming part of the scoring process, we introduced the Feedback Bank. This feature is designed to expedite the feedback process, enabling teachers to provide feedback more efficiently and effectively. We did a preference testing on the way to give inline feedback, and 100% users choose the option on the left.
👎 Comment box floating
👍 Comment box on the side
- 😤 The feedback content covers texts
- 😤 Teachers can not see previous feedback
- 🥰 Similar to users' habits of other word processor layout
- 🥰 Clear view of feedback content on the side of the student’s work
☑️ Associate the scoring dimension and feedback
Recognizing the central importance of the rubric as a cornerstone product for LDC, we sought to maximize its impact. Consequently, we conducted preference testing to determine the most effective way to link a scoring dimension before providing inline feedback.
👎 Tab
👎 Drop-down menu
👍 Multiple-choice question
😤 Tabs are too small to see; the information consist in each tab is incomplete which makes it hard to fully understand.
😤 It is inconvenient to go through two clicks to complete the matching action.
🥰 The format of multiple-choice question list is very straightforward.
Final Design
Create and Manage Classes
Teachers can create and manage classes under My Classes panel. They can add class name, year, discipline, and, of course, students.
Manage Students' Works
Teachers can upload students' works and track their scoring status in the assignment detail page.
Clickable Rubrics
Teachers can use the clickable rubric to help them scoring. It allows teachers to only focus on one scoring dimension at a time, and scoring criteria are in bullet points which reduce the cognitive load of teachers.
Feedback Bank
Teachers can use the feedback bank for suggestions for feedback. There are pre-designed feedback and previous feedback in the feedback bank.
Export Scores and Feedback
After scoring, teachers can export the scores and feedback according to their needs.
Corner Cases
User Experience is not just about happy path...
🗑️ Delete or Transfer students
What if a student drop out of the class or move to another class during the semester? According to our user research, those unhappy situations happen.
Testings
Teachers like the new design...
Almost all tasks got completed without much difficulties
Save teachers' scoring time 50% compared to scoring offline
Teachers are happy with the scoring experience interface
Next step
🗂️ Collaborate with the subject matter experts to design the feedback bank
If I have more time, I would collaborate with more subject matter experts to further develop and design the content for the feedback bank, which would better help teachers to save time for giving feedback.
📚 Design an Onboarding Tutorial
Given our users' limited technical background, primarily those accustomed to offline scoring, creating an onboarding tutorial will guide teachers through the digital scoring process, enhancing their understanding and comfort with our product.
Takeaways
📑 User Research: Choose the Right Research Methods for the Audience and Goal
In this project, I recognized the complexity of the teachers' scoring process. To gain authentic insights, I insisted our approach beyond typical surveys and interviews to include contextual inquiries, diving into the teachers' real-world scenarios.
Considering their limited tech background, I also introduced preference testing alongside usability testing. This simpler, choice-based method encouraged teachers to share their opinions more openly, significantly enriching our feedback collection and understanding of their needs.
⚖️ Design with Constrains: Find the Balance
The scoring process, by nature, is time-consuming and filled with numerous tedious steps. It was crucial to acknowledge these constraints rather than envisioning an ideal scenario. For instance, acknowledging that most student work is paper-based, I couldn't solely focus on digital solutions. My approach was to accommodate these realities, striving to simplify the process within these bounds, ensuring practicality and usability in our design solutions.