Artificial Intelligence Teaching Lab

Session 2: Rethinking Assessment Design with GenAI

The LUMS Learning Institute (LLI) continued its AI Teaching Lab (AITL) series with an interactive second session on GenAI in Assessment Design, led by Dr. Farah Nadeem. Building on the conversations initiated in Session 1, where faculty reflected on the relationship between AI, learning, and student thinking, the second session shifted focus toward a practical challenge: designing assessments that remain meaningful in an AI-enabled learning environment. Bringing together faculty members from across disciplines, the session created space for practical reflection on one important question: 

As AI makes it easier to produce polished academic work, how can we design assessments that still reveal what students are truly learning? 

Rather than focusing only on whether students should or should not use AI, the session encouraged faculty to think more carefully about assessment purpose, student thinking, and the kinds of evidence that show meaningful learning.

The session focused on the AI Assessment Scale (AIAS), which helps instructors define the level of AI used expected in an assessment. Faculty explored how different levels of AI use require different kinds of assessment design, rubric language, and student guidance. 

This is closely connected with the Guiding AI at LUMS policy document, which encourages faculty to design process-based assessments that make student thinking visible. The discussion highlighted that AI should support learning where appropriate, but should not replace the intellectual effort, judgment, and reasoning that students are expected to develop.

When AI Can Answer, What Should Assessment Ask?

One of the strongest themes of the session was the need to move beyond final answers.

Faculty reflected on how AI tools can generate confident responses to many common assessment questions, especially when prompts are broad, generic, or focused only on explanation. This raised an important design challenge: if AI can answer a question convincingly, what does the assessment reveal about the studen

Through the first activity, participants worked with their own course learning outcomes and existing assessment questions. They drafted, refined, and stress-tested assessment prompts by asking whether AI could answer them at a higher level than intended. Where this happened, the prompt revealed a vulnerability in the assessment design. 

Faculty then explored ways to strengthen assessment questions by adding elements such as: 

• Personal or local context 
• Course-specific concepts or readings 
• Evidence of student decision-making 
• Process logs or reflection 
• Discipline-specific reasoning 
• Explanation of how conclusions were reached 

The goal was not simply to make assessments “AI-proof.” It was to make them more clearly aligned with the kind of learning faculty want students to demonstrate.

Designing Rubrics for the AI Era

The second part of the session focused on evaluation. Faculty explored how rubrics need to shift depending on the selected AIAS level. 

A Level 1 assessment may require evidence of independent student thinking and authentic personal context. A higher-level assessment may ask students to use AI as part of the task, but then evaluate the quality of their critique, judgment, revision, or reflection. 

Participants worked on building evaluation packages that included rubrics, model answers, and calibration checks. They also considered how an “AI-typical” response might score against a rubric. If such a response received a high score, it signaled that the rubric may be rewarding surface-level correctness rather than deeper learning. 

This helped faculty think about rubrics not only as marking tools, but also as communication tools. A strong rubric tells students what kind of thinking matters.

A Space for Practical Redesign

The AI Teaching Lab is designed as a faculty-centered space for honest conversation, experimentation, and shared learning. 

In this session, faculty moved from broad concerns about AI toward concrete changes they could make in their own courses. The conversation emphasized that assessment design now needs to make learning processes more visible: how students interpret a task, use evidence, make decisions, revise their thinking, and exercise judgment. 

As the series continues, one message from the second session remains clear: 

The goal is not just to assess work in the age of AI; it is to design assessments that protect and strengthen student learning.

Continue the Conversation

Missed our first session? Read Session 1 Highlights.

Interested in joining the next AI Teaching Lab session? Register here.