Designing Computer Science Exam Questions That AI Cannot Easily Answer

With the widespread availability of tools like ChatGPT, Computer Science exams should focus on reasoning, construction, justification, and originality rather than memorization. The following question types are effective for undergraduate CS courses.

1. Trace-Based and Step-by-Step Reasoning

Example (Automata Theory):

Given the PDA transitions shown,
simulate the PDA for input aabb.
Show the stack contents after each transition.
    
Why AI struggles:

2. Design + Justification Questions

Example (Parsing):

Design a grammar where left recursion is intentionally preserved.
Justify why eliminating left recursion would harm your parsing strategy.
    
Why AI struggles:

3. Error Detection in Code or Proofs

Example (Algorithms):

for(i=0; i<n; i++) {
    for(j=1; j<n; j=j*2) {
        sum++;
    }
}
    

(a) Identify the mistake in the claimed time complexity O(n log n).
(b) Correct it with proper justification.

Why AI struggles:

4. Questions my Lectures/Book

Using the argument discussed in Chapter 5,
explain why a DPDA cannot recognize a certain CFL.
Provide a different example of your own.
    
Why AI struggles:

5. Partial Construction Problems

A DFA is partially defined.
Add only two transitions so that it recognizes
strings where the number of 1s is congruent to 2 mod 3.
Explain why no additional transitions are needed.
    
Why AI struggles:

6. Comparing Two Incorrect Answers

Two students gave answers on NP-completeness.
(a) Which answer is closer to correctness?
(b) Identify one deep conceptual flaw in the other.
    
Why AI struggles:

7. Change-One-Assumption Questions

If context switching time becomes zero,
which scheduling algorithms lose their advantage, and why?
    
Why AI struggles:

8. Constrained Derivation Questions

Convert the given grammar to CNF
without eliminating epsilon productions.
Explain how correctness is preserved.
    

9. Short, Precise Conceptual Questions

Why does determinism in DPDA matter more for parsing
than for language generation?
(Answer in 4–5 lines.)
    

10. Reflective + Technical Questions

After studying Pushdown Automata,
name one concept that became clearer
and one that became more confusing, with reasons.
    

Recommended Exam Pattern