Idea: skill-based refresher flash-quiz game
- Topics and Question Database:
- Generate on demand using LLMs based on user-selected topics
- Ability to ground/reference a document collection
- Question validation should be tracked, allowing verification and diputes by user, agents, and experts
- Questions and results should support rich types:
- Text
- Markdown
- Mathjax
- Mermaid
- Code snippets with syntax highlighting
- Types of questions:
- Simple (fast-touch interface questions)
- multiple choice
- multiple true/false
- Multi-multi match questions (e.g. “Link the following terms to their definitions”)
- Free-answer (must include rubric for auto-grading)
- Coding (free answer + solution code + test cases)
- Adaptive difficulty:
- Questions should adapt in difficulty based on user performance
- The quizzer should generate a topic-specific score that is used for further testing, feedback, and dashboarding
- Topic/difficulty may be visible, may be selectable, depending on system config
- Answers should be selected from a pool of plausible distractors generated by the LLM for repeatability
- Answers should list why they are correct/incorrect after each question
- Adaptive difficulty engine:
- Questions are tagged with various topic tags tied to difficulty ratings
- Adaptive testing involves building a robust model of the user’s skill levels, refining that model by querying high-uncertianty questions
- Every e.g. 5 questions, the system should re-evaluate the user’s skill model and find the next best question to ask
- Modelling method may vary