CPS LEARN Lab · Northeastern University
AI-Powered Rubric
Evaluation at Scale.
Evaluation at Scale.
Upload your rubric. Upload your CSV. RubricAI evaluates every participant's transcript — delivering evidence-based scores, direct quotes, and actionable feedback at institutional scale.
80%
Time saved
Any
Rubric format
Any
CSV layout
100%
Researcher ctrl
Powered by Claude AI
No data stored
Real-time streaming
RubricAI v2 — Class Overview
16 evaluated
See It In Action
Real output.
Real evidence.
Real evidence.
Every participant gets scored on every rubric indicator — with a rationale citing specific criteria, actionable improvement feedback, and verbatim transcript quotes automatically extracted by AI.
✓
Score + Rationale
AI explains exactly why the student earned each score using rubric language.
✓
Actionable Feedback
Specific, improvement-focused suggestions tied to the rubric criteria.
✓
Transcript Quotes
Direct evidence pulled verbatim from the student's transcript.
C2_I3 — Active Listening
The student demonstrates clear reflection on the client's perspective, acknowledging the concern about pricing directly before pivoting to problem-solving. However, the response could more explicitly validate the emotional dimension of the concern.
Consider using the client's exact words when reflecting back their concern — this signals deeper listening and builds trust more effectively.
"I hear you on the cost — let me walk you through what that covers and where we can flex."
How It Works
Four steps. Full evaluation.
From setup to scored results — no manual grading, no spreadsheets, no guesswork.
01
Configure Data Setup
Set your course, cohort, session labels, and number of sessions. Give the AI context about what you're evaluating.
02
Upload Your Rubric
Upload any markdown rubric. Clusters, indicators, and 4-level descriptors are auto-parsed. Select which indicators to evaluate.
03
Upload Participant CSV
Upload any CSV with transcript columns. Columns are auto-detected or mapped in one click. Supports CSV and Excel.
04
Get Scored Results
AI evaluates all participants in parallel. Every score includes rationale, actionable feedback, and direct transcript evidence.
Comparison
Manual grading vs RubricAI
Same result. Radically different time investment.
Without RubricAI
Manual
Est. 25–30 min per student
Progress42%
✓
Read the transcript✓
Match to rubric criteria○
Write score rationale In progress○
Write improvement feedback○
Find transcript evidence○
Repeat for 199 more studentsWith RubricAI
Automated
Est. ~10 minutes for 200 students
Progress100%
✓
Upload rubric✓
Upload CSV✓
AI scores every indicator✓
Rationale + feedback written✓
Transcript quotes extracted✓
Cohort analytics readyCapabilities
Everything you need for rigorous assessment.
Not a template engine. A full AI evaluation pipeline built for academic research.
Any Rubric Format
Upload markdown rubrics. Clusters, indicators, and 4-level descriptors are auto-parsed. Override session assignments with checkboxes.
Flexible Indicator Selection
Choose exactly which indicators to evaluate per run. Assign each indicator to one or more sessions independently.
Evidence-Based Scores
Every score includes a rationale citing rubric criteria, actionable improvement feedback, and verbatim transcript quotes.
Any CSV or Excel
Upload any CSV or Excel file. Columns are auto-detected or mapped manually. Handles quoted fields, multi-session layouts, and batch columns.
Subgroup Analytics
Filter cohort analytics and class overview by batch/class. KPIs, indicator charts, and completion rates update per subgroup instantly.
Professional Exports
Per-participant and full-cohort exports in CSV and PDF — academic, print-ready format with indicators, scores, and evidence.
Use Cases
Built for anyone evaluating at scale.
If you have transcripts and a rubric, RubricAI handles the rest.
Researchers
Run consistent, repeatable rubric-based evaluation across large cohorts. Export structured data for statistical analysis.
Course Instructors
Score simulation transcripts for every student. Get AI-generated cohort summaries and identify students who need support.
Teaching Assistants
Eliminate hours of manual transcript review. Focus on teaching, not grading — the AI handles scoring with full evidence.
FAQ
Common questions.
What file formats does RubricAI accept?
RubricAI accepts CSV and Excel (.xlsx, .xls) files for participant data, and Markdown (.md) or plain text files for rubrics. Column names are auto-detected — you can override them in the column mapping modal if needed.
How does it handle different CSV column names?
The system uses smart pattern matching to auto-detect columns like participant IDs, transcript fields, completion status, and batch groups. You can always review and correct the mapping before running an evaluation.
Can I choose which indicators to evaluate?
Yes — all indicators are selected by default. You can deselect any indicator in the Rubric Framework tab. You can also assign each indicator to a specific session using the checkbox-based session mapping table.
How are scores calculated?
Claude AI evaluates each transcript against the rubric criteria and assigns a score of 1–4 (Beginning, Developing, Applying, Mastery) per indicator. Communication and Critical Thinking aggregate scores are computed server-side from the individual indicator scores — the AI does not average them.
Can I filter results by class or batch?
Yes — if your CSV has a batch or group column, batch filter chips appear automatically on the Class Overview and Summary & Charts pages. Clicking a chip filters all KPIs, charts, and the participant table to that class only.
Is my data stored anywhere?
No data is stored persistently. All evaluation runs in memory for the duration of your session. The rubric file is temporarily saved on the backend server only to process your run. No participant data is retained after the session ends.
Your next evaluation is
one upload away.
one upload away.
Upload your rubric and CSV. Get evidence-based scores for every participant. No setup required beyond what you already have.
Powered by Claude AI · Built at CPS LEARN Lab · Northeastern University
⚡ Quick Start — 4 Steps
Follow these steps in order for a complete evaluation run.
1
Data Setup
Set course, cohort, simulation type. Gives the AI context — more detail = better scores.
2
Rubric Framework
Upload your .md rubric. Clusters, indicators, and level descriptors are auto-parsed. Select which indicators to score.
3
Upload & Run
Upload transcript CSV. Columns are auto-detected. Click "Edit Mapping" to verify, then hit Run Evaluation.
4
Review Results
Class Overview for all scores. Click any participant for full rationale + quotes. Export CSV or PDF.
📐
Rubric Format Guide
How to structure your rubric markdown file.
Required Structure
## Domain Name
### Cluster 1: Name
#### Indicator 1: Name
| Level 1 | descriptor |
| Level 2 | descriptor |
| Level 3 | descriptor |
| Level 4 | descriptor |
#### Indicator 2: Name
...
## Domain 2
...
### Cluster 1: Name
#### Indicator 1: Name
| Level 1 | descriptor |
| Level 2 | descriptor |
| Level 3 | descriptor |
| Level 4 | descriptor |
#### Indicator 2: Name
...
## Domain 2
...
✓
Auto-detected heading levels
Any consistent ## / ### / #### structure is supported. The parser finds indicators automatically.
✓
Any domains work
Define any domain structure — KPI cards and analytics adapt automatically to your rubric.
⚠
If indicators aren't detected
Use the Parser Level dropdown in Rubric Framework to manually set the heading level for indicators.
📊
CSV Format Guide
What columns your transcript CSV needs. Column names are auto-detected.
💡
Auto-detection: Column names are matched automatically. If yours differ, use Edit Mapping after uploading to assign them manually.
🎛️
Session–Indicator Mapping
Control which indicators run in which session.
What is session mapping?
Each indicator can run in one or more sessions. E.g. Communication → User Interview only; Critical Thinking → both sessions.
Default: all indicators → all sessions
Every indicator is evaluated in every session that has a transcript, unless you change the mapping.
To change: Rubric Framework → Session Mapping table
Uncheck sessions for any indicator to exclude it from that session's evaluation.
💡
Tips for Best Results
Get the most accurate scores.
✓
Fill in Data Setup
More context = more accurate and relevant AI scores.
✓
Use specific rubric descriptors
The AI scores strictly against your level descriptors — make them distinct and precise.
✓
Always verify column mapping
Click "Edit Mapping" before every run to confirm transcript columns are correctly assigned.
✓
Use the AI Assistant
Ask about who needs support, patterns across classes, or session comparisons.
🔧
Troubleshooting
Common issues and how to fix them.
My rubric uploaded but no indicators were found
The parser couldn't detect your heading levels. Go to Rubric Framework → find the "Parser Level" dropdown → manually select the heading level for your indicators (e.g. #### for Level 4). Try each option until your indicators appear.
My CSV uploaded but transcript columns weren't detected
Click "Edit Mapping" on the Upload & Evaluate page. A modal shows all CSV columns — assign the correct one to each session transcript. Click "Confirm Mapping" to save.
Some participants show N/A scores
N/A means the transcript was too short (under 50 characters) or empty for that participant. Check your CSV — make sure the transcript columns are mapped correctly and contain full transcript text.
Client conversation transcripts are not being evaluated
Check the column mapping — the client transcript column may not have been auto-detected. Click "Edit Mapping" and assign the correct column to "Session 2 Transcript". Also verify those participants actually have client transcripts in the CSV.
The evaluation is slow or timing out
RubricAI evaluates up to 10 participants in parallel. A 16-participant run with 29 indicators typically takes 3–5 minutes. Make sure your backend is running and your Anthropic API key has sufficient credits.
A domain is not showing in KPI cards
Domain KPI cards are generated from your rubric's ## headings. Make sure each domain uses ## (e.g. ## Professional Agency) and ### for clusters within it. Re-upload the rubric after fixing the structure.
How do I set up my API key?
Create a .env file in the backend folder with: ANTHROPIC_API_KEY=your-key-here. Get an API key at console.anthropic.com. A 16-participant run with 29 indicators costs approximately $1.
🤖
AI Transparency
How scoring works under the hood.
Built at CPS LEARN Lab
Developed by Tanvi Kadam at the CPS LEARN Lab, Northeastern University. Designed for internal research use — not a commercial product.
FastAPI
Vanilla JS
Claude Haiku 4.5
ReportLab PDF
Render
Vercel
Version 2.0 · © 2026 CPS LEARN Lab
Data Setup
Configure evaluation context — fed directly to AI to improve scoring accuracy.
Evaluation Context
Basic course and cohort information for labelling reports.
Simulation Context
Help the AI understand the scenario participants were evaluated in.
Researcher Expectations
Tell the AI what to pay attention to — this directly improves evaluation quality.
Rubric Framework
Upload your rubric. Use checkboxes to select indicators — all selected by default.
Upload Rubric File
.md · .txt · .pdf · .docx — any format
No rubric uploaded
Upload a rubric file to see the full framework and select indicators.
Upload a rubric to get started
Any markdown rubric will be parsed automatically.
Upload & Evaluate
Upload your transcript CSV and run AI evaluation.
Rubric
Not uploaded — go to Rubric Framework
Transcript File
Not uploaded
Indicators Selected
Upload rubric first
Upload Transcript File
CSV · Excel (.xlsx / .xls) — columns auto-detected
Scoring Guide optional
Scenario-specific level anchors (.md or .txt) — passed to AI alongside the rubric
Evaluating with AI...
Uploading...
No results yet
Upload a transcript CSV and run an evaluation to see results here.
No data to summarize
Run an evaluation first.
Who needs most support?
Cohort average?
Lowest indicator?
Cohort summary
Struggling participants?
Compare sessions
AI
Hello. I can help you analyse evaluation results or answer questions about the rubric. Run an evaluation first, then ask me anything about performance, scores, patterns, or feedback.