Nehal Jain

Designing responsible AI oversight in education

01 · Context

The tool was designed to help monitor student engagement and performance and surface potential safety risks related to AI.

02 · The challenge

In education, AI-generated assessments and risk signals can influence how teachers understand students. But an AI rubric cannot fully capture genuine engagement or learning. Students may also adapt their behavior to match what the system rewards, making its signals less reliable.

03 · My role

I shaped product strategy and examined the governance implications of AI-enabled monitoring in education.

04 · Product considerations

• Transparency: Make the tool’s role and use of student data understandable.
• Accountability: Clarify who reviews and acts on potential risk flags.
• Human oversight: Keep teachers involved in interpreting AI-generated signals.
• Behavioral effects: Consider how the system’s rubric could influence the behavior it measures.

05 · Unresolved risk

Students could optimize for the rubric to improve grades or engagement scores, without a corresponding improvement in learning. This feedback loop remained an open product and safety challenge.

06 · Outcome

I brought transparency, data use, accountability, and human oversight into product strategy for the monitoring tool.

07 · Takeaway

AI-based grading is here to stay, but its rubric cannot fully represent a student’s genuine engagement or learning. These systems should support teachers by surfacing patterns and automating parts of their work, not make judgments on their own. Teachers need to interpret the signals and remain responsible for decisions that affect students.

Governing human likeness in GenAI

The challenge

Likenesses can appear in generated content or training data without creators having clear visibility or control. This is an AI safety, ethics, risk, and governance problem.

The product

A protected, encrypted digital representation helps monitor for likeness use in GenAI content and traceable training data. Potential uses are flagged for review against consent and licensing terms.

My role

I research with actors and studios, translate requirements across the UK, EU, US, and India into the first version of the Regulation and Consent Engine, and define KPIs for detection accuracy, consent verification, and enterprise adoption.

Constraint

Detection and training data traceability have limits. Flags need human review and cannot be treated as definitive proof.

Takeaway

Automated governance can make consent actionable, monitor use, and flag potential violations at scale, giving creators more visibility and control.

Rebuilding a Governance Platform

01 · The Problem

Govenda helps boards make decisions and manage their work. Committee governance was difficult to manage consistently because each organization had different structures, charters, roles, and responsibilities.

02 · Governance challenge

Board professionals needed to turn each committee’s rules into clear responsibilities, membership, meetings, and access to information. Govenda had to support those differences for both administrators and members.

03 · Committee Suite

I designed committee hierarchies, role and tenure tracking, and workflows for meetings, documents, tasks, and discussion.

04 · Product management

I turned 200+ client insights into product opportunities, wrote requirements, defined 12 KPIs, and partnered with product and R&D teams on 4 launches.

05 · Outcome

The work became Govenda Committee Suite, a standalone product. The case materials report clearer collaboration, less time managing committees after setup, and a 50% increase in customer satisfaction at market entry.

06 · Takeaway

Governance products need a consistent foundation that can accommodate different rules and responsibilities.

Company milestone: On May 23, 2024, Passageways, parent company of OnBoard, acquired Govenda in a strategic move to expand boardroom AI capabilities across a platform serving nearly 6,000 boards worldwide.