Content Moderation

Content moderation screens the messages people type to Pria (chat and meetings) before they reach the AI model. When a message is flagged as unsafe, Pria returns a polite refusal instead of forwarding the prompt to the model — protecting your learners, your brand, and the underlying providers from policy violations. This page is the Admin-side reference for the Moderation feature. End users do not configure anything; they only see the refusal copy when their message trips the filter.

​

What Moderation Does

When enabled on a Digital Twin, every user message is sent to a dedicated moderation model before any other processing. The moderation model returns a classification across several unsafe categories:

• Hate or harassment
• Violence or graphic content
• Self-harm
• Sexual content
• Illicit or illegal acts

If any category exceeds the provider’s threshold, the message is rejected with a safe response and never reaches the conversation model, the assistant’s prompt, your tools, or RAG. Each blocked chat message is emailed to the Digital Twin’s Contact Emails (“Unsafe Content with User Request”: the person’s address, the blocked text and the flagged categories), so you know who tripped the filter and why.

​

Enabling Moderation

Moderation is enabled per Digital Twin:

1. Go to Admin → Instances and edit the Digital Twin.
2. On the Configuration and Integrations tab, find the Enable Moderation toggle under AI Behavior (on for new Digital Twins).
3. Save.

Keep the toggle on — and double-check it after creating a new Digital Twin. Disabling it sends every prompt straight to the conversation model; only do this for narrowly scoped internal Digital Twins where users are fully trusted.

​

Selecting a Moderation Model

The Moderation field (Personalization and AI Models tab, System section) selects which model performs the safety check. Different providers offer different moderation surfaces: When you leave the field empty, Pria uses the platform default.

​

What Happens When Content Is Flagged

When the moderation model returns “unsafe”:

1. The user sees a refusal in the chat — “Unsafe content, please try different language. Unsafe content:” followed by the flagged categories (the classic interface adds that the attempt may be audited). The original prompt is not forwarded to the model, so no credits are spent on the rejected turn.
2. The blocked turn never becomes part of the conversation, so it does not appear in Histories.

​

Customizing Thresholds

The moderation endpoints expose a single per-category yes/no decision; there is no admin-tunable threshold in the Pria UI. If a category of legitimate language for your domain (clinical, legal, harm-reduction, historical) keeps tripping the filter, try the alternative moderation model — different classifiers draw the line in different places — or contact the Praxis AI team at humans@praxis-ai.com.

​

Moderation vs. Praxis Shield

Moderation and Praxis Shield are complementary safety layers:

• Moderation (this page) blocks individual unsafe messages in the moment, before they reach the AI model.
• Praxis Shield monitors conversations after the fact and raises security incidents — with severity, status, and category — in a dedicated triage panel where admins review them, add notes, take action, and mark false positives.

If you are looking for the dashboard where flagged activity is reviewed and triaged, that’s Praxis Shield. This page covers the per-message blocking configuration only.

​

Acting on Violations

A flagged prompt is a signal — not always a punishment. The alert email tells you who sent it and what was blocked. From there:

• Open the user’s conversations in the Histories view to see the context around the blocked attempt. The blocked message itself will not be there; the turns either side of it will.
• Determine whether the user was testing the filter, asking a legitimate question that was misclassified, or genuinely attempting misuse.
• Contact the user via the email associated with their account if intervention is needed.
• Adjust your Digital Twin’s onboarding or assistant prompt if a category of legitimate question keeps tripping the filter.

For a Digital Twin that serves an organization, consider pairing moderation with a clear acceptable-use statement during user onboarding — this reduces accidental violations and gives you a clean basis for enforcement.

​

Limitations

No moderation model is perfect. A few honest caveats:

• False positives happen — clinical, legal, harm-reduction, and historical contexts can trip safety filters even when the underlying intent is educational. The moderation models cannot be tuned; switching between them is the main lever.
• False negatives also happen — a determined adversary can phrase requests to slip past any moderator. Moderation is a layer, not a fortress.
• Output moderation is provider-dependent. The moderation model scores user inputs; some conversation providers (notably Google’s Gemini family) additionally apply safety filters to model outputs. For the providers that don’t, lean on the assistant’s prompt to constrain behaviour.

The most resilient setup combines: (1) Moderation on inputs, (2) an Assistant prompt that explicitly refuses out-of-scope or unsafe requests, (3) clear onboarding guidance to users about what the Digital Twin will and won’t do.

​

Compliance Considerations

If your deployment is subject to a regulatory regime, moderation interacts with it as follows:

• FERPA — the alert email includes the student’s flagged input. Treat those emails with the same access controls as the rest of the conversation log, and keep Contact Emails tight.
• GDPR — under “right to be forgotten” requests, a user’s conversation data must be deleted. Pria’s standard user-deletion flow covers this; blocked prompts are not stored in the conversation history (they do travel in the alert email).
• COPPA — for products serving children under 13, keep moderation on and configure your assistant to refuse personal-data prompts. The conversation log in Histories supports the operator’s record-keeping obligations.

If you operate under an explicit Data Processing Agreement, contact the Praxis AI team at humans@praxis-ai.com for region-specific guidance.

​

Related

• Praxis Shield — review and triage security incidents detected across conversations.
• AI Models — pick the conversation, moderation, and supporting models for your Digital Twin.
• Feedback — review user-reported issues alongside automatic moderation flags.
• Histories — inspect the conversations that produced a moderation event.
• Privacy & Educational Consent — the policy framing for student data and flagged content.