sifaka

Critics Overview

Critics are the heart of Sifaka’s text improvement system. Each critic implements a specific evaluation strategy based on cutting-edge AI research.

What Are Critics?

Critics analyze text and provide structured feedback for improvement. They:

• Evaluate text quality
• Identify specific issues
• Suggest improvements
• Guide iterative refinement

Available Critics

Core Critics

Critic	Purpose	Best For
SELF_REFINE	General improvement	All-purpose text enhancement
REFLEXION	Learning from attempts	Complex reasoning tasks
CONSTITUTIONAL	Ethical evaluation	Safety-critical content
SELF_CONSISTENCY	Consensus building	Balanced perspectives

Advanced Critics

Critic	Purpose	Best For
SELF_RAG	Fact verification	Academic/factual content
META_REWARDING	Quality assurance	High-stakes content
N_CRITICS	Multiple perspectives	Comprehensive review
STYLE	Tone adaptation	Audience-specific content

How Critics Work

1. Analysis: Critic examines the text
2. Evaluation: Identifies strengths and weaknesses
3. Feedback: Provides specific improvement suggestions
4. Iteration: Process repeats until satisfaction

# Basic critic flow
from sifaka import improve
from sifaka.core.types import CriticType

result = await improve(
    "Your text",
    critics=[CriticType.SELF_REFINE],
    max_iterations=3
)

# Access critique history
for critique in result.critiques:
    print(f"Critic: {critique.critic}")
    print(f"Feedback: {critique.feedback}")
    print(f"Confidence: {critique.confidence}")

Choosing Critics

By Content Type

Technical/Academic:

• REFLEXION - Deep analysis
• SELF_RAG - Fact checking
• SELF_CONSISTENCY - Balanced claims

Creative/Marketing:

• STYLE - Tone adaptation
• SELF_REFINE - General polish
• N_CRITICS - Multiple perspectives

Business/Professional:

• SELF_REFINE - Clarity and professionalism
• CONSTITUTIONAL - Ethical considerations
• META_REWARDING - Quality assurance

By Goal

Improve Clarity:

critics=[CriticType.SELF_REFINE]

Ensure Accuracy:

critics=[CriticType.SELF_RAG, CriticType.REFLEXION]

Maintain Safety:

critics=[CriticType.CONSTITUTIONAL]

Adapt Style:

critics=[CriticType.STYLE]

Combining Critics

Critics can work together for comprehensive improvement:

# Accuracy + Safety
result = await improve(
    text,
    critics=[
        CriticType.SELF_RAG,        # Verify facts
        CriticType.CONSTITUTIONAL   # Ensure safety
    ]
)

# Style + Quality
result = await improve(
    text,
    critics=[
        CriticType.STYLE,          # Match tone
        CriticType.SELF_REFINE,    # Polish
        CriticType.META_REWARDING  # Verify quality
    ]
)

Research Foundation

Each critic is based on peer-reviewed research:

• SELF_REFINE: Madaan et al., 2023
• REFLEXION: Shinn et al., 2023
• CONSTITUTIONAL: Bai et al., 2022
• SELF_CONSISTENCY: Wang et al., 2022
• SELF_RAG: Asai et al., 2023
• META_REWARDING: Wu et al., 2024

Understanding Critique Results

Each critique provides:

class CritiqueResult:
    critic: str              # Which critic
    feedback: str            # Detailed feedback
    suggestions: list[str]   # Specific improvements
    needs_improvement: bool  # Continue or stop
    confidence: float        # 0.0-1.0 confidence

Example output:

CritiqueResult(
    critic="self_refine",
    feedback="The text lacks specific examples and data.",
    suggestions=[
        "Add concrete examples",
        "Include relevant statistics",
        "Clarify the main argument"
    ],
    needs_improvement=True,
    confidence=0.75
)

Performance Characteristics

Speed

Fast (< 2s per iteration):

• SELF_REFINE
• STYLE

Medium (2-5s per iteration):

• REFLEXION
• CONSTITUTIONAL
• META_REWARDING

Slower (5-10s per iteration):

• SELF_CONSISTENCY (multiple samples)
• N_CRITICS (multiple perspectives)
• SELF_RAG (retrieval required)

Quality Impact

High Impact:

• REFLEXION (for reasoning)
• N_CRITICS (for comprehensiveness)
• META_REWARDING (for quality)

Targeted Impact:

• STYLE (for tone)
• CONSTITUTIONAL (for safety)
• SELF_RAG (for accuracy)

Custom Critics

Create domain-specific critics:

from sifaka.plugins import CriticPlugin
from sifaka.core.models import CritiqueResult

class SEOCritic(CriticPlugin):
    """Critic for SEO optimization."""

    async def critique(self, text: str, result):
        # Analyze for SEO factors
        issues = []
        if len(text.split()) < 300:
            issues.append("Add more content (300+ words)")
        if not any(keyword in text.lower()
                  for keyword in ["keyword", "phrases"]):
            issues.append("Include target keywords")

        return CritiqueResult(
            critic="seo_critic",
            feedback="SEO analysis complete",
            suggestions=issues,
            needs_improvement=len(issues) > 0,
            confidence=0.8
        )

Best Practices

1. Start Simple: Begin with SELF_REFINE
2. Add Specificity: Layer specialized critics
3. Consider Context: Match critics to content type
4. Balance Speed/Quality: More critics = better but slower
5. Monitor Confidence: High confidence = diminishing returns
6. Test Combinations: Find what works for your use case

Next Steps

• Read the detailed Critics Guide
• Try the examples
• Learn about custom critics

This site is open source. Improve this page.