ADR-002: Plugin Architecture for Critics and Validators
Status
Accepted
Context
Sifaka needs to support multiple text improvement strategies (critics) and quality validators. The system should be extensible to allow:
- Adding new critics without modifying core code
- Custom validators for specific use cases
- Third-party extensions
- Easy configuration of which critics to use
We considered several approaches:
- Hard-coded critics in the core library
- Registry-based plugin system
- Entry point-based plugin discovery
- Configuration-driven plugin loading
Decision
We will implement a registry-based plugin system with both automatic discovery and manual registration.
# Automatic registration (in plugin modules)
from sifaka.critics import register_critic
@register_critic("my_critic")
class MyCritic(BaseCritic):
# implementation
# Manual registration
from sifaka.critics import CriticRegistry
CriticRegistry.register("custom_critic", CustomCritic)
# Usage
result = await improve("text", critics=["my_critic", "custom_critic"])
Rationale
- Extensibility: Easy to add new critics without core changes
- Modularity: Critics can be developed independently
- Discoverability: Registry allows listing available critics
- Configuration: Users can easily choose which critics to use
- Testing: Each critic can be tested in isolation
Design Principles
- Protocol-based: Critics implement a common interface
- Composable: Multiple critics can be combined
- Configurable: Each critic can have its own configuration
- Isolated: Critics should not depend on each other
Implementation Details
Base Classes
class BaseCritic(ABC):
@abstractmethod
async def critique(self, text: str, result: SifakaResult) -> CritiqueResult:
pass
@property
@abstractmethod
def name(self) -> str:
pass
Registry System
class CriticRegistry:
_critics: Dict[str, Type[BaseCritic]] = {}
@classmethod
def register(cls, name: str, critic_class: Type[BaseCritic]):
cls._critics[name] = critic_class
@classmethod
def get(cls, name: str) -> Type[BaseCritic]:
return cls._critics.get(name)
Plugin Discovery
- Automatic registration through decorators
- Manual registration for dynamic loading
- Configuration-based critic selection
- Validation of critic implementations
Consequences
Positive
- Easy to extend with new critics
- Clean separation of concerns
- Testable and maintainable
- Supports both built-in and custom critics
- Enables community contributions
Negative
- Additional complexity in the core system
- Plugin discovery overhead
- Potential version compatibility issues
- Need for plugin documentation standards
Mitigation
- Provide clear base classes and interfaces
- Document plugin development guidelines
- Implement plugin validation
- Version compatibility checking
- Comprehensive examples and templates
Built-in Critics
The system will include several built-in critics:
- ReflexionCritic: Self-reflection based improvement
- ConstitutionalCritic: Principle-based evaluation
- SelfRefineCritic: Iterative refinement
- SelfRAGCritic: Retrieval-augmented critique
- NCriticsCritic: Ensemble of multiple critics