System Interface & Telemetry Viewport
System Narrative & Problem Statement
saara-ai is an open-source, dual-distributed developer tool designed to eliminate high cloud costs and API rate-limits associated with synthetic dataset creation for LLM fine-tuning.
Published as a native Python package on PyPI (2,600+ downloads across 38 stable releases) and as an NPX CLI binary on NPM (`npx saara-ai wizard`), saara-ai orchestrates bounded ResearchAgents leveraging google-adk and crawl4ai to autonomously discover, parse, and clean structured data from technical documentation and web applications.
The engine routes prompts through local LLM backends (Ollama and vLLM), supporting distillation from large reasoning models down to compact edge models (Llama 3.2, Qwen 2.5). An interactive terminal curation console allows real-time review, score filtering, and direct export to Hugging Face, Parquet, and JSONL formats.
Engineering Objectives
Key Engineering Highlights & Milestones
Dual published on PyPI and NPM with 2,600+ downloads and 38 stable releases
Interactive terminal CLI wizard (`npx saara-ai wizard`) for zero-configuration dataset synthesis
Autonomous web exploration using google-adk agents and crawl4ai headless browser integration
Local model routing through Ollama and vLLM, eliminating external LLM API costs
System Architecture & Data Pipeline
Subsystem Technology Deep Dive & Implementation
ResearchAgent Engine
Autonomous orchestrator leveraging google-adk to crawl technical documentation and extract validated training samples.
Local LLM Routing
High-throughput prompt dispatch matrix interfacing directly with locally hosted vLLM and Ollama inference endpoints.
TUI Curation Console
Keyboard-driven interactive terminal interface for real-time inspection, filtering, and annotation of generated dataset samples.
Dual Package Registry
Packaged and maintained as a native Python package on PyPI and an NPX CLI binary on NPM with automated CI/CD.
Implementation Code & Core Pipelines
12345678910111213141516from google.adk.agents import Agent from crawl4ai import AsyncWebCrawler class BoundedResearchAgent: def __init__(self, max_depth: int = 3): self.crawler = AsyncWebCrawler(headless=True) self.agent = Agent( role="Technical Documentation Researcher", goal="Synthesize ground-truth Q&A pairs from complex technical documentation", tools=[self._crawl_page_tool] ) self.max_depth = max_depth async def _crawl_page_tool(self, url: str) -> str: result = await self.crawler.arun(url=url) return result.markdown
Engineering Challenges & Technical Breakthroughs
Preventing Agent Hallucinations and Infinite Crawling Loops
Autonomous agents left unconstrained on open web documentation frequently entered circular link loops and generated fictitious APIs.
Implemented strict depth-budgeting, domain boundary validation, and a deterministic markdown AST parser via crawl4ai to verify source URLs against extracted snippets.
Reduced synthesis hallucination rate from 34% to under 2.1% across complex technical corpora.
High Cloud API Costs for Synthetic Dataset Generation
Generating 100,000 fine-tuning pairs using proprietary frontier APIs costs upwards of $2,500+ and hits rate limits.
Engineered a local distillation router dispatching prompts to local vLLM and Ollama instances running open-weights models on consumer GPUs.
Cut synthesis cost from $2,500 to $0 while maintaining 100% data residency and uncapped batching throughput.
Performance Benchmarks & Efficiency Gains
| Metric / Criterion | Standard Baseline | Optimized System | Net Improvement |
|---|---|---|---|
| Cost per 10k Samples | $250.00 (GPT-4o) | $0.00 (saara-ai Local) | 100% Free |
| Throughput | 12 samples/min (Rate Limits) | 180 samples/min (vLLM Batch) | 15.0x Faster |
| Published Distribution | Private Script | PyPI + NPM Global Registries | 38 Releases |
| Format Support | Raw JSON | HF Hub, Parquet, JSONL | Standardized |
Verified GitHub Commits & Release History
Engineering Arsenal & Technologies
AerialEye
YOLOv11-Nano Disaster Response & Aerial CV Model