[360Labs.ai]
{ 360Labs }Case Study
{ Research and Development }· Live

Promptctl

CLI for prompt engineering: version control, test-driven evaluation, A/B testing with statistical confidence.

Promptctl
// overview

Prompts as Code. Markdown + YAML frontmatter, JSON test fixtures, bulk evaluation with pass/fail. A/B testing with 95% confidence intervals via bootstrapped sampling. Local React dashboard for traces and cost analysis.

// specs

Specifications

FormatMarkdown with YAML frontmatter, Git-tracked
TestingExact match, regex, semantic similarity, JSON schema
A/B Testing95% confidence intervals via bootstrapped sampling
ProvidersGoogle Gemini, OpenAI, Anthropic, Ollama
// features

Features

  • 01Prompt versioning with Markdown + YAML frontmatter
  • 02Four assertion types: exact match, regex, semantic similarity, JSON schema
  • 03Parallel execution across providers
  • 04A/B testing with 95% confidence intervals
  • 05Local React dashboard for history and cost analysis
// tech stack

Tech Stack

TypeScriptNode.jsReactViteExpressGoogle GenAI

Interested
in Promptctl?

Let's discuss how this can work for you.

Get in Touch →
// end of case studyPromptctl
Contact