Innovative Software Technology-Structured Testing for AI Prompts and LLMs with promptfoo

Elevating LLM Prompt Testing: A Comprehensive Guide with Promptfoo

The journey from a basic LLM prompt to a robust, production-ready AI application is often fraught with unexpected challenges. While initial “vibe checks”—manual testing with a few examples—might suffice for early development, real-world user-generated content introduces a level of complexity that demands a more systematic and automated evaluation framework. This guide builds on foundational prompt testing concepts, diving deeper into promptfoo to demonstrate how to build sophisticated, repeatable regression tests for your AI systems.

The Imperative for Rigorous Testing

Imagine you’re developing an e-commerce feature designed to process user product reviews. This system needs to accurately classify sentiment, extract key features, identify potentially fake submissions, and recommend moderation actions. A prompt that seems perfect in a controlled environment will quickly falter when confronted with the “messy reality” of live reviews. This includes handling:

Mixed sentiments: (“Loved the product, hated the delivery experience.”)
Deceptive content: Fake or suspicious reviews.
Inappropriate language: Profanity or offensive phrases.
Subtle communication: Sarcasm or nuanced expressions.
Competitor mentions: Reviews that reference rival brands.

Addressing these complexities requires a systematic approach to testing with diverse scenarios.

Structuring Your Testing Strategy

Before diving into the technical implementation, it’s crucial to define what successful prompt behavior looks like. A human-readable format, such as Gherkin, can effectively outline these requirements. For our e-commerce review analyzer, the core scenarios might include:

Feature: Product Review Analysis Prompt

Scenario Outline: Prompt analyzes product reviews correctly
Given a product review analysis prompt
And a “<review_type>” product review
When the prompt processes the review
Then the sentiment should be classified as “<expected_sentiment>”
And fake review indicators should be “<fake_indicators>”
And the recommendation should be “<expected_recommendation>”
And key features should be extracted

review_type	expected_sentiment	expected_fake_indicators	expected_recommendation
positive	positive	absent	approve
negative	negative	absent	approve
mixed	mixed	absent	flag_for_review
suspicious	positive	present	flag_for_review

While promptfoo doesn’t directly consume Gherkin, these scenarios serve as a blueprint for crafting your automated tests.

Organized Project Setup for Scalable Testing

For maintainability and collaboration, organizing your test assets into a structured project is highly recommended. Instead of embedding everything in a single YAML file, separate your prompts and test data.

promptfoo-product-reviews/
├── prompts/
│   └── analyze-review.txt
├── test-data/
│   ├── positive-review.txt
│   ├── negative-review.txt
│   ├── mixed-review.txt
│   └── suspicious-review.txt
├── analyze-review-spec.yaml
└── package.json

This structure clearly delineates prompt logic from test inputs and configuration.

Crafting the Review Analysis Prompt

Our core AI component is a prompt designed to extract structured information from product reviews. This prompt should guide the LLM to provide specific data points, such as sentiment, confidence, key features, potential fake indicators, and a moderation recommendation.

prompts/analyze-review.txt

You are an expert product review analyzer for an ecommerce platform. Analyze the following product review and provide a structured assessment.

Product Review:
{{review_text}}

Provide your analysis in the following JSON format. Return ONLY the JSON object, no markdown code blocks, no explanations, no additional text:
{
  "sentiment": "positive|negative|mixed",
  "confidence": 0.0-1.0,
  "key_features_mentioned": ["feature1", "feature2"],
  "main_complaints": ["complaint1", "complaint2"],
  "main_praise": ["praise1", "praise2"],
  "suspected_fake": boolean,
  "fake_indicators": ["indicator1", "indicator2"],
  "recommendation": "approve|flag_for_review|reject",
  "summary": "Brief 1-2 sentence summary"
}

Focus on:
- Accurate sentiment classification, especially for mixed reviews
- Extracting specific product features mentioned
- Identifying potential fake review indicators such as generic language without specific details, suspicious patterns, overly positive language, and extreme superlatives, overly negative language
- Providing actionable moderation recommendations

IMPORTANT: Return ONLY valid JSON. Do not wrap in markdown code blocks or add any other text.

The prompt clearly defines the desired JSON output format and emphasizes key analytical tasks, including the detection of fake review characteristics.

Designing Realistic Test Scenarios

To effectively evaluate the prompt, we need a diverse set of test inputs that mirror real-world complexities. These examples can be synthesized or drawn from actual production data.

Scenario 1: Genuine Positive Review (test-data/positive-review.txt)
A detailed review praising specific aspects like battery life and sound quality, with a minor, realistic complaint.

Scenario 2: Detailed Negative Review (test-data/negative-review.txt)
A review articulating clear dissatisfaction with specific product flaws like connectivity issues and poor sound quality.

Scenario 3: Mixed Sentiment Review (test-data/mixed-review.txt)
A review balancing both positive observations (e.g., sound quality) with significant drawbacks (e.g., connectivity, fit). This is often the most challenging for sentiment analysis.

Scenario 4: Suspicious/Fake Review (test-data/suspicious-review.txt)
A review characterized by overly generic, exaggerated praise, lacking specific details, and using extreme superlatives – common indicators of inauthenticity.

Implementing Comprehensive Test Configuration with Promptfoo

With our prompt and test data ready, we can configure promptfoo to run these scenarios and assert the expected outcomes.

analyze-review-spec.yaml

description: Product Review Analysis Testing

prompts:
  - file://prompts/analyze-review.txt

providers:
  - openai:chat:gpt-4o-mini

tests:
  # Test 1: Genuine Positive Review
  - vars:
      review_text: file://test-data/positive-review.txt
    assert:
      - type: is-json
      - type: javascript
        value: |
          const response = JSON.parse(output);
          response.sentiment === 'positive' && response.confidence > 0.7
      - type: contains-json
        value:
          suspected_fake: false
      - type: llm-rubric
        value: "Should identify key positive features like battery life, sound quality, and comfort. Should not flag as fake since it contains specific details and minor complaints."

  # Test 2: Detailed Negative Review  
  - vars:
      review_text: file://test-data/negative-review.txt
    assert:
      - type: is-json
      - type: javascript
        value: |
          const response = JSON.parse(output);
          response.sentiment === 'negative' && response.confidence > 0.7
      - type: contains-json
        value:
          suspected_fake: false
      - type: llm-rubric
        value: "Should identify specific complaints about connection, battery, sound quality, and comfort. Should extract main issues for product team review."

  # Test 3: Mixed Sentiment Review
  - vars:
      review_text: file://test-data/mixed-review.txt
    assert:
      - type: is-json
      - type: javascript
        value: |
          const response = JSON.parse(output);
          response.sentiment === 'mixed'
      - type: llm-rubric
        value: "Should correctly identify mixed sentiment, extracting both positive aspects (sound quality, build) and negative aspects (connectivity, fit). This is the most challenging scenario for sentiment analysis."

  # Test 4: Suspicious/Fake Review
  - vars:
      review_text: file://test-data/suspicious-review.txt
    assert:
      - type: is-json
      - type: contains-json
        value:
          suspected_fake: true
      - type: javascript
        value: |
          const response = JSON.parse(output);
          response.fake_indicators && response.fake_indicators.length > 0
      - type: llm-rubric
        value: "Should detect fake review indicators: overly positive language, lack of specific details, generic praise, and extreme superlatives."

This promptfoo configuration defines four distinct tests, each corresponding to a scenario. The file:// syntax efficiently loads the review content into the review_text variable for each test.

Multi-Layered Assertions for Robust Validation

A key strength of promptfoo is its diverse assertion types, enabling thorough validation:

is-json: Verifies that the LLM’s output is syntactically valid JSON.
contains-json: Checks for the presence and value of specific key-value pairs within the JSON output (e.g., suspected_fake: true).
javascript: Provides immense flexibility for custom validation logic. For instance, it can check if the sentiment is ‘positive’ AND if the confidence score exceeds a certain threshold (response.sentiment === 'positive' && response.confidence > 0.7), catching cases where the model might be unsure.
llm-rubric: Leverages another LLM instance to evaluate the output against human-readable criteria, providing a qualitative assessment.

Setting Up and Running Your Tests

First, install promptfoo as a development dependency:

npm install --save-dev promptfoo

Then, execute your tests and view the results:

npx promptfoo eval -c promptfoo-product-reviews/analyze-review-spec.yaml --no-cache
npx promptfoo view -y

The promptfoo web viewer offers a detailed grid display of test outcomes. For each scenario, you can examine the prompt’s input, the LLM’s raw JSON response, and the pass/fail status of each assertion. This granular view helps in quickly diagnosing issues and understanding the AI’s behavior. For example, a successful positive review analysis might show:

{
  "sentiment": "positive",
  "confidence": 0.9,
  "key_features_mentioned": ["battery life", "sound quality", "comfort", "touch controls"],
  "main_complaints": ["case is bulky"],
  "main_praise": ["excellent battery life", "crisp and clear sound quality", "comfortable during workouts", "reliable touch controls"],
  "suspected_fake": false,
  "fake_indicators": [],
  "recommendation": "approve",
  "summary": "The reviewer expresses high satisfaction with the wireless earbuds, highlighting their excellent battery life and sound quality while noting a minor complaint about the case size."
}

Integrating Tests into Your CI/CD Pipeline

The command-line nature of promptfoo makes it an ideal candidate for integration into your continuous integration (CI) pipeline. By adding a promptfoo eval step, you can automate regression testing for your prompts. This ensures that any changes to the prompt, the underlying LLM provider, or even new model versions, are validated against your established test suite, preventing unexpected regressions and maintaining the quality of your AI system. As requirements evolve, so too can your test suite, ensuring your system remains aligned with business needs.

Conclusion

Establishing a comprehensive testing framework for LLM prompts is no longer optional for reliable AI applications. By leveraging tools like promptfoo and adopting a systematic approach to scenario definition and multi-layered assertions, developers can move beyond anecdotal testing. This methodology, akin to Test-Driven Development (TDD), streamlines the iteration process, enabling quicker diagnosis of issues and fostering confidence in your AI’s performance across diverse, real-world inputs. The initial effort in setting up these tests yields significant returns in terms of stability, maintainability, and ultimately, the success of your AI-powered features.