when-to-use-z3
Your Questions Answered
1. Do we need models for every hallucination scenario?
NO! You only use automated reasoning for specific high-stakes domains.
❌ Don’t Use Z3 For:
- Creative writing
- Summarization
- General Q&A
- Content generation
- Most typical LLM tasks
✅ DO Use Z3 For:
- Security policies (AWS IAM, access control)
- Financial calculations (pricing, invoicing, accounting)
- Legal compliance (EU AI Act, regulations)
- Safety-critical systems (medical, aviation)
- Mathematical proofs
2. Why Z3 vs Normal Validator Class?
The Key Difference
Normal Validator: Checks specific values
class PricingValidator: def validate(self, original, discount, final): expected = original * (1 - discount / 100) return final == expected # Just checks THIS valueProblem: Only checks the specific value you give it.
Z3: Proves for ALL possible values
from z3 import *
def prove_pricing_formula_is_correct(): """Proves the formula works for ALL inputs""" original = Real('original') discount = Real('discount') final = Real('final')
solver = Solver()
# The formula we claim is correct solver.add(final == original * (1 - discount / 100))
# Add constraints solver.add(original >= 0) solver.add(And(discount >= 0, discount <= 100))
# Try to find a counterexample solver.add(final < 0) # This should be impossible
if solver.check() == unsat: return "✅ PROVEN: Formula never produces negative prices" else: return "❌ FOUND COUNTEREXAMPLE!"Z3 proves it works for ALL values, not just test cases.
Real-World Example: When You NEED Z3
Scenario: AWS IAM Policy Validation
Problem: Does this policy accidentally grant admin access?
{ "Effect": "Allow", "Action": ["s3:*"], "Resource": "arn:aws:s3:::my-bucket/*", "Condition": { "IpAddress": { "aws:SourceIp": ["203.0.113.0/24", "198.51.100.0/24"] } }}Normal Validator (Won’t Catch Bug):
def validate_policy(policy): # Check specific test cases assert has_ip_condition(policy) assert action_is_scoped(policy) # Misses: What if someone adds another condition?Problem: You can only test cases you think of!
Z3 Approach (Catches All Cases):
from z3 import *
# Model the policy as logical formuladef verify_policy_never_grants_admin(): # Variables action = String('action') ip = String('source_ip') resource = String('resource')
solver = Solver()
# Policy rules (encoded as logic) solver.add(Or( action == "s3:GetObject", action == "s3:PutObject" )) solver.add(Or( PrefixOf("203.0.113.", ip), PrefixOf("198.51.100.", ip) ))
# Try to find case where admin is granted solver.add(action == "iam:CreateUser") # Admin action
if solver.check() == unsat: return "✅ PROVEN: Policy never grants admin access" else: return "❌ SECURITY HOLE FOUND!"Z3 proves it’s impossible to get admin access, for ALL combinations.
3. Dynamic Constraints Based on Context
YES! Z3 can handle dynamic constraints.
Example: Restaurant with Variable Rules
from z3 import *
class DynamicConstraintValidator: def __init__(self): self.solver = Solver()
def validate_restaurant(self, people, dishes, custom_rules): """ Validate with context-specific rules
custom_rules example: { "Alice": {"allergies": ["dairy", "gluten"]}, "Bob": {"diet": "vegetarian", "exclude": ["salmon"]}, "Carol": {"copies": "Bob"} } """
# Create variables dynamically assignments = { person: Int(f'{person}_dish') for person in people }
# Add dynamic constraints based on context for person, rules in custom_rules.items():
# Allergy constraints (dynamic based on person) if "allergies" in rules: allowed_dishes = [ dish_id for dish_id, dish in enumerate(dishes) if not any(allergen in dish["contains"] for allergen in rules["allergies"]) ] self.solver.add(Or([ assignments[person] == dish_id for dish_id in allowed_dishes ]))
# Diet constraints (vegetarian, vegan, etc.) if "diet" in rules: if rules["diet"] == "vegetarian": veg_dishes = [ i for i, d in enumerate(dishes) if d["vegetarian"] ] self.solver.add(Or([ assignments[person] == i for i in veg_dishes ]))
# Dependency constraints (eat same as someone) if "copies" in rules: other_person = rules["copies"] self.solver.add( assignments[person] == assignments[other_person] )
return self.solver.check() == sat
# Usage with different contextsvalidator = DynamicConstraintValidator()
# Context 1: Just allergiesresult1 = validator.validate_restaurant( people=["Alice", "Bob"], dishes=[...], custom_rules={ "Alice": {"allergies": ["dairy"]} })
# Context 2: Complex rulesresult2 = validator.validate_restaurant( people=["Alice", "Bob", "Carol"], dishes=[...], custom_rules={ "Alice": {"allergies": ["dairy", "gluten"]}, "Bob": {"diet": "vegetarian"}, "Carol": {"copies": "Bob"} })The constraints are built dynamically based on context!
When Z3 Actually Matters: Real Examples
1. Financial System (Stripe-like)
Problem: Verify pricing calculation is correct for ALL possible inputs
# Normal validator - only checks specific casesdef validate_invoice(subtotal, tax_rate, discount, total): expected = (subtotal - discount) * (1 + tax_rate) return abs(total - expected) < 0.01 # Checks ONE case
# Z3 - proves formula is correct for ALL casesdef prove_invoice_formula(): subtotal = Real('subtotal') tax = Real('tax_rate') discount = Real('discount') total = Real('total')
solver = Solver() solver.add(total == (subtotal - discount) * (1 + tax))
# Business rules solver.add(subtotal >= 0) solver.add(And(tax >= 0, tax <= 1)) solver.add(And(discount >= 0, discount <= subtotal))
# Prove total is never negative solver.add(total < 0)
if solver.check() == unsat: return "✅ PROVEN: Formula never produces negative invoices"Why it matters: In a financial system, you can’t just test - you need mathematical proof.
2. Access Control (AWS-like)
Problem: Does combining 5 different policies accidentally grant unauthorized access?
# Normal validator - can't handle policy combinationsdef check_access(user, action, resource): # You'd have to enumerate all combinations # Misses edge cases! pass
# Z3 - proves no unauthorized access is possibledef verify_no_unauthorized_access(): # Model all policies as logical formulas # Z3 searches ALL possible combinations # Returns proof or counterexample passWhy it matters: Security bugs in policy combinations are hard to catch with testing.
3. Medical Device Dosage Calculator
Problem: Verify drug dosage calculation is safe for ALL patient weights/ages
# Normal validator - dangerous to only test specific cases!def validate_dosage(weight, age, dose): max_safe = weight * 0.5 return dose <= max_safe # Only checks THIS case
# Z3 - proves dosage is ALWAYS safedef prove_dosage_always_safe(): weight = Real('weight') age = Real('age') dose = Real('dose')
solver = Solver()
# Dosage formula solver.add(dose == weight * 0.5 * (age / 100))
# Constraints solver.add(And(weight >= 0, weight <= 200)) # kg solver.add(And(age >= 0, age <= 120)) # years
# Prove dose never exceeds safe limit solver.add(dose > weight * 0.5)
if solver.check() == unsat: return "✅ PROVEN: Dosage formula is always safe" else: model = solver.model() return f"❌ DANGEROUS! Counterexample found: {model}"Why it matters: Lives depend on this being correct for ALL cases, not just tests.
Bottom Line: When to Use What
| Scenario | Use Normal Validator | Use Z3 |
|---|---|---|
| Blog post generation | ✅ | ❌ |
| Summarization | ✅ | ❌ |
| Customer support chat | ✅ | ❌ |
| Financial calculations | ❌ | ✅ |
| Security policies | ❌ | ✅ |
| Safety-critical systems | ❌ | ✅ |
| Legal compliance | ❌ | ✅ |
| Simple data validation | ✅ | ❌ |
| Business logic | ✅ (usually) | ✅ (if high-stakes) |
The Real Value: Z3 Finds Bugs You Didn’t Think Of
Example: Bug in Restaurant Riddle
What if we had this constraint:
“If Alice eats fish, then David must eat vegetarian”
Normal validator:
def validate(assignments): if assignments["Alice"] == "Grilled Salmon": assert is_vegetarian(assignments["David"])You’d only test the cases you thought of!
Z3 automatically finds:
- What if David is already committed to meat?
- What if Alice’s only option is fish?
- What if this creates an impossible situation?
solver = Solver()
# Existing constraintssolver.add(alice_dish == 2) # Must eat salmon (only option)
# New constraintsolver.add(Implies( alice_dish == 2, # If Alice eats salmon Or(david_dish == 1, david_dish == 3, david_dish == 4) # David vegetarian))
# David must be uniquesolver.add(david_dish != bob_dish) # Bob has dish 1solver.add(david_dish != carol_dish) # Carol has dish 1
if solver.check() == unsat: print("❌ IMPOSSIBLE! Constraints contradict each other") # Z3 found the bug automatically!Summary: Answers to Your Questions
1. Do we need models for every situation?
NO! Only for:
- Financial/billing
- Security/access control
- Safety-critical (medical, aviation)
- Legal compliance
2. Why Z3 vs normal class?
Z3 proves for ALL cases, normal class only checks specific values.
| Feature | Normal Class | Z3 |
|---|---|---|
| Checks specific values | ✅ | ✅ |
| Proves ALL values | ❌ | ✅ |
| Finds solutions automatically | ❌ | ✅ |
| Detects impossible constraints | ❌ | ✅ |
| Simpler code | ✅ | ❌ |
3. Dynamic constraints?
YES! Build constraints at runtime based on context. See example above.
When You Should Actually Use This
Use automated reasoning when:
- ✅ Stakes are high (money, security, safety, legal)
- ✅ Complexity is high (many interacting rules)
- ✅ You need mathematical proof, not just testing
- ✅ Edge cases are hard to predict
- ✅ Regulatory compliance requires it (EU AI Act, SOC2, etc.)
Don’t use it when:
- ❌ Simple validation is enough
- ❌ Creative/subjective output
- ❌ Low stakes
- ❌ Performance is critical (Z3 can be slow)
What You Should Build
For most LLM applications:
- Start with normal validation (simple, fast)
- Add Z3 for high-stakes parts (billing, access control)
- Use LLM for everything else (content, summaries)
Example Architecture:
User Query → LLM → Response ↓ Is it high-stakes? ↙️ ↘️ YES NO ↓ ↓ Z3 Verification Return ↓ Valid/InvalidLast Updated: October 31, 2025