Your Questions Answered

1. Do we need models for every hallucination scenario?

NO! You only use automated reasoning for specific high-stakes domains.

Don’t Use Z3 For:

  • Creative writing
  • Summarization
  • General Q&A
  • Content generation
  • Most typical LLM tasks

DO Use Z3 For:

  • Security policies (AWS IAM, access control)
  • Financial calculations (pricing, invoicing, accounting)
  • Legal compliance (EU AI Act, regulations)
  • Safety-critical systems (medical, aviation)
  • Mathematical proofs

2. Why Z3 vs Normal Validator Class?

The Key Difference

Normal Validator: Checks specific values

class PricingValidator:
def validate(self, original, discount, final):
expected = original * (1 - discount / 100)
return final == expected # Just checks THIS value

Problem: Only checks the specific value you give it.

Z3: Proves for ALL possible values

from z3 import *
def prove_pricing_formula_is_correct():
"""Proves the formula works for ALL inputs"""
original = Real('original')
discount = Real('discount')
final = Real('final')
solver = Solver()
# The formula we claim is correct
solver.add(final == original * (1 - discount / 100))
# Add constraints
solver.add(original >= 0)
solver.add(And(discount >= 0, discount <= 100))
# Try to find a counterexample
solver.add(final < 0) # This should be impossible
if solver.check() == unsat:
return "✅ PROVEN: Formula never produces negative prices"
else:
return "❌ FOUND COUNTEREXAMPLE!"

Z3 proves it works for ALL values, not just test cases.


Real-World Example: When You NEED Z3

Scenario: AWS IAM Policy Validation

Problem: Does this policy accidentally grant admin access?

{
"Effect": "Allow",
"Action": ["s3:*"],
"Resource": "arn:aws:s3:::my-bucket/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": ["203.0.113.0/24", "198.51.100.0/24"]
}
}
}

Normal Validator (Won’t Catch Bug):

def validate_policy(policy):
# Check specific test cases
assert has_ip_condition(policy)
assert action_is_scoped(policy)
# Misses: What if someone adds another condition?

Problem: You can only test cases you think of!

Z3 Approach (Catches All Cases):

from z3 import *
# Model the policy as logical formula
def verify_policy_never_grants_admin():
# Variables
action = String('action')
ip = String('source_ip')
resource = String('resource')
solver = Solver()
# Policy rules (encoded as logic)
solver.add(Or(
action == "s3:GetObject",
action == "s3:PutObject"
))
solver.add(Or(
PrefixOf("203.0.113.", ip),
PrefixOf("198.51.100.", ip)
))
# Try to find case where admin is granted
solver.add(action == "iam:CreateUser") # Admin action
if solver.check() == unsat:
return "✅ PROVEN: Policy never grants admin access"
else:
return "❌ SECURITY HOLE FOUND!"

Z3 proves it’s impossible to get admin access, for ALL combinations.


3. Dynamic Constraints Based on Context

YES! Z3 can handle dynamic constraints.

Example: Restaurant with Variable Rules

from z3 import *
class DynamicConstraintValidator:
def __init__(self):
self.solver = Solver()
def validate_restaurant(self, people, dishes, custom_rules):
"""
Validate with context-specific rules
custom_rules example:
{
"Alice": {"allergies": ["dairy", "gluten"]},
"Bob": {"diet": "vegetarian", "exclude": ["salmon"]},
"Carol": {"copies": "Bob"}
}
"""
# Create variables dynamically
assignments = {
person: Int(f'{person}_dish')
for person in people
}
# Add dynamic constraints based on context
for person, rules in custom_rules.items():
# Allergy constraints (dynamic based on person)
if "allergies" in rules:
allowed_dishes = [
dish_id for dish_id, dish in enumerate(dishes)
if not any(allergen in dish["contains"]
for allergen in rules["allergies"])
]
self.solver.add(Or([
assignments[person] == dish_id
for dish_id in allowed_dishes
]))
# Diet constraints (vegetarian, vegan, etc.)
if "diet" in rules:
if rules["diet"] == "vegetarian":
veg_dishes = [
i for i, d in enumerate(dishes)
if d["vegetarian"]
]
self.solver.add(Or([
assignments[person] == i
for i in veg_dishes
]))
# Dependency constraints (eat same as someone)
if "copies" in rules:
other_person = rules["copies"]
self.solver.add(
assignments[person] == assignments[other_person]
)
return self.solver.check() == sat
# Usage with different contexts
validator = DynamicConstraintValidator()
# Context 1: Just allergies
result1 = validator.validate_restaurant(
people=["Alice", "Bob"],
dishes=[...],
custom_rules={
"Alice": {"allergies": ["dairy"]}
}
)
# Context 2: Complex rules
result2 = validator.validate_restaurant(
people=["Alice", "Bob", "Carol"],
dishes=[...],
custom_rules={
"Alice": {"allergies": ["dairy", "gluten"]},
"Bob": {"diet": "vegetarian"},
"Carol": {"copies": "Bob"}
}
)

The constraints are built dynamically based on context!


When Z3 Actually Matters: Real Examples

1. Financial System (Stripe-like)

Problem: Verify pricing calculation is correct for ALL possible inputs

# Normal validator - only checks specific cases
def validate_invoice(subtotal, tax_rate, discount, total):
expected = (subtotal - discount) * (1 + tax_rate)
return abs(total - expected) < 0.01 # Checks ONE case
# Z3 - proves formula is correct for ALL cases
def prove_invoice_formula():
subtotal = Real('subtotal')
tax = Real('tax_rate')
discount = Real('discount')
total = Real('total')
solver = Solver()
solver.add(total == (subtotal - discount) * (1 + tax))
# Business rules
solver.add(subtotal >= 0)
solver.add(And(tax >= 0, tax <= 1))
solver.add(And(discount >= 0, discount <= subtotal))
# Prove total is never negative
solver.add(total < 0)
if solver.check() == unsat:
return "✅ PROVEN: Formula never produces negative invoices"

Why it matters: In a financial system, you can’t just test - you need mathematical proof.


2. Access Control (AWS-like)

Problem: Does combining 5 different policies accidentally grant unauthorized access?

# Normal validator - can't handle policy combinations
def check_access(user, action, resource):
# You'd have to enumerate all combinations
# Misses edge cases!
pass
# Z3 - proves no unauthorized access is possible
def verify_no_unauthorized_access():
# Model all policies as logical formulas
# Z3 searches ALL possible combinations
# Returns proof or counterexample
pass

Why it matters: Security bugs in policy combinations are hard to catch with testing.


3. Medical Device Dosage Calculator

Problem: Verify drug dosage calculation is safe for ALL patient weights/ages

# Normal validator - dangerous to only test specific cases!
def validate_dosage(weight, age, dose):
max_safe = weight * 0.5
return dose <= max_safe # Only checks THIS case
# Z3 - proves dosage is ALWAYS safe
def prove_dosage_always_safe():
weight = Real('weight')
age = Real('age')
dose = Real('dose')
solver = Solver()
# Dosage formula
solver.add(dose == weight * 0.5 * (age / 100))
# Constraints
solver.add(And(weight >= 0, weight <= 200)) # kg
solver.add(And(age >= 0, age <= 120)) # years
# Prove dose never exceeds safe limit
solver.add(dose > weight * 0.5)
if solver.check() == unsat:
return "✅ PROVEN: Dosage formula is always safe"
else:
model = solver.model()
return f"❌ DANGEROUS! Counterexample found: {model}"

Why it matters: Lives depend on this being correct for ALL cases, not just tests.


Bottom Line: When to Use What

ScenarioUse Normal ValidatorUse Z3
Blog post generation
Summarization
Customer support chat
Financial calculations
Security policies
Safety-critical systems
Legal compliance
Simple data validation
Business logic✅ (usually)✅ (if high-stakes)

The Real Value: Z3 Finds Bugs You Didn’t Think Of

Example: Bug in Restaurant Riddle

What if we had this constraint:

“If Alice eats fish, then David must eat vegetarian”

Normal validator:

def validate(assignments):
if assignments["Alice"] == "Grilled Salmon":
assert is_vegetarian(assignments["David"])

You’d only test the cases you thought of!

Z3 automatically finds:

  • What if David is already committed to meat?
  • What if Alice’s only option is fish?
  • What if this creates an impossible situation?
solver = Solver()
# Existing constraints
solver.add(alice_dish == 2) # Must eat salmon (only option)
# New constraint
solver.add(Implies(
alice_dish == 2, # If Alice eats salmon
Or(david_dish == 1, david_dish == 3, david_dish == 4) # David vegetarian
))
# David must be unique
solver.add(david_dish != bob_dish) # Bob has dish 1
solver.add(david_dish != carol_dish) # Carol has dish 1
if solver.check() == unsat:
print("❌ IMPOSSIBLE! Constraints contradict each other")
# Z3 found the bug automatically!

Summary: Answers to Your Questions

1. Do we need models for every situation?

NO! Only for:

  • Financial/billing
  • Security/access control
  • Safety-critical (medical, aviation)
  • Legal compliance

2. Why Z3 vs normal class?

Z3 proves for ALL cases, normal class only checks specific values.

FeatureNormal ClassZ3
Checks specific values
Proves ALL values
Finds solutions automatically
Detects impossible constraints
Simpler code

3. Dynamic constraints?

YES! Build constraints at runtime based on context. See example above.


When You Should Actually Use This

Use automated reasoning when:

  1. Stakes are high (money, security, safety, legal)
  2. Complexity is high (many interacting rules)
  3. You need mathematical proof, not just testing
  4. Edge cases are hard to predict
  5. Regulatory compliance requires it (EU AI Act, SOC2, etc.)

Don’t use it when:

  1. ❌ Simple validation is enough
  2. ❌ Creative/subjective output
  3. ❌ Low stakes
  4. ❌ Performance is critical (Z3 can be slow)

What You Should Build

For most LLM applications:

  1. Start with normal validation (simple, fast)
  2. Add Z3 for high-stakes parts (billing, access control)
  3. Use LLM for everything else (content, summaries)

Example Architecture:

User Query → LLM → Response
Is it high-stakes?
↙️ ↘️
YES NO
↓ ↓
Z3 Verification Return
Valid/Invalid

Last Updated: October 31, 2025