Your Questions Answered

1. Do we need models for every hallucination scenario?

NO! You only use automated reasoning for specific high-stakes domains.

❌ Don’t Use Z3 For:

  • Creative writing
  • Summarization
  • General Q&A
  • Content generation
  • Most typical LLM tasks

✅ DO Use Z3 For:

  • Security policies (AWS IAM, access control)
  • Financial calculations (pricing, invoicing, accounting)
  • Legal compliance (EU AI Act, regulations)
  • Safety-critical systems (medical, aviation)
  • Mathematical proofs

2. Why Z3 vs Normal Validator Class?

The Key Difference

Normal Validator: Checks specific values

class PricingValidator:
def validate(self, original, discount, final):
expected = original * (1 - discount / 100)
return final == expected # Just checks THIS value

Problem: Only checks the specific value you give it.

Z3: Proves for ALL possible values

from z3 import *
def prove_pricing_formula_is_correct():
"""Proves the formula works for ALL inputs"""
original = Real('original')
discount = Real('discount')
final = Real('final')
solver = Solver()
# The formula we claim is correct
solver.add(final == original * (1 - discount / 100))
# Add constraints
solver.add(original >= 0)
solver.add(And(discount >= 0, discount <= 100))
# Try to find a counterexample
solver.add(final < 0) # This should be impossible
if solver.check() == unsat:
return "✅ PROVEN: Formula never produces negative prices"
else:
return "❌ FOUND COUNTEREXAMPLE!"

Z3 proves it works for ALL values, not just test cases.


Real-World Example: When You NEED Z3

Scenario: AWS IAM Policy Validation

Problem: Does this policy accidentally grant admin access?

{
"Effect": "Allow",
"Action": ["s3:*"],
"Resource": "arn:aws:s3:::my-bucket/*",
"Condition": {
"IpAddress": {
"aws:SourceIp": ["203.0.113.0/24", "198.51.100.0/24"]
}
}
}

Normal Validator (Won’t Catch Bug):

def validate_policy(policy):
# Check specific test cases
assert has_ip_condition(policy)
assert action_is_scoped(policy)
# Misses: What if someone adds another condition?

Problem: You can only test cases you think of!

Z3 Approach (Catches All Cases):

from z3 import *
# Model the policy as logical formula
def verify_policy_never_grants_admin():
# Variables
action = String('action')
ip = String('source_ip')
resource = String('resource')
solver = Solver()
# Policy rules (encoded as logic)
solver.add(Or(
action == "s3:GetObject",
action == "s3:PutObject"
))
solver.add(Or(
PrefixOf("203.0.113.", ip),
PrefixOf("198.51.100.", ip)
))
# Try to find case where admin is granted
solver.add(action == "iam:CreateUser") # Admin action
if solver.check() == unsat:
return "✅ PROVEN: Policy never grants admin access"
else:
return "❌ SECURITY HOLE FOUND!"

Z3 proves it’s impossible to get admin access, for ALL combinations.


3. Dynamic Constraints Based on Context

YES! Z3 can handle dynamic constraints.

Example: Restaurant with Variable Rules

from z3 import *
class DynamicConstraintValidator:
def __init__(self):
self.solver = Solver()
def validate_restaurant(self, people, dishes, custom_rules):
"""
Validate with context-specific rules
custom_rules example:
{
"Alice": {"allergies": ["dairy", "gluten"]},
"Bob": {"diet": "vegetarian", "exclude": ["salmon"]},
"Carol": {"copies": "Bob"}
}
"""
# Create variables dynamically
assignments = {
person: Int(f'{person}_dish')
for person in people
}
# Add dynamic constraints based on context
for person, rules in custom_rules.items():
# Allergy constraints (dynamic based on person)
if "allergies" in rules:
allowed_dishes = [
dish_id for dish_id, dish in enumerate(dishes)
if not any(allergen in dish["contains"]
for allergen in rules["allergies"])
]
self.solver.add(Or([
assignments[person] == dish_id
for dish_id in allowed_dishes
]))
# Diet constraints (vegetarian, vegan, etc.)
if "diet" in rules:
if rules["diet"] == "vegetarian":
veg_dishes = [
i for i, d in enumerate(dishes)
if d["vegetarian"]
]
self.solver.add(Or([
assignments[person] == i
for i in veg_dishes
]))
# Dependency constraints (eat same as someone)
if "copies" in rules:
other_person = rules["copies"]
self.solver.add(
assignments[person] == assignments[other_person]
)
return self.solver.check() == sat
# Usage with different contexts
validator = DynamicConstraintValidator()
# Context 1: Just allergies
result1 = validator.validate_restaurant(
people=["Alice", "Bob"],
dishes=[...],
custom_rules={
"Alice": {"allergies": ["dairy"]}
}
)
# Context 2: Complex rules
result2 = validator.validate_restaurant(
people=["Alice", "Bob", "Carol"],
dishes=[...],
custom_rules={
"Alice": {"allergies": ["dairy", "gluten"]},
"Bob": {"diet": "vegetarian"},
"Carol": {"copies": "Bob"}
}
)

The constraints are built dynamically based on context!


When Z3 Actually Matters: Real Examples

1. Financial System (Stripe-like)

Problem: Verify pricing calculation is correct for ALL possible inputs

# Normal validator - only checks specific cases
def validate_invoice(subtotal, tax_rate, discount, total):
expected = (subtotal - discount) * (1 + tax_rate)
return abs(total - expected) < 0.01 # Checks ONE case
# Z3 - proves formula is correct for ALL cases
def prove_invoice_formula():
subtotal = Real('subtotal')
tax = Real('tax_rate')
discount = Real('discount')
total = Real('total')
solver = Solver()
solver.add(total == (subtotal - discount) * (1 + tax))
# Business rules
solver.add(subtotal >= 0)
solver.add(And(tax >= 0, tax <= 1))
solver.add(And(discount >= 0, discount <= subtotal))
# Prove total is never negative
solver.add(total < 0)
if solver.check() == unsat:
return "✅ PROVEN: Formula never produces negative invoices"

Why it matters: In a financial system, you can’t just test - you need mathematical proof.


2. Access Control (AWS-like)

Problem: Does combining 5 different policies accidentally grant unauthorized access?

# Normal validator - can't handle policy combinations
def check_access(user, action, resource):
# You'd have to enumerate all combinations
# Misses edge cases!
pass
# Z3 - proves no unauthorized access is possible
def verify_no_unauthorized_access():
# Model all policies as logical formulas
# Z3 searches ALL possible combinations
# Returns proof or counterexample
pass

Why it matters: Security bugs in policy combinations are hard to catch with testing.


3. Medical Device Dosage Calculator

Problem: Verify drug dosage calculation is safe for ALL patient weights/ages

# Normal validator - dangerous to only test specific cases!
def validate_dosage(weight, age, dose):
max_safe = weight * 0.5
return dose <= max_safe # Only checks THIS case
# Z3 - proves dosage is ALWAYS safe
def prove_dosage_always_safe():
weight = Real('weight')
age = Real('age')
dose = Real('dose')
solver = Solver()
# Dosage formula
solver.add(dose == weight * 0.5 * (age / 100))
# Constraints
solver.add(And(weight >= 0, weight <= 200)) # kg
solver.add(And(age >= 0, age <= 120)) # years
# Prove dose never exceeds safe limit
solver.add(dose > weight * 0.5)
if solver.check() == unsat:
return "✅ PROVEN: Dosage formula is always safe"
else:
model = solver.model()
return f"❌ DANGEROUS! Counterexample found: {model}"

Why it matters: Lives depend on this being correct for ALL cases, not just tests.


Bottom Line: When to Use What

ScenarioUse Normal ValidatorUse Z3
Blog post generation✅❌
Summarization✅❌
Customer support chat✅❌
Financial calculations❌✅
Security policies❌✅
Safety-critical systems❌✅
Legal compliance❌✅
Simple data validation✅❌
Business logic✅ (usually)✅ (if high-stakes)

The Real Value: Z3 Finds Bugs You Didn’t Think Of

Example: Bug in Restaurant Riddle

What if we had this constraint:

“If Alice eats fish, then David must eat vegetarian”

Normal validator:

def validate(assignments):
if assignments["Alice"] == "Grilled Salmon":
assert is_vegetarian(assignments["David"])

You’d only test the cases you thought of!

Z3 automatically finds:

  • What if David is already committed to meat?
  • What if Alice’s only option is fish?
  • What if this creates an impossible situation?
solver = Solver()
# Existing constraints
solver.add(alice_dish == 2) # Must eat salmon (only option)
# New constraint
solver.add(Implies(
alice_dish == 2, # If Alice eats salmon
Or(david_dish == 1, david_dish == 3, david_dish == 4) # David vegetarian
))
# David must be unique
solver.add(david_dish != bob_dish) # Bob has dish 1
solver.add(david_dish != carol_dish) # Carol has dish 1
if solver.check() == unsat:
print("❌ IMPOSSIBLE! Constraints contradict each other")
# Z3 found the bug automatically!

Summary: Answers to Your Questions

1. Do we need models for every situation?

NO! Only for:

  • Financial/billing
  • Security/access control
  • Safety-critical (medical, aviation)
  • Legal compliance

2. Why Z3 vs normal class?

Z3 proves for ALL cases, normal class only checks specific values.

FeatureNormal ClassZ3
Checks specific values✅✅
Proves ALL values❌✅
Finds solutions automatically❌✅
Detects impossible constraints❌✅
Simpler code✅❌

3. Dynamic constraints?

YES! Build constraints at runtime based on context. See example above.


When You Should Actually Use This

Use automated reasoning when:

  1. ✅ Stakes are high (money, security, safety, legal)
  2. ✅ Complexity is high (many interacting rules)
  3. ✅ You need mathematical proof, not just testing
  4. ✅ Edge cases are hard to predict
  5. ✅ Regulatory compliance requires it (EU AI Act, SOC2, etc.)

Don’t use it when:

  1. ❌ Simple validation is enough
  2. ❌ Creative/subjective output
  3. ❌ Low stakes
  4. ❌ Performance is critical (Z3 can be slow)

What You Should Build

For most LLM applications:

  1. Start with normal validation (simple, fast)
  2. Add Z3 for high-stakes parts (billing, access control)
  3. Use LLM for everything else (content, summaries)

Example Architecture:

User Query → LLM → Response
↓
Is it high-stakes?
↙️ ↘️
YES NO
↓ ↓
Z3 Verification Return
↓
Valid/Invalid

Last Updated: October 31, 2025