The £800k AI System That Made Everything Worse
Every week, another company announces their "AI-powered decision-making solution." But most AI applications in decision-making are either solving the wrong problem or creating new problems they don't advertise.
A logistics company implemented an "AI-driven route optimization system" for £800k. The system was supposed to reduce delivery times by 15% and fuel costs by 20%.
Results after 6 months:
- Delivery times increased by 8%
- Customer complaints doubled
- Driver satisfaction plummeted
- Fuel costs stayed flat
What went wrong? The AI optimized for mathematical efficiency but ignored human factors: driver experience with local routes, customer preferences for delivery timing, and real-world constraints like traffic patterns and parking availability.
The algorithm was technically correct and practically useless.
Where AI Actually Helps Decision-Making
1. Pattern Recognition in Large Datasets
What AI does well: Identifying patterns humans can't see in complex data.
Real example: A retail chain uses AI to predict which products will need restocking. The system considers 47 variables (weather, local events, historical trends, social media sentiment) to make predictions that are 23% more accurate than human buyers.
Why this works: Clear success criteria (accuracy), abundant historical data, measurable outcomes.
2. Processing Speed for Time-Critical Decisions
What AI does well: Making consistent decisions quickly when time matters.
Real example: Financial trading algorithms that can process market information and execute trades in milliseconds. Human traders can't match this speed.
Why this works: Speed is the primary advantage, rules are well-defined, outcomes are immediately measurable.
3. Reducing Cognitive Bias in Routine Decisions
What AI does well: Applying consistent criteria without being influenced by irrelevant factors.
Real example: Resume screening AI that focuses on relevant qualifications without being influenced by names, photos, or university prestige.
Why this works: Human biases in routine decisions are well-documented, criteria can be made explicit.
Where AI Fails Spectacularly
1. Context-Dependent Decisions
The problem: AI systems struggle with context that isn't explicitly programmed.
Example: An AI recruitment system rejected all qualified candidates because it learned to prefer candidates who mentioned "baseball" or "basketball" in their resumes—sports more commonly mentioned by male candidates in historical data.
Why this fails: The AI optimized for historical patterns rather than actual job performance predictors.
2. Ethical or Value-Based Decisions
The problem: AI systems can't make value judgments or navigate ethical trade-offs.
Example: An AI-powered loan approval system consistently rejected applicants from certain neighborhoods, effectively perpetuating redlining practices even though it wasn't explicitly programmed to consider race.
Why this fails: Ethical decisions require human judgment about fairness, impact, and societal values.
3. Novel or Unprecedented Situations
The problem: AI systems fail when they encounter situations outside their training data.
Example: During early COVID-19, many AI-powered demand forecasting systems failed catastrophically because they had never encountered global pandemic conditions.
Why this fails: AI systems can't truly reason about new situations—they can only pattern-match to past experience.
The Reality Check Framework for AI Decisions
Question 1: "What evidence do we have that someone wants this AI solution?"
Often, AI projects are technology-driven rather than problem-driven.
Good evidence:
- Current decision-making process has measurable problems
- Human decision-makers are asking for this type of support
- Similar AI applications have proven ROI in comparable contexts
Bad evidence:
- "AI is the future"
- "Competitors are using AI"
- "We have lots of data so we should use AI"
Question 2: "What surprised us this week about how the AI performs?"
AI systems often behave unexpectedly. Regular surprise-detection helps catch problems early.
Monitor for:
- Decisions that are technically correct but practically problematic
- Performance degradation as conditions change
- Unintended consequences on human behavior
- Edge cases where the system fails
Question 3: "What are we pretending not to know about AI limitations?"
Many AI implementations ignore known limitations.
Common ignored limitations:
- Need for continuous retraining as conditions change
- Difficulty explaining AI decisions to stakeholders
- Dependence on data quality and representativeness
- Potential for amplifying existing biases
Practical Framework: When to Use AI vs. Human Judgment
Use AI When:
- Decision volume is high: Thousands of similar decisions needed
- Speed matters: Decision must be made faster than humans can process
- Consistency is crucial: Human variance is a bigger problem than AI limitations
- Pattern recognition: Complex data patterns that humans miss
- Bias reduction: Well-documented human biases in the decision domain
Use Human Judgment When:
- Context is crucial: Decisions require understanding of nuanced situations
- Ethics matter: Value judgments or fairness considerations
- Stakes are high: Consequences of wrong decisions are severe
- Novel situations: Unprecedented conditions or circumstances
- Stakeholder relationships: Decisions affect trust, morale, or relationships
Use Hybrid Approach When:
- AI provides analysis, humans make decisions: AI surfaces patterns and recommendations, humans make final choices
- Humans handle exceptions: AI makes routine decisions, humans handle unusual cases
- Multiple perspectives needed: AI and human judgment provide different valuable inputs
Red Flags in AI Decision-Making
Technical Red Flags:
- System can't explain its decisions
- Performance degrades over time without obvious cause
- High confidence in obviously wrong decisions
- Consistent failure in specific scenarios
Business Red Flags:
- Implementation focused on technology rather than outcomes
- No clear success metrics defined upfront
- Resistance from people who will use the system
- "Black box" system that can't be audited or understood
Process Red Flags:
- AI system implemented without testing on small scale first
- No human oversight or intervention capability
- Training data isn't representative of real-world conditions
- No plan for monitoring and continuous improvement
Case Study: Hybrid Decision-Making Done Right
Company: Insurance claims processing
Challenge: Process 10,000+ claims monthly, reduce processing time while maintaining accuracy
Hybrid approach:
- AI first screening: System automatically approves straightforward claims (60% of volume)
- AI-assisted review: System flags potential fraud or complexity for human review (25% of volume)
- Human-only review: Complex cases requiring judgment (15% of volume)
Results:
- Average processing time: 3 days → 1.2 days
- Accuracy rate maintained at 99.2%
- Customer satisfaction increased 18%
- Staff satisfaction increased (less routine work, more interesting cases)
Key success factors:
- Clear boundaries between AI and human decisions
- AI explainability for human reviewers
- Continuous monitoring and adjustment
- Staff training and buy-in
The Bottom Line on AI Decision-Making
AI is a powerful tool for specific types of decisions, but it's not magic. The most successful implementations combine AI efficiency with human judgment, rather than trying to replace human decision-making entirely.
Before implementing AI for decisions, ask:
- What specific problem are we solving? (Not "how can we use AI?")
- What evidence shows AI is better than current approaches for this specific problem?
- How will we know if the AI system is working correctly?
- What safeguards exist for when the AI system fails?
- How do we maintain human oversight and intervention capability?
Remember: The goal isn't to use AI—it's to make better decisions. Sometimes that involves AI, sometimes it doesn't.