When Leaders Don't Believe Their Own Data

• 5 min read
Data & Analytics Leadership Employee Experience

I was in a boardroom last month looking at employee engagement scores that any HR leader would envy. Overall engagement: 78%. Manager effectiveness: 82%. Career development satisfaction: 75%. The numbers painted a picture of a thriving organization.

But the CEO kept shaking his head.

“These numbers don’t match what I’m seeing,” he said. “People seem stressed. Our best performers are leaving. Something’s not adding up.”

He was right to be skeptical. The data looked good, but the organization was struggling. This disconnect happens more often than you’d think, and it reveals something important about how we measure employee experience.

The Problem with Looking Good on Paper

Most employee surveys are designed to produce data that looks reasonable to executives. They ask questions that people can answer positively without being dishonest, even when their actual experience is problematic.

Consider these common survey questions:

  • “I have the resources I need to do my job effectively”
  • “My manager communicates expectations clearly”
  • “I understand how my work contributes to company goals”

Employees can honestly answer “agree” to all of these while still being deeply frustrated with their workplace experience. They might have resources but not the right ones. Clear expectations that constantly change. A sense of purpose coupled with exhaustion.

The result? Data that technically tells the truth while missing the real story.

What the Numbers Miss

In my follow-up interviews at this organization, I discovered what the survey hadn’t captured:

The Pace Problem: While people felt they had clear goals and adequate resources, the pace of change was unsustainable. New priorities emerged weekly, making it impossible to do anything well.

The Innovation Paradox: Employees felt supported to share ideas but noticed that the same types of solutions were always chosen. Risk-taking was encouraged in theory but punished in practice.

The Growth Ceiling: Career development opportunities existed, but they required relocating to headquarters. For the 70% of employees in regional offices, growth meant leaving.

None of these nuances showed up in the survey data because the questions weren’t designed to uncover them.

The CEO’s Gut vs. The Dashboard

When leaders say “the data doesn’t match what I’m seeing,” they’re usually picking up on patterns that traditional engagement surveys miss:

  • Energy levels: People can be engaged but exhausted
  • Retention risk: High performers can score positively while actively job searching
  • Innovation potential: Teams can feel supported but still play it safe
  • Resilience: Organizations can function well under normal conditions but struggle when stressed

These are the signals that show up in hallway conversations, meeting dynamics, and the quality of questions people ask during all-hands meetings. They’re harder to quantify but often more predictive of future performance.

The Measurement Trap

Here’s what I’ve learned: organizations often measure what’s easy to measure, not what matters most.

Employee engagement surveys excel at capturing attitudes about established organizational processes. They’re much worse at revealing:

  • Unintended consequences of well-intentioned policies
  • Cultural dynamics that people can’t or won’t name directly
  • Early warning signs of problems that haven’t fully emerged yet
  • The gap between espoused values and actual experience

This is why purely quantitative approaches to employee listening miss so much of the story.

A Better Approach

The most insightful leaders I work with don’t abandon survey data—they triangulate it with other sources:

Qualitative Deep Dives

Regular focus groups and listening sessions that explore the “why” behind survey responses. Not just “Do you feel supported?” but “Tell me about a time when you didn’t feel supported and how that affected your work.”

Leading Indicators

Tracking metrics that predict future problems: project timeline changes, meeting frequency, cross-department collaboration patterns, informal network changes.

Exit Interview Evolution

Instead of asking departing employees why they’re leaving, ask them what would need to change for someone like them to stay and thrive.

Leadership Observation

Encouraging executives to spend structured time with different parts of the organization, not just their direct reports. What questions do people ask? What do they avoid talking about? How do they respond to new initiatives?

What This CEO Did

Back to my client: instead of dismissing the survey data, we used it as a starting point for deeper investigation.

We identified the teams with the highest engagement scores and spent time understanding what made them different. We looked at the gap between stated values and resource allocation. We tracked how decisions moved through the organization and where they got stuck.

What we found was an organization with good intentions and broken execution. People felt positive about their managers and clear about their goals, but the system wasn’t set up for them to succeed at the pace leadership expected.

The engagement scores weren’t wrong—they were incomplete.

The Real Question

When your gut tells you something’s off despite positive data, don’t ignore either signal. Instead, ask: “What would we need to know to understand the full story?”

Sometimes great engagement scores mask organizational fragility. Sometimes they reflect genuine strength in some areas while problems brew in others. Sometimes they reveal that people are adapting to dysfunction so well that it looks like health.

The goal isn’t to get better survey scores—it’s to build organizations where people can do their best work sustainably. That requires looking beyond the dashboard to understand what’s really happening in the day-to-day experience of your people.

Your employees are already telling you what they need. The question is whether you’re listening with the right tools.