"Five years of engineering management experience" reads like a qualification. It appears in job postings as a filter, at the top of CVs and in promotion cases as evidence. What it records is a duration: how long someone held the seat. It says nothing about what they did there or how good they became.
Take two managers. One has spent five years running a stable team at a company with a predictable roadmap, few incidents and low attrition. Most decisions are low-stakes and mistakes are easy to absorb. The other has spent eighteen months leading a team through a reorganisation, two serious production incidents and a round of layoffs, making consequential calls under pressure most weeks. On paper the first has more than three times the experience. We'd expect the second to have the sharper judgment, and a tenure line can't tell them apart.
Exposure and judgment are different things
Years in role measure exposure. Someone can run the same 1:1s, approve the same pull requests and sit in the same planning meetings for years without meeting a hard trade-off, a difficult performance conversation, or a moment where the safe answer and the right one point in different directions. They accumulate tenure without being tested. Someone with less time, dropped into a hard situation, may develop more judgment in a year because they had to use it.
Experience does matter. Over a career it tends to go with skill, and nobody sensible would hire a director who has never managed anyone. The problem is that it's a noisy, indirect measure of what an organisation wants to know, which is whether this person makes good calls when it counts.
The selection research points the same way. Frank Schmidt and John Hunter's 1998 meta-analysis in Psychological Bulletin, drawing on 85 years of studies, found years of job experience to be one of the weaker predictors of job performance, with a validity of about .18. Structured interviews came in at about .51 and work-sample tests at about .54. Their review covered jobs in general rather than engineering management, so treat it as direction rather than a precise figure for this role. The direction is clear enough.
How it plays out in hiring
A posting that requires "5+ years of engineering management experience" is a blunt instrument. It screens out people who developed strong judgment quickly, who are often the ones worth hiring, and lets through people whose five years were spent in a setting that never stretched them.
Hiring teams reach for the threshold because tenure is easy to verify and judgment is hard to see on a CV. That's a reasonable instinct for a first screen. It goes wrong when the threshold becomes the evidence, and the organisation is surprised a year later that the manager with the long record struggles with the parts of the job the record never tested.
What to look at instead
Leadership judgment shows up in specific, observable places: how someone handles an ambiguous trade-off, gives and takes feedback, sets technical direction without full information, supports a person through a bad stretch, and explains a decision to stakeholders who don't share their context. The Body of Knowledge sets these out across five domains, and none of them needs a tenure count to assess.
The practical version is a realistic scenario. Describe a situation the role will produce (a strong engineer whose behaviour is wearing the team down, a date the team won't hit) and ask the candidate what they'd do and why. Ask every candidate the same scenarios and score their answers against a rubric agreed beforehand, before the panel compares notes. That is the structured interview, and it works the same whether the candidate has managed for eighteen months or eighteen years.
Promotion has the same trap. The useful question about readiness is whether the person is already exercising the next level's judgment on real stakes. How long they've waited, or how much they want it, is a poor substitute.
Where this sits in the standard
Hiring is competency PL-3 in the EMA Competency Framework, and the PL-3 unit covers structured, calibrated evaluation and Schmidt and Hunter's findings. Judging readiness by demonstrated judgment rather than tenure is part of PL-5, career development.
The Academy's own levels use years as a recommendation only. EMA-I has no experience requirement; EMP-II recommends three or more years leading teams and EME-III seven or more, and neither checks. Each credential is awarded on a scenario-based examination. Compare the three levels.