Search Explore

Dev productivity metrics suck. Ops reviews are key for AI-accelerated eng orgs

https://www.cortex.io/drive

gsdatta · 18 hours ago

4 comments

  • gsdatta · 18 hours ago

    Dev productivity metrics frameworks are (and always have been, IMO) bad for measuring organizational effectiveness. Operational Excellence reviews are the future of measuring and improving organizational health, and I wrote a paper making the case for this + a framework that helps make the OE review effective.

    • zihotki · 17 hours ago

      That sound interesting. Would you mind sharing it?

      • gsdatta · 15 hours ago

        Should be available for download at the link in the post!

      • gsdatta · 15 hours ago

        I'm the author – I spent 6 months writing this myself, and, if it helps, our product doesn't even do the stuff in the framework. I'm not sure what I'd be selling?

      • xutopia · 17 hours ago

        Their web site has a missing image.

        • CraigJPerry · 17 hours ago

          I'm trying to disentangle this from the established / proven / trusted "dx core 4" (ask your local devops person if you don't recognise the name).

          I found "initiatives" was added. What does this new initiatives measure bring. Why do we care about otherwise unqualified initiatives, how do i know that doesn't just mean using the other 4 proven measures as cover for pushing pet projects without merit?

          I'm super cynical tonight it seems. This is just rubbing me up the wrong way i guess and i can't really put my finger on why.

          • gsdatta · 15 hours ago

            I know I'm biased but Core 4 (and similar) rub me the wrong way – measuring individual developers as the atomic unit IMO is always meaningless. It's a proxy for organizational health but not directly correlated.

            Especially now with AI, what do metrics like "prs/engineer" even mean when you have background agents open/reviewing/releasing PRs without human intervention? what is the right unit for measuring health of the org?

            FWIW I write a lot more about "why not existing frameworks" in depth in the full paper.

            Initiatives are defined specifically as non-productive, technical leverage-producing initiatives that affect the org's health as a whole and are often left behind. For example, we recently ran an initiative around feature flag cleanups and full rollouts that we tracked religiously during our weekly OpEx review – without which we probably would not have had the same success with that cleanup initiative.

            I understand your cynicism with "yet another framework" but (and I know I'm biased) this framework is intimately tied to ops reviews as a mechanism for both measurement and organizational change.