Evaluating software development is measuring the invisible

Dmitry Yakovlev, a software architect at DataArt, contributes a bylined article to VentureBeat, where he questions the findings of a recent McKinsey report on measuring efficiency in application development, and suggests a different set of metrics.

“Most of the project managers will admit that running IT projects without any metrics is evil, and I completely agree with them. However, I should note that all widely-adopted software development methodologies do have such metrics to ensure certain sanity level at least.

What surprises me is when the value an engineering team brings to business in measured in lines of code or story points. Having been working for a vendor providing custom software development services for a long time, I strongly disagree with associating the value a software team contributes to business with X use case points delivered over N months.

Why do organizations undertake application development projects? Because they feel pain or see new opportunities. Solving these problems defines the success of the project. The result is always a joint effort of the engineering team and business stakeholders. From such an angle, the most valuable skill of a development team is ability to collaborate with business…

…The values a good engineering team brings to the table.

  • Facilitating communications with stakeholders
  • Challenging business requirements
  • Creative thinking and proactively

DataArt, as a custom software service provider, has dozens of engineering teams running in parallel. There were a number of attempts made on elaborate metrics to measure how healthy the teams and clients were. No surprise, all story point-based approaches failed miserably, being not able to identify problematic projects. Finally, we arrived to a set of simple questions business stakeholders are asked when we need to assess how they value the outcome of the project:

  • Did project go live at the right time?
  • Do you think the team did all their best? Did you congratulate the team?
  • Would you undertake another project with the same development team?

View original article or download PDF