Where IT All Went Wrong

There was a time when a big IT project cost $5-10 million, now it’s north of $50m, big programmes run to hundreds of millions, even past a billion. At the same time, getting things done has gone from turning up and doing stuff to an insane bureaucracy where even a tiny piece of work can take weeks and involve vast numbers of people. How did this happen?

To some extent the cost increase is due to the ever-increasing scope and complexity of corporate systems, but that is by no means the whole story. I believe that there are two key, related, changes which have brought a substantial but hidden loss of productivity: the increase in communication span between individuals, especially those who need to collaborate closely, and loss of trust in experienced technicians. By communication span, I mean the ease with which people can work together, ranging from sitting next to each other and talking frequently to being separated by continents, time zones and contracts between companies. The loss of trust has been driven by the fragmentation of organisations and has led to jobs (or actually non-jobs) that used not to exist. These non-productive people I call Ladder Shakers who do nothing productive but actively obstruct people who do by constantly making them justify and get approval for what they are doing.

A company of any significant size inevitably divides itself into smaller units and this is a normal ‘growing pain’ as a company expands. These organisational units will have some form of boundaries between them, ranging from loyalty to one’s own department or division (office politics), through increasing levels of physical separation up to separation between continents and across time zones. The effect of these boundaries is to hinder the interactions between the people involved (interactions always occur between people, not organisations). Historically boundaries were largely geographical and line of business. Supporting functions such as IT, HR and Finance were embedded into entities such as refineries, factories and regional offices; these supporting functions were operated by staff directly employed by the company and physically, socially and culturally close to the entities they supported. This was inevitable because communication was slow and expensive; there was no video conferencing, mobile phones, instant messaging and so on. That meant the people had to sit near the people they were building or running systems for, which was not a bad thing.

From the 1990s on, improvements in communication technology enabled consolidation, outsourcing and offshoring as gullible or ambitious managers were persuaded by silver-tongued consultants to destroy their organisations on the promise of reducing costs. These changes that were designed to reduce costs and lower the management attention spent on IT systems did the exact opposite when you look at total cost to the organisation rather than a single unit in isolation. For example, centralised help desks look cheaper, but no-one ever looks at the costs of the time wasted by the poor sods on the other end of the phone navigating mind-boggling menus, then explaining the same problem to three people none of whom share their native language before they give up and phone a friend. The ladder shakers can show management lovely reports though, making them appear to be useful.

Fragmentation changed the boundaries. Each of these changes took the staff in supporting functions further away from the businesses they supported, physically, organisationally, socially. These functions now likely work in a different office, a different country, a different time zone, have a different native language and work for a different company. This matters, because each of these factors makes interaction less efficient which is why the promised benefits are not seen.

Complex organisations destroy trust. New artefacts such as contracts and service level agreements are brought in in an attempt to align the objectives of the different parties; people who used to sit next to each other are moved to different teams, even different companies. The resulting increase in communication span between people who need to work closely together also reduces trust; as each of them has less sight of the context within which the other works, they often put inaction down to incompetence, political games, even laziness. The erosion of the trust in technicians under the oversight of a single experienced manager has led to slow and burdensome authorisation processes. This has been exacerbated by the fragmentation of IT (and likely other functions) into specialist silos. Where once a multi-skilled team focussed on a common purpose could operate largely independently, many activities now span multiple silos and the interface to each silo has to be navigated to get things done. These interfaces manifest as portals, tickets, relationship managers and so on.

So that’s the problem. My aim with this blog is to go into this in a lot more detail then attempt to offer solutions.

While this blog focusses on IT organisations, many of the lessons will be applicable to other service functions such as HR and Finance.