An agile culture clash and the ambidextrous organisation

The public sector is capable of bringing great stability to the way our country operates. However, the very culture that produces this stability is terrible at working in an agile manner. One solution is to create two distinct working cultures (supported from the top) as a way to foster innovation without it getting stymied from traditional ways of thinking.

Our public sector institutions have a cultural leaning towards formality, stable processes, and a power-based hierarchy (where a leader’s authority is absolute). This approach provides consistency for government and education at local levels. This culture has also allowed nations to become empires and for armies to operate over vast distances.

A true organisational culture is revealed in unspoken norms and rules. These are often hard to see by employees because they are the underlying assumptions for how things are done. Things that are easy to observe might be a culture of presenteeism or a certain dress code.

It is the internal norms which are harder to spot. In the public sector, this can be the assumption that:

there is one right way to do something, we just need to find it
power-based hierarchy is vital to effect change
if I complete my tasks each day, I’m doing a good job
we can control the future if only we follow the processes correctly

These internalised rules and norms encourage conformity. While this keeps things steady, it also reduces (almost to zero) the mindset which discovers and invents new approaches to problems. Anyone innovating in this environment will feel the pain of working against the status quo.

Have we succeeded in recruiting only non-thinkers into our public sector institutions? Absolutely not! Many of those who work in the public sector will have incredibly rich, expressive, and inventive interests outside of their work. It’s that moment when they enter the office each day where they feel a pressure to switch off the creative aspects of themselves. The reduction in self means that we’re getting only the procedural and conformist aspects of their being – a huge loss in my opinion.

So, with a culture where the momentum is always towards the status quo, our public sector has been able to provide a relative level of stability to its citizens while inadvertently stifling innovation.

Innovation in a conformist culture

Power-based hierarchy

With any power-based hierarchy, the broad concept is that the thinking happens at the top, and the doing at the bottom. The limitation here is that the thinkers at the top don’t have the time to immerse themselves in all of the complexity involved in digital – technology, users, analytics, service design – let alone how this all hangs together.

Beyond their time constraints, those at the top generally can’t cope with the approaches that digital requires to work. The fluidity, iterations, ambiguity, and lack of paperwork seems like chaos to them. These leaders got to their positions because they (on some level) prioritise the idea of process, control, and conforming to organisational structures. Anything else will feel deeply uncomfortable to them.

I’m being careful to use the phrase power-based hierarchy because not all hierarchy is detrimental. When it comes to being an authority on a particular topic, we have natural hierarchies between people. And, this comes to the fore in an agile team, where people take turns to lead in different areas. A generalised, power-based hierarchy is often too broad, where those at the top have the power to dictate on almost all subjects.

On digital projects, with the thinkers at the top, we develop a significant communication overhead in getting alignment between users’ needs and leadership. Reports need to be written, questioned, refined, signed off, handed over, digested, then translated into a solution. By the time the product is finished, we’re simply developing something that will adhere to the vision of a senior person – who spends relatively little time with those that will use the product. We fall into the trap of designing to the idea that a senior person has of a user, not the real user. What a wasted opportunity, let alone a waste of resources.

Big-bang projects

Not everything is suited to agile. At least, I’m not aware of large infrastructure projects being run in a completely agile way. Big-bang projects, however, are problematic when it comes to initiatives with lots of unknowns, and where the cost of change is relatively low, i.e. digital.

Taking an agile approach means admitting that we don’t know what the best solution is from the outset. We take the time to discover, prototype, iterate, and test to check that what we learn is working. The alternative, waterfall (big bang) projects require us to over predict the solution, often defining and building features that are not heavily used once launched – à la MS Office.

The danger with waterfall projects in the public sector is that they are long, drawn-out, and politically infused. Processes are being reviewed, broken, and argued over while the outside world moves on. Opportunities are missed due to bureaucracy, maintaining the power of the organisation chart, and the need to work within annual budgetary cycles. The people that suffer are the communities we serve and the civil servants doing the work, who go home discouraged due to lack of progress.

Continuing to plough energy and resources into projects that produce little value, or little value for money, simply doesn’t make sense. However, it’s the natural output from a culture seeking certainty and a level of control over the future. After all, planning ahead is the breakthrough that made this culture so revolutionary when it first emerged.

Digital won’t settle down after a bit

Believing that normality will resume at some point can be limiting, particularly when it won’t happen. The level of change and complexity in our working lives is increasing. We need to develop the skill of navigating complexity. We shouldn’t try to reduce or control it.

I often come across people who see ‘digital transformation’ as a thing that will be unsettling for a while, then things will settle down once we are all digital. This attitude makes sense in a world where adopting a change to tax legislation, clinical standards, or curriculum implies a period of adjustment followed by business as usual.

Digital (by most definitions) brings more than a technological change. It brings with it a new way of viewing the world, a new approach to working with, rather than controlling reality. Digital isn’t just a phase of adjustment we’re going through – change is the new norm. Get used to ambiguity, uncertainty, testing, improving, and iterating.

This new paradigm represents an existential threat to the traditional ideas of a best way and fixed processes.

Thinkers and doers

I would argue that we need doers and thinkers in the same person. This reduces the need for reporting, handover, and translation between individuals. An employee who has met with your audience and interviewed them, will make excellent (intuitive) decisions during the design and build phases of a project. They should be involved throughout. Think of the time and effort saved if they can just get on with it, make mistakes, and improve the product over time. These people should be free from the constraints inherent to traditional project-management approaches. An interesting phrase used by the department for work and pensions is “trust and verify.” This seems to support the need for checks without preventing people from acting. It seems a little paradoxical, as verifying doesn’t imply trust, but it could be seen as a pragmatic approach in the culture we’ve been describing. The transparency in agile methodologies helps support verification.

The downfall of the power-based hierarchy is that the talent you need will simply leave. A thinker and doer will not enjoy adopting ideas from above without discussion – particularly when they've spent time with real users, analysed the objectives of the organisation, and started building a product. Suddenly, someone at the top has an opinion, and regardless of its merit, they have to treat this new requirement as top priority.

This arrangement makes no sense to someone capable of the research, analysis, design, and build of a product. When you expand this issue to a whole team, their morale will constantly suffer in this clash of worldviews.

There is a solution to this culture clash, and it’s been working in both the public and private sector. It allows for new ways of thinking to exist alongside the work of the established organisation. The terms that’s used is the ambidextrous organisation. Leadership creates a new part of the organisation where a different culture can exist, where those involved have permission to act in a new way.

The ambidextrous organisation

In many ways you’re creating a bubble. Inside this space exists whatever culture, processes, and ideas needed to meet a particular aim. In the the case of innovation/digital, we need a culture that allows for failure, trial, not-knowing, radical transparency, and trust.

Trust is an interesting point to consider, because most traditional processes exist to circumvent the need for trust. In addition to the emotional toll this lack of trust takes, it also creates a lot of overhead in terms of procedure. Broadly speaking, the incumbent approaches towards work come from a place of fear – what if something goes wrong? Newer working cultures are designed to build trust between people working together. We can look at agile as 'people over process’.

Examples of organisations taking this approach are the Department for Work and Pensions, GDS, and British Gas. The Harvard Business Review published a good article on this, which goes into a good level of detail.

One danger to highlight is that existing processes can easily destroy the new model. For example, the digital team may be asked to supply a 12-month plan for which features will be delivered for online services. The team probably don’t know what they’ll be doing in 12 months, so they could be tempted to create a pseudo plan to satisfy the bureaucrats. This then becomes a rod for their own back when someone assesses them against it further down the line.

The need here is for leadership to safeguard the concept of the two cultures working alongside each other. To stop the established approach from taking over this fledgling idea. Without conscious effort, the new culture and approach will be eroded to the extent that it is indistinguishable from the rest of the organisation.

Leaders should also support efforts of transparency from the digital team to communicate their work. The transparency inherent in agile/scrum methodologies should work to soften any disquiet from the rest of the organisation. The team may seem to work in a strange way, but everyone can all see their priorities and progress because they are available for all to see.

Above all, leadership needs to have the courage to not fall back on traditional approaches when it all goes wrong. When an issue arises, it is tempting to blame the new methodology. When a waterfall project goes wrong, we check whether the ideas and processes were followed correctly. We don’t throw out the whole methodology. Protect new ways of working, don’t throw them out the first time a challenge appears.

In conclusion

It’s not realistic to believe that public-sector organisations are capable of changing a culture which inherently resists change. A culture which gravitates towards the status quo will not suddenly work in a more agile manner. Think about Yes, Minister. The premise of that series was to demonstrate the skills and culture of the UK civil service to prevent change (for good or ill).

We need to embrace a new way of working in order to develop new and affordable services. The approach that seems to be working is the idea of the ambidextrous organisation where the old and new cultures don’t clash, they co-exist.