Building for the Frontline: Human Machine Teaming
About 5 years ago, whilst we were all just starting to see the first hints of normality return to life, I remember being sat with a very select few around a whiteboard (socially distanced of course), looking to build what was then to be the 15 year strategy for realising human machine teams in the British Army. It was a room full of optimism and excitement, HMT felt like something that if we played our cards right could be an epoch shift in how we fought and define how battles were fought in 2035. We laid out a careful, calculated plan to deliver exactly that.
As is almost always the way, no sooner had we wrapped up our carefully researched plan, the character of the modern operating environment took a seismic shift. Human machine teams were no longer something that could be carefully nurtured, they were forming, fighting and evolving on our social media feeds right in front of us. They were no longer the future of warfare, they were a very present and immediate reality.
I reflect quite often on how it has unfolded. I still firmly believe that human machine teams are central to the future character of warfare. They have emerged rapidly through necessity, but I think we all have a part to play from industry to operator, in helping them evolve to deliver their full potential.
What's required to form an effective human machine team?
I think it's four key characteristics: competency, common understanding, trust and the appropriate division of labour. If a team thrives on positive behaviours, then the team characteristics are what makes the behaviours possible.
From a competency perspective, I think this often focused on the system itself. Is the system capable of doing the tasks with which it is going to be asked to do? This is fundamentally a technology maturity problem. The human element should not be underestimated, how capable is the user of getting the system to do the tasks in which it really excels. Common understanding captures how well the user understands the strengths, weaknesses and vulnerabilities of the system. Trust defines the likelihood a user is to afford the system delegated responsibility or decision agency in a given task. This is all rolled up into division of labour, in a strong team, the humans and machines have a well distributed workshare that directly reflects their competencies.
The characteristics are particularly interesting as they aren't independent, and they are to some degree sequential. Trust depends on competency from both sides, and the ability for the human to clearly understand the systems capabilities, only when we have trust can a division of labour approaching optimal can be defined. So that really means, we have three levers to build effective teams, system competency, operator competency and a clear articulation of the system competency to the user.
So how can we build to enable human machine teams?
I think too often it can be assumed that machine + human = team, but this is flawed. High performing human machine teams can hugely scale mass, capability and strength but setting the conditions for them to emerge takes careful planning, consideration and focus from all sides. I think there are considerations we can all make, to ensure our teams are the ones that create the advantage:
Software systems are machines too. When people envisage human machine teams they tend to imagine the Robocop version of the battle space where robots are operating alongside humans seamlessly. Whilst we're on that path (see the volume of UAS in UKR) we're a long way from that level of teaming. Software currently sits in the hinterland between human and physical machine, but software systems also have a large part to play in capability in general. Considering software as part of the team is really important in my view, and we could learn a lot from how we integrate the hardware into a team, which is sometimes lost in the software environment. Hardware has a rigorous user testing pipeline, test deployments and specialist users dedicating to improving the functionality as part of the trials process. This doesn’t always happen to the same level in software and is to the detriment of the human machine team.
Exposing weakness is a strength. Soapbox number 2… I’ve personally felt these pressures, especially in AI. To trust a system, users need to understand its strengths and weaknesses. Industry excels at identifying strengths, but exposing weaknesses is counterproductive. This can lead to a dangerous misconception that models can perform everywhere. This is false; models will always fail somewhere. Understanding this allows users to tailor the division of labour accordingly. Without trust, unpredictable performance can collapse the team. Exposing model weaknesses, software, or platform weaknesses shouldn’t be negative; it increases trust. Both customers and industry must remove the taboo of weaknesses. Buyers should encourage and reward exposure, and sellers should openly offer it. If we don’t, we’ll always be behind.
Explain, explain and explain again. Another point where I am not sure we pay as much attention in the operational environment to AI software as we do hardware, is that of human factors, particularly in justifying why or how a specific prediction was made. I find this particularly challenging, as if a model is providing a prediction, it triggers biases and heuristics in a human, meaning accepting that decision is far more likely. In some instances, that might be fine, but if the system isn’t explaining how a decision was made, how do you really know what the model considers? It boils down again to having a common understanding of capabilities and building trust. I fully appreciate that there are explainability limitations in some systems, but there are still things that can be done. I think this should be the standard, not the exception.
Trust is a function of trustworthiness. Trust is crucial for any team’s operation, but it’s a challenging concept to articulate. There’s often an assumption that more trust is better, but that’s an oversimplification. Too much unwarranted trust can lead to dangerous deployments, while too little can cause underutilisation. The optimal level of trust is determined by the system’s capability and performance in a given situation, or its trustworthiness. This hidden truth is only understood through interaction and experience. Users’ representation of this truth is trust. If this is correct, the objective of trust is to make it best reflect the system’s trustworthiness, which is optimal trust. Focusing on the factors mentioned helps curate it.
Looking back at that small, socially distanced group around a whiteboard, it’s clear we’ve made significant progress. Human-machine teams have evolved from slides and sketches to systems that shape our operations. However, we must remain committed to securing enduring operational advantage. This requires a relentless focus on performance, of the machines, people, and teams they form. This demands a long-term perspective and a willingness to prepare for tomorrow’s challenges today.
-- Tom Scott.
