In this series, Claire Leverne—outdoor rescue expert and engineer—shares insights that SREs can draw from rescue operations. Check out the previous parts of the series for more context:
When a search and rescue call goes out, it’s common for multiple teams to get dispatched.
Most will be a collection of volunteer and professional SAR organizations within responding distance—usually within three hours, depending on the scenario and remoteness of the location
But often, the responding teams are from altogether different organizations or disciplines: firefighters, sheriffs, dog teams, or even ad-hoc volunteers.
Working with many hands and multiple teams can have its own challenges and benefits. When a scenario is inherently high-stress, the difficulties (or payouts) are only amplified.
Teamwork Maketh the Dream Work
I don’t think I’m surprising anyone with the assertion that people accomplish more as a team.
We’ve known this since the cognitive revolution, when our species’ evolution into abstract thought and social constructs allowed us to collaborate and accomplish things heretofore undreamed of: from the mass extinction of terrestrial megafauna (way to go, Team) to the invention of the microchip.
Still, we can struggle on the daily when we’re asked to work with teams we’re not used to.
In the world of incident response, there are a myriad of benefits to cross-functional collaboration, but also a few red flags for which to watch.
Coordinating across teams can allow us to transcend to higher levels of efficiency; collaboration across teams increases the thoroughness at which we address a task.
On the flip side, there can be challenging interpersonal dynamics, or inefficiencies may develop from poorly defined workflows and tasks. In every multi-team effort, effective management is critical to proper utilization of these teams.
Coordination: Links in the Chain
In SAR operations, one way to think about a successful multi-team operation is as links in a chain. Similar to how an in-series circuit allows high-current flow, the chain model allows high levels of specialized energy at each step of the process: dog teams or air support are able to help with fine searches and point-searching of the lost hiker.
Wilderness First Responders step in next to provide medical response, and hand off the bundled patient to a technical high-angle extraction team (should it be needed). When the patient is transported to a trail, a fresh team of muscled and high-power support is ready for transport to a road, another team may be utilized for atv transport, and an ambulance of trained EMS professionals can take over transport at the road. Each team did a specific job that they were uniquely trained and suited to do, with a handoff to the next level of transport or care, for an efficient and masterful rescue operation.
For other forms of incident response, this ‘Links in the Chain’ approach is akin to an incident lifecycle, where different teams handle specific stages of an incident. Each team specializes in a particular aspect of the incident management, ensuring a seamless transition and efficient resolution.
Collaboration: Grid Searches
A totally different, yet effective utilization of multiple teams is to use a many-hands-light-work method of divvying up a colossal task into manageable pieces.
Often in SAR this looks like the deployment of grid searches. Grid searches are designed to very thoroughly cover land area when quick-search methods prove fruitless.
Not only is it useful to have many hands, but it’s useful to have many minds. In a multi-disciplinary collaboration, team members look at the same problem differently; a solution that may never have occurred to one team may be a total no-brainer to the next team.
Similarly in tech responses, large-scale incidents often require cross-functional collaboration. Resolving a complex outage could involve network engineers, database specialists, and application developers working together.
Incident management platforms help facilitate this collaboration by providing a centralized system where all responding teams can communicate, share insights, and coordinate their efforts to restore services swiftly.
Unified Command
No matter what approach is taken for deploying multiple teams – collaboration or coordination, or a mixture of the above – the truly essential component is an effective and central command structure to facilitate the operation. This includes planning, allocation of resources, communication, real-time tracking, and critical decision-making.
In SAR, this might be the Incident Commander overseeing all teams, while for reliability engineers, it could be the incident lead coordinating efforts across different engineering teams to ensure a cohesive response.
Be wary of 🚩
Disintegrating Existing Teams
Much of the function of a high-performing team comes at the social level. When social structures begin to disintegrate, the team can quickly become dysfunctional.
Good teams spend a lot of time building this dynamic, and – barring an unusual circumstance – it does little good to break apart a team ad-hoc to mix-and-match with other teams.
Fragmented teams can lead to confusion and miscommunication during incidents, slowing down the response and increasing the risk of errors.
Re-work and wheel-spinning
When several teams are working on the same problem, overlapping responsibilities or redundant work is a real risk, lowering overall efficiency of the mission.
Be careful to use a good management strategy to avoid redundant work while ensuring that each team is working on something meaningful and productive.
Establish clear incident scopes and role definitions to prevent overlapping responsibilities.
Conclusion
Collaboration and coordination across multiple teams are essential for both successful SAR operations and effective SRE incident response. By leveraging specialized teams and fostering strong communication channels, organizations can handle complex, high-stress scenarios with greater efficiency and resilience.
Whether coordinating the efforts of diverse rescue teams in the wilderness or managing cross-functional SRE teams during a system outage, play to the strengths of each team and the cumulative effort will be powerful and efficient.
Practical advice for adventurers: this one is about looking out for your neighbor. My time working with SAR has made me much more cognizant of other people I pass on the trail; for every lost hiker, there’s usually a handful of people who saw them throughout the day and had no idea they were in trouble. In the great outdoors, we’re all on the same team. When you see someone sitting by the trail, check in – ask them how their day is going.
Make good choices, and remember to pack snacks!