BACKGROUND

Among the principal responsibilities of the U.S. Army’s Combat Capabilities Development Command/Data and Analysis Center (CCDC/DAC) is supporting many Army test and evaluation (T&E) programs.These programs come in a wide variety of forms and may attempt to characterize the effectiveness, safety, reliability, lethality, vulnerability, survivability, and/or susceptibility of relevant combat systems. DAC’s T&E responsibilities include planning, executing, and assessing the results of associated testing and increasingly include the performance of modeling and simulation (M&S) work for T&E leverage.

One common type of DAC-performed M&S exercise is the survivability/vulnerability/lethality (SVL) analysis. A typical vulnerability-focused SVL analysis contemplates a single encounter of a target system and a specific threat under controlled conditions and predicts the outcome from an SVL perspective. Modeled threat disciplines may include ballistics, electromagnetic warfare, non conventional threats, cyber attacks, and others.

SVL analyses tend to be relatively granular (i.e., initial calculations [analytic, numerical, or empirical] focus on determining the probability of dysfunction occurring at the component level). This probability is often smoothed over a domain such as threat incidence angle. Quantification of system-level “kill,” or loss, occurs via consulting with a damage assessment list (DAL). The DAL provides an indication of how component dysfunction correlates to the kill of a system’s broadly-defined mobility, firepower, or overall availability. Kill results are then combined or averaged, as appropriate, and reported out as the product of the analysis.

Several drawbacks to the single-system, DAL-based approach have motivated our current work on developing the system capabilities analytic process (SCAP) in the context of DAC’s Advanced Teaming Analysis Concept (ATAC).

DAL output often fails to capture the “so what?” of the results. More so than a holistic loss-of-function description, whether a damaged system retains one or more particular capabilities is often the information of greatest interest to evaluators, program managers, downstream analysts, and other customers. Additionally, using  a single quantity for a broad category of capabilities (e.g., firepower) often elides the differences between distinct capabilities within that category. (A 0.6 firepower kill could mean many different things in the context of a vehicle with multiple weapons.) These are serious handicaps for determining how a system will operate.

Another issue is that traditional SVL analyses tend to consider the target isolated from operational context. This approach is straightforward but leaves factors such as emergent capabilities and redundancies of the teamed assets, the effects of other encounters within the mission scenario, and miscellaneous effects (reliability issues, operator error, logistical problems, or even environmental factors) ill considered.

These drawbacks limit the ability of DAC analysis products to transition from answering narrow questions about SVL performance in controlled conditions—the direct focus of much of T&E support—to becoming more broadly applicable in modeling complex engagement scenarios. As the importance of modeling multi-domain and teamed systems acting in concert gains increasing acceptance in the Army, approaches that tend to isolate the system in question become less relevant. SCAP development is therefore intended to affect a better correspondence between DAC analyses and the M&S requirements of our partners and customers.

THE SCAP ANALYSIS AND ATAC IMPLEMENTATION

ATAC is DAC’s effort to bring a capabilities-based analysis process
to the complex problem of assessing teamed assets. DAC is employing SCAP [1] for team-centered analyses because it is seen as inherently suitable; many collective characteristics of a teamed force are difficult to define solely in terms of individual actors. Work on the ATAC program was performed in fiscal year (FY) 2019 by a multidisciplinary team of DAC engineers: Stephen Abbott, Kevin Agan, Andrew Drysdale, William Landis, SFC Tonio Pearce, and Gina Schafer [2].

When applied to ATAC implementation, the SCAP methodology requires three kinds of input data for model processing:

1. The functional tree, a bidirectional map between individual components and the capabilities they enable.
2. The event script, which defines the mission scenario and dictates which functionality changes will occur, when, and under what conditions.
3. The status updater, which handles contingency conditions in the event script and the calculation of additional state values (e.g., battery power consumption) at each time step.

The SCAP functional tree is a set of entities (grouped into components/subsystems, functions, and capabilities) that serves to describe how individual components are combined to affect
a system’s capabilities. By way of example, a portion of a tree is shown in Figure 1. Critical components, and the subsystems they comprise, are the lowest-level entities. Functions are intermediate entities defined by a unique combination of required components or other functions. Conceptually, a function is a “unit of accomplishment” that is observable, measurable, and not normally considered an end unto itself, such as lubricant regulation or power supply.

High-level entities are called capabilities; these are, in turn, defined by a combination of required functions. A capability represents a complete action, such as identifying a target or communicating with base. The most overarching capabilities are sometimes defined by a set of other capabilities, often involving more than one system, and representing complicated actions such as “engage armored enemy.”

Because most groups of components function in series (e.g., a drivetrain requires each of its components working in turn), most functions and capabilities in a SCAP model require the availability of each of their constituent entities. Thus, the great majority of functional tree connections use exclusively “AND” relationships, which are shown in Figure 1 as arrows.

For ATAC, DAC implemented functional trees as signal-processing models in MATLAB’s Simulink module. To aid with editing, the unified tree for the team was split into two steps—one tree maps component/subsystem availability to function availability, and a second tree maps that output to capability availability. The output of the second tree is the updated capability status for the teamed systems.

Figure 2 shows a simple example of how the subsystem-to-function implementation works in practice. The availability of components, grouped into subsystems, is input via the data tags (yellow) to the left. Those availability values are separated and then logically combined to form the availability of the function. A final input (lower left) is a “virtual” entity; it is required to be “on” (the default) to satisfy the requirements of the function’s availability logic but can be switched off in the event script.

This gives the analyst the opportunity to disable a function (or subsystem or capability) without specifying which constituent is unavailable.

The event script is essentially the encoded narration of the modeled scenario. In other words, it provides the actions performed upon the actors. The script’s simple format arranges information by column as follows:

1. Event ID: an arbitrary, unique number.
2. Time of event occurrence (dimensioned consistently with the status updater).
3. Narrative branch ID.
4. Event type: what level of entity is affected and whether its availability is gained or lost.
5. Object ID:  identifies the entity affected within its level.

The script is read into the SCAP processor before execution and stored as a two-dimensional table; this allows editing the script (inserting or removing events) during runtime if the scenario requires that flexibility.

The “narrative branch” of a scenario assesses which combination of conditional statements is currently valid. Using narrative branching allows analysts to dictate that something in the scenario will occur if one or more conditions are met. Different paths that the scenario may take—depending on a random draw, a choice of initialization values, or other methods—are mapped out ahead of time. The event script will only recognize events that occur on the current branch. Narrative branching helps analysts build many closely-related scenarios in a batch and will allow stochastic modeling in the future. The transition logic that governs switching between branches is stored in the status updater, which is called at every time step in the scenario to check for a transition.

The status updater is the final input in a SCAP-based ATAC analysis. This is a code section unique to the specific scenario. It is where narrative branches are switched, nonbinary state values are calculated, and mission objectives are stored. It also measures mission completion or other evaluation metrics.

The processing algorithm used for ATAC is itself quite straightforward. Each processing iteration represents one step forward through the scenario at a time interval specified by the analyst. At each time step, the event script is checked for a new occurrence. If one is found, the functional trees are re-executed. In either case, the status updater is called in order to record progress through the scenario. The scenario’s timeline is played through this way until the end time or when an exit condition is reached.

The ultimate product of SCAP-based modeling is a time history of system capabilities.  If the status updater is set up accordingly, it is also a verdict on mission completion or other objectives.  For our development work, output consists of the histories of analyst-selected entities in scatterplots, with green corresponding to availability and red to dysfunction.  A notional sample is shown in Figure 3.

ADVANTAGES OF CAPABILITIES-BASED ANALYSIS

SCAP, especially as applied to teaming scenarios germane to ATAC, is well positioned to answer the DAL-related issues mentioned in the opening section. These advantages and other merits of the methodology are discussed next.

In contrast to the DAL-based paradigm, SCAP attempts to express the availability of every entity on every level of granularity as a binary value. By avoiding partial, probabilistic, or otherwise nonbinary availability values, the propagation of a dysfunction through the functional tree is made less ambiguous. (The availability of an indirectly affected entity switches from 1 to 0 instead of, perhaps, from 0.20 to 0.16.)

Additionally, functions and capabilities themselves are finely categorized and descriptive enough that their status is more informative than DAL outputs. (Losing the “traverse off-road at full speed” capability but retaining “traverse off-road at minimum speed” provides more salient information to the analyst than “assess a 0.4 mobility kill.”) Thus, the SCAP methodology represents a significant improvement in the ability to preemptively answer the “so what” question of how a certain damage level affects the system’s remaining capabilities.  This represents an important added value for various DAC analyses consumers.

The other drawback identified with DAL-based methodologies is their propensity for modeling the target system(s) in a vacuum, both in terms of separating from other battlefield assets and isolating from other events that affect the system’s operability.  The SCAP methodology can respond to both senses of this problem.

One particular advantage of SCAP is in the way information is organized; it calls for each analysis to operate on a single, unified functional tree, which tends to emphasize high-level capabilities of the holistic team.  Actors are considered as a single aggregated team from the beginning, instead of being modeled separately and then pasted together  for modeling purposes.  As a result,  analysts are less likely to mischaracterize  or entirely overlook the emergent properties, functional redundancies,  and nonlinear capability changes that occur with teams of multiple actors.

Modeling the networking of a great number of interchangeable actors, such as a fleet of unmanned aerial vehicles, may be especially simplified by looking through the lens of the team’s capabilities as opposed to the status of individual systems.

When modeling a coherent narrative where the outcome of a threat encounter influences later event outcomes, SCAP methodology was very compatible with the sort of event scripting utilized to this point in ATAC development. A capability-based approach can even offer certain advantages. For example, decision logic in traditional analyses can become quite complicated when the relevant criteria distributes between multiple teamed systems. However, the SCAP functional tree serves as a labeled array of system states across the entire team, and the array remains available throughout scenario execution.

Thus, the logic becomes more intuitive to programming and verifying. Instead of requiring new ad hoc variables at each evaluation point, decisions can be defined to require input only from values already calculated when processing the tree. This keeps decisions linked to the tree’s plain-language descriptions of the team’s states and capabilities and limits ambiguities as to why a scenario’s narrative branch is conditioned a certain way.

SCAP’S PLACE IN THE SVL ANALYSIS TOOLBOX

It is important to note that SCAP methodology does not determine whether damage occurs due to a threat encounter but only what the capability losses would be if damage occurs. It completely abstracts the actor(s) into lists of capabilities and lower-level functional entities, the logical interdependencies of these entities, and their current states of availability.

Since a SCAP model does not generally carry the information required to “play” an encounter as a component-level vulnerability model, it mandates dysfunctions via an event schedule. In some types of analysis, a mandated event schedule is useful because it controls and standardizes the input damage. In many others (particularly as associated with T&E support), however, part of the premise of the analysis is that component-level damage is not known beforehand. Thus, SCAP cannot replace component-vulnerability modeling, such as penetration or fire-initiation codes, and should be seen as occupying a different niche in the SVL analysis ecosystem.

One promising method where SCAP advantages might be leveraged to allow the interaction of models of different scope is shown in Figure 4.  An engagement model or other means of setting the terms of a target-threat interaction are used to generate the parameters necessary for SVL modeling.  This information is then fed to the ballistic-penetration model, cybermodel, or other means of determining component-level damage.  A damage prediction becomes the input for a SCAP model (essentially becomes a single-line event script) that updates the system(s) accordingly.  These updates are then fed back to the engagement model, which can then adjust its agents as appropriate and continue.

This method is far more feasible under a SCAP paradigm than with DAL-based methods because DAL output is too generalized to be useful to the engagement model.  By contrast, capability-based modeling outputs exactly the type of capability losses that can inform how a high-level engagement model controls the actors for the model controls the actors for the remainder of the scenario.

TEST CASE AND NEXT STEPS

Through the ATAC effort, DAC performed analysis of a hypothetical “route recon” mission that verified certain programming strategies in SCAP/ATAC implementation and served as a methodology proof of concept. The teamed actors were a Bradley Fighting Vehicle (B-FiST variant) and a generic unmanned aerial vehicle (UAV) based on the RQ-20 Puma 3 AE. Functional trees and several event scripts were created specifically for the exercise. This test case was geared toward demonstrating that a SCAP-based analysis could process a series of contingent events as a unified, coherent narrative. As such, the event script mandated one of four damage outcomes to an initial threat encounter during the scenario. Each outcome, in turn, led to disparate capability losses later in the mission.

Mission success was assessed in the status updater in the team’s reconnaissance-related capabilities late in the scenario. As expected, the SCAP model successfully assessed different levels of mission success based on the initial levels of dysfunctionality sustained.  Event script variations for this exercise and associated changes in mission outcome are shown in Table 1.

To build on this early work, several initiatives are planned for FY 2020. First, the library of systems with populated functional trees will expand to include additional rotorcraft and artillery systems. Although the emergent properties of a team mean that a system cannot be fully “drag and dropped” into an ATAC analysis, building out SCAP input for individual systems is an effective way to partially prepare for future exercises. Second, the implementation of more complicated forms of narrative branching will be tested so that decision making can be shown in an operationally realistic context. Finally, DAC will determine the best way to incorporate SCAP/ATAC methodology into a larger program for addressing the analysis requirements of multidomain operations and other teamed-system engagement cases. Together, these initiatives will help position DAC to remain at the forefront of Army analysis as the battlefield further evolves.

REFERENCES

1. Agan, K. S. “An Emerging Methodology: The System Capability Analytic Process (SCAP).” ARL-TR-5415, U.S. Army Research Laboratory, Aberdeen Proving Ground (APG), MD, December 2010.
2. Abbott, S., K. Agan, A. Drysdale, W. Lardis, SFC T. Pearce, and G. Shafer. ATAC program, CCDC/DAC, 2019.

BIOGRAPHY

ANDREW DRYSDALE is an aerospace engineer at CCDC/DAC, APG, MD. With 14 years’ experience with the Army, he has focused on modeling and simulation, primarily in methodology development for survivability and vulnerability models. His previous work includes projects related to helicopter autorotation, buried-blast vehicle response, and blunt-force head trauma. Mr. Drysdale holds a B.S. in mechanical engineering from the University of Delaware and an M.S. in aerospace engineering from the University of Maryland College Park.