Development of a Portable, Field-Usable Cognitive Test Battery (ACRA)

Even as military conflict becomes more complex and technologically sophisticated, it is evident that there is a need to focus attention on the individual as both warrior and system controller. The Department of Defense has requested the development of cognitive assessment/readiness test batteries based on a variety of diverse theoretical underpinnings. NTI has surveyed current models of cognition and test batteries in order to assess the current state-of-the art, and to develop a final recommended approach to test battery development that will encompass behavioral and physiological measures of skills critical to military and civilian missions. In a Phase I Small Business Innovative Research (SBIR) effort managed out of the Army Proving Ground at Aberdeen, MD, NTI recommended a complete series of physiological and performance tests, innovative hardware/software approaches, and test selection procedures. In this SBIR, these tests were implemented as a test battery (ACRA) as part of the Army's "Cognitive Fightability" program.


ACRA Enhancement


The Army Cognitive Readiness Assessment (ACRA) system currently consists of 12 neuropsychological tests designed to evaluate the cognitive fightability of the dismounted warrior. Although ACRA has been used successfully in a number of operational test and evaluation experiments, it demands considerable expert input from the user, as well as sophistication in interpretation.  Since delivery of the ACRA system to the Army, there have been several technological breakthroughs that could expand the applicability of ACRA to all Army (and DoD) jobs and missions, incorporate physiological measures into the system, and simplify test selection, presentation, and interpretation. These developments also offer the possibility that traditional static cognitive assessment testing approach could, for the first time, be expanded into predictive, dynamic models permitting estimates of soldier cognitive fightability for some time into the future.  The present SBIR effort explored these possibilities as well as the addition of physiological metrics, carried out proof-of-concept demonstrations for several enhanced applications and prepared for implementation of the final product.



NASA Cognitive Assessment Tool (CAT)


This contract with NASA developed an entirely new approach to cognitive performance testing, and a prototype system employing this approach was implemented and delivered. 


This new approach constitutes a “next generation” technique for quantifying the cognitive demands of specific jobs or missions, and selecting batteries of cognitive tests tailored to probing an individual’s ability to meet those demands.  It builds on previous developments by NTI, Inc., primarily under SBIR sponsorship, that had produced the basic testing paradigm and mathematical analysis techniques.  In the present effort, a test battery – the NASA Cognitive Assessment Tool (NCAT) - was created to specifically address the cognitive demands of three representative mission activities – extra-vehicular activity (EVA), remote arm manipulation (RMA), and rendezvous/docking (DOCK).  A computer-based model, the Person-System-Mission (PSM) model, was also created. This used the output of the original tests to estimate the operational impact of an individual’s current and future cognitive status on the outcome of an activity or mission carried out in space over extended periods (six months or more). In creating this model, sub-modules evaluating the impact of sleep, rest, and workload were designed and its integration into the PSM model described.  In the prototype delivered under this contract, the model was exercised on the three representative mission activities.


Fatigue Avoidance Scheduling Tool (FAST)


Prior to 2002, considerable research had documented the effects of sleep and circadian rhythm on human cognitive performance and scientists were beginning to create models of those relationships.  In an AF SBIR contract, NTI implemented the Sleep, Activity, Fatigue, and Task Effectiveness (SAFTE) model of Dr. Steven Hursh in a graphical user interface called the Fatigue Avoidance Scheduling Tool (FAST™).  At the end of the Phase II and enhancement contracts, F33615-00-C-6013, FAST a user could predict cognitive performance effectiveness based on the timing and amount of sleep an individual received prior to and during a mission.  FAST provides the military planner, mishap investigator, or shift work scheduler the ability to optimize performance under conditions of limited sleep, thus minimizing the need for pharmacological aids while indicating work periods where additional fatigue countermeasures may be necessary.  Features included shifting the circadian rhythm for shift work or transmeridian travel, automatic insertion of sleep into a schedule when no data are available, predicting lapses of attention, an algorithm to predict the impact of stimulants on performance, a window showing levels of fatigue factors affecting performance at any time within a schedule, and an algorithm to predict performance variability.  The underlying model was validated in a field study using a C-130 aircrew flying from airbases in Japan to the US.  A study of the effects of sleep aids on performance was also completed.



Aircrew Medication and Flight Assessment Program (AMAP)


In this contract, NTI developed, demonstrated, and delivered a testing program to evaluate the effects of FDA approved therapeutic medications on aviators in order to determine waiver status. To accomplish this goal, NTI researched, developed, and validated a battery of neurocognitive tests (the Aircrew Medication Assessment Program – AMAP) which can be administered to an individual aviator off medication, and then again 3-5 days following resumption of his or her drug. Test results are to be compared to a set of program criteria to determine whether or not the aviator should have a change in medication. This effort used Commercial-off-the-shelf hardware and software for program implementation to the maximum extent possible. The system developed is portable with a test duration goal of less than two work weeks.


To determine the effectiveness of the test battery, an experimental study was planned and conducted.  The study included all system components and procedures. In spite of an extremely conservative experimental design and limitations on response timing resolution imposed by the use of certain “off the shelf” tests, the study showed that the battery is able to detect alcohol impairment at BAC levels as low as 0.04. The development of a “Pass/Refer” criterion has been started. The goal is to correctly identify impairment in individual patients while minimizing the number of false positives.



NTI has designed a multi-faceted assessment tool to be used in the centrifuge facilities at Wright Patterson Air Force Base. The G-Performance Assessment Simulation System (G-PASS) provides training and evaluation of the flight tasks and skills considered critical to safety and mission completion. Furthermore, G-PASS serves as the state-of-the-art tool used to validate a cognitive performance under gravitational stress model that is also being developed in this contract. G-Tool to Optimize Performance (G-TOP) dynamically predicts critical flight performance skills while under acceleration force. This technique allows for one to analyze the effect of acceleration force on the performance of critical cognitive processes based on a self-input +Gz profile.

OSD Cognitive Test Battery

NTI has created an "armory" of traditional and innovative cognitive test batteries chosen by a panel of subject matter experts to measure various levels of job-related cognitive skills. Fronting the test "armory" is a user-guided interface that allows for one to choose the skills and the levels of those skills that are necessary for a certain job. Based on the input given by the user, a list of tests is generated that matches the demands of the job. Each user-generated battery includes the minimum number of tests needed to address each skill demanded by the job producing the most efficient to date assessment tool to be used in the workforce.


The objective of this Small Business Innovation Research (SBIR) Phase II effort was to assess the feasibility of collecting sensor imagery of the air-to-surface weapon-impact event and the target environment, just prior to and following weapon impact, to support bomb damage assessment (BDA) analysis in near real-time by use of a free-fall sensor package deployed from the munition. Achieving the goal of a low-cost final production system required an innovative technical approach and maximum usage of commercial-off-the-shelf (COTS) components. Trade and parametric studies were performed, and prototype systems and subsystems were designed, fabricated, and tested. Innovative, low-cost flight test methodologies were developed and employed, including captive-carry and drop tests using a Cessna 172 aircraft. An innovative sensor-tracking concept was developed and demonstrated, and in-flight video of the target environment was collected during several drop tests. In the final drop test, a prototype sensor package and deployment mechanism was attached to an inert Mk-82 Low Drag General Purpose (LDGP) bomb and dropped from a contracted A-4N fighter-bomber on a Government-provided weapons training range.


Increasingly, helmet-mounted displays (HMDs) are replacing "head-down" and Head-Up Displays (HUDs) in advanced cockpit interface designs. HMDs offer potential advantages by providing pilots with more direct access to critical visual information, while offering greater flexibility of head movements, less weight, and less consumption of cockpit space. Much of the symbology, functionality, and mechanization found in current HMDs can be traced directly to HUDs. But, because HMDs are decoupled from the longitudinal axis of the aircraft, different kinds of information can be presented on HMDs. Thus, questions arise concerning the best manner in which to present the additional information, and its interaction with traditional HUD information. The purposes of this effort were to define the human performance requirements for both HUD and HMD interfaces as utilized in military missions, and to produce a preliminary HMD symbology design for a no-HUD aircraft. To establish the functional specifications for the interface design, a user-centered design approach employing cognitive work analysis was employed.

Federal Railroad Administration

Under contract to the Federal Railroad Administration (FRA), NTI, SAIC, the US Air Force, Burlington Northern Santa Fe Railway, New Jersey Transit and railroad consultants partnered to advance the application of FAST™ for railroad operations. The goal of the project was to analyze historical schedule data and sleep logs, collect on-the-job performance data to validate the tool, and design a workable approach to fatigue and alertness management tailored to specific modes of transportation. Initially, this project focused on railroad applications. As part of the process, FAST™ was used to predict performance effectiveness under current work schedules and then various hypothetical schedule alternatives were analyzed. Using available data sets that contain alertness data, the performance predictions of the FAST™ model were validated. Thus, the scheduling tool is a validated aid for finding solutions to fatigue related problems in transportation operations. The project focused on two major initiatives: (1) Development and validation of a model-based work schedule evaluation and design tool, and (2) development and validation of a model-based accident investigation tool for fatigue analysis.


Development of a Test of Cognitive Function for Space Shuttle Research

NTI developed, implemented, and tested a complex test of higher cognitive functions compatible with a lap-top computer. This test was incorporated in the NASA PAWS test battery, and was used in the International Microgravity Laboratory (IML-2) shuttle flight which was launched from Cape Kennedy on July 8, 1994, and in a second shuttle flight on August 6, 1996.


Development of a Desk-top Flight Simulator For Assessing Pilot Situation Awareness

In this Phase II SBIR effort, NTI utilized its proprietary real-time software system to generate realistic flight scenarios for the F-16 aircraft. Multiple measures of situation awareness and cognitive function are embedded into the flight simulation. This will allow enhanced training of new pilots and provide a research tool for exploring cognition during pilot performance.

See the recent USAF Press Release (PDF file) on SAFTE and FAST!

Validation and Customizing of A Performance-Based Test of Impairment

NTI developed an FAA-specific performance test to detect job readiness of employees in safety-sensitive positions. This included validation of expected/required levels of sensitivity, verification of sensitivity to multiple stressors (e.g., alcohol, drugs, fatigue, sleep loss, etc.), verification of test stability and maintainability, and confirmation of compatibility with standard FAA equipment.

Aircrew Team Performance

For eight years, NTI has supported aircrew team performance research in the AESOP facility at the former Brooks Air Force base. Activities include definition of individual and team performance measures and conduct of manned simulation research on the effects of drugs on human performance.

Enhanced Soldier Performance Capabilities Study

This Phase I SBIR effort included analysis of current workload theories and definition of the factors limiting an individual's capacity to respond to increased task demands. Realizing that the cognitive capacities of the human constitute the newest barrier to efficient personnel utilization, internal "state" and "trait" variables were assessed within a conceptual framework, and a test battery identified to assess performance and, later, enhance required skills.

Development and Validation of a Test Battery for Predicting Air Traffic Controller Trainee Success

Under subcontract to Aerospace Sciences, Inc., NTI developed and implemented, during a validation study, a computerized battery of cognitive/behavioral tests predictive of air traffic controller skills in the FAA. This battery was implemented in lieu of the traditional nine-week process previously utilized during FAA air traffic controller trainee selection. The program greatly reduces both the FAA's investment of committed resources and the individual's risk.

Development and Validation of Performance Based Testing

Under contract (1989) to a major petrochemical company, NTI designed and produced a five-minute computerized test to serve as a performance screen for skill degradation of workers in safety-sensitive positions. Studies were performed in both flight and driving simulators, under conditions of both alcohol and sobriety.

Systems Design and Software for the Tri-Service Standardized Performance Assessment Battery

NTI has designed and developed a standardized performance assessment battery. This product was designed for tri-service research to evaluate the impact on human performance of protective drugs for chemical defense. A unique feature of the software is that it is composed of elementary task program modules so that battery additions can be developed rapidly and easily by researchers.

Certification of Civilian Pilots

NTI evaluated the capabilities of current mini-mental status tests to screen general aviation pilots. NTI reviewed available brief exams to determine their suitability and acceptance by examiners and pilots. In addition, NTI conducted validation efforts in which available clinical mental status exams were tested to assess their sensitivity to various cognitive and psychological deficits, as well as their ability to predict performance on behavioral tests of information processing functions relevant to pilot characteristics.

Federal Aviation Administration Human Factors

Over a period of 13 years NTI has provided human factors support to a number of FAA programs concerned with the performance of pilots and air traffic controllers. The most significant of these has involved development, operational testing and implementation of Controller-Pilot Data Link Communications (CPDLC). By providing digital information transfer along with the ability to discretely address individual receivers, CPDLC is expected to reduce congestion on existing voice radio communication channels and enhance the safety and productivity of ATC operations.

NTI conducted initial high fidelity simulation research with air traffic controllers to guide the design of effective ATC services and human computer interfaces, and to evaluate the impact of CPDLC on system performance. In preparation for implementation, NTI participated in the development of CPDLC controller procedures and training as well as research to promote optimal coordination between air traffic controller and aircrew users. NTI conducted Air Traffic effectiveness and Airway Facilities testing during the OT&E process for CPDLC Build I. NTI also completed an operational evaluation of CPDLC Build I at the Miami Air Route Traffic Control Center in support of the Center's decision to declare Initial Daily Use.

Continuing NTI human factors efforts for the FAA include the testing of a new graphical interactive controller interface and the assessment of an expanded set of ATC services that will be delivered by CPDLC in the future.


Army BAA

The intent of this effort (funded by the U.S. Army Medical Research and Materiel Command) was to extend NTI's model-based, software tool that predicts cognitive performance from sleep and naps to include transmeridian travel and to conduct three experiments to assess the effect of pharmacological compounds on sleep and performance in military scenarios of sustained operations. Study 1 assessed flumazenil, a benzodiazepine antagonist, for reversing the effects of zolpidem, a commonly used sleep aid. Study 2 assessed the relative effects of zolpidem and melatonin against placebo for their effects on performance after sudden awakening. Study 3 assessed sleep aids and stimulants used in combination on a multi-day, military scenario. Parameters for transmeridian travel and shiftwork changes were added to the homeostatic model to account for cognitive performance effects found in the literature.