JSEM: a framework for identifying and evaluating indicators.

Abstract

There are two issues in indicator development that have not been adequately addressed: (1) how to select an optimal combination of potentially redundant indicators that together best represent an endpoint, given cost constraints; (2) how to identify and evaluate indicators when the endpoint is unmeasured. This paper presents an approach to identifying and evaluating combinations of indicators when the mathematical relationships between the indicators and an endpoint may not be quantified, a limitation common to many ecological assessments. The approach uses the framework of Structural Equation Modeling (SEM), which combines path analysis with measurement models, to formalize available information about potential indicators and to evaluate their potential adequacy for representing an endpoint. Unlike traditional applications of SEM which require data on all variables, our approach---judgement-based SEM (JSEM)--can utilize expert judgement regarding the strengths and shapes of indicator-endpoint relationships. JSEM is applied in two stages. First, a conceptual model that relates variables in a network of direct and indirect linkages is developed, and is used to identify indicators relevant to an endpoint. Second, an index of indicator strength--i.e., the strength of the relationship between the endpoint and a set of indicators--is calculated from estimates of correlation between the modeled variables, and is used to compare alternative sets of indicators. The second stage is most appropriate for large, long-term assessments. Although JSEM is not a statistical technique, basing JSEM on SEM provides a structure for validating the conceptual model and for relining the index of indicator strength as data become available. Our main objective is to contribute to a rigorous and consistent selection of indicators even when knowledge about the ability of indicators to represent an endpoint is limited to expert judgement.