Logic models are an important tool for policy and program developers, managers and evaluators.
One serious problem, however, with logic models is that they usually leave out external influences and feedback effects, even when they are likely to be important, because they make the model “too complex”. It is good to simplify, but ignoring important influences on program success when planning a program or an evaluation is poor strategy. Failing to consider potentially important external influences and other complexities essentially places hopes for success on a best case scenario. Moreover, this may lead evaluators to fail to collect important data and to misinterpret program results.
Trying to embrace complexity by simply drawing a web of boxes and arrows like this is not helpful: it’s too complex to use and explain; will drive your audience away; and will probably come only from the mind of the evaluator or program manager, thereby easily missing important external influences and other complexities.
Fortunately, the same technology as used in social network analysis is perfect for uncovering and highlighting the patterns and structure of complex systems. Using existing research and/or facilitated sessions with experts and stakeholders, you can map out elements of cause and effect for evaluation, research and planning.
This network analysis-based methodology can build logic models and theory of change mappings that untangle complex relationships and external effects that would otherwise not be possible or practical, for policy planning, performance measurement and program evaluation.
In this simple demonstration version, the links between a program, its resources and activities, and its outcomes, as well as external factors, can be seen as parts of a system. Moreover, the centrality of certain parts of the system can be clearly seen.
My approach is to develop a first draft mapping using existing material that can be then validated in a facilitated session with staff, experts or stakeholders, as appropriate. I bring a varied set of tools to help in illustrating the connections between elements of a system or logic model, including visuals that can easily be revised during an expert or stakeholder data validation session, or used in reporting, like this interactive matrix displaying strengths of connection (read the causality as “y-axis drives x-axis”) and real-time-revisable network analysis software. The interactive matrix is used along with a network mapping, onscreen to help guide validators through potentially complex chains of influence and effect.
Mapping of dynamic systems is equally powerful for understanding complex webs of organizations, functions and results, such as, for example, my recent mapping of Ontario’s immunization system. This approach allowed the client to focus on key connections in the midst of a large and complex set of factors.
As the 2016 Conference Chair of the American Evaluation Association Topical Interest Group on Social Network Analysis I was very happy to give a presentation entitled ”Getting comfortable with complexity: a network analysis approach to program logic, design and evaluation”, covering this subject in detail.