In Jorge Luis Borges’ parable “On Rigor in Science” (“Del rigor en la ciencia”), a lost empire attains such perfection in the art of cartography that a one-for-one scale map of the empire is produced, laid-out across the land. In time, the creation is understandably deemed useless and subsequent generations ambivalently witness the decay of the map (Borges, 1975). Borges’ critique is that of a science which seeks to perfectly validate, one-to-one, its experiments such that the efforts and impositions of testing and documentation overwhelm the utility and impact of the proofs. Effective inquiry must enact efficiencies between the overhead of validation and the utility achieved from experimental models. This notion sits as the foundation of this proposal regarding business model validation: given the impracticality, and indeed impossibility, of ‘perfect model validation’, what can be considered ‘robust’ from a commercial organizational standpoint? Is it possible to improve upon limitations in current model validation practices, particularly when organizational validation is viewed as an unstructured confidence-building exercise?
Organizations, struggling in an age of information overload and complexity, are seeking methods with which to better forecast, strategize, and prepare for lurking risks and to identify hidden opportunities. The specific domain of concern here is particularly sensitive to model validation overhead: business analytics, a practitioner discipline uniting management science techniques and information technology solutions to guide decision making in commercial settings. A fast growing discipline, business analytics practitioners are increasingly active embedding complex decision models in enterprise information technology systems: software-driven statistical analysis, large data-set management, business intelligence solutions, and decision support systems (DSS).
The rise of increasingly complex analytics systems are accompanied by intricate decision models. These models require validation: testing which confirms that the model approximates the behavior of the system under assessment (Pidd, 2004). While the techniques and technologies for managing these models and systems are advancing rapidly, the methods for organizationally validating the underlying models themselves remain quite rudimentary and unstructured. Business analytics model validation processes can be broadly characterized as ad hoc, consensus-based, confidence building exercises (Sargent, 2013). As such, model validation can be seen as an organizational sensemaking effort: an attempt to deem a model loosely ‘good enough’ in its concordance with target phenomenon. Beyond this, it is generally recognized that exhaustive validation of an analytical model is not formally achievable, being at best falsifiable (Balci, 1998; Pidd, 2004).
As firms rely on increasingly complex analytical models to drive strategic decisions, yet often lack robust model validation processes, it is proposed that emerging understandings of sensemaking in organizations, particularly socio-structural understandings, can be applied to bolster robustness in the practice of business analytics model validation. It is proposed to enhance as-is practices for validating business analytics models via the application of Social Network Analysis (SNA). SNA, as a method which can facilitate socio-structural understandings of organizational consensus building, offers the potential to improve robustness and outcomes in model validation processes.
Business model validation may initially appear to be a somewhat obscure domain. However, the guiding motivation of this research is that poor model validation, and the ensuing consequences, poor business decisions, are a growing threat to organizations, and thus of central concern to both business practitioners and researchers alike. The need to manage increasingly complex decisions in business is growing due to the combined forces of globalization, technological advancement, and competition (Castellani & Hafferty, 2010). Global organizations are beset by expanding complexity bearing accompanying uncertainty (Hubbard, 2009; Taleb, 2007). Growing global business complexity has resulted in increasingly complex models for strategic decision making. Decision models in large enterprise settings are attempting to embrace broader stakeholder groups, sets of variables, and frames for uncertainty. Increasingly complex analytical models, spanning broad timeframes, interconnected markets, and diverse supply chains, are being utilized. With larger and more complex models comes higher sensitivity to initial inputs and assumptions.
Managers are acutely aware that, despite strengthened risk protocols and structured processes, global organizations are often flying blind in environments of uncertainty and doubt, with little warning concerning lurking crises. Concurrently, a new awareness of the limitations of individual and group decision making are emerging: behavioral economics research has uncovered inbuilt decision biases which commonly short-circuit attempts to improve decision quality, even for the best prepared (Kahneman, 2011; Kahneman & Klein, 2009; Miller & Page, 2007; Nutt, 2002). Faced with increasing complexity and data overload, beset managers are easily tempted to retreat into the comfortable, but often misleading, domain of personal instinct and intuition (Bonabeau, 2003). The myth of the omniscient business leader, guided by superior intuition and instinct, comes under assault in conditions where the amount of interacting variables and available data outstrips the capacity of individuals to effectively correlate and process meaningful and significant results. In an attempt to strengthen controls, marshal complexity, and bolster against poor judgment, organizations are keenly interested in identifying and adopting model-driven, repeatable analytical decision making processes.
To manage complex decisions, many organizations are embracing advanced business analytics tools and techniques. A rising elite of forward-thinking firms are embracing concerted systems-driven business analytics decision making as a strategic value driver. Celebrated high-performers such as Wal-Mart, Amazon, Apple, Dell, FedEx, and Zara have ushered in a revolution in strategic supply chain analytics, deriving value from concerted computational forecasting and prediction. Recent research explores the foundations of analytics as a powerful competitive differentiator (Davenport, Harris, & Morison, 2010; Kiron, Shockley, Kruschwitz, Finch, & Haydock, 2011; LaValle, Lesser, Shockley, Hopkins, & Kruschwitz, 2011). Higher levels of organizational analytical capability are correlated with value creation via competitive outperformance (Kiron et al., 2011; LaValle et al., 2011). In the wake of these dramatic success stories, technical and methodological advancements associated with business analytics have taken root in forward-thinking organizations (Kiron et al., 2011). As such, there is a great interest in the methods by which organizations can spur the adoption and implementation of analytics-based organizational decision making (Hey, 2010).
The success of business analytics as a value driver has spawned growing practitioner and academic interest. Key research themes frame notions of evidenced-based management, decision-driven organizations, and the concept of organizational analytics maturity levels. The rising trend towards analytics is an emergent phenomenon driven by a desire to not only survive, but to thrive amongst synergistic forces: advancing information and communication technologies, expanding datasets resulting from increased collection and storage capacities, globalization facilitated by growing low-cost communication and transport feasibility, virtual companies woven together by partnerships and outsourced services, expanding supply chain and market complexities, and broadening stakeholder factors associated with sustainability. The emerging status quo for global business represents a complex agglomeration of third and fourth level factors which make traditional intuition-based managerial judgment not only untenable, but increasingly dangerous in terms of provoking unanticipated consequences.
However, with the adoption of advanced business analytics approaches comes increasingly complex models, stakeholder involvement, and decision processes. Complex business analytics-driven decision processes pose organizational challenges. Business analytics practitioners are faced with challenges related to decision process and model management. Given lack of time, funding, and high overhead, practitioner analytics model design, structure, validation, and valorization processes are often ad hoc and cursory exercises. These emerging technologies and methods put fresh demands upon organizational leadership: new decision processes must be married to organizational structure and culture to be truly viable. It has been recognized that increasingly impactful economic decisions are made on the basis of shaky, at times flawed, business decision making models. Forging a tighter organizational contextual link between analytical models and organizational validation processes would improve the robustness of complex business decision making programs.
Given the organizational challenge of validating and valorizing complex models, an integrated analytical modeling and validation approach linked explicitly to organizational socio-structural context would provide practitioners with a method for streamlining and fostering stronger consensus regarding strategic decisions.
