Bio-Inspired Self-Regenerating Data Warehouses

Abstract

Modern data warehouse architectures are increasingly challenged by the growth in data volume and the diversity in data workloads.As data volume increases exponentially and data workloads become more diverse, operational continuity, query performance and storage efficiency are becoming increasingly difficult problems for modern architectures of data warehouse systems. Drawing inspiration from concepts in immunology, evolutionary biology, swarm intelligence, and symbiotic ecology, these bio-inspired computational paradigms offer an exciting approach to building systems that are capable of self-regenerating data warehouse systems that can autonomously detect and alleviate faults, adaptively select views, and continuously optimize themselves. This paper consolidates basic and applied research on the topic of bio-inspired self-regenerating data warehouses from 2002 to 2020, and proposes a common theoretical and architectural framework for the bio-inspired data warehouse. Artificial immune systems (AIS) are used for anomaly detection and self-healing, ant colony optimisation (ACO), artificial bee colony (ABC), particle swarm optimization (PSO) and coral reefs optimization (CRO) are applied to materialised view selection and autonomic computing principles are used for self-management. The results shown in the references prove that hybrid bio-inspired architectures can achieve 47% query processing improvement and 15-22% storage overhead reduction, and also have high self-healing rates of 95% or more. The paper also suggests a layered autonomic architecture that satisfies the four pillars of self-configuration, self-healing, self-optimisation, and self-protection, and covers the scalability, convergence behaviour and deployment aspects of large-scale data environment up to 2020.