What is Big Data?

Big Data refers to exceptionally large and complex data sets that traditional tools and methods cannot efficiently process, store, or analyse. It encompasses structured, unstructured, and semi-structured information generated from diverse sources, including sensors, social media, transactions, and more. The defining characteristics of big data are often summarised by the “3 Vs”: Volume, Velocity, and Variety.

Key Terms Related to Big Data

• Small Data: Manageable data sets that can be easily processed by humans or basic tools like spreadsheets.
• Structured Data: Quantitative data organised in clear formats like tables, e.g., sales figures or inventory lists.
• Unstructured Data: Qualitative data like images, videos, or emails that don’t fit into standard formats, requiring advanced analysis tools.
• Open Data: Publicly available information, often provided by governments or corporations, to encourage transparency and innovation.

How Big Data Works

Big data is only useful when it is stored, processed, and analysed effectively. This requires:

• Storage: Cloud storage and distributed frameworks (e.g., Apache Hadoop) make it possible to store massive data sets affordably and securely.
• Processing: Specialised software handles the scale and complexity of big data, enabling insights through techniques like predictive analytics and machine learning.
• Security: Robust cybersecurity measures and disaster recovery plans are critical to protecting data from threats, both virtual and environmental.

Applications of Big Data

Big data drives innovation across industries, including smart buildings. For example, historical data on energy use in a building can inform predictive models to optimise efficiency, reduce costs, and enhance user comfort.