How can Databricks be used to analyse smart building data?

Databricks is a cloud data and analytics platform built around a "lakehouse", which stores large volumes of data and lets engineers and analysts process it with SQL, Python and machine-learning tools. For buildings, it can be a good fit when a portfolio needs to combine large volumes of operational data, such as meter intervals, BMS trends and sensor readings, with business data such as leases, floor areas, occupancy, maintenance and costs, and to analyse all of it at scale.

It is an analytics environment, not a building management system. It does not replace local control, and it is not the right answer for every owner. The business case should begin with analytical needs that justify both the platform and the team required to run it.

When it fits, and when it is overkill

A simpler energy-management or analytics product may deliver the first decisions faster and at lower cost. A general-purpose platform pays off when the analytical ambition and the skills to match are both present.

Bronze, silver and gold, for buildings

Databricks' documentation describes the "medallion" architecture, a pattern of layers that progressively improve the structure and quality of data, which it calls a recommended best practice but not a requirement. Translated into building terms, it works like this.

The bronze layer holds raw data as received, with minimal validation, retained so that it can be reprocessed and audited. For a portfolio, that means meter files, BMS trend exports and API responses, unchanged, with their source and arrival time recorded.

The silver layer holds cleaned and validated data: consistent timestamps (preferably UTC) and units, de-duplicated readings, invalid values quarantined, and points mapped to assets and spaces, ideally using a semantic model such as Brick.

Business-ready datasets sit in the gold layer: energy per square metre per site per month, comfort compliance per tenancy, or fault counts per asset type.

Most of the building-specific work happens in silver. That is where late readings and corrections are handled, cumulative registers are distinguished from interval consumption, estimated values are flagged and meter boundaries are kept consistent. Neglect it, and a sophisticated model built on inconsistent meter boundaries will produce precise-looking but misleading comparisons.

Where the platform's responsibilities end

Essential control has no place in an analytics platform. Heating, cooling, ventilation and life-safety functions stay in the BMS and local systems. If analytical outputs are to influence operations, they should travel by a controlled, reviewed route, such as a recommendation that an engineer approves and applies through the BMS.

Ingestion needs the same care. Connections from operational networks to a cloud platform are information flows across security boundaries and should be designed with the OT security team.

Then there is cost, which goes beyond compute and storage to the engineering time needed to build pipelines and keep them working as building systems change. A test with a representative workload before committing is the only reliable estimate.