The Knowledge That Cannot Transfer
For years, the enterprise information technique was easy: transfer every thing to the cloud. Migrate the info lakes and the warehouses to the cloud, after which governance follows. It was a clear story — till it wasn’t.
At the moment, a number of the world’s most subtle enterprises are telling us clearly: they can not — and won’t — transfer all of their information to the cloud. Main semiconductor producers are coaching fashions on engineering-classified datasets that mustn’t ever go away their premises. International buying and selling companies sit on large volumes of historic tick information the place the economics of cloud egress make migration unattainable. Tier-1 banks have adopted “Hybrid Perpetually” methods, modernizing on-premises storage whereas sustaining strict information sovereignty. Main pharmaceutical corporations run tens of millions of every day drug experiments in opposition to petabyte-scale on-premises information estates topic to stringent regulatory controls.
These aren’t edge instances. They symbolize a structural shift in how enterprises take into consideration information: from “Migrate All the things” to “Govern All the things.”
The drivers are actual and compounding:
- Knowledge sovereignty & regulation: Monetary providers, healthcare, and authorities organizations function underneath mandates — GDPR, HIPAA, NIS2, sector-specific information residency guidelines — that require information to stay inside particular jurisdictions or air-gapped environments. Cloud migration shouldn’t be optionally available; it’s legally prohibited for sure datasets.
- Knowledge gravity & prices: At petabyte and exabyte scale, the economics of cloud migration break down completely. Egress charges, storage prices, and sheer information quantity make the “transfer it as soon as” mannequin financially unsustainable. A few of the world’s largest retailers are actively repatriating analytics workloads from cloud again to on-premises infrastructure for exactly this cause.
- Latency & edge workloads: Retail, manufacturing, and telco workloads require low-latency entry to on-premises and edge information. Telecommunications suppliers ingest monumental volumes of community telemetry on-premises every day to energy AI-driven community operations that can’t tolerate cloud round-trips.
- AI on darkish information: Huge shops of backup information, unstructured archives, and secondary datasets — representing a whole bunch of exabytes throughout the enterprise — comprise immense AI worth that has by no means been unlocked as a result of governance did not attain it.
The sign is unmistakable. We’ve acquired requests from a whole bunch of shoppers explicitly requesting on-premises and hybrid storage connectivity to Unity Catalog. The Software program-Outlined Storage (SDS) market stands at a whole bunch of billions of {dollars} in 2026, and the enterprise companions who handle this property — collectively holding greater than 2 Zettabytes of knowledge underneath administration — are constructing with us.
Introducing the Databricks Storage Ecosystem
At the moment, we’re excited to announce the Databricks Software program-Outlined Storage (SDS) Ecosystem — a brand new companion class purpose-built to deliver Databricks Intelligence Platform to enterprise information wherever it lives: on-premises, in personal clouds, and on the edge environments. In case you are an enterprise operating petabytes of knowledge on these platforms as we speak, you not have to decide on between your current non-cloud storage infrastructure and Databricks AI.
For too lengthy, enterprises had to decide on between the on-premises storage infrastructure they depend on and the cloud-native AI they need to construct. Forcing clients emigrate large quantities of knowledge utilizing advanced pipelines simply to unlock that intelligence is a damaged mannequin. By uniting these industry-leading companions, we’re ending that compromise and delivering Databricks Intelligence on to the place the enterprise information lives. However this launch is simply day one. We’re constructing the inspiration to make sure that quickly, each piece of hybrid information–structured or unstructured–is immediately prepared for generative AI with out ever copying a byte. — Stephen Orban, SVP, Product Partnerships & Ecosystem, Databricks
On the coronary heart of this ecosystem is OpenSharing, an open-source protocol for safe, ruled information sharing. Our storage companions are implementing OpenSharing servers to show their information estates on to Databricks Serverless Compute. The trail is easy: the storage companion stands up a OpenSharing endpoint, you join it to Unity Catalog, and also you immediately acquire safe, ruled entry to your on-premise information in Databricks with out information migration.
This integration gives a single, unified catalog throughout your total hybrid surroundings. Clients can now use Databricks Serverless Compute, Genie, AgentBricks, and mannequin coaching to question and cause over information that by no means leaves the premises. The end result? Zero information motion, no duplication of knowledge and 0 compliance danger.
This isn’t a roadmap aspiration. Clients can strive these integrations as we speak. Companions constructing these integrations comply with the Associate Nicely-Architected Framework — a technical blueprint masking structure, safety, and certification standards.
Clients need to break down information silos and unify all of their Knowledge and AI property – together with massive quantities of knowledge that also sits on-premises. Due to on-premises storage companions leveraging the open supply Open Sharing protocol, clients can now seamlessly unify, govern, and analyze all of their information property in Databricks Unity Catalog – unlocking the total worth of their information within the Databricks Knowledge Intelligence Platform. — Jonathan Keller, VP, Product Administration, Databricks
Our Launch Companions
We’re proud to announce integrations with the next main storage suppliers:
MinIO — Common Availability (demo, weblog)
MinIO AIStor is the bridge that seamlessly connects the Databricks Knowledge Intelligence Platform with enterprise information that may’t transfer to the cloud. By natively implementing the open Open Sharing protocol on the storage layer, AIStor eliminates complexity and allows Databricks clients to effectively question stay on-premises Apache Iceberg™️ and Delta tables underneath full Unity Catalog governance. It extends Serverless Compute, Genie, and Agent Bricks to on-premises information, bringing the total energy of the Databricks Platform to an enterprise’s most crucial information.
AI and analytics initiatives are sometimes constrained by the place information resides, notably in environments with strict safety, sovereignty, or operational necessities. By bringing native OpenSharing to AIStor, we’re enabling organizations to securely expose information the place it lives whereas giving Databricks seamless entry by means of open requirements. This removes a significant barrier between enterprise information and AI, permitting organizations to activate beforehand inaccessible information for AI, analytics, and agentic purposes with out compromising management. — Ugur Tigli, Chief Know-how Officer, MinIO
Everpure (previously Pure Storage) — Personal Preview (demo, weblog)
Everpure and Databricks allow organizations to make use of on-prem information straight within the cloud eradicating the necessity for information replication or duplication.That is delivered by means of an OpenSharing connector that bridges information in object storage with databricks core workspaces in a safe and gated method.
Everpure and Databricks allow organizations to entry and analyze on-premises information straight from the cloud with out the necessity for replication or duplication. Constantly shifting information between environments is expensive and unsustainable at scale. Clients are in search of an easier strategy that balances price, compliance, and information sovereignty whereas lowering operational complexity. — Chadd Kenney, VP of Product Administration, Everpure
Qumulo — Personal Preview in July 2026 (weblog)
Qumulo has built-in OpenSharing with its new NeuralSearch, permitting clients to securely share Qumulo-stored information with Databricks throughout core, cloud, and edge environments—with out replication, additional prices, or complexity. Utilizing NeuralSearch, customers can uncover related datasets, together with unstructured content material, by way of natural-language queries and seamlessly share these curated tables with Databricks by way of OpenSharing.
Organizations can not afford the fee, complexity, and delays of copying large datasets throughout environments simply to help AI and analytics. By combining Qumulo NeuralSearch with Databricks OpenSharing, clients can securely uncover, govern, and share each tabular and unstructured information throughout core information facilities, edge areas, and public clouds – in actual time, with out shifting the info itself. Collectively, we’re serving to organizations speed up AI initiatives, unify governance, and unlock quicker time-to-insights from globally distributed information whereas sustaining a single supply of reality. — Brandon Whitelaw, SVP and Head of Product at Qumulo
VAST Knowledge — Personal Preview in August 2026
VAST Knowledge is extending the VAST AI Working System with OpenSharing help to assist enterprises bridge Databricks workflows with information that resides throughout on-premises and hybrid infrastructure – with out requiring large information motion or migration. The mixing will give clients extra flexibility to entry, course of and operationalize information throughout cloud, information heart and rising AI infrastructure environments whereas supporting trendy hybrid AI and analytics workloads.
AI infrastructure is turning into basically hybrid. Clients more and more need the power to course of information wherever it makes probably the most sense economically and operationally, whereas nonetheless sustaining seamless entry throughout environments. OpenSharing help extends the VAST AI Working System’s skill to bridge Databricks workflows with information that resides throughout cloud and on-premises infrastructure for contemporary AI and analytics purposes. In contrast to conventional storage platforms, VAST combines information providers, distributed processing and AI infrastructure orchestration right into a unified working system for AI information at scale. — John Mao, Vice President, International Know-how Alliances at VAST Knowledge
What’s Subsequent
Integrations Coming Quickly
Along with our launch companions, momentum throughout the storage ecosystem continues to speed up. We’ve secured commitments from Cohesity, Commvault, HPE, NetApp, Nutanix, and Rubrik —to construct native integrations by the tip of the yr.
Collectively, these companions, together with launch companions, handle a whole bunch of exabytes of enterprise information, spanning high-performance unstructured media, secondary backup archives, cost-effective cloud storage, and hyperconverged personal cloud estates.
Unlocking Unstructured Knowledge
At the moment’s launch establishes structured, tabular information as absolutely ruled and accessible throughout this ecosystem. However we all know that thrilling alternative lies forward in unstructured information: the pictures, PDFs, movies, medical scans, engineering simulations, and backup archives that symbolize nearly all of enterprise information underneath administration — and the uncooked materials for the subsequent technology of RAG pipelines and fine-tuned fashions.
We’re actively working to increase the OpenSharing protocol with Volumes APIs — exposing unstructured information from on-premises storage on to Databricks for GenAI workloads. With this coming, companions managing large unstructured estates — from media and imaging archives to enterprise backup repositories — will unlock a wholly new class of AI use instances for his or her clients.
That is what it means to manipulate every thing.
Be a part of the Ecosystem
In case you are a storage vendor considering constructing an OpenSharing integration, go to the Associate Nicely Architected Framework or attain out to the Databricks Associate workforce to get began.
In case you are an enterprise buyer who desires to attach your on-premises storage property to Databricks, contact your account workforce to study extra.
The period of “Migrate All the things” is over. The period of “Govern All the things” begins as we speak.
