Databricks launches Project Lightspeed, its next-gen Spark streaming engine

At its Data + AI Summit, Databricks today made the requisite number of announcements one would expect from a company’s flagship developer events. Among those are the launch of Delta Lake 2.0, the next version of its platform for building data lakehouses, MLflow 2.0, the next generation of its platform for managing the machine learning pipeline, which now includes MLflow Pipelines with templates for bootstrapping model development, and a couple of announcements around the Apache Spark data analytics engine, which forms part of the core of the Databricks platform.
With Spark Connect, Datab

リンク元