Spring Cloud Data Flow
2.9.5Microservice based Streaming and Batch data processing for Cloud Foundry and Kubernetes.
Spring Cloud Data Flow provides tools to create complex topologies for streaming and batch data pipelines. The data pipelines consist of Spring Boot apps, built using the Spring Cloud Stream or Spring Cloud Task microservice frameworks.
Spring Cloud Data Flow supports a range of data processing use cases, from ETL to import/export, event streaming, and predictive analytics.
Features
The Spring Cloud Data Flow server uses Spring Cloud Deployer, to deploy data pipelines made of Spring Cloud Stream or Spring Cloud Task applications onto modern platforms such as Cloud Foundry and Kubernetes.
A selection of pre-built stream and task/batch starter apps for various data integration and processing scenarios facilitate learning and experimentation.
Custom stream and task applications, targeting different middleware or data services, can be built using the familiar Spring Boot style programming model.
A simple stream pipeline DSL makes it easy to specify which apps to deploy and how to connect outputs and inputs. The composed task DSL is useful for when a series of task apps require to be run as a directed graph.
The dashboard offers a graphical editor for building data pipelines interactively, as well as views of deployable apps and monitoring them with metrics using Wavefront, Prometheus, Influx DB, or other monitoring systems.
Getting Started
The Spring Cloud Data Flow Microsite is the best place to get started.
Quickstart Your Project
Documentation
2.9.5 CURRENT GA | Reference Doc. | API Doc. |
2.10.0-SNAPSHOT SNAPSHOT | Reference Doc. | API Doc. |
2.10.0-M1 PRE | Reference Doc. | API Doc. |
2.9.6-SNAPSHOT SNAPSHOT | Reference Doc. | API Doc. |
2.9.4 GA | Reference Doc. | API Doc. |
2.9.3 GA | Reference Doc. | API Doc. |
2.9.2 GA | Reference Doc. | API Doc. |
2.8.3 GA | Reference Doc. | API Doc. |
2.7.2 GA | Reference Doc. | API Doc. |
2.6.4 GA | Reference Doc. | API Doc. |
2.5.3.RELEASE GA | Reference Doc. | API Doc. |
2.4.2.RELEASE GA | Reference Doc. | API Doc. |
2.3.1.RELEASE GA | Reference Doc. | API Doc. |
Branch | Initial Release | End of Support | End Commercial Support * |
---|---|---|---|
2.9.x
|
2021-10-12 | 2022-10-12 | 2023-10-12 |
2.8.x
|
2021-06-11 | 2022-06-11 | 2023-06-11 |
2.7.x
|
2020-11-30 | 2021-11-30 | 2022-11-30 |
OSS support
Free security updates and bugfixes with support from the Spring community. See VMware Tanzu OSS support policy.
Commercial support
Business support from Spring experts during the OSS timeline, plus extended support after OSS End-Of-Life.
Publicly available releases for critical bugfixes and security issues when requested by customers.
Future release
Generation not yet released, timeline is subject to changes.
About commercial support (*)
A few examples to try out:
- Twitter Analytics In this demonstration, you will learn how to build a data pipeline using Spring Cloud Data Flow to consume data from TwitterStream and compute simple analytics over data-in-transit using Counter sink applications
- Predictive Analytics In this demonstration, you will learn how to use PMML model in the context of streaming data pipeline orchestrated by Spring Cloud Data Flow
- HTTP -> Cassandra In this demonstration, you will learn how to build a data pipeline using Spring Cloud Data Flow to consume data from an HTTP endpoint and write the payload to a Cassandra database
- HTTP -> MySQL In this demonstration, you will learn how to build a data pipeline using Spring Cloud Data Flow to consume data from an http endpoint and write to MySQL database using JDBC sink.
- HTTP -> Gemfire In this demonstration, you will learn how to build a data pipeline using Spring Cloud Data Flow to consume data from an http endpoint and write to Gemfire using the Gemfire/Geode/PCC sink
- Batch File Ingest in CF/K8s In this demonstration, you will learn how to create a data processing application using Spring Batch which will then be run within Spring Cloud Data Flow.
- SCDF, InfluxDB, and Metrics In this demonstration, you will learn how Micrometer can help to monitor your Spring Cloud Data Flow streams using InfluxDB and Grafana
- SCDF, Prometheus, and Metrics In this demonstration, you will learn how Micrometer can help to monitor your Spring Cloud Data Flow Streams using Prometheus and Grafana