Eric Bottard

TAB completion

Keyboard shortcuts

Quotes handling in Spring Shell

Quotes handling in Data Flow DSL

Stability Improvements The instability problems when using SCDF’s 1.1.0.RELEASE on PCF manifested to the users in the following forms.

Error Handling Improvements The improved error handling and logging support include the operations and its call chain details. This is especially useful when performing 100s of stream/task operations in SCDF, and if anything fails in the process, it will be easy to correlate and determine the root-cause more efficiently. There has been error handling improvements in SCDF’s cf-deployer, cf-java-client, and reactor-core projects.

Network Call Optimizations While invoking stream list from the shell or navigating to the equivalent stream list page in the Dashboard, we were performing a network call for each application associated with the stream(s). This is now refactored and replaced to efficiently query for all the application statuses in a single network call if the backing runtime supports it (and the Cloud Foundry deployer certainly does). This change significantly improves the user-experience of the stream listing operation.

Stress/Load Testing In the process of addressing above mentioned instability patterns, we have built a comprehensive integration and acceptance test harness. With the load-testing routines simulating 100s of concurrent operations on the server including “stream creates”, “stream deploys”, “stream destroys”, and “stream lists”, we were able to monitor the overall health of the SCDF server and as well correlate to confirm the successful execution of operations in entirety.

Memory Optimizations: After much troubleshooting, we are pleased to deliver a memory efficient SCDF server in this release. With the changes now, regardless of how many stream/task deployments and concurrency, we have observed the memory usage of the server stay within the 200MB range as opposed to 1G or over to an eventual crash in previous 1.0.0.RELEASE. If you experienced spurious crashes previously, we highly recommend upgrading to 1.1.0.RELEASE right away.

Maven Cache and Disk Optimizations: We have improved the way Maven caching is applied and used. There is a new LRU (Least Recently Used) design on top of the maven cache to allow us to be more proactive about the cache size as opposed to letting it grow indefinitely, which if not governed, leads to out of disk space errors for the application container where the server is running.

Tasks Support: In the latest PCF 1.9 release, the Task component graduated from being experimental to production-ready feature capability. Spring Cloud Data Flow’s Task support for Cloud Foundry closely followed this development and with this release, we have removed the “experimental” clause from our side, too. Spring Cloud Data Flow’s Task support is production-ready when used on PCF 1.9. One can still try this functionality on older PCF releases, PCF 1.8 for example, but the feature capabilities neither guaranteed nor supported on older PCF releases since is it classified as ”experimental”.

Improved Error Handling: Building on the latest version of the Spring Cloud Deployer for Cloud Foundry project, interaction with the Cloud Controller REST API has been made more resilient, being more pro-active in dealing with the occasional server-side error.

Builds upon 2.0.1.RELEASE of Cloud Foundry’s Java Client and 3.0.2.RELEASE of Reactor

Builds upon 1.0.1.RELEASE of Spring Cloud Data Flow core

Gets general clarifications and improvements in the reference documentation

Fixes a bug pertaining to multiple ORG + Space combination causing task launch failures

Fixes a bug where environment variables were not propagated correctly when launching a task

Builds upon the 1.0.0.RC1 release of Spring Cloud Deployer Cloud Foundry, which itself builds upon Project Reactor 3.0.0.RELEASE and of CF-Java-Client 2.0.0.RELEASE

Adds the ability to orchestrate short-lived tasks including Spring Batch Jobs in Cloud Foundry, which can be enabled as an experimental feature toggle

Adds support for command line arguments as a separate set of properties to be passed to a Task when it’s launched

Adds support to separate stream and task specific service bindings. This allows pinning stream and tasks specific services to stream and task applications respectively

Adds the ability to override the buildpack used on a per-app deployment basis

Significant improvements to the Spring Cloud Deployer Cloud Foundry TCK test coverage allowed refinements in app-state calculation

Adds documentation about rolling upgrades and how to integrate with Spring Cloud Config Server "service" in Pivotal Cloud Foundry

Builds upon 1.0.0.RELEASE of Spring Cloud Data Flow Core (shell, UI, REST-APIs, etc.)

Streamlines application vs. deployment property semantics

Adds a migration guide and other documentation fragments to the reference documentation

Adds a project site

We are looking forward to Reactor and Cloud Foundry Java Client’s RC and GA releases and by this month, we shall release 1.0.0.RELEASE of Spring Cloud Data Flow for Cloud Foundry.

We have exciting work-in-progress to orchestrate short-lived microservices through TaskLauncher constructs in the Spring Cloud Deployer for Cloud Foundry project. We are hopeful to deliver the BETA release of this functionality alongside the upcoming Pivotal Cloud Foundry’s 1.7.x release.

Alignment with Cloud Foundry’s global environment properties such as ORG, API, and URL as opposed to previous naming variants

Adds security integration to authenticate against OAUTH backend server that’s compatible with Cloud Foundry

Publishes security configurations for easier overrides when connecting to OAUTH backend server that’s compatible with Cloud Foundry

Preliminary iterations on performance testing on Cloud Foundry with and without taps’s in real-time streaming pipeline

Builds upon feature-toggle support from Spring Cloud Data Flow’s core project to conditionally refine what features get included in each release