SD Occasions information digest: Apache Spark launched, OpenAPI RC0, Pinpoint Agile ceremonies


The brand new launch contains an accelerator-aware scheduler, adaptive question execution, dynamic partition pruning, in addition to efficiency and SQL compatibility enhancements.

Spark SQL is the highest energetic part on this launch with 46% of the resolved tickets are for Spark SQL. This 12 months can also be the 10-year anniversary of the Spark venture.

Within the TPC-DS 30TB benchmark, Spark is roughly two instances quicker than Spark 2.four.

Further particulars on the brand new launch can be found right here.


The primary launch candidate of OpenAPI now helps 100% compatibility with the most recent draft of JSON Schema.

It features a new top-level component for describing Webhooks which might be registered and managed out of band. It additionally contains help for figuring out API licenses utilizing the SPDX identifier. 

The PathItems object is now elective to make it easier to create reusable libraries of elements. Reusable PathItems could be described within the elements object. There may be additionally help for describing APIs secured utilizing shopper certificates.