Onyx 0.8.0 发布,此版本最主要是增加了自动化状态管理功能,恢复,Windowing 和触发器。状态会被记录在 BookKeeper 日志记录,segments 可以使用嵌入式 RocksDB 删除处理。开箱支持多个类型的聚合。 升级注意事项: Onyx 0.8.0 不兼容之前的版本 改进列表: Breaking change nyx.messaging/peer-port-range and nyx.messaging/peer-ports are deprecated in favour of a single nyx.messaging/peer-port port. The Aeron layer multiplexed all communication over a single port so multiple port selection is longer required. Important change: default task batch timeout was reduced from 1000ms to 50ms. New major feature: Windowing and Triggers. New feature: Fixed windows New feature: Sliding windows New feature: Session windows New feature: Global windows New feature: Timer triggers New feature: Segment-based triggers New feature: Watermark triggers New feature: Percentile Watermark triggers New feature: Punctuation triggers New feature: Accumulating refinement mode New feature: Discarding refinement mode New feature: BookKeeper & embedded RocksDB automatic "exactly once" filtering. New feature: friendlier error messages. Added custom schema error handlers to print out documentation and required types in a more customized format. New flux policy: Recover New lifecycle: after-read-batch New metric: peer-send-bytes Bug fix: fixed an issue where the percentage job scheduler would misallocate. Bug fix: fixed :data key being removed from exception messages in the logs. 详细改进内容请看发行说明。 Onyx 是什么? Onyx 是一个无中心、支持云、容错的分布式计算系统 使用 Clojure 编写 支持批处理和流处理混合 提供信息模型用于描述和构建分布式工作流 竞争对手: Storm, Cascading, Map/Reduce, Dryad, Apache Sqoop, Twitter Crane 等等 用途? 实时的事件流处理 连续计算 ETL 在 MapReduce 进行数据转换 数据清理 数据摄入和存储媒介传输 Onyx 0.8.0 发布,分布式计算系统下载地址