content/blog/2022/06/hop-2.0.0/index.adoc (79 lines of code) (raw):

--- title: "Apache Hop 2.0 is available!!" date: 2022-06-02T07:49:56+02:00 authors: ["bamaer"] categories: ["2.0", "Release", "Apache Hop"] preview: "Apache Hop 2.0 Released" description: "Apache Hop 2.0.0 adds support for Java 11, Chinese translations, new Formula, Avro, Drools and Apache Doris plugins and lots of improvements and bug fixes." draft: false --- :imagesdir: ../../../../../static :toc: macro :toclevels: 3 :toc-title: Let's take a closer look at what Hop 2.0 brings: :toc-class: none After three months of work, the Apache Hop community is pleased to announce the general availability of Apache Hop 2.0.0. + A huge thank you to everyone who made this possible. image::/img/Release-2.0/hop-20.svg[ width="45%"]   + toc::[] or https://hop.apache.org/download/[**download Hop 2.0.0 **] right away. == Upgrade to Java 11 Upgrading all of the Apache Hop code to not only run but more importantly build reliably on Java 11 instead of Java 8 is not a small task, and therefore justified a major release. The upgrade to Java 11 has been in te making for months, meticulously fixing and extending the tests and code issues. With this 2.0 release, Hop runs reliably on Java 11. We used the necessary code changes for the Java 11 upgrade as an opportunity to make some code cleanup and breaking API changes. Check https://hop.apache.org/dev-manual/latest/upgrading-to-20.html[the docs^] to see which changes you need to make to upgrade your Hop plugins to 2.0. == Chinese translations Apache Hop is increasingly popular in Asia, with a large and growing Chinese community. Hop 2.0.0 includes a major contribution (thanks, https://github.com/shlxue[Shl Xue^]!) to translate Apache Hop to Simplified Chinese (zh_CN). This contribution includes updates and improvements to Hop Translator. image:/img/Release-2.0/hop-gui-simplified-chinese.png[Hop Gui - Simplified Chinese, width="90%"] == New transform plugins === Apache Avro File Output The https://hop.apache.org/manual/next/pipeline/transforms/avro-file-output.html[Apache Avro File Output^] transforms writes to binary files or fields in Avro Binary or JSON formats. image:/img/Release-2.0/apache-avro-file-output-transform.png[Avro File Output transform, width="90%"] === Apache Doris Bulk Loader https://doris.apache.org[Apache Doris^] is a modern MPP analytical database product. It can provide sub-second queries and efficient real-time data analysis. With it’s distributed architecture, up to 10PB level datasets will be well-supported and easy to operate. The https://hop.apache.org/manual/next/pipeline/transforms/dorisbulkloader.html[Apache Doris Bulk Loader^] transform allows you to insert data into Apache Doris at high speed and volume, making it a faster way to load data than using the traditional database insert statements. The Apache Doris Bulk Loader transform was kindly contributed by the Apache Doris community. image:/img/Release-2.0/apache-doris-bulk-loader-transform.png[Avro File Output transform, width="90%"] === Drools Rules Accumulator The https://hop.apache.org/manual/next/pipeline/transforms/rulesaccumulator.html[Drools Accumulator^] transform collects incoming rows and executes them against a rule set. This may be useful to determine the answer to a question or otherwise analyze a dataset. image:/img/Release-2.0/drools-rules-accumulator-transform.png[Drools Rules Accumulator transform, width="90%"] === Drools Rules Executor The https://hop.apache.org/manual/next/pipeline/transforms/rulesexecutor.html[Drools Rule Executor^] transform allows fields of incoming rows to be executed against a rule set. This may be useful to determine additional information or route rows onto another transform. image:/img/Release-2.0/drools-rules-executor-transform.png[Drools Rules Executor transform, width="90%"] Many thanks to add these transforms to Apache Hop, https://twitter.com/sramazzina[Sergio^]! === Formula The https://hop.apache.org/manual/next/pipeline/transforms/formula.html[Formula^] transform allows you to apply Excel-like formulas and functions on fields in a pipeline. The screenshot below shows a couple of examples of date and time functions with the Apache Hop graduation date. image:/img/Release-2.0/formula-transform.png[Formula transform, width="90%"] == Apache Beam upgrade https://beam.apache.org[Apache Beam^] is an advanced unified programming model that allows you to implement batch and streaming data processing jobs that run on any execution engine. Popular execution engines are for example Apache Spark, Apache Flink or Google Cloud Platform Dataflow. Apache Beam has been an important plugin in Apache Hop, and was upgraded to 2.38.0 with https://spark.apache.org[Apache Spark^] 3.1.3 and https://flink.apache.org[Apache Flink^] 1.14.4. image:https://beam.apache.org/images/beam_logo_navbar.png[Apache Beam, width="25%"] == Various === Variable Annotations A new `@Variable` annotation makes it easier for Hop developers to declare variables at the right place in the code. Static final fields annotated with "Variable" are automatically recognized and registered as a described variable. Thanks, https://github.com/nadment[Nicolas^]! === Dimension Lookup/Update new dialog The https://hop.apache.org/manual/next/pipeline/transforms/dimensionlookup.html[Dimension Lookup/Update^] transform has been around for ages. Over time, so much functionality was added to the transform's dialog that it became cluttered. The dialog has been cleaned up and now shows the available options in 4 tabs: keys, fields, technical key and versioning. image:/img/Release-2.0/dimension-lookup-update-transform.png[Dimension Lookup Update transform, width="90%"] === Integration tests The library of integration tests continues to grow. Hop 2.0 was tested on a daily basis with close to 200 integration tests. These integration tests make sure that bugs that are fixed remain fixed (prevent regressions), to make sure all existing workflows and pipelines continue to work as Hop evolves, and help to keep Hop as a platform as robust and reliable as possible. Feel free to follow the status of the integration tests at any moment on the https://ci-builds.apache.org/job/Hop/job/Hop-integration-tests/[Hop CI Builds^]. == Community The Hop community continues to grow! The overview below shows the community growth compared to the 1.2.0 release in March: * chat: 355 registered members (up from 300) link:https://chat.project-hop.org[join] * LinkedIn: 911 followers (up from 840) link:https://www.linkedin.com/company/hop-project[follow] * Twitter: 714 followers (up from 670) link:https://twitter.com/ApacheHop[follow] * YouTube: 519 subscribers (up from 452) link:https://www.youtube.com/ApacheHop[subscribe] * 3Hx Meetup: 236 members (up from 219) link:https://www.meetup.com/3hx-apache-hop-incubating-hot-hop-hangouts/[join] Without community interaction and contribution, Hop is just a coding club! Please feel free to join, participate in the discussion, test, file bug tickets on the software or documentation, ... Contributing is a lot more than writing code. Check out our link:/community/contributing/[contribution guides^] and http://hop.apache.org/community/ethos/[Code of Conduct^] to find out more. == JIRA Hop 2.0.0 contains work on 151 tickets: * Resolved: 143 * Closed: 8 Check the https://issues.apache.org/jira/browse/HOP-3962?jql=project%20%3D%20%22Apache%20Hop%22%20and%20fixVersion%20%3D%202.0.0[Hop Jira^] for a full overview of all tickets.