Apache IGNITE 2.4

Usually, Ignite community rolls out a new version once in 3 months, but we had to make an exception for Apache Ignite 2.4 that consumed five months in total. We could easily blame Thanksgiving, Christmas and New Year holidays for the delay and would be forgiven, but, in fact, we were forging the release you can't simply pass by.

Discover the benefits of 2.4 by reading this blog post and only then flip through the features and improvements below.

Features and Improvements

Spark Data Frames: support saving a data frame in Ignite [#IGNITE-7337]

DataSet API [#IGNITE-7205]

.NET: Support additional Data Region metrics [#IGNITE-7171]

Implement Multilayer Perceptron [#IGNITE-7129]

Apache Ignite RPM packages [#IGNITE-7107]

Ability to disable WAL (Non recoverable case) [#IGNITE-7003]

Add a flag for Visor batch mode to output only command results without additional info [#IGNITE-6999]

Implement new JMX metrics for Memory Regions [#IGNITE-6902]

KNN(k nearest neighbor) algorithm [#IGNITE-6880]

Implement new JMX metrics for partitions map monitoring [#IGNITE-6871]

Implement new JMX metric for cache topology validation monitoring [#IGNITE-6870]

Implement new JMX metric for jobs monitoring [#IGNITE-6869]

Implement new JMX metrics for TcpCommunicationSpi monitoring [#IGNITE-6868]

Implement new JMX metrics for topology monitoring [#IGNITE-6867]

Ability to turn BaselineTopology handling off [#IGNITE-6694]

Introduction of models import/export [#IGNITE-6640]

Introduction of distributed neural networks [#IGNITE-6386]

BaselineTopology is created on first cluster activation [#IGNITE-6308]

DDL: Support ALTER TABLE DROP COLUMN [#IGNITE-5949]

Thin client protocol [#IGNITE-5896]

Validate baseline topology change history during node joins [#IGNITE-5854]

Automatically activate cluster if baseline topology is reached [#IGNITE-5851]

Introduce an API for baseline topology for persistence-enabled clusters [#IGNITE-5850]

Introduce ignite node persistent meta-store [#IGNITE-5849]

API for Models [#IGNITE-5847]

Fuzzy c-means (FCM) [#IGNITE-5246]

Decision trees [#IGNITE-5218]

Gradient descent for OLS lin reg [#IGNITE-5217]

Add ability to start rebalancing from management console [#IGNITE-4835]

Spark Data Frames Support in Apache Ignite [#IGNITE-3084]

TcpCommunicationSpi metrics causes cache operations slowdown [#IGNITE-7866]

SQL: COPY command should try to create cache in case table is not found [#IGNITE-7714]

Fix windows *.bat scripts Java 9 compatibility [#IGNITE-7550]

Rename KEYS.txt [#IGNITE-7543]

ODBC: use native batching capabilities on the server side [#IGNITE-7443]

Use various backgrounds for Ignite main page's main blocks [#IGNITE-7404]

Improve content on What's Ignite page [#IGNITE-7403]

Java 9: rework "sun.misc.SharedSecrets" and "sun.misc.JavaNioAccess" usages [#IGNITE-7352]

Move all SQL examples out of "data grid" package [#IGNITE-7318]

Update RocketMQ dependencies to 4.2.0 [#IGNITE-7296]

Optimize Ignite main page load performance [#IGNITE-7292]

Apache Ignite "in use at" section [#IGNITE-7291]

performance measurement for FCM and KNN algorithms [#IGNITE-7214]

SQL: Improvde DML encapsulation [#IGNITE-7200]

performance measurement for SparseDistributedMatrix multiplication [#IGNITE-7097]

Add examples for kNN classification and for kNN regression [#IGNITE-7079]

Rework Features menu and page [#IGNITE-7061]

Avoid fsync on WAL segment header write in non-default mode [#IGNITE-7016]

SQL: in-place update should be allowed when indexing is enabled [#IGNITE-7014]

Decision tree code cleanup [#IGNITE-7007]

Refactor dynamic indexes tests to use SQL API [#IGNITE-7001]

GridQueryProcessor.querySqlFieldsNoCache should check for cluster active/inactive state [#IGNITE-6932]

IgniteH2Indexing#rebuildIndexesFromHash should pass old value to indexing engine [#IGNITE-6901]

Linear regression should implement Model API [#IGNITE-6872]

SQL: integrate index inline size to CREATE INDEX syntax [#IGNITE-6850]

SQL parser: support DROP INDEX command [#IGNITE-6848]

SQL parser: support CREATE INDEX command [#IGNITE-6840]

Create common mechanism for group training [#IGNITE-6783]

SQL: prototype for hand-made parser [#IGNITE-6769]

JDBC thin: connection property refactoring [#IGNITE-6752]

Java 9: rework usages of URLClassLoader.getURLs() [#IGNITE-6745]

Java 9: create TC infrastructure to run unit tests [#IGNITE-6744]

Java 9: rework DirectBuffer.cleaner().clean() usage in GridNioServer [#IGNITE-6743]

Java 9: rework Cleaner usage in PlatformMemoryPool class [#IGNITE-6742]

Java 9: rework DirectBuffer.address() usages [#IGNITE-6740]

Java 9: rework GridCacheMapEntry synchronization logic to avoid Unsafe.monitor* methods [#IGNITE-6736]

Java 9: fix Java version resolution/check logic [#IGNITE-6735]

Java 9: do not use sun.misc.BASE64Encoder [#IGNITE-6734]

Java 9: Hadoop module should not start on Java 9 [#IGNITE-6732]

Java 9: remove sun.misc.PerfCounter usages [#IGNITE-6731]

Java 9: fix project build procedure [#IGNITE-6730]

Java 9: fix IgniteUtils#close(URLClassLoader, IgniteLogger) [#IGNITE-6729]

Support Java 9 [#IGNITE-6728]

Web console: Check demo node configuration [#IGNITE-6670]

SQL: optimize primary key equality lookup [#IGNITE-6663]

SQL: move IgniteDistributedJoinTestSuite to query suite(s) [#IGNITE-6634]

SQL: simplify GridH2Row infrastructure [#IGNITE-6632]

Optimize GridH2KeyValueRowOnheap.getValue() method [#IGNITE-6631]

SQL: Avoid materializing rows when possible [#IGNITE-6626]

SQL: IndexingQueryCacheFilter should use immediate partition data if possible [#IGNITE-6624]

SQL: Merge and optimize backup query filter [#IGNITE-6605]

SQL: DDL tests should not check internal structures, but rather rely on expected public behavior [#IGNITE-6416]

SQL: CREATE INDEX should fill index from multiple threads [#IGNITE-6406]

WebConsole: support ClientConnectorConfiguration [#IGNITE-6398]

ODBC: support multiple SQL statements [#IGNITE-6357]

SQL: ANTLR performance assessment [#IGNITE-6320]

BaselineTopology versioning [#IGNITE-6310]

SQL: Add ability to do INSERT without specifying the column names [#IGNITE-6111]

SQL: add native batch execution support for DML statements [#IGNITE-6022]

Web Console: In addition to export also implement copy result set to clipboard [#IGNITE-5641]

Web Console: Show query execution progress [#IGNITE-5635]

DDL needs to support DEFAULT operator [#IGNITE-5623]

Make sure that cache-less execution works as good as cache-based [#IGNITE-5571]

Optimize DML for fast INSERT and MERGE [#IGNITE-4490]

Web console - Admin panel: show different message for 'Grant\Revoke admin' actions [#IGNITE-4466]

Web console: add information on query panel UI about node query was executed on [#IGNITE-4454]

Web console: add execution time to results panel on Queries screen [#IGNITE-4452]

Cache manager should switch cache statisticsEnabled property globaly [#IGNITE-369]

Read external split from V2 Context and V1 Reporter #getInputSplit [#IGNITE-358]

Change example for decision trees [#IGNITE-7590]

Baseline topology should affect only persistent caches [#IGNITE-7502]

Implement metrics for Memory Regions in UI tools [#IGNITE-7492]

Use soft reference for checkpoint entry contents to avoid excessive memory usage [#IGNITE-7471]

Web console: improve list-editable scope variables access [#IGNITE-7395]

Avoid updating PagePartitionCounters in case all counters were not modified [#IGNITE-7380]

WAL converter: add records statistic to WAL reader dev-util [#IGNITE-7378]

Web Console: Add support for custom SMTP servers [#IGNITE-7357]

Create selftest suite for ml examples [#IGNITE-7332]

Use java.util.Base64 instead of current implementations of Base64Encoder interface [#IGNITE-7315]

BaselineTopology: API to reset branching history [#IGNITE-7303]

Thin client: partial cache configuration [#IGNITE-7288]

Thin client: idle connection timeout [#IGNITE-7287]

Web Agent: Support multiple statements on Queries screen [#IGNITE-7274]

Web Console: Improve notification management [#IGNITE-7256]

Add abilities to monitor custom thread pools [#IGNITE-7217]

Java 8 by default [#IGNITE-7203]

Web Console: some improvments [#IGNITE-7199]

Limit the number of collection elements printed in S.toString() [#IGNITE-7194]

Web console: Add example to queries page [#IGNITE-7172]

Web Console: Cleanup webAgentDownloadVisibleStates [#IGNITE-7168]

Implement connected clusters component [#IGNITE-7147]

Web console: Improve ignite icon service interface to extendable [#IGNITE-7133]

Deprecate embedded mode in IgniteRDD [#IGNITE-7092]

S3 IP finder: support server side encryption [#IGNITE-7053]

S3 IP finder: add an ability to provide endpoint address [#IGNITE-7052]

Add support for spring3 [#IGNITE-7050]

Web Console: add column resizer to pinned columns [#IGNITE-7020]

Make cache creation slightly more verbose [#IGNITE-6984]

Visor CMD: Add ability to put/get/remove data to caches via command line Visor [#IGNITE-6976]

Ignite Logger type & logging file config indication [#IGNITE-6971]

Update com.google.code.simple-spring-memcached:spymemcached to 2.8.4 [#IGNITE-6955]

Simplify index rebuild [#IGNITE-6931]

Simplify cache metrics activation [#IGNITE-6925]

Cache metrics are updated in timeout-worker potentially delaying critical code execution due to current implementation issues [#IGNITE-6923]

Add support for baseline manipulations to controls.sh [#IGNITE-6913]

ODBC: Add support for SQL_ATTR_CONNECTION_TIMEOUT [#IGNITE-6876]

Web console: some error messages are too technical [#IGNITE-6873]

Additional JMX metrics for cluster monitoring [#IGNITE-6844]

ODBC: Add support for SQL_ATTR_QUERY_TIMEOUT [#IGNITE-6836]

Web Console: migrate from angularjs 1.5 to angular 1.6 [#IGNITE-6824]

Detailed memory consumption on start and OOM reporting [#IGNITE-6814]

Ignite Kafka Connector certification with Confluent [#IGNITE-6801]

Simplifying and optimising GridNioServer [#IGNITE-6787]

Change block numeration in SparseBlockMatrixStorage [#IGNITE-6761]

Persist system cache when persistent data regions are present [#IGNITE-6746]

Source release should be fixed [#IGNITE-6725]

Get rid of values deserialization in H2TreeIndex.getRowCount [#IGNITE-6702]

Do not deserialize previous value during indexes update [#IGNITE-6701]

DiscoverySpi: Clientmode Ignite should not fail on handshake errors [#IGNITE-6690]

Allow discovery cache instance reuse if only minor topology change has occured [#IGNITE-6667]

Add a version of Ignite instance to logger category [#IGNITE-6658]

Web Console: Implement schema migration [#IGNITE-6647]

PageId handling improvement: PARTITION_ID masking [#IGNITE-6641]

Add an ability for auto activate BLT [#IGNITE-6638]

Allow multiple caches use one SQL schema [#IGNITE-6572]

Add new snapshot record type (SnapshotRecord) [#IGNITE-6550]

Need to support ability log timestamp for wal tx and data records [#IGNITE-6547]

Configure NotNull fields with annotations [#IGNITE-6534]

Add an option to start caches in inactive state [#IGNITE-6512]

performance measurement of decision trees algorithms [#IGNITE-6495]

Add flags to ClientConnectorConfiguration which enable/disable different clients [#IGNITE-6456]

[IGNITE-6390] Web console: Implement component for cluster selection on pages where it make sense [#IGNITE-6390]

Create example for local and distributed k-means algorithm [#IGNITE-6373]

Web Console: Implement control for selecting and activating cluster [#IGNITE-6348]

Use direct IO or libaio for file page store where applicable [#IGNITE-6341]

SQL: Investigate parser generators [#IGNITE-6276]

Rename user facing CREATE TABLE param names to split words properly [#IGNITE-6270]

ODBC: Implement SQL_ATTR_QUERY_TIMEOUT statement attribute [#IGNITE-6156]

First performance measurement of implemented ml algorithms [#IGNITE-6123]

Implement WAL logs compaction and compression after checkpoint [#IGNITE-5938]

Web console: Implement editable list [#IGNITE-5909]

Web Console: Improve the ï¿½ï¿½luster active/inactive states control [#IGNITE-5881]

Add support of distributed matrices for OLS regression [#IGNITE-5846]

Parallel plus/multiply matrix algorithms [#IGNITE-5114]

Support CLEAR command with REST [#IGNITE-4585]

Fixed

ClusterGroupExample duplicate [#IGNITE-7534]

Partition update counters may be inconsistent after rebalancing [#IGNITE-7500]

Fix flaky GridServiceProcessorMultiNodeConfigSelfTest#checkDeployOnEachNodeUpdateTopology [#IGNITE-7340]

Caches info is updated on cluster change finish message in discovery thread [#IGNITE-7308]

Log of thin client is not disabled in special case [#IGNITE-7295]

JDBC thin driver may list table name twice [#IGNITE-7277]

BinaryMetadata may be lost if node with outdated metadata becomes a coordinator [#IGNITE-7258]

"Schema not found" error when setting streaming mode for JDBC driver [#IGNITE-7248]

In case of not serializable cache update response, future on node requester will never complete [#IGNITE-7239]

Nodes may ping each other on stop for a failureDetectionTimeout period [#IGNITE-7206]

GridToStringBuilder should limit large collections output to first N elements [#IGNITE-7195]

ODBC: ODBC return weird names for tables created with DDL [#IGNITE-7189]

Ignite fails to start on non x86/64 architectures [#IGNITE-7178]

Custom discovery messages from plugins are handled incorrectly [#IGNITE-7177]

Throttling is not applied to page allocation [#IGNITE-7175]

Fix javadoc MemoryConfiguration (20% instead of 80%) [#IGNITE-7170]

SQL join with partition and replicated caches fails if number of partitions is too low [#IGNITE-7166]

CPP: Can not insert zero decimal value with the ODBC driver [#IGNITE-7143]

IgniteCluster.startNodes() returns successful ClusterStartNodeResult even though the remote process fails [#IGNITE-7135]

visorcmd: there is no output for last command in batch mode in case of using -nq option [#IGNITE-7121]

Test cases inherited by AbstractSchemaSelfTest became flucky after the refactoring to use SQL API [#IGNITE-7120]

CPP: C++ node can't start without java examples folder [#IGNITE-7114]

Javadoc package descriptions missed for ML [#IGNITE-7110]

JdbcConnection hard linked in IgniteH2Indexing preventing old connections from closing [#IGNITE-7095]

Wrong implementation of DIRECT comparator for ordering cache start operations [#IGNITE-7088]

When PDS is enabled and IGNITE_BINARY_SORT_OBJECT_FIELDS property is set and IgniteCache#put is called a node hangs [#IGNITE-7085]

Text query for a particular field not working [#IGNITE-7055]

Optimistic transaction is not properly rolled back if timed out before sending prepare response [#IGNITE-7049]

IgniteOOM is not propogated to client in case of implicit transaction [#IGNITE-7021]

Ignite MR problem with accessing hdfs with enabled Kerberos [#IGNITE-6992]

Wrong value of RebalancingStartTime cache metric [#IGNITE-6985]

Race in GridClientPartitionTopology may cause NPE and partition map exchange hang [#IGNITE-6979]

Node restarts with enabled persistence lead to affinity assignment mismatch on different nodes [#IGNITE-6973]

TotalAllocatedPages metric does not match PhysicalMemoryPages when persistence is disabled [#IGNITE-6963]

Ignite fails to start on non x86/64 architectures [#IGNITE-6952]

Fail to execute task with immutable list string [#IGNITE-6944]

Web console: SimpleWorkerPool select wrong worker in special case [#IGNITE-6926]

SQL: partition reservations are released too early in lazy mode [#IGNITE-6904]

Visor doesn't show the list of caches after persistence enabled cluster restart [#IGNITE-6897]

visorcmd: 'cache -a' shows nodes that are not matched by nodeFilter [#IGNITE-6863]

Wait for exchange inside GridReduceQueryExecutor.query which never finishes due to opened transaction [#IGNITE-6858]

ODBC: Add new version for multiple result set functionality [#IGNITE-6841]

ODBC driver should handle ungraceful tcp disconnects [#IGNITE-6835]

handle IO errors while checkpointing [#IGNITE-6832]

Ignite SPI: Test local port range failed on TC agents [#IGNITE-6831]

Confusing messages "SLF4J: Failed to load class" at Ignite start [#IGNITE-6828]

Unhandled interruption in GridH2Table [#IGNITE-6825]

In case of incoming communication connection ping the old one if it's alive [#IGNITE-6818]

Handle IO errors in LFS files [#IGNITE-6797]

Ignite WAL reader fails on Tx marker record for persistent store with new style folder naming [#IGNITE-6788]

GridCacheDatabaseSharedManager#persistenceEnabled used incorrectly [#IGNITE-6778]

NearCache#localPeek(key, NEAR) always returns 'null' if the node, which owns the primary partition for the given key, left the cluster [#IGNITE-6767]

ODBC: Error when closing executed DML statement [#IGNITE-6765]

SparseDistributedMatrixExample failed with NPE [#IGNITE-6762]

Get rid of noop segment lock-unlock in PageMemoryImpl [#IGNITE-6719]

Ignite Cache 5 flaky test CacheRebalancingSelfTest.testDisableRebalancing() [#IGNITE-6713]

DataRegionMetrics#totalAllocatedPages is not valid after node restart [#IGNITE-6711]

[Web Console] Wrong java type used in generated config from DB schema [#IGNITE-6671]

CacheStoreSessionListener#onSessionStart() #onSessionEnd() methods are called by GridCacheStoreManagerAdapter even if a store operation should not be performed [#IGNITE-6669]

Ignite node can try to join to itself [#IGNITE-6639]

Web console: Do not show client nodes in node selection modal [#IGNITE-6618]

Web console: there is no error message that cluster is not active on Queries screen [#IGNITE-6617]

WebConsole cache config parse [#IGNITE-6616]

TCP port in bin/control.sh differs from default [#IGNITE-6590]

During start cache we should check temporary file existence besides lock file [#IGNITE-6577]

When a CacheStore with a @SpringResource annotated field is configured Ignite fails to start via igniteSpringBean [#IGNITE-6555]

Standalone WAL iterator fails to handle delete WAL data records [#IGNITE-6553]

SocketChannel may not be closed in createTcpClient() [#IGNITE-6542]

NPE on registerClassName() with MappedName [#IGNITE-6536]

Client node does not reconnect to server node when the latter is restarted [#IGNITE-6496]

Web console: Invalid result on query result with big number [#IGNITE-6463]

Select * doesn't return new field name after concurrent ALTER TABLE [#IGNITE-6448]

DataStructure can not be obtained on client if it is created on server node [#IGNITE-6437]

NPE in Log4J2Logger [#IGNITE-6362]

If getAll() fails with NPE, onHeap entry is not removed, for local cache [#IGNITE-6307]

Check Java 7 builds for compatibility with Ignite and update documentation [#IGNITE-6248]

[Test failure] GridCacheClientModesTcpClientDiscoveryAbstractTest$CaseNearReplicatedTransactional.testGetFromClientNode [#IGNITE-6234]

IgniteContext starts server nodes on executors [#IGNITE-5981]

Client disconnects if server it is connected to goes unresponsive [#IGNITE-5860]

Web console: use byte array type instead of java.lang.Object for binary JDBC types [#IGNITE-5767]

Wrong msg: calculation of memory policy size [#IGNITE-5755]

Wrong metrics calculation in ClusterMetricsSnapshot [#IGNITE-5731]

Web console: Wrong generation of marshaller on version switch [#IGNITE-5407]

DataStreamer can fails if non-data node enter\leave the grid [#IGNITE-5195]

Unexpected error occurs when node left topology [#IGNITE-5189]

NPE in reading data from IGFS [#IGNITE-4862]

SQL: Add support for Java 8 Time API classes in date\time functions [#IGNITE-4172]

ClassLoaders are not switched during object deserialization [#IGNITE-3935]

Ignite does not recognize the right number of CPU cores when running under Docker [#IGNITE-2092]

.NET: Features and Improvements

.NET Core support (run on Linux) [#IGNITE-2662]

.NET: Propagate WAL management API [#IGNITE-7493]

.NET: Ensure that Ignite works on Mono platform [#IGNITE-1628]

.NET: Use InternalsVisibleTo for Core projects [#IGNITE-7390]

.NET: Cluster auto activation (baseline topology) [#IGNITE-7301]

.NET: Rename ICache.QueryFields to Query, add FieldNames [#IGNITE-7229]

.NET: Thin client: Cache benchmark [#IGNITE-7124]

.NET: Verify metrics API parity with a test [#IGNITE-7123]

.NET: Thin client: Async cache operations [#IGNITE-7109]

.NET: Thin client: LINQPad example [#IGNITE-7089]

.NET: Add cross-platform examples on .NET Core [#IGNITE-7002]

.NET: Thin client: Move ClientStatus to public API [#IGNITE-6997]

.NET: Thin client: Remove OP_SQL_QUERY [#IGNITE-6990]

.NET: Thin client: Group operation codes by purpose [#IGNITE-6989]

.NET: Migrate to latest NUnit [#IGNITE-6982]

.NET: Create cross-platform examples on .NET Core [#IGNITE-6975]

.NET: Support pointer serialization [#IGNITE-6951]

.NET: Optionally disable Java console redirect [#IGNITE-6911]

.NET: Thin client: SQL & LINQ example [#IGNITE-6776]

.NET: DataStorageConfiguration.WalAutoArchiveAfterInactivity [#IGNITE-6715]

.NET: CacheConfiguration missing properties [#IGNITE-6705]

.NET: CacheConfiguration.KeyConfiguration [#IGNITE-6704]

.NET: Thin client: XML configuration [#IGNITE-6615]

.NET: DotNetCore examples [#IGNITE-6614]

.NET: Thin client: Create dotnet core unit test project, run on Linux [#IGNITE-6613]

.NET: IIgnite.AddCacheConfiguration [#IGNITE-6598]

.NET: Set up NDepend analysis on TeamCity [#IGNITE-6382]

.NET: Thin client example [#IGNITE-6371]

.NET: Thin client: LINQ [#IGNITE-6338]

.NET: Thin client: SQL queries [#IGNITE-6337]

.NET: Thin client: Create cache [#IGNITE-6336]

.NET: Thin client: cache binary mode [#IGNITE-6335]

.NET: Propagate multiple services deployment [#IGNITE-6272]

.NET: Verify Java API parity with a unit test [#IGNITE-6264]

.NET: Verify Java config parity with a unit test [#IGNITE-6263]

.NET: Get rid of BinaryStreamBase [#IGNITE-5928]

.NET: Pure SQL & DDL example [#IGNITE-5353]

.NET: Interoperate with JVM directly, get rid of C++ layer [#IGNITE-5343]

.NET: Fields query cursor metadata [#IGNITE-5253]

.NET: Custom platform toolset in build script [#IGNITE-5130]

.NET: Fixed

SqlDmlExample: Incorrect result for Delete if run with standalone nodes (Java & .NET) [#IGNITE-7413]

.NET: consoleWrite error during application shutdown [#IGNITE-6974]

.NET: support Multidimensional Arrays in binary serializer [#IGNITE-6896]

.NET: cache deserialization fails with complex value type & enum [#IGNITE-6627]