_posts/2014-04-11-the_effect_of_columnfamily

--- layout: post status: PUBLISHED published: true title: The Effect of ColumnFamily, RowKey and KeyValue Design on HFile Size id: 6ac717dd-f6ad-457d-a2b9-7dae88a5733e date: '2014-04-11 18:29:54 -0400' categories: hbase tags: - row - keys - hfile - hbase permalink: hbase/entry/the_effect_of_columnfamily_rowkey --- By Doug Meil, HBase Committer and Thomas Murphy Intro One of the most common questions in the HBase user community is estimating disk footprint of tables, which translates into HFile size – the internal file format in HBase. We designed an experiment at Explorys where we ran combinations of design time options (rowkey length, column name length, row storage approach) and runtime options (HBase ColumnFamily compression, HBase data block encoding options) to determine these factors’ effects on the resultant HFile size in HDFS. HBase Environment CDH4.3.0 (HBase 0.94.6.1) Design Time Choices <ol> <li> Rowkey <ol type="a"> <li> Thin <ol type="i"> <li> 16-byte MD5 hash of an integer. </li> </ol> </li> <li> Fat <ol type="i"> <li> 64-byte SHA-256 hash of an integer. </li> </ol> </li> </ol> </li> </ol> <ol> <ol type="a" start="3"> <li> Note: neither of these are realistic rowkeys for real applications, but they chosen because they are easy to generate and one is a lot bigger than the other. </li> </ol> </ol> <ol start="2"> <li> Column Names <ol type="a"> <li> Thin <ol type="i"> <li> 2-3 character column names (c1, c2). </li> </ol> </li> <li> Fat <ol type="i"> <li> 10 characters, randomly chosen but consistent for all rows. </li> </ol> </li> </ol> </li> </ol> <ol> <ol type="a" start="3"> <li> Note: it is advisable to have small column names, but most people don’t start that way so we have this as an option. </li> </ol> </ol> <ol start="3"> <li> Row Storage Approach <ol type="a"> <li> Key Value Per Column <ol type="i"> <li> This is the traditional way of storing data in HBase. </li> </ol> </li> <li> One Key Value per row. <ol type="i"> <li> Actually, two. </li> <li> One KV has an Avro serialized byte array containing all the data from the row. </li> <li> Another KV holds an MD5 hash of the version of the Avro schema. </li> </ol> </li> </ol> </li> </ol> Run Time <ol> <li> Column Family Compression <ol type="a"> <li> None </li> <li> GZ </li> <li> LZ4 </li> <li> LZO </li> <li> Snappy </li> </ol> </li> </ol> <ol> <ol type="a" start="6"> <li> Note: it is generally advisable to use compression, but what if you didn’t? So we tested that too. </li> </ol> </ol> <ol start="2"> <li> HBase Block Encoding <ol type="a"> <li> None </li> <li> Prefix </li> <li> Diff </li> <li> Fast Diff </li> </ol> </li> </ol> <ol> <ol type="a" start="5"> <li> Note: most people aren’t familiar with HBase Data Block Encoding. Primarily intended for squeezing more data into the block cache, it has effects on HFile size too. See HBASE-4218 for more detail. </li> </ol> </ol> 1000 rows were generated for each combination of table parameters. Not a ton of data, but we don’t necessarily need a ton of data to see the varying size of the table. There were 30 columns per row comprised of 10 strings (each filled with 20 bytes of random characters), 10 integers (random numbers) and 10 longs (also random numbers). HBase blocksize was 128k. Results The easiest way to navigate the results is to compare specific cases, progressing from an initial implementation of a table to options for production. Case #1: Fat Rowkey and Fat Column Names, Now What? This is where most people start with HBase. Rowkeys are not as optimal as they should be (i.e., the Fat rowkey case) and column names are also inflated (Fat column-names). Without CF Compression or Data Block Encoding, the baseline is: <table width="712" cellpadding="8" cellspacing="0"> <colgroup> <col width="299" /> <col width="83" /> <col width="83" /> <col width="83" /> <col width="83" /> </colgroup> <tbody> <tr valign="bottom"> <td width="299" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> psudorandom-table-R1000-i10-s10_20-l10-NONE-NONE-FATKEY-FATCOL </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 6,293,670 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> </tr> </tbody> </table> What if we just changed CF compression? This drastically changes the HFile footprint. Snappy compression reduces the HFile size from 6.2 Mb to 1.8 Mb, for example. <table width="397" cellpadding="8" cellspacing="0"> <colgroup> <col width="83" /> <col width="83" /> <col width="83" /> <col width="83" /> </colgroup> <tbody> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,362,033 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> GZ </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,803,240 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> SNAPPY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,919,265 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> LZ4 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,950,306 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> LZO </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> </tr> </tbody> </table> However, we shouldn’t be too quick to celebrate. Remember that this is just the disk footprint. Over the wire the data is uncompressed, so 6.2 Mb is still being transferred from RegionServer to Client when doing a Scan over the entire table. What if we just changed data block encoding? Compression isn’t the only option though. Even without compression, we can change the data block encoding and also achieve HFile reduction. All options are an improvement over the 6.2 Mb baseline. <table width="397" cellpadding="8" cellspacing="0"> <colgroup> <col width="83" /> <col width="83" /> <col width="83" /> <col width="83" /> </colgroup> <tbody> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,491,000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> DIFF </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,492,155 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> FAST_DIFF </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 2,244,963 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> PREFIX </td> </tr> </tbody> </table> Combination The following table shows the results of all remaining CF compression / data block encoding combinations. <table width="397" cellpadding="8" cellspacing="0"> <colgroup> <col width="83" /> <col width="83" /> <col width="83" /> <col width="83" /> </colgroup> <tbody> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,146,675 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> GZ </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> DIFF </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,200,471 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> GZ </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> FAST_DIFF </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,274,265 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> GZ </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> PREFIX </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,350,483 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> SNAPPY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> DIFF </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,358,190 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> LZ4 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> DIFF </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,391,016 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> SNAPPY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> FAST_DIFF </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,402,614 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> LZ4 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> FAST_DIFF </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,406,334 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> LZO </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> FAST_DIFF </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,541,151 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> SNAPPY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> PREFIX </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,597,440 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> LZO </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> PREFIX </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,622,313 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> LZ4 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> PREFIX </td> </tr> </tbody> </table> Case #2: What if we re-designed the column names (and left the rowkey alone)? Let’s assume that we re-designed our column names but left the rowkey alone. After using the “thin” column-names without CF compression or data block encoding it results in an HFile 5.8 Mb in size. This is an improvement from the original 6.2 Mb baseline. It doesn’t seem like much, but it’s still a 6.5% reduction in the eventual wire-transfer footprint. <table width="712" cellpadding="8" cellspacing="0"> <colgroup> <col width="299" /> <col width="83" /> <col width="83" /> <col width="83" /> <col width="83" /> </colgroup> <tbody> <tr valign="bottom"> <td width="299" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> psudorandom-table-R1000-i10-s10_20-l10-NONE-NONE-FATKEY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 5,778,888 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> </tr> </tbody> </table> Applying Snappy compression can reduce the HFile size further: <table width="397" cellpadding="8" cellspacing="0"> <colgroup> <col width="83" /> <col width="83" /> <col width="83" /> <col width="83" /> </colgroup> <tbody> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,349,451 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> SNAPPY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> DIFF </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,390,422 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> SNAPPY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> FAST_DIFF </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,536,540 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> SNAPPY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> PREFIX </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,785,480 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> SNAPPY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> </tr> </tbody> </table> Case #3: What if we re-designed the rowkey (and left the column names alone)? In this example, what if we only redesigned the rowkey? After using the “thin” rowkey the result is an HFile size that is 4.9 Mb down from the 6.2 Mb baseline, a 21% reduction. Not a small savings! <table width="712" cellpadding="8" cellspacing="0"> <colgroup> <col width="299" /> <col width="83" /> <col width="83" /> <col width="83" /> <col width="83" /> </colgroup> <tbody> <tr valign="bottom"> <td width="299" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> psudorandom-table-R1000-i10-s10_20-l10-NONE-NONE-FATCOL </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 4,920,984 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> </tr> </tbody> </table> Applying Snappy compression can reduce the HFile size further: <table width="397" cellpadding="8" cellspacing="0"> <colgroup> <col width="83" /> <col width="83" /> <col width="83" /> <col width="83" /> </colgroup> <tbody> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,295,895 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> SNAPPY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> DIFF </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,337,112 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> SNAPPY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> FAST_DIFF </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,489,446 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> SNAPPY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> PREFIX </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,739,871 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> SNAPPY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> </tr> </tbody> </table> However, note that the resulting HFile size with Snappy and no data block encoding (1.7 Mb) is very similar in size to the baseline approach (i.e., fat rowkeys, fat column-names) with Snappy and no data block encoding (1.8 Mb). Why? The CF compression can compensate on disk for a lot of bloat in rowkeys and column names. Case #4: What if we re-designed both the rowkey and the column names? By this time we’ve learned enough HBase to know that we need to have efficient rowkeys and column-names. This produces an HFile that is 4.4 Mb, a 29% savings over the baseline of 6.2 Mb. <table width="397" cellpadding="8" cellspacing="0"> <colgroup> <col width="83" /> <col width="83" /> <col width="83" /> <col width="83" /> </colgroup> <tbody> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 4,406,418 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> </tr> </tbody> </table> Applying Snappy compression can reduce the HFile size further: <table width="397" cellpadding="8" cellspacing="0"> <colgroup> <col width="83" /> <col width="83" /> <col width="83" /> <col width="83" /> </colgroup> <tbody> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,296,402 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> SNAPPY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> DIFF </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,338,135 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> SNAPPY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> FAST_DIFF </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,485,192 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> <a name="_GoBack"></a>1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> SNAPPY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> PREFIX </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,732,746 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> SNAPPY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> </tr> </tbody> </table> Again, the on-disk footprint with compression isn’t radically different from the others, as Compression can compensate to large degree for rowkey and column name bloat. Case #5: KeyValue Storage Approach (e.g., 1 KV vs. KV-per-Column) What if we did something radical and changed how we stored the data in HBase? With this approach, we are using a single KeyValue per row holding all of the columns of data for the row instead of a KeyValue per column (the traditional way). The resulting HFile, even uncompressed and without Data Block Encoding, is radically smaller at 1.4 Mb compared to 6.2 Mb. <table width="712" cellpadding="8" cellspacing="0"> <colgroup> <col width="299" /> <col width="83" /> <col width="83" /> <col width="83" /> <col width="83" /> </colgroup> <tbody> <tr valign="bottom"> <td width="299" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> psudorandom-table-R1000-i10-s10_20-l10-NONE-NONE-AVRO </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,374,465 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> </tr> </tbody> </table> Adding Snappy compression and Data Block Encoding makes the resulting HFile size even smaller. <table width="397" cellpadding="8" cellspacing="0"> <colgroup> <col width="83" /> <col width="83" /> <col width="83" /> <col width="83" /> </colgroup> <tbody> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,119,330 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> SNAPPY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> DIFF </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,129,209 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> SNAPPY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> FAST_DIFF </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,133,613 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> SNAPPY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> PREFIX </td> </tr> <tr valign="bottom"> <td width="83" height="7" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1,150,779 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> 1000 </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> SNAPPY </td> <td width="83" bgcolor="#ffffff" style="border: none; padding: 0in;"> NONE </td> </tr> </tbody> </table> Compare the 1.1 Mb Snappy without encoding to the 1.7 Snappy encoded Thin rowkey/Thin column-name. Summary Although Compression and Data Block Encoding can wallpaper over bad rowkey and column-name decisions in terms of HFile size, you will pay the price for this in terms of data transfer from RegionServer to Client. Also, concealing the size penalty brings with it a performance penalty each time the data is accessed or manipulated. So, the old advice about correctly designing rowkeys and column names still holds. In terms of KeyValue approach, having a single KeyValue per row presents significant savings both in terms of data transfer (RegionServer to Client) as well as HFile size. However, there is a consequence with this approach in having to update each row entirely, and that old versions of rows also be stored in their entirety (i.e., as opposed to column-by-column changes). Furthermore, it is impossible to scan on select columns; the whole row must be retrieved and deserialized to access any information stored in the row. The importance of understanding this tradeoff cannot be over-stated, and is something that must be evaluated on an application-by-application basis. Software engineering is an art of managing tradeoffs, so there isn’t necessarily one “best” answer. Importantly, this experiment only measures the file size and not the time or processor load penalties imposed by the use of compression, encoding, or Avro. The results generated in this test are still based on certain assumptions and your mileage may vary. Here is the data if interested: <a href="http://people.apache.org/~dmeil/HBase_HFile_Size_2014_04.csv" target="_blank" style="color: #1155cc; font-family: Calibri, sans-serif; font-size: 14px;">http://people.apache.org/~dmeil/HBase_HFile_Size_2014_04.csv</a>

_posts/2014-04-11-the_effect_of_columnfamily_rowkey.html (997 lines of code) (raw):