content/blog/mesos-0-17-0-released-featuring-autorecovery/index.html

<!DOCTYPE html> <html> <head> <meta charset="utf-8"> <title>Mesos 0.17.0 Released, Featuring Autorecovery for the Replicated Log</title> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <meta property="og:locale" content="en_US"/> <meta property="og:type" content="website"/> <meta property="og:title" content="Apache Mesos"/> <meta property="og:site_name" content="Apache Mesos"/> <meta property="og:url" content="http://mesos.apache.org/"/> <meta property="og:image" content="http://mesos.apache.org/assets/img/mesos_logo_fb_preview.png"/> <meta property="og:description" content="Apache Mesos abstracts resources away from machines, enabling fault-tolerant and elastic distributed systems to easily be built and run effectively."/> <meta name="twitter:card" content="summary"/> <meta name="twitter:site" content="@ApacheMesos"/> <meta name="twitter:title" content="Apache Mesos"/> <meta name="twitter:image" content="http://mesos.apache.org/assets/img/mesos_logo_fb_preview.png"/> <meta name="twitter:description" content="Apache Mesos abstracts resources away from machines, enabling fault-tolerant and elastic distributed systems to easily be built and run effectively."/> <link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet"> <link rel="alternate" type="application/atom+xml" title="Apache Mesos Blog" href="/blog/feed.xml"> <link href="../../assets/css/main.css" media="screen" rel="stylesheet" type="text/css" />  <script type="text/javascript"> var _gaq = _gaq || []; _gaq.push(['_setAccount', 'UA-20226872-1']); _gaq.push(['_setDomainName', 'apache.org']); _gaq.push(['_trackPageview']); (function() { var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true; ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s); })(); </script> </head> <body>  <div class="topnav"> <div class="container"> <ul class="breadcrumb"> <li> <div class="dropdown"> <a data-toggle="dropdown" href="#">Apache Software Foundation <span class="caret"></span></a> <ul class="dropdown-menu" role="menu" aria-labelledby="dLabel"> <li><a href="http://www.apache.org">Apache Homepage</a></li> <li><a href="http://www.apache.org/licenses/">License</a></li> <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li> <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li> <li><a href="http://www.apache.org/security/">Security</a></li> </ul> </div> </li> <li><a href="http://mesos.apache.org">Apache Mesos</a></li> <li><a href="/blog /">Blog </a></li> </ul> </div> </div>  <div class="navbar navbar-default navbar-static-top" role="navigation"> <div class="container"> <div class="navbar-header"> <button type="button" class="navbar-toggle collapsed" data-toggle="collapse" data-target="#mesos-menu" aria-expanded="false"> <span class="sr-only">Toggle navigation</span> <span class="icon-bar"></span> <span class="icon-bar"></span> <span class="icon-bar"></span> </button> <a class="navbar-brand" href="/"><img src="/assets/img/mesos_logo.png" alt="Apache Mesos logo"/></a> </div> <div class="navbar-collapse collapse" id="mesos-menu"> <ul class="nav navbar-nav navbar-right"> <li><a href="/gettingstarted/">Getting Started</a></li> <li><a href="/blog/">Blog</a></li> <li><a href="/documentation/latest/">Documentation</a></li> <li><a href="/downloads/">Downloads</a></li> <li><a href="/community/">Community</a></li> </ul> </div> </div> </div> <div class="content"> <div class="container"> <div class="row"> <div class="col-md-3"> <div class="meta"> <span class="author"> <span class="author_contact"> <p><strong>Jie Yu</strong></p> <p><a href="http://twitter.com/jie_yu">@jie_yu</a></p> </span> </span> <p><em>Posted March 13, 2014</em></p> </div> <div class="share"> <span class="social-share-button"><a href="https://twitter.com/share" class="twitter-share-button" data-via="apachemesos">Tweet</a></span> <span><script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script></span> <span><div class="g-plusone" data-size="medium"></div></span>  <script type="text/javascript"> (function() { var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = 'https://apis.google.com/js/plusone.js'; var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s); })(); </script> <script src="//platform.linkedin.com/in.js" type="text/javascript"> lang: en_US </script> <script type="IN/Share" data-counter="right"></script> </div> </div> <div class="post col-md-9"> <h1>Mesos 0.17.0 Released, Featuring Autorecovery for the Replicated Log</h1> <p>The latest Mesos release, <a href="http://mesos.apache.org/downloads/">0.17.0</a> was made available for <a href="http://mesos.apache.org/downloads">download</a> last week, introducing a new autorecovery feature for the replicated log that handles additional disk failure and operator error scenarios.</p> <h2>What is the replicated log?</h2> <p>The replicated log is a distributed <em>append-only</em> log that stores arbitrary data as entries. Replication comes into play as at least a quorum of log replicas are stored on different machines, protecting the data from individual disk or network failures. The replicated log uses the <a href="http://en.wikipedia.org/wiki/Paxos_(computer_science%29">Paxos algorithm</a>. We will describe additional implementation details in both an upcoming blog post and project documentation.</p> <p>Mesos has had replicated log since version 0.9.0 and <a href="http://aurora.incubator.apache.org/">Apache Aurora</a>, a framework used in production at Twitter, uses replicated log for its persistent storage. Additionally, the Mesos master will start using the replicated log to persist some cluster state with the introduction of the <a href="https://issues.apache.org/jira/browse/MESOS-764">registrar</a>.</p> <h2>New feature in Mesos 0.17.0: <em>autorecovery</em></h2> <p>Before Mesos version 0.17.0, the replicated log could become inconsistent if a replica’s disk was replaced or the data was accidentally deleted by an operator. That meant that an operator couldn’t safely, or easily, replace a replica’s disk while the other replicas kept running. Instead, an operator had to stop all replicas, replace the disk, copy a pre-existing log, then restart all the replicas.</p> <p>As of this release, a replica with a new disk can automatically recover without operator involvement. As long as a quorum of replicas are available, applications using the replicated log won’t notice a thing!</p> <h2>Future work</h2> <p>The new autorecovery feature enables replacing a single replica at a time, but it does not allow an operator to easily add or remove replicas, known as <em>reconfiguration</em>. You can follow <a href="https://issues.apache.org/jira/browse/MESOS-683">MESOS-683</a> for the latest on this future work.</p> </div> </div> </div> </div> <hr>  <div class="footer"> <div class="container"> <div class="col-md-4 social-blk"> <span class="social"> <a href="https://twitter.com/ApacheMesos" class="twitter-follow-button" data-show-count="false" data-size="large">Follow @ApacheMesos</a> <script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script> <a href="https://twitter.com/intent/tweet?button_hashtag=mesos" class="twitter-hashtag-button" data-size="large" data-related="ApacheMesos">Tweet #mesos</a> <script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script> </span> </div> <div class="col-md-8 trademark"> <p>© 2012-2017 <a href="http://apache.org">The Apache Software Foundation</a>. Apache Mesos, the Apache feather logo, and the Apache Mesos project logo are trademarks of The Apache Software Foundation. <p> </div> </div> </div>  <script src="//code.jquery.com/jquery-1.11.0.min.js" type="text/javascript"></script> <script src="//netdna.bootstrapcdn.com/bootstrap/3.1.1/js/bootstrap.min.js" type="text/javascript"></script> </body> </html>

content/blog/mesos-0-17-0-released-featuring-autorecovery/index.html (156 lines of code) (raw):