tag:tonyarcieri.com,2014:/feedTony Arcieri2019-11-07T07:30:37-08:00Tony Arcierihttp://tonyarcieri.comSvbtle.comtag:tonyarcieri.com,2014:Post/rust-2020-towards-a-1-0-crate-ecosystem2019-11-07T07:30:37-08:002019-11-07T07:30:37-08:00Rust 2020: towards a 1.0 crate ecosystem<p><a href="https://svbtleusercontent.com/dpbCKrTMxwqwWNwF1eNDuh0xspap.png"><img src="https://svbtleusercontent.com/dpbCKrTMxwqwWNwF1eNDuh0xspap_small.png" alt="letsgobroncos.png"></a></p>
<p>The Rust language itself went 1.0 in 2015. And yet, almost 5 years later, at least in my anecdotal assessment talking to and reading things written by current or potential users, there is still a sense that the language is still in some way unfinished and a bit rough around the edges.</p>
<p>I sympathize with these concerns.</p>
<p>There are various language-level issues which are partly to blame for this. The absence of a comprehensive story around asynchronous I/O and event-driven programs in general has been one of them, but that all changes… today… with the long awaited <a href="https://blog.rust-lang.org/2019/11/07/Rust-1.39.0.html">stabilization of <code class="prettyprint">async</code>/<code class="prettyprint">await</code> in the Rust 1.39 release</a> along with the release of <a href="https://github.com/rust-lang-nursery/futures-rs/blob/master/CHANGELOG.md#030---2019-11-5"><code class="prettyprint">futures</code> v0.3.0</a> earlier this week.</p>
<p>There are other core features which many desire. <a href="https://tonyarcieri.com/rust-in-2019-security-maturity-stability#const-generics-and-cryptography_2">In my Rust in 2019 blog post I wrote about const generics</a> as the #1 in my top 3, along with <code class="prettyprint">async</code>/<code class="prettyprint">await</code> and <code class="prettyprint">Pin</code>, which both shipped this year.</p>
<p>Others long for even more advanced type system features, primarily those around the general vicinity of associated type constructors, generic associated types, and higher kinded types, which would elevate the corner Rust occupies on the <a href="https://en.wikipedia.org/wiki/Lambda_cube">Lambda cube</a>.</p>
<p>But if you were to ask me what’s really holding Rust back, and the true source of the perception that Rust is “rough” or “unfinished”, I honestly don’t think it’s core language features, at least, as of today: my foremost concern there, <code class="prettyprint">async</code>/<code class="prettyprint">await</code>, is shipping right now.</p>
<p>To me the biggest “unfinished” part of Rust is the crate ecosystem.</p>
<p>This isn’t to say that people aren’t writing crates, and that they aren’t writing damn good crates! No, quite the opposite, the crate ecosystem is blossoming.</p>
<p>And yet looking at all these amazing crates, across many projects, there’s a recurring theme: the majority of crates I’m building my project on are versioned 0.x.</p>
<p>Why does 1.0 matter? Here are a couple of reasons why 0.x crates are bad:</p>
<ul>
<li>Every minor release of a 0.x crate is a semver breaking change</li>
<li>Large numbers of 0.x crates add to the perception of an immature/incomplete ecosystem</li>
</ul>
<p>With that prompt, I would like to ask everyone in the Rust community, including myself: why aren’t your crates versioned 1.x yet?</p>
<ul>
<li>Not feature complete?</li>
<li>Still want to make breaking changes?</li>
<li>Waiting on a forthcoming language feature?</li>
<li>Just not polished enough to “deserve” it?</li>
</ul>
<p>Whether or not a crate should be 1.0 or not is a judgment call only an author can make. But here’s a rubric I think can be used to test if you should label your crate 1.0 or not:</p>
<ul>
<li>Several years old, locked at the same 0.x minor version it’s been for years</li>
<li>Still receiving active maintenance / releases</li>
<li>Hundreds of transitive dependencies / downloads per day</li>
</ul>
<p>I think these are great candidates for 1.0 crates!</p>
<p>If your crate fits roughly into that mould, but you still have lingering items on your 1.0 milestone, here’s a simple thing I think you can do: move those lingering items to a new 2.0 milestone, and <em>release 1.0</em>.</p>
<p>That’s not to say this is a bar to which 1.0 crates should be held. Once upon a time 1.0 used to mean the “first version”. Now I would argue it means the “first stable semver guarantee”. I think that’s a bar which is attainable for many crates, and which all crates should aspire to.</p>
<p>Let’s make 2020 the year of 1.0.</p>
tag:tonyarcieri.com,2014:Post/factual-inaccuracies-of-facebook-libra-is-architecturally-unsound2019-11-05T09:07:53-08:002019-11-05T09:07:53-08:00Factual inaccuracies of "Facebook Libra is Architecturally Unsound"<p>A <a href="http://www.stephendiehl.com/posts/libra.html">post by Stephen Diehl about Facebook Libra</a> is making the rounds this morning. It makes a number of claims about flaws in the software. Many of these claims are factually inaccurate. I consider it to be a <a href="https://en.wikipedia.org/wiki/Gish_gallop">gish gallop</a> that bombards you with a bunch of negative claims, many of which are false or at best, specious.</p>
<p>For context, I am a contributor to <a href="https://github.com/open-libra">OpenLibra</a>, a group independent of Facebook (or should I say “FACEBOOK”) who are experimenting with operating the Libra software. My company also operates other BFT-based cryptocurrencies, such as the <a href="https://cosmos.network/">Cosmos Network</a> which is based on <a href="https://tendermint.com/docs/introduction/what-is-tendermint.html#abci-overview">Tendermint BFT</a>. Prior to that I worked extensively in credit card processing at companies like <a href="https://squareup.com/us/en">Square</a> and <a href="https://www.livingsocial.com/">LivingSocial</a>, also doing card processing integrations at several companies before that.</p>
<p>This post, like the original, is intended to be purely technical, and if it were to talk about my opinions of a hypothetical production Libra network, they wouldn’t be positive: the day Libra came out and I was first able to review the papers and code, my tl;dr was “<a href="https://twitter.com/bascule/status/1141106902740299776">some interesting looking tech with terrifying social implications</a>”.</p>
<p>That said, allow me to review some of the technical claims:</p>
<h2 id="libras-byzantine-tolerance-on-a-permissioned_2">Libra’s byzantine tolerance on a permissioned network is an incoherent design. <a class="head_anchor" href="#libras-byzantine-tolerance-on-a-permissioned_2">#</a>
</h2>
<p>The claim is that Libra is a centralized, “permissioned” network, and therefore doesn’t need BFT.</p>
<p>This claim would be a good one, but ignores some properties of the Libra software: it supports running in <a href="https://github.com/libra/libra/blob/master/config/src/config.rs#L304">either a permissioned <em>or permissionless</em> mode as one of the settings in the configuration</a>. I suspect Facebook’s longer term plans, were they to actually launch Libra, are to move to a permissionless proof-of-stake model much like the one used by Cosmos (which managed to launch in a fully permissionless mode).</p>
<h2 id="libra-hotstuff-bft-is-not-capable-of-achievin_2">Libra HotStuff BFT is not capable of achieving the throughput necessary for a payment rail <a class="head_anchor" href="#libra-hotstuff-bft-is-not-capable-of-achievin_2">#</a>
</h2>
<p>This section throws a few numbers out: it claims the UK Bacs system does 580,000,000 transactions a month, or about 223 transactions per second.</p>
<p>Curiously it claims Libra can’t do this, but posts no numbers about Libra’s throughtput. Libra’s throughput target is 1,000 transactions/sec (see <a href="https://developers.libra.org/docs/assets/papers/the-libra-blockchain.pdf">Section 8: Performance in the Libra paper</a>) with 10 second latencies on consensus time. I don’t think that number is unrealistic, and some of the testnets we are running for other BFT protocols are operating at those performance levels or above.</p>
<p>As another data point: the Open Libra testnet has been running for about a day and has made nearly 100,000 blocks (i.e. approximately 1/6th the number in the 10+ year history of the Bitcoin blockchain), operated by volunteers on the open Internet. Though I didn’t take the time to do so for this post, I would love to measure the transaction throughput we’re able to achieve on this testnet.</p>
<h2 id="libras-move-language-is-unsound_2">Libra’s Move language is unsound <a class="head_anchor" href="#libras-move-language-is-unsound_2">#</a>
</h2>
<p>The core argument of this section is the <a href="https://github.com/libra/libra/tree/master/language/compiler">Move language claims to use linear types, but the Move IR compiler contains no prover to assert the soundness</a>, ala something like the Rust borrow checker.</p>
<p>This is true, but also highly misleading: the Move IR compiler does not perform these checks, and can emit bytecode which is unsound, however that code is immediately <a href="https://github.com/libra/libra/blob/master/language/bytecode-verifier/README.md">checked by a bytecode verifier</a> which asserts these properties, including <a href="https://github.com/libra/libra/blob/master/language/bytecode-verifier/README.md#reference-safety">reference safety</a>. To me this provides better guarantees than doing these checks in the Move IR compiler alone: it ensures all bytecode is checked, regardless of what compiler emitted it.</p>
<p>Is it actually sound? I don’t know, it’s brand new and looks unfinished to me, and even if it were finished I’m unqualified to make that assertion, but it’s certainly quite a bit more existing machinery than the original blog post would lead you to believe.</p>
<p>Beyond that, they have <a href="https://github.com/libra/libra/tree/master/language/stackless-bytecode">experimental support for translating their bytecode</a> to <a href="https://boogie-docs.readthedocs.io/en/latest/">Boogie IVL</a> in order to facilitate formal verification.</p>
<h2 id="libras-cryptography-engineering-is-unsound_2">Libra’s cryptography engineering is unsound <a class="head_anchor" href="#libras-cryptography-engineering-is-unsound_2">#</a>
</h2>
<p>And… here we get to the part of the post which really takes the cake. Stephen Diehl doesn’t understand cryptography, but sure has a lot to say about it.</p>
<blockquote>
<p>It is impossible to say whether the dependencies on the following tools are secure or not as none of these libraries have had security audits nor do they have standard practice disclosure policies. Several core libraries in particular are indeterminate about their vulnerabilities to side channel and timing attacks.</p>
<ol>
<li>ed25519-dalek</li>
<li>curve25519-dalek</li>
</ol>
</blockquote>
<p>My goodness, where to begin. I guess the part where the claims that these libraries have never had a security audit are completely incorrect: these libraries have been audited by both Quarkslab and NCC group, with only minor findings (the <a href="https://blog.quarkslab.com/security-audit-of-dalek-libraries.html">Quarkslab audit is public</a>, the NCC audit is not but was paid for by a former employer of mine and had only minor findings).</p>
<p>Shortly before that, the post extolls the virtues of formally verified cryptography:</p>
<blockquote>
<p>The tools to build verifiable primitives exist today and while expensive to do, are certainly not outside the economic capacity of Facebook to build. Yet the team has chosen not to on a project stated to be reliable for global finance.</p>
</blockquote>
<p>False. The <a href="https://github.com/calibra/curve25519-dalek/tree/fiat">Calibra fork of <code class="prettyprint">curve25519-dalek</code></a> is integrating formally verified implementations from the <a href="https://github.com/mit-plv/fiat-crypto">fiat-crypto</a> project.</p>
<p>That said, formally verified cryptography is limited in application and scope: it can’t prove code is free of sidechannels, for instance, because those are a property of the combined software-hardware physical system, not a property of the formal model.</p>
<p>This harkens back to the famous Don Knuth quote: “Beware of bugs in the above code; I have only proved it correct, not tried it.” While I’m a fan of formal verification and will soon be attending a seminar on formally verifying cryptographic code, it has a number of limitations. Personally I’ll be sticking with upstream <code class="prettyprint">curve25519-dalek</code>, despite the lack of formal verification (in absence of that the field arithmetic contains prose proofs of correctness in the comments).</p>
<blockquote class="short">
<p>The library gets even more experimental and ventures quite outside the Cryptography Standard Model</p>
</blockquote>
<p>This sounds like Facebook has ventured straight into the realm of <a href="https://twitter.com/yolocrypto">YOLO cryptography</a>! But really it’s a meaningless claim: the standard model is not a particularly useful one, and much of the cryptography we rely on every day uses constructions proven in different but still rather boring models such as the <a href="https://blog.cryptographyengineering.com/2011/09/29/what-is-random-oracle-model-and-why-3/">random oracle model</a>. Protocols like TLS rely on several constructions which are proven in the random oracle model: without it we wouldn’t have things like digital signatures, message authentication codes, or transcript hashing needed to build a secure authenticated key exchange.</p>
<p>I guess this is where I should point this out (and also that I get a hostname verification error accessing the site over HTTPS):</p>
<p><a href="https://svbtleusercontent.com/jcZDxY9rNtkBN1eXHpf2UW0xspap.png"><img src="https://svbtleusercontent.com/jcZDxY9rNtkBN1eXHpf2UW0xspap_small.png" alt="Screen Shot 2019-11-05 at 9.59.14 AM.png"></a></p>
<p>Anywho… back to the post. For some context on that “Cryptography Standard Model” stuff:</p>
<blockquote>
<p>The library gets even more experimental and ventures quite outside the Cryptography Standard Model by folding in very novel techniques like verifiable random functions, bilinear pairings and threshold signatures.</p>
</blockquote>
<p>Verifiable Random Functions (VRFs) are effectively a type of non-malleable signature which are secure in the random oracle model. While they are poorly known, I find them to be simultaneously a very powerful and very boring construction. If anything, they provide better security properties than typical digital signatures across the board, such as the afforementioned non-malleability and message preimage resistance from an attacker who holds both a public key and a VRF output.</p>
<p>Regarding bilinear pairings and threshold signatures: <a href="https://github.com/libra/libra/issues/276">Libra doesn’t use these for either consensus or transaction signing</a>, and instead uses Ed25519 for this purpose. This is about the most boring way you can build a BFT blockchain like Libra today, and is the same approach used by Tendermint. Ed25519 is quite boring: it was selected by the <a href="https://tools.ietf.org/html/rfc8422">IRTF CFRG as the next-generation signature algorithm for TLS</a>, and <a href="https://csrc.nist.gov/publications/detail/fips/186/5/draft">NIST’s latest draft of FIPS 186-5 now includes Ed25519 as a soon-to-be recommended algorithm</a>.</p>
<p>Ironically, one of the things the things the post complained with earlier was the lack of transaction privacy. This is something I <a href="https://twitter.com/bascule/status/1141077382503079936">agree with and care about</a>, however the reference to bilinear pairings is ostensibly in regard to the <a href="https://github.com/libra/libra/blob/master/crypto/crypto/src/bls12381.rs">BLS12-381 elliptic curve</a>. There are a few different ways they could potentially use this, such as for <a href="https://tools.ietf.org/html/draft-irtf-cfrg-bls-signature-00">BLS threshold signatures</a>, but the main way the BLS12-381 curve is leveraged today is as part of the <a href="https://github.com/zcash/zips/blob/master/protocol/protocol.pdf">Zcash Protocol</a> for private/shielded transactions expressed as zero-knowledge proofs. I sure would love to see Libra integrate something like that.</p>
<p>The section concludes:</p>
<blockquote>
<p>These techniques and libraries might be sound, but the amalgamation of all of them into one system should raise some serious concerns about attack surface area. The combination of all of these new tools and techniques makes the burden of proof much higher.</p>
</blockquote>
<p>The core Libra protocol is computing additions to a Merkle tree constructed using SHA-3 with Ed25519 signatures from validators. To me that is an extremely boring construction and works the same way as several other production proof-of-stake blockchains such as Cosmos.</p>
<p>Yes, there’s some wild “next gen crypto” in their cryptography library, but it’s not being used by the core protocol (or practically anything in their codebase) yet, and its purpose may very well be to experiment with solving issues like transaction privacy.</p>
<p>That brings us to our last section…</p>
<h2 id="libra-has-no-capacity-for-consumer-protection_2">Libra has no capacity for consumer protection mechanisms <a class="head_anchor" href="#libra-has-no-capacity-for-consumer-protection_2">#</a>
</h2>
<p>This section is talking about payment reversal mechanisms like <a href="https://en.wikipedia.org/wiki/Chargeback">chargebacks</a>. Right now Libra is missing many features of other proof-of-stake blockchains like <a href="https://cosmos.network/docs/spec/governance/">governance</a>, however these mechanisms can be used to implement things like payment reversals.</p>
<p>We can imagine a model where a Libra partner brokered some transactions they want to reverse, and have determined they were fraudulent as part of an existing disputes process. They can put this reversal to a governance vote, and if it passes reverse the desired transactions.</p>
<p>While it’s true Libra doesn’t have these features, it’s being worked on, and it certainly has “the capacity” for them.</p>
<h2 id="conclusion_2">Conclusion <a class="head_anchor" href="#conclusion_2">#</a>
</h2>
<p>Stephen Diehl’s ends with “the final conclusion one must take away after doing technical due diligence on this project is this simply that it would not pass muster in any respected journal on distributed systems research or financial engineering.”</p>
<p>Well, that’s more or less how I feel about his blog post. His post is filled with a number of accusations and claims which are completely wrong.</p>
<p>There are a lot of reasons not to like Libra, and in particular there are reasons to worry about the social impact of a Libra network operated by Facebook.</p>
<p>But from a technical perspective, I find Libra to be one of the best enterprise grade BFT-based blockchain systems in development today.</p>
tag:tonyarcieri.com,2014:Post/rust-in-2019-security-maturity-stability2019-01-15T23:37:32-08:002019-01-15T23:37:32-08:00Rust in 2019: Security, Maturity, Stability<p><a href="https://svbtleusercontent.com/kdfeBxqyacCcQst17caeiR0xspap.png"><img src="https://svbtleusercontent.com/kdfeBxqyacCcQst17caeiR0xspap_small.png" alt="hermes.png"></a></p>
<p>I’ve been a day-to-day Rust user for nearly 5 years now. In that time I’ve perpetually felt like Rust is “the language that will be awesome tomorrow”. Rust has always felt like it had so much promise and potential, but if you actually sat down and tried to use it, the result was akin to trying to assemble a complex mechanism but you were always missing a few pieces. </p>
<p>I managed to suffer through it in the long long ago of 2014, and each year it’s gotten better, until in 2018 I could confidently say “Now’s a great time to start learning Rust”. But how about actually writing a real-world Rust application and deploying it to production?</p>
<p>2018 marked my second year as a full-time Rust developer, and I’m happy to say it’s the first year I shipped production Rust applications (not just one but three!), and added the <a href="https://iqlusion.io">startup I cofounded</a> to the <a href="https://www.rust-lang.org/production/users">Friends of Rust (nee Production Users)</a> page. After years of using Rust as a hobbyist, and aspiring to build “a real app” with it someday, I finally shipped.</p>
<p>Thus far these applications have been unproblematic, <a href="https://twitter.com/iqlusioninc/status/1075269069882834944">quietly and efficiently doing their job</a>, and just as boring and predictable as I’d hope they’d be.</p>
<p>I would like to hope my experience is repeatable, and that 2019 is the year that Rust can transcend the notion of being a bleeding-edge science experiment and instead settle down into being a “boring in a good way” practical language with a growing community developing and deploying production-quality software.</p>
<p>Some of the biggest names in tech are now using Rust for some very interesting projects, including:</p>
<ul>
<li><a href="https://aws.amazon.com/blogs/aws/firecracker-lightweight-virtualization-for-serverless-computing/">Amazon’s Firecracker VM</a></li>
<li>
<a href="https://chromium.googlesource.com/chromiumos/platform/crosvm/">Google’s crosvm</a> which now powers Linux VMs on Chromebooks. Rust is also leveraged heavily within Google’s new <a href="https://en.wikipedia.org/wiki/Google_Fuchsia">Fuchsia operating system</a>, and from what I can gather is their language of choice for things like userspace tooling in this new OS.</li>
<li><a href="https://github.com/Azure/iotedge/tree/master/edgelet">Microsoft’s “Edgelet” IoT Security Proxy</a></li>
</ul>
<p>All of these are very positive signs for Rust’s overall development, but is it fair to say that Rust is “mature” yet?</p>
<p>Almost.</p>
<p>2019 is the year I hope that line will finally be crossed. Things have certainly matured considerably since the early days of “nightly broke your code and all your dependencies”, but even after the release of Rust 1.0, it still felt “bleeding edge”. Year-over-year things have gotten better, and while there are still a number of high priority foundational problems with the language,<br>
I wouldn’t consider any of them showstoppers, and many seem like low-hanging fruit that I hope will get fixed this year!</p>
<p>Looking ahead to 2019, I’d like to being by asking: who could benefit the most from Rust who isn’t already using it? Why aren’t they using it? Is Rust missing something they need?</p>
<h1 id="who-needs-rust-the-most_1">Who needs Rust the most? <a class="head_anchor" href="#who-needs-rust-the-most_1">#</a>
</h1>
<p><a href="https://svbtleusercontent.com/phx6Q6aSdCg9RpGGhoGBKr0xspap.png"><img src="https://svbtleusercontent.com/phx6Q6aSdCg9RpGGhoGBKr0xspap_small.png" alt="rustc has your back"></a></p>
<p><em>Image originally from <a href="http://leftoversalad.com/c/015_programmingpeople/">Leftover Salad</a></em></p>
<p>Rust places a high bar on what code it considers correct and in doing so asks a lot from a programmer when authoring software. Where many other languages would just let many (potentially severe / security-critical) errors slide, Rust’s philosophy leans towards eliminating runtime errors through a much more meticulous set of rules.</p>
<p>The Rust compiler’s <a href="https://www.reddit.com/r/rustjerk/comments/8olhu2/the_warn_police/">relentless nitpicking</a> is great if you are writing any software which demands correctness or applications which are nontrivial and which you intend to maintain for some time, but contrary to the impressions given by certain Rust community members who shall remain nameless, there are an awful lot of useful programs which don’t need to be written in Rust, and while Rust is potentially useful for practically anything, Rust is overkill for many projects (or possibly even most projects).</p>
<p>If you just need something quickly and are fine with a garbage collector, there are any number of languages which might better fit your needs depending on the circumstances. They might be slower, with a resource-hungry garbage collected runtime unsuitable for use on certain classes of embedded devices, riddled with thread safety, type safety, integer overflow, and null pointer errors, but the reality is in a lot of cases we can tolerate bloated, buggy programs, because they may be temporary, written for a one-off purpose, or simply aren’t that important. Hardware is cheap when your program is trivial and seldom utilized.</p>
<p>Back to the question: who isn’t using presently using Rust, but would greatly benefit from it? How could Rust help them, and why aren’t they using it?</p>
<p>In broad strokes I am personally thinking about authors of software that meets the following three criteria:</p>
<ol>
<li>Widely deployed, foundational, mission-critical software</li>
<li>Presently authored in C, C++, or Ada</li>
<li>Critical memory safety bugs still an ongoing issue in practice</li>
</ol>
<p>Generally you might describe the above as “high-assurance software”. Here are some specific examples:</p>
<ul>
<li>Bootloaders, management controllers, and mobile baseband radios</li>
<li>Virtual machine managers, container managers, hypervisors</li>
<li>SmartChips (e.g. EMV credit cards), Authentication Tokens (U2F / WebAuthn), PIV Cards</li>
<li>OS Kernels / memory managers / filesystems / network stacks</li>
<li>Databases</li>
<li>Cryptography
<ul>
<li>Algorithm / protocol implementations (ciphers, signature algorithms, key exchange algorithms, zero-knowledge proofs, transport encryption protocols, cryptocurrencies)</li>
<li>Key Management Services (Signatures, Decryption, HMAC)</li>
<li>Trusted Platform Modules (TPMs), Hardware Security Modules (HSMs), Secure Enclave Processors (SEPs).</li>
<li>Trusted Execution Environments (SGX, TrustZone)</li>
<li>Cryptocurrencies: Nodes, Wallets, Relay Networks</li>
</ul>
</li>
<li>Libraries designed for FFI embedding (in-process DBMS, TLS stacks, parsers)</li>
<li>Network devices (routers, switches, firewalls)</li>
<li>Medical devices (pacemakers, imaging devices, radiation therapy machines)</li>
<li>Aerospace / Defense (I’m personally a peacenik, but so long as we have computers controlling weapons systems, I don’t want them to be buggy or easy to compromise!)</li>
<li>Physical Security: conditional access (e.g. ticketing), door locks, alarm systems, bomb/weapons detectors, security imaging machines / scanners</li>
<li>Transportation: Planes, trains, self-driving cars</li>
<li>Industrial control systems / SCADA</li>
<li>Consumer electronics with Internet connectivity / IoT</li>
<li>Those programs that continuously download attacker-controlled code and execute it, constructing attacker-controlled interfaces to present the user while slowly consuming all system resources and user attention. What are they called again? Oh right, web browsers</li>
</ul>
<p>Software like this is ubiquitous in our daily lives, and often must run on low-powered embedded hardware which until recently there weren’t practical alternatives, including Rust. Memory safety is of utmost importance: any such vulnerability is treated as high severity, mitigated, catalogued, and the fix broadcasted to the relevant system operators for immediate deployment. </p>
<p>It’s high stakes software, but also software developed with constraints where there often aren’t practical alternatives to languages like C, C++, or Ada, owing to demands like the need to run on cheap, low-resource devices, or requiring precise low-level control of memory not possible in other languages, including non-nightly versions of Rust!</p>
<p>Should developers working on this kind of software consider Rust? Absolutely. Will large numbers of them do that in 2019? I’m not sure yet.<br>
Having worked directly and indirectly with various embedded developers, I’ve generally gotten the feeling that C is so thoroughly entrenched into every toolchain that suggesting a new language is practically a non-starter. </p>
<p>I’ve personally managed to squeeze Rust into a few “unusual” embedded platforms, shoving it through decades-old toolchains, and everything worked great once the initial legwork is done. But that’s very much off the “happy path” for embedded development, and C is the devil they know.</p>
<p>One key area where I think Rust needs to improve is getting embedded developers to make the initial leap to actually prototyping real-world code. I think Rust’s benefits are sufficient to keep them using it if we can just get them to start!</p>
<p>From a technical perspective, Rust is finally, as of the 2018 edition, mostly in great shape for embedded development. However, using embedded Rust in any kind of production capacity is still highly unusual. Until recently embedded Rust development required a nightly version of the compiler.</p>
<p>The <a href="https://github.com/rust-embedded/wg">Rust Embedded Working Group</a> has been doing some amazing work improving the ergonomics of embedded Rust development creating foundational libraries for commonly used platforms. Embedded development is now possible with stable Rust!</p>
<p>Again, we’re almost there. Many of the technical blockers are now gone. However, I think where there is a modicum of interest in the embedded space, most developers who are curious about Rust are in a “wait and see” mode. The Rust community needs to demonstrate that others can come in, the water is fine!</p>
<p>The notorious “Rewrite it in Rust” mantra generally suggests throwing away an existing project and starting over from scratch in Rust, I think Rust has huge potential as a supplemental language, where existing C/C++ codebases can be largely left alone, but Rust integrated into the build workflow, and FFI wrappers created (potentially autogenerated with tools like <code class="prettyprint">bindgen</code> and <code class="prettyprint">cbindgen</code>).</p>
<p>If projects can tolerate the work required to get Rust integrated into their build workflow, development environments, and the legwork of setting up the existing shims/bindings between Rust and the language their existing codebase is already written in, the result is hopefully a low-impact change to existing C/C++ codebases which enables an entirely <em>new</em> set of developers (potentially ones who have never done systems/embedded programming before) to work on <em>new</em> feature development in Rust.</p>
<p>Instead of “rewrite it in Rust”, how about “write the new parts in Rust”, or perhaps “only rewrite the most complicated, scary, security critical bits in Rust and leave the rest as it is”. The old parts stay stable and well-understood by its original authors, and teams can attract new members not by telling them to rewrite a bunch of crufty old legacy C/C++ code, but write brand new features in an interesting new language.</p>
<p>I gave a talk at a 2014 Bay Area Rust meetup talking about a specific case of this kind of software and how Rust might help: Transport Layer Encryption stacks. The core of the talk, however, focuses on the difficulty of developing high-assurance applications in languages which make it difficult to express correct software not just in terms of memory safety, but also other areas like numeric overflows, or even just boolean logic:</p>
<p><a href="https://www.youtube.com/watch?v=fO_ox-DGDqw"><img src="https://svbtleusercontent.com/a63TXQmfhYCdgdqYNm1iZC0xspap_small.png" alt="Bay Area Rust December 2014 Meetup"></a></p>
<p>As I put it in my talk, in addition to critical memory safety-related vulnerabilities in C programs, we also see many critical vulnerabilities where we’re “losing on the easy stuff”. These are problems that could’ve been easily caught by a better type system, first-class mechanisms for managing integer overflow behavior, or even just universal agreement on what <code class="prettyprint">true</code> and <code class="prettyprint">false</code> should be.</p>
<p>It’s been 20 years since C99, which added <code class="prettyprint">stdbool.h</code>, and not only is it still not universally embraced, but also implicated in security bugs in practice. C seems to be stuck at a local maximum where it’s just not going to get any better, and its many deficiencies result in us seeing the same types of security critical bugs over and over.</p>
<p>The talk, which while a bit dated in terms of Rust specifics is just as relevant ever topically, surveyed TLS vulnerabilities in 2014 and how Rust could’ve helped. Looking back at 2018, microarchitectural vulnerabilities like Meltdown and Spectre were the security highlight of the year, however we also saw a lot of of the same type of vulnerabilities I had covered in my talk before which, despite not specifically occurring inside of a TLS stack, are quite similar to the ones in my talk as the share certificate verification in common with TLS, and these sorts of extremely high severity certificate verification failures, dare I say, a recurring pattern.</p>
<p>Next I’ll cover a couple of the notable security vulnerabilities of 2018. This section of the post is full of a bunch of 20/20 hindsight and preachiness, so if you’re just here for the roadmap you might want to skip ahead.</p>
<h2 id="2018-case-study-bootloaders-and-management-co_2">2018 Case Study: Bootloaders and Management Controllers <a class="head_anchor" href="#2018-case-study-bootloaders-and-management-co_2">#</a>
</h2>
<p>We started off 2018 with a <a href="https://seclists.org/fulldisclosure/2018/Jan/12">remote code execution vulnerability in the AMD Platform Security Processor (PSP)</a>, an ARM TrustZone-based security coprocessor for AMD CPUs which provides management and security functionality for the host CPU and also the lynchpin for its Secure Boot functionality.</p>
<p>In 2017, there was a <a href="https://www.intel.com/content/www/us/en/security-center/advisory/intel-sa-00086.html">similarly severe remote code execution vulnerability in the Intel management controller</a>. Management controllers, despite being some of the most security critical components in modern computers, are routinely subject to high severity vulnerabilities. Sometimes the vulnerabilities arise from memory corruption, other times from logic bugs, but regardless of the specifics of the vulnerability, total compromise of the entire system is a routinely common failure mode.</p>
<p>In the case of the AMD PSP, it was a remote code execution vulnerability resulting from a failure to parse ASN.1 correctly in C, something which has become so exceedingly routine all I can say is “ASN.1 is a serialization format which enables C programmers to turn binary data into remote code execution vulnerabilities”.</p>
<p>While there are ways ASN.1 could be parsed more safely in C, like using ASN.1 compilers or failing that a library, there’s just something about ASN.1 (and BER/DER in particular) which for specific small ASN.1 messages makes the problem seem simple enough to handroll, and yet the format has so many edge cases and convolutions so as to resist correct handrolled implementations in C. As it were, I tried handrolling a few Rust ASN.1 parsers this way (for similar signature verification use cases), and I can say that while Rust’s slices were immensely helpful, I managed to screw it up several times. At least the failure mode was just a panic, and not an RCE vulnerability! Also the Rust ecosystem has some <a href="https://crates.io/crates/untrusted">good crates for writing panic-free parsers</a> which I should really look into using.</p>
<p>While I’m on the subject of bootloaders written in C with 2018-dated remote code execution vulnerabilities arising from a failure to parse ASN.1 correctly, yet another example of this is just such a vulnerability in the <a href="https://arxiv.org/abs/1802.00359">Nintendo 3DS Boot ROM</a> loader. This vulnerability resulted in an unpatchable total compromise of the 3DS’s secure boot process. As yet another example of these sorts of vulnerabilities repeating themselves over and over, this particular vulnerability greatly resembles one of the TLS vulnerabilities I covered in my 2014 talk, <a href="https://www.imperialviolet.org/2014/09/26/pkcs1.html">BERserk</a>, which impacted the NSS library used at the time by both Firefox and Chrome. Both the 3DS vulnerability and BERserk resulted from buggy code for parsing ASN.1 encoded RSA signatures.</p>
<p>Should developers of bootloaders and management controllers be looking at Rust? Absolutely. Do I think these developers are going to start widely embracing Rust? I think some eager early adopters will, but I’m not sure the rest are ready to leave the “devil they know” with C and venture into the relatively unknown territory of bare metal Rust development.</p>
<h2 id="2018-case-study-embedded-databases-sqlite_2">2018 Case Study: Embedded Databases: SQLite <a class="head_anchor" href="#2018-case-study-embedded-databases-sqlite_2">#</a>
</h2>
<p>C is the lingua franca for writing libraries which can be easily wrapped/consumed by other languages, particularly for cases where performance and memory usage matter. <a href="https://sqlite.org/whyc.html">For these reasons, legacy reasons, and a handful of other fairly sensible reasons, SQLite is written in C</a>. The page giving SQLite’s rationale for choosing C as an implementation also makes an attempt to answer the question of why they aren’t using a memory safe language:</p>
<blockquote class="short">
<p>Safe languages are often touted for helping to prevent security vulnerabilities. True enough, but SQLite is not a particularly security-sensitive library.</p>
</blockquote>
<p>The <a href="https://www.sqlite.org/testing.html">SQLite web site also covers the rigorous testing the library undergoes</a>, including its 100% branch test coverage and testing for memory safety issues:</p>
<blockquote>
<p>SQLite contains a pluggable memory allocation subsystem. The default implementation uses system malloc() and free(). However, if SQLite is compiled with SQLITE_MEMDEBUG, an alternative memory allocation wrapper (memsys2) is inserted that looks for memory allocation errors at run-time. The memsys2 wrapper checks for memory leaks, of course, but also looks for buffer overruns, uses of uninitialized memory, and attempts to use memory after it has been freed. These same checks are also done by valgrind (and, indeed, Valgrind does them better) but memsys2 has the advantage of being much faster than Valgrind, which means the checks can be done more often and for longer tests.</p>
</blockquote>
<p>At the tail end of 2018 the <a href="https://www.tenable.com/blog/magellan-remote-code-execution-vulnerability-in-sqlite-disclosed">CVE-2018-20346: “Magellan”</a> vulnerability was disclosed: a remote code execution vulnerability impacting SQLite’s “FTS3” fulltext search functionality. The vulnerability was fundamentally just an integer overflow, but in C integer overflows are eager to evolve into buffer overflows and eventually reach their final form as remote code execution vulnerabilities.</p>
<p>This vulnerability was remotely exploitable in several versions of Chrome and other Chromium-based applications. In stark contrast to the SQLite statement about security, headlines proclaimed “SQLite RCE vulnerability exposes billions of apps, including all Chromium-based browsers”. Sensationalistic as that may sound, I don’t think it’s too far off.</p>
<p>I don’t think it makes sense to rewrite SQLite in Rust. It’s written in C, and I don’t see any reason for that to change. So why even bring it up? In this particular case, I’d like to focus on how Rust could help someone who might be considering using a SQLite-like database, and how they might potentially be better off with one written in Rust.</p>
<p>Despite SQLite’s admirably rigorous test suite with 100% test coverage and memory sanitization, it was unable to catch this vulnerability. In my opinion, for applications which can’t afford a garbage collected runtime, <em>there is no substitute for the borrow checker</em>, and nothing short of complete formal verification (which I consider incredibly onerous, labor intensive, doable only by a limited set of experts, and suitable only for greenfield projects) will ever find all of these issues in a C program.</p>
<p>There’s another problem going on here though: from the perspective of the SQLite project quoted above, apps which allowed this vulnerability to be remotely exploited were using SQLite unsafely. This is something we see time and time again when determining which project is responsible for a particular vulnerability: “<a href="http://www.metzdowd.com/pipermail/cryptography/2019-January/034740.html">security is somebody else’s problem</a>”, the “Doctor, it hurts when I use SQLite to execute attacker-controlled fulltext searches” approach to security.</p>
<p>Empirically we have seen that though SQLite employed a multitude of different testing strategies we hope would’ve caught that particular issue, none of them were able to. The safe subset of Rust (in particular, crates which transitively are free of <code class="prettyprint">unsafe</code>, i.e. they show up clean on <a href="https://github.com/anderejd/cargo-geiger">cargo-geiger</a>) empowers developers of embeddable, multi-language FFI-friendly libraries to make claims about the security properties of their code which are ultimately predicated on rustc’s soundness and the correctness of the borrow checker. Or to put it another way: safe Rust makes memory safety rustc’s problem. These are the memory safety benefits we’ve seen for years from garbage collected languages with managed runtimes, but now applicable to software projects like embedded databases where a runtime + GC are a nonstarter.</p>
<p>As for those integer overflows, by default Rust arithmetic is checked in debug builds, but unchecked in release builds for performance reasons. That said, if you’re using Rust in any security critical application, consider adding these lines to your <code class="prettyprint">Cargo.toml</code> to enable overflow checks in release builds too:</p>
<pre><code class="prettyprint lang-toml">[profile.release]
overflow-checks = true
</code></pre>
<p>That’s right, with only two lines of TOML you can at least ensure that integer overflows in your project are at worst a DoS attack.</p>
<p>That’s all well and good, but if we’re not going to rewrite SQLite in Rust, how can Rust help? Instead of rewriting SQLite in Rust, how about the following:</p>
<ul>
<li>Take an existing awesome pure Rust storage engine like <a href="https://github.com/spacejam/sled">sled</a>
</li>
<li>Add indexing and an expressive (SQL) query layer</li>
<li>Expose a C API/ABI and/or language-specific bindings (both of which can be generated safely and automatically in many cases) for projects written in other languages to consume.</li>
</ul>
<p>I think there are a number of projects considering use of a roughly SQLite-shaped database who might be potentially interested in a pure Rust alternative because they’re security critical and concerned about future RCE vulnerabilities. This is particularly true in greenfield Rust applications, but I also think users of all languages could benefit from such a project.</p>
<p>There are many noteworthy crates which could benefit more languages than just Rust if they had proper C bindings. And fortunately, there’s <a href="https://github.com/eqrion/cbindgen">cbindgen for automatically building C bindings</a>.</p>
<p>I think if the Rust ecosystem embraces adding C APIs/FFIs to noteworthy crates, that where C is presently the lingua franca for embedded libraries and reusing code across multiple languages, Rust can at least become an alternative people think about where they might otherwise reach for a C library. Can we start getting developers of all languages, when they’re about to reach for a noteworthy C library, to go “hey, I wonder if there’s a Rust library for that”?</p>
<h1 id="2019-roadmap-suggestions_1">2019 Roadmap Suggestions <a class="head_anchor" href="#2019-roadmap-suggestions_1">#</a>
</h1>
<p>Finally, to the actual roadmap suggestions! I’ll quickly summarize what I said above for those of you who may have skipped over parts:</p>
<ul>
<li>Rust is finally approaching a level of maturity where it’s suitable for use in “high assurance” software applications, including mission critical embedded security and cryptography applications</li>
<li>However, it must overcome perceptions of being a bleeding edge science experiment by establishing a track record of successful production usage and ecosystem maturity</li>
<li>Rust seems uniquely positioned to be the next-generation language of choice for this particular space</li>
</ul>
<p>Tangibly, how do we improve Rust, and perceptions of Rust, to finally get the Rust-curious developers who have lingering doubts about the stability and maturity of the language to finally give it a whirl in real-world projects?</p>
<p>I have a few ideas, particularly around security, and looking at some of the 2019 roadmap posts others have written, it seems many of us are on the same wavelength in that regard.</p>
<p>But let me get “Stability & Maturity” out of the way first. Let’s start with core language features.</p>
<h2 id="core-language-features_2">Core Language Features <a class="head_anchor" href="#core-language-features_2">#</a>
</h2>
<p>My top 3 Rust features I’d like to see stabilized in 2019:</p>
<ul>
<li><a href="https://github.com/rust-lang/rust/issues/44580">Const generics (RFC 2000)</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/55766">Pin (RFC 2349)</a></li>
<li><a href="https://github.com/rust-lang/rust/issues/50547">async/await (RFC 2394)</a></li>
</ul>
<p>These all seem like things a lot of people want, and from my perspective I’d hope there’s a good chance at least some if not all of them are doable in a 2019 timeframe, but perhaps that’s just wishful thinking.</p>
<p>That said, I can offer some insights on why these three matter for my particular use cases.</p>
<h2 id="const-generics-and-cryptography_2">Const Generics and Cryptography <a class="head_anchor" href="#const-generics-and-cryptography_2">#</a>
</h2>
<p>Cryptography and const generics go together like peanut butter and chocolate. When you look at the names of cryptographic algorithms, you start to notice a pattern:</p>
<ul>
<li>AES-128</li>
<li>AES-256</li>
<li>SHA-256</li>
<li>SHA-384</li>
<li>SHA-512</li>
<li>ECDSA P-256</li>
<li>ECDSA P-384</li>
<li>ECDSA P-521 (not a typo, just a Mersenne prime)</li>
<li>Ed25519</li>
<li>Ed448</li>
</ul>
<p>The norm for cryptographic algorithm naming is an acronym/word describing the algorithm followed by a number. Algorithms are offered at different security levels with different tradeoffs, particularly in regard to things like key size and/or message length. Tradeoffs are everywhere in security: do we want things more secure? faster? smaller messages? </p>
<p>Particularly in regard to things like message sizes, digest sizes, MAC sizes, etc. having something that’s smaller on the wire (for e.g. a credential which is sent repeatedly with each request) can be helpful, or maybe you’d prefer the extra security and don’t care about message size. It’s nice to have knobs for these things. </p>
<p>Many algorithms can be implemented generically, such that a single generic implementation of an algorithm can be used at many different security levels. For example, internally SHA-384 is almost the same algorithm as SHA-512, but with some slight tweaks and truncated output. We can hoist those differences out into e.g. associated constants, and share a single implementation for both digest functions.</p>
<p>With const generics, interfaces to cryptographic primitives can be described generically as traits which operate on arrays whose size is determined by an associated constant of that trait, e.g.:</p>
<pre><code class="prettyprint lang-rust">pub trait NewStreamCipher: Sized {
const KeySize: usize;
const NonceSize: usize;
fn new(
key: &[u8; Self::KeySize],
nonce: &[u8, Self::NonceSize]
) -> Self;
}
</code></pre>
<p>This allows for abstract traits for very general cryptographic concepts which are usable for different sized keys, messages, and security levels. The API is error-free and panic-free, because the compiler can statically assert everything we need to eliminate any potential for a failure case. The best way to avoid runtime errors is to make potentially erroneous cases unrepresentable.</p>
<p>Const generics are needed desperately enough for these cases that many Rust cryptography crates have adopted <a href="https://crates.io/crates/typenum">typenum</a> and <a href="https://crates.io/crates/generic-array">generic-array</a> to express them. I use this approach in two of my cryptography crates: <a href="https://github.com/tendermint/signatory">Signatory</a> (a multi-provider digital signature library) and <a href="https://github.com/miscreant/miscreant.rs">Miscreant</a> (a misuse-resistant authenticated encryption library).</p>
<p>While a <code class="prettyprint">typenum</code>-based approach gets the job done, it utilizes what I’ll politely call “clever hacks”. This has a number of drawbacks: it’s heavy with boilerplate, the type signatures are unreadable, and doing any sort of nontrivial arithmetic, if even possible, is incredibly unwieldy. Here’s an example of a trait where that sort of type level arithmetic would be helpful:</p>
<p><a href="https://github.com/tendermint/signatory/blob/master/src/ecdsa/curve/mod.rs">https://github.com/tendermint/signatory/blob/master/src/ecdsa/curve/mod.rs</a></p>
<pre><code class="prettyprint lang-rust">pub trait WeierstrassCurve {
type ScalarSize: ArrayLength<u8>;
type CompressedPointSize: ArrayLength<u8>;
type UntaggedPointSize: ArrayLength<u8>;
type UncompressedPointSize: ArrayLength<u8>;
type Asn1SignatureMaxSize: ArrayLength<u8>;
type FixedSignatureSize: ArrayLength<u8>;
}
</code></pre>
<p>😱Aack! 7 different associated type-based constants?! Why? Well really we should only need one (i.e. <code class="prettyprint">ScalarSize</code>), and the rest can be trivially computed from it using a little bit of arithmetic. But where <code class="prettyprint">typenum</code> supports that arithmetic in theory, it comes at a considerable cost in regard to ease of debugging, and in attempting to even try it I couldn’t figure out the incantation.</p>
<p>Here is what I hope this trait will look like after const generics:</p>
<pre><code class="prettyprint lang-rust">pub trait WeierstrassCurve {
const ScalarSize: usize;
}
</code></pre>
<p>The Miscreant encryption library provides a bit more compelling example. Here is its Authenticated Encryption with Associated Data (AEAD) API:</p>
<pre><code class="prettyprint lang-rust">pub trait Aead {
type KeySize: ArrayLength<u8>;
type TagSize: ArrayLength<u8>;
fn new<K>(key: K) -> Self
where
K: Into<&GenericArray<u8, Self::KeySize>>;
[...]
}
</code></pre>
<p>Here associated type-level constants are used to determine the size of a particular parameter, namely the key size. This means this trait can be generic over key sizes, ensures that the key is always the correct size (i.e. no chance of a panic) using the type system, and could be trivially converted to const generics:</p>
<pre><code class="prettyprint lang-rust">pub trait Aead {
const KeySize: usize;
const TagSize: usize;
fn new<K>(key: K) -> Self
where
K: Into<&[u8; Self::KeySize]>;
[...]
}
</code></pre>
<p>Another noteworthy project using <code class="prettyprint">generic-array</code> for cryptography like this are the <a href="https://github.com/RustCrypto">Rust Cryptography Project</a>’s <a href="https://docs.rs/digest/">digest</a>, <a href="https://docs.rs/block-cipher-trait/">block-cipher</a>, and <a href="https://docs.rs/stream-cipher">stream-cipher</a> crates.</p>
<p>This project has developed a set of traits which are close to ones which, eventually, I would like to see get rolled in-tree as a solution to the <a href="https://github.com/rust-lang/rfcs/issues/766">Figure out in-tree crypto story</a>. But that’s something I’ve been saying for years, and it always comes with the caveat “…but that doesn’t make sense until const generics land”.</p>
<p>Fortunately, as I hope I’ve shown in this post, it’s pretty easy to rewrite code already written using <code class="prettyprint">generic-array</code> to use const generics once they hit <code class="prettyprint">stable</code>. As soon as that happens, converting the existing RustCrypto crates to use them should be fairly straightforward, and once that’s done, we can hopefully ship 1.0 versions of them, as well as Signatory and Miscreant!</p>
<p>While I’m on the subject of this particular project, perhaps you can excuse a small sidebar for an important message:</p>
<h2 id="rustcrypto-is-dead-long-live-rustcrypto_2">rust-crypto is dead. Long live RustCrypto! <a class="head_anchor" href="#rustcrypto-is-dead-long-live-rustcrypto_2">#</a>
</h2>
<p><em>NOTE: Those simply interested in roadmap items can skip this section</em></p>
<p>In Rust’s earlier years, the <a href="https://github.com/DaGenix/rust-crypto">rust-crypto</a> crate provided perhaps the first noteworthy effort to provide pure-Rust cryptographic implementations. I covered it in my Bay Area Rust talk I mentioned above, and at the time said it was a good effort but needed a lot more work to be considered mature and robust. To this day it is the most widely used Rust cryptography crate, with <a href="https://crates.io/crates/rust-crypto/reverse_dependencies">213 downstream dependencies</a>, which is approximately twice that of, say, <a href="https://crates.io/crates/ring"><em>ring</em></a>, a much more mature cryptography crate.</p>
<p>The <code class="prettyprint">rust-crypto</code> crate filled a bit different niche though: where <em>ring</em> presents high-level, misuse resistant APIs, which is a fantastic thing for a cryptography crate to do, the <code class="prettyprint">rust-crypto</code> crate contains what I’ll call “shoot yourself in the foot” crypto, and sometimes you need that. For example, I recently implemented the <a href="https://naeemgik.blogspot.com/2017/12/globalplatform-secure-channel-protocol.html">GlobalPlatform Secure Channel Protocol</a>, a purely symmetric transport encryption protocol based on AES-CTR and AES-CMAC.</p>
<p>While <em>ring</em> is great for modern protocols, there are a lot of gross old protocols which lack a nicely designed modern equivalent. I also used these crates to implement <a href="https://github.com/miscreant/miscreant.rs">Miscreant</a>, which is probably the best use of “shoot yourself in the foot” crypto: using it to build high-level misuse-resistant constructions.</p>
<p>So what’s wrong with <code class="prettyprint">rust-crypto</code>? Does it contain a horrible security vulnerability?</p>
<p>Worse: <u>it’s unmaintained!</u></p>
<p>Where in my 2014 talk I suggested that <code class="prettyprint">rust-crypto</code> was rough around the edges but could be a good crate with a lot of effort to polish it up, work around <code class="prettyprint">rust-crypto</code> stalled shortly thereafter, and it hasn’t seen a release or even a commit in nearly 3 years. I think it’s fairly safe to say that <code class="prettyprint">rust-crypto</code> is dead.</p>
<p>Fortunately it has a replacement - the Rust Crypto GitHub organization:</p>
<p><a href="https://github.com/RustCrypto/">https://github.com/RustCrypto/</a></p>
<p>Rather than one monolithic “crypto” crate, this organization contains several topical subprojects covering most popular (at least symmetric) cryptographic concepts like hash/digest functions, block ciphers, stream ciphers, key derivation functions, message authentication codes, and more.</p>
<p>If you’ve been using the [rust-crypto] crate in your project, 2019 is a great year to get rid of it! Instead consider using <a href="https://crates.io/crates/ring"><em>ring</em></a> (if it supports the algorithms you need), a <a href="https://github.com/RustCrypto/">RustCrypto</a> crate, or Google’s new <a href="https://github.com/google/mundane">Mundane Crypto</a> project.</p>
<p>Last but not least, check out <a href="https://github.com/dalek-cryptography/curve25519-dalek">curve25519-dalek</a> (perhaps the first particularly noteworthy 1.0+ Rust cryptography crate) and its almost post-1.0 companion crate <a href="https://github.com/dalek-cryptography/ed25519-dalek">ed25519-dalek</a> for generating digital signatures (using the Ed25519 signature algorithm) and <a href="https://github.com/dalek-cryptography/x25519-dalek">x25519-dalek</a> for Diffie-Hellman key exchanges.</p>
<h2 id="cryptographic-applications-of-code-classprett_2">Cryptographic applications of <code class="prettyprint">Pin</code> <a class="head_anchor" href="#cryptographic-applications-of-code-classprett_2">#</a>
</h2>
<p><a href="https://github.com/rust-lang/rust/issues/55766">Pin</a> is a mechanism for describing immovable types, “pinning” them to a particular memory region. In many ways it’s an answer to a longstanding frustration with kernel and embedded developers: Rust’s memory model was too high level and could not express the needed constraints for certain applications, like userspace synchronization primitives.</p>
<p>I’ll talk very briefly about what I’d like to use <code class="prettyprint">Pin</code> for: zeroing memory. In 2018 I wrote the <a href="https://crates.io/crates/zeroize">zeroize</a> crate. It had a simple goal: provide APIs for reliably zeroing out memory in cryptographic contexts. Originally it provided C FFI-based wrappers to OS-specific zeroing operations, but after talking with several people about the particular problem, I felt confident enough to rewrite it in Rust, leveraging Rust’s volatile semantics and compiler/memory fences to attempt to describe to rustc exactly what I wanted.</p>
<p>The story around all of that now is fairly good, I think, but less so is the story about making copies of data you might potentially zeroize. Calling a magic API that promises “There, it’s zeroed!” is all fine and good, except for the part where Rust made several copies behind your back and you didn’t even notice. Indeed <code class="prettyprint">Copy</code> is implemented on many types that are interesting for cryptographic purposes. Most symmetric cipher keys will ultimately look like <code class="prettyprint">[u8; 16]</code> or <code class="prettyprint">[u8; 32]</code>, both of which <code class="prettyprint">impl Copy</code>, and that makes it so very easy to make lots of copies of things like cryptographic keys if you aren’t paying attention.</p>
<p>With <code class="prettyprint">Pin</code>, things become quite interesting. Here is the core trait of the <code class="prettyprint">zeroize</code> crate:</p>
<p><a href="https://docs.rs/zeroize/latest/zeroize/trait.Zeroize.html">https://docs.rs/zeroize/latest/zeroize/trait.Zeroize.html</a></p>
<pre><code class="prettyprint lang-rust">pub trait Zeroize {
fn zeroize(&mut self);
}
</code></pre>
<p>A <code class="prettyprint">ZeroizeWithDefault</code> marker trait notes which types can be zeroed out with their <code class="prettyprint">Default</code> value. The crate provides some fairly broad impls, like <code class="prettyprint">impl<Z: ZeroizeWithDefault> Zeroize for Z</code> and <code class="prettyprint">impl<Z: ZeroizeWithDefault> Zeroize for [Z]</code>.</p>
<p>With <code class="prettyprint">Pin</code>, these impls could be changed to <em>only</em> work on <code class="prettyprint">Pin</code>-ed types, e.g. <code class="prettyprint">impl<Z: ZeroizeWithDefault> Zeroize for Pin<Z></code>. This would make <code class="prettyprint">Pin</code> a mandatory part of the <code class="prettyprint">zeroize</code> API, but it would also enforce a particularly powerful property: we can ensure that the data wasn’t moved since it was pinned. This allows the type system to ensure that we didn’t accidentally make a bunch of extraneous copies of data we tried to zero out.</p>
<p>Whenever <code class="prettyprint">Pin</code> lands in stable Rust, I’d be excited to check this out.</p>
<h2 id="awaiting-code-classprettyprintasynccode-code_2">Awaiting <code class="prettyprint">async</code> / <code class="prettyprint">await</code> <a class="head_anchor" href="#awaiting-code-classprettyprintasynccode-code_2">#</a>
</h2>
<p>I’ve been a huge fan of using <code class="prettyprint">async</code> / <code class="prettyprint">await</code> in TypeScript (I don’t really write vanilla JavaScript anymore), and having used several different approaches to concurrency over the years including “microthread” approaches in Erlang and Go as well as various future / promise abstractions in many languages and the boring old “handwritten I/O loop and thread pool” approach, I have to say that the way Rust composes <code class="prettyprint">async</code> / <code class="prettyprint">await</code>, <code class="prettyprint">futures</code>, and <code class="prettyprint">tokio</code> is among my favorites. Things are definitely still unstable and rough around the edges, but the foundation is solid.</p>
<p>Rust’s approach neatly wraps up the problems of multithreaded asynchronous I/O in a way which actually feels ergonomic in Rust. When we look at how runtimes like Erlang and Go handle multithreaded scheduling, particularly in regard to things like FFI calls, things are a bit complex, messy and gross. To (roughly) quote Chris Meiklejohn on Erlang:</p>
<blockquote class="short">
<p>200,000 processes are cheap but opening a file is expensive</p>
</blockquote>
<p>Erlang and Go are both “stackless” languages (that is to say, they don’t use the C stack) with schedules that are internally event loops. Things which can potentially block these event loops are verboten, and need to be scheduled to run on a separate thread pool, e.g. the <a href="https://docs.basho.com/riak/kv/2.0.1/using/performance/erlang/#asynchronous-thread-pool">Erlang async pool</a>, or the <a href="https://dave.cheney.net/2016/01/18/cgo-is-not-go">CGo thread pool</a> in Go.</p>
<p>There are ways to work around this, like trying to make inline FFI calls by constructing C stack frames and then calling the C target from the “emulated” stack frame. Erlang supports this (NIFs), and there are experimental Go projects to do this like <a href="https://blog.filippo.io/rustgo/">rustgo</a>. This approach provides a bridge between the stackless and stackful worlds: these calls hang the language runtime’s microthread scheduler, blocking all other “microthreads” that are trying to execute on it (hence Chris’s quote above).</p>
<p>To avoid <em>that</em> problem the runtime needs to detect that a scheduler thread is stuck on an inline FFI call and that other scheduler threads need to steal work from it while it’s stuck, and now that separate thread pool is worthless and we have something that looks a lot like Tokio. I haven’t kept up on the latest Erlang developments, but I believe this is where they’re at after many decades of work. Go seems to be at the prototyping stage with this sort of work.</p>
<p>The result is we wind up with two completely different worlds: the stackless world where Erlang and Go code execute, and the C stack where everything else executes. Once upon a time Rust had something like this: <a href="https://stackoverflow.com/questions/29791031/what-happened-to-libgreen">libgreen</a>. It wasn’t nearly as sophisticated as Erlang or Go’s schedulers: the microthreads it implemented weren’t “preemptive”. But even if they were, the divide between these two different worlds was stark, and trying to gloss over it for things like <code class="prettyprint">std::io</code> was the exact opposite of the sort of “zero cost abstraction” Rust prides itself on.</p>
<p>With <code class="prettyprint">async</code> / <code class="prettyprint">await</code>, instead of having a completely different stackless world where our Rust program runs, and a far away thread pool using “real” stacks, our program runs inside of that thread pool and is free to use the C stack and make blocking FFI calls as a first-class feature. We get the benefits of an asynchronous execution model, and while we do wind up with a separate <code class="prettyprint">async</code> world which is “infectious” within a codebase, all code everywhere is still using the C stack, and therefore FFI calls are “zero cost” and can block indefinitely.</p>
<p>The “infectious” nature of <code class="prettyprint">async</code> is a common complaint with this approach, where Erlang <code class="prettyprint">receive</code> and Go <code class="prettyprint">:= <-</code> as a sort of “virtual blocking call” built on top of a stackless runtime. But Rust supports both an <code class="prettyprint">async</code> execution model and a non-<code class="prettyprint">async</code> one simultaneously, so we need an annotation to describe that, and to me it’s a small price to pay to avoid having to worry about mixing two different calling conventions in the same program, especially when trying to debug. Some may lament the need to write <code class="prettyprint">async</code>, but it’s worth keeping in mind that <code class="prettyprint">async</code> needs a runtime to power it, and it’s nice to (still) have the option to write Rust code that doesn’t need that runtime.</p>
<p>Earlier I mentioned three Rust apps I wrote (or collaborated on) and deployed to production in 2018. None of them are presently using <code class="prettyprint">futures</code> or <code class="prettyprint">tokio</code>, and ideally all of them probably should be. One of them in particular is doing complex subprocess management and talking on lots of pipes, presently using threads and blocking I/O. But there’s a lot more asynchronous behavior and complex interactions I’d like to add, and the more complex those interactions get, the grosser that sort of thing becomes when using blocking I/O, and you start to wish there was an event loop scheduling everything.</p>
<p>At one point I took a look at rewriting one of these apps with <code class="prettyprint">tokio-subprocess</code>, and after some issues with the <code class="prettyprint">tokio-core</code> -> <code class="prettyprint">tokio</code> transition were addressed, and <code class="prettyprint">futures</code> was upgraded and it even became possible to even attempt to rewrite it, I was unhappy with the result. After years of using <code class="prettyprint">nightly</code> Rust, I have finally moved onto <code class="prettyprint">stable</code> for day-to-day use, and now that these apps are in production, I don’t want to be using <code class="prettyprint">nightly</code> features. And so I suffer through multithreaded blocking I/O, waiting for <code class="prettyprint">async</code> / <code class="prettyprint">await</code> to land on <code class="prettyprint">stable</code>.</p>
<p>I could go on, but I think we’re all aware that <code class="prettyprint">async</code> / <code class="prettyprint">await</code> is a huge missing piece of the ecosystem which is a blocker for using Rust in any asynchronous application, and members of the core team have done some outstanding work on it in 2018. When it lands, I think it will unlock Rust’s true potential as a powerhouse language for high-performance asynchronous applications which are ergonomic, maintainable, and seamlessly scalable across multicore CPUs.</p>
<p>I am eager to embrace <code class="prettyprint">async</code> / <code class="prettyprint">await</code> whenever it lands on <code class="prettyprint">stable</code>, but in the meantime I will probably continue to use blocking I/O rather than going back to <code class="prettyprint">nightly</code>. One of the great things about the <code class="prettyprint">async</code> / <code class="prettyprint">await</code> approach is code written in this style can be easily adapted from code already doing blocking I/O by adding the proper <code class="prettyprint">await</code> points, so I’d like to hope it will be easy to prototype with blocking I/O today, and <code class="prettyprint">async</code>-ify my code tomorrow.</p>
<p>That wraps it up for core language features. Onto Cargo!</p>
<h2 id="cargo-embedded-rust39s-biggest-pain-point_2">Cargo: Embedded Rust’s Biggest Pain Point <a class="head_anchor" href="#cargo-embedded-rust39s-biggest-pain-point_2">#</a>
</h2>
<p>There are two big <code class="prettyprint">cargo</code> pain points for embedded Rust developers right now, and one I’m not sure is properly triaged within the core team:</p>
<ul>
<li>
<strong>xargo</strong>: Cargo cross-compiler. An essential tool for embedded Rust development, with amazing features like being able to swap out <code class="prettyprint">std</code> in the sysroot. That’s all well and good, except xargo is unmaintained!</li>
<li>
<strong><code class="prettyprint">build-</code>/<code class="prettyprint">dev-dependencies</code> + <code class="prettyprint">std</code>-dependent cargo features + <code class="prettyprint">no_std</code> = 😱</strong>: Cargo has a bug which makes developing with <code class="prettyprint">no_std</code> extremely difficult and is extremely confusing to diagnose and debug. People keep encountering this bug, and I see reports trickle in through GitHub issues on my <code class="prettyprint">no_std</code>-friendly crates.</li>
</ul>
<p>Let me start with the latter, since it’s the more pressing of the two, and then I’ll circle back on xargo.</p>
<h2 id="bad-interactions-between-code-classprettyprin_2">Bad interactions between <code class="prettyprint">build-/dev-dependencies</code> and cargo features impacting <code class="prettyprint">no_std</code> crates <a class="head_anchor" href="#bad-interactions-between-code-classprettyprin_2">#</a>
</h2>
<p><strong>ATTENTION RUST CORE TEAM!</strong></p>
<p>If I can <code class="prettyprint">sudo rust core please ...</code> one thing in this post, it would be drawing this particular bug to your attention. It’s a pretty big problem, but conversation about it is highly disorganized.</p>
<p>There are several open issues about this on the cargo bug tracker, which I believe to all be the same underlying issue:</p>
<p><a href="https://github.com/rust-lang/cargo/issues/2589">https://github.com/rust-lang/cargo/issues/2589</a><br>
<a href="https://github.com/rust-lang/cargo/issues/4664">https://github.com/rust-lang/cargo/issues/4664</a><br>
<a href="https://github.com/rust-lang/cargo/issues/4866">https://github.com/rust-lang/cargo/issues/4866</a></p>
<p>Of these open issues, I think the best description of the problem is whitequark’s report in <a href="https://github.com/rust-lang/cargo/issues/4866">rust-lang/cargo#4866</a>, and think it might make sense to close #2589 and #4664 as dupes of #4866 to at least move discussion to a single place.</p>
<p>Here’s a Reddit thread about it:</p>
<p><a href="https://www.reddit.com/r/rust/comments/a91qv0/for_a_no_std_crate_libstd_is_still_included_in/">https://www.reddit.com/r/rust/comments/a91qv0/for_a_no_std_crate_libstd_is_still_included_in/</a></p>
<p>…and some organic reports I’ve seen on projects I contribute to:</p>
<p><a href="https://github.com/dalek-cryptography/ed25519-dalek/issues/35">https://github.com/dalek-cryptography/ed25519-dalek/issues/35</a></p>
<p>This is a bit of a tricky bug to explain, but let me give it a shot. Right now cargo feature activation applies to a project as a whole. If any dependency in a particular crate activates a particular cargo feature, it’s “on” for the entire project.</p>
<p>This is a big problem for <code class="prettyprint">no_std</code> crates, because it effectively breaks <code class="prettyprint">no_std</code> builds whenever a <code class="prettyprint">no_std</code> crate shares a dependency with one of its <code class="prettyprint">build-</code>/<code class="prettyprint">dev-dependencies</code>, and that dependency wants to make use of that dependency’s <code class="prettyprint">std</code> feature. If any <code class="prettyprint">build-</code>/<code class="prettyprint">dev-dependencies</code> activate the <code class="prettyprint">std</code> feature of a shared dependency, that dependency now links <code class="prettyprint">std</code> for the target binary, thereby breaking <code class="prettyprint">no_std</code> builds.</p>
<p>This is a weird bit of spooky-action-at-a-distance between <code class="prettyprint">build-</code>/<code class="prettyprint">dev-dependencies</code>: requirements of build tooling are getting mixed in with the requirements of the <code class="prettyprint">target</code>. Ensuring all of the dependencies of a <code class="prettyprint">no_std</code> crate is tricky to begin with (although <a href="https://github.com/hobofan/cargo-nono">cargo-nono</a> is helpful there).</p>
<p>It seems to me like cargo features used by <code class="prettyprint">build-</code>/<code class="prettyprint">dev-dependencies</code> (or proc macros or other kinds of build-time tooling) should have no effect on what cargo features are activated when building the target. That sounds like the sort of thing that might potentially have a very small fix, if there’s some way to omit <code class="prettyprint">build-</code>/<code class="prettyprint">dev-dependiences</code> from the target feature calculation.</p>
<p>It also seems like cargo workspaces already support a unified dependency calculation across all cargo features used by a workspace, while still keeping cargo features used by independent crates within a workspace isolated from each other. If the fix turns out to be complex, perhaps it could lean on whatever workspaces are already doing. I haven’t looked at any of the relevant code, just seeing the symptoms and spitballing about fixes, so apologies if I’m way off here. </p>
<p>If all of that turns out to be too complicated to fix, here’s another solution I think might be simple and “good enough”: it seems this particular problem comes up due to some tool which is a full-blown command line application like <code class="prettyprint">bindgen</code> which is installable via <code class="prettyprint">cargo install</code>. If that’s the case, another option might be a way for crates to specify build-time <code class="prettyprint">cargo install</code>-able helper tools which are spawned as plain old child processes. This isolates these tools and their dependencies from the rest of the build, and might work well in tandem with something like <a href="https://github.com/sagiegurari/cargo-make">cargo make</a> to drive them.</p>
<p>Regardless, this is a big pain point that from my perspective seems to be coming up a lot, and I’d like to help get it fixed!</p>
<h2 id="the-future-of-code-classprettyprintxargocode_2">The future of <code class="prettyprint">xargo</code> <a class="head_anchor" href="#the-future-of-code-classprettyprintxargocode_2">#</a>
</h2>
<p>Let me begin by saying that <code class="prettyprint">xargo</code> is a fantastic tool which, from my perspective, already does everything I want.</p>
<p>I use <code class="prettyprint">xargo</code> to cross-compile to a number of different targets embedded targets as well as Intel SGX. For the latter I use xargo to swap in a replacement <code class="prettyprint">std</code> crate (i.e. <code class="prettyprint">sgx_tstd</code>) into the sysroot which is able to call out of the SGX enclave (using an “OCALL”) into untrusted userspace, which in turn calls the “real” <code class="prettyprint">std</code> function (which can then, it turn, make system calls to the kernel). It’s a complicated trampoline-like system, but the result is somewhat amazing: SGX enclaves can transparently leverage the greater <code class="prettyprint">std</code>-dependent crate ecosystem (provided the desired <code class="prettyprint">std</code> functionality is actually wrapped and all transitive dependencies also compile inside SGX).</p>
<p>The current plan, as I understand it, is to fold all of <code class="prettyprint">xargo</code>‘s features into cargo. That’s a fantastic plan and I want to live in that future, but in the meantime there is no “official” fork of <code class="prettyprint">xargo</code> as far as I know.</p>
<p>It’d be nice if there were an “official” fork of <code class="prettyprint">xargo</code> under the <code class="prettyprint">rust-lang-nursery</code> or perhaps <code class="prettyprint">rust-embedded</code> GitHub organizations which could at least serve as a coordination point for people interested in continuing to use <code class="prettyprint">xargo</code> until <code class="prettyprint">cargo</code> can completely replace it. I have no idea who has time to or should “own” that project, but perhaps people in the the Rust Embedded WG would be interested?</p>
<h1 id="improving-rust-ecosystem-security-in-2019_1">Improving Rust ecosystem security in 2019 <a class="head_anchor" href="#improving-rust-ecosystem-security-in-2019_1">#</a>
</h1>
<p>Security is definitely a passion of mine. Sadly it seems I’ve written an enormous post and some of the stuff I want to say the most is buried way down here at the bottom. Whoops, guess I buried the lede.</p>
<p>A couple quick things before I start with some specific security improvements I’d like to see in Rust:</p>
<p>The newly-formed <a href="https://github.com/rust-secure-code/wg">Rust Secure Code Working Group</a> has been a great place to discuss issues relating to Rust security. Check out the <a href="https://github.com/rust-secure-code/wg/issues">open issues on GitHub</a> and chat with us on Zulip.</p>
<p>While there are always security improvements to be made, Rust is not in a state of crisis, and in looking to improve its security, I personally prefer to adopt a “do no harm” approach. This means we shouldn’t be making onerous changes to existing workflows, tools, and services, or if we do, we should have very good reasons why it will improve the overall ecosystem.</p>
<p>Finally, some of the things I’m going to talk about below, well, I’ve tried to work on them in other languages. They are extremely difficult, almost never done well, and if we can actually pull them off they will make Rust somewhat unique. I will start with ideas I think are practical and high-value first, and then gradually work my way to the unicorn items.</p>
<h2 id="sandboxing-for-code-classprettyprintbuildrsco_2">Sandboxing for <code class="prettyprint">build.rs</code>, proc macros, and other build-time code execution <a class="head_anchor" href="#sandboxing-for-code-classprettyprintbuildrsco_2">#</a>
</h2>
<p>Rust provides a lot of features for executing code at compile time. The <code class="prettyprint">build.rs</code> file is commonly used to run code at build-time by many projects, and procedural macros are very popular. Right now there are no constraints on what this code is allowed to do. Most security professionals I know who work hands on with Rust seem to agree that it would be nice to apply some form of sandboxing.</p>
<p>Figuring out exactly what that looks like will be tricky, especially given we aren’t greenfielding but instead adding constraints to a living system. Perhaps we can find an acceptable sandbox profile to begin with that actually manages to pass <a href="https://github.com/rust-lang-nursery/crater">crater</a>, or maybe we’ll need for build sandboxing is initially an opt-in feature.</p>
<p>But perhaps I should back up a bit and talk about what benefits we’d hope to get out of sandboxing.</p>
<p>At first glance, it might seem a bit silly to sandbox build tooling when we’re using that to build potentially attacker-controlled code. If we’re building a malicious crate, does it really matter if the malicious code is in the build script, or in the resulting executable?</p>
<p>Here are some of the reasons I feel build scripts are a bit more worrisome than their targets, from the perspective of an attacker. I’ll go ahead and put on my attacker hat. As an attacker, one of my primary concerns is <strong>stealth</strong>. I want my attack to be silent as possible (otherwise I’ll get caught!), and I want to erase my tracks as I go as much as possible.</p>
<p>Given these constraints, there are many reasons why I would place a malicious payload into build scripts, instead of the target code:</p>
<p><strong>Less forensic evidence</strong>: when we compile binary targets from Rust code in a build system, the resulting binary is almost always first uploaded to some sort of artifact storage, be it a Docker registry, some sort of cloud storage bucket, or what have you. This means if there is a malicious payload hidden somewhere in that binary, we can at least examine it retroactively as part of a data forensics and incident response (DFIR) process to determine the nature of a malicious payload. If we find such a payload, we have the advantage of it being effectively “data at rest”, and that it won’t become actively malicious until we try to execute it.</p>
<p>By comparison, build scripts and proc macros are downloaded, compiled, and executed on the spot. We probably don’t keep copies of the resulting binaries. Perhaps there are logs from which a Cargo.lock can be reassembled, but where with a target binary we have the exact compiler output, unless we always save off a copy of the binaries for every build script and proc macro, it’s just going to be our best guess.</p>
<p>An attacker can always place the malicious payload in something innocuous-looking and difficult to audit, like a resource fetched over HTTP inside of <code class="prettyprint">build.rs</code> which looks like a legitimate part of the build when fetched from a browser, but serves a malicious payload when fetched by that <code class="prettyprint">build.rs</code> script. They could even blacklist IP addresses once they’ve grabbed the malicious payload, so subsequent attempts to access while perfectly impersonating the <code class="prettyprint">build.rs</code> file as part of an investigation do not reveal the payload, confusing DFIR responders.</p>
<p>Finally, the build script can clean up after itself and delete or falsify forensic evidence, and we’ll be none the wiser. With a target binary, we’ll have a static artifact for a human or potentially an automated auditor process to examine, potentially before it’s even deployed!</p>
<p><strong>Shorter window of compromise</strong>: When compiling a target binary on a build system, the result is data-at-rest which is immediately stored somewhere, and will remain stored somewhere until deployed. This means that if a dependency contains a malicious payload, there will be a minute amount of time between when the payload is downloaded and when it is executed.</p>
<p><strong>Lateral movement</strong>: Build systems often build a mixture of high-value and low-value executables which are deployed differently, possibly in different firewall zones. This makes it a great place to escalate privilege between these sorts of apps. A compromise of a minor dependency in a low-value app can provide access to a build server as a build user. As an attacker, I can either try to directly attack the higher value app as the build user, or also attempt local privilege escalation attacks on the kernel and escalate to root privilege. Once I have root there are many options for stealthy persistence,</p>
<p><strong>Fewer eyeballs</strong>: You may read the code of your dependencies, but did you read all of the build scripts? The proc macros? The latter in particular can be quite difficult to read, especially if you’re looking for a malicious payload.</p>
tag:tonyarcieri.com,2014:Post/the-tether-conundrum-part-2-the-plot-thickens2018-06-26T08:00:18-07:002018-06-26T08:00:18-07:00The Tether Conundrum Part 2: The Plot Thickens<p><em>NOTE: For more background including a brief introduction to Tether itself, see part 1 of this series: <a href="https://tonyarcieri.com/the-tether-conundrum">The Tether Conundrum: A Quick Backstory</a>. This post will largely assume you have read that first, or are at least familiar with the material it covers. Also thanks to <a href="https://twitter.com/bitfinexed">Bitfinex'ed</a> for surfacing much of the material covered in this post.</em></p>
<p><a href="https://svbtleusercontent.com/xnMdDwCcFH8yU9Y3w4nTmp0xspap.jpg"><img src="https://svbtleusercontent.com/xnMdDwCcFH8yU9Y3w4nTmp0xspap_small.jpg" alt="0gkpricidqing_retina.jpg"></a></p>
<p>There have been plenty of developments surrounding Tether in the months since I wrote the first post in this series in January:</p>
<ul>
<li>Tether’s previous “auditor” (who never completed an audit), Friedman LLP, <a href="https://twitter.com/Bitfinexed/status/956042851921616906">removed all mentions of Tether from their web site</a>. Shortly thereafter Tether would announce that they are <a href="http://uk.businessinsider.com/cryptocurrency-tether-splits-with-its-auditors-friedman-llp-2018-1">dissolving their relationship with Friedman</a>. Despite claims of “frequent professional audits” on their web site, to date Tether has never completed an audit.</li>
<li>The US Commodity Futures Trading Commission (CFTC) sent <a href="https://www.coindesk.com/report-cftc-sends-subpoenas-bitfinex-tether/">subpoenas to Bitfinex and Tether</a> and subsequently denied a <a href="https://www.coindesk.com/cftc-denies-bitfinex-foia-request-citing-possible-investigation/">freedom of information act (FOIA) request to obtain them</a> citing what the linked CoinDesk article described “a number of exemptions in denying the request, including one commonly applied to law enforcement investigations”. That said, some <a href="https://twitter.com/Silver_Watchdog/status/1011477796172886017">public statements made by the CFTC give us a glimpse into how Tether may have manipulated Bitcoin’s price</a>.</li>
<li>A study based on statistical methods called <a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3195066">Is Bitcoin Really Un-Tethered?</a> suggests Tether was used to manipulate Bitcoin’s price, lending credence to the speculation which I covered in my previous blog post on Tether. Their conclusions are certainly not a slam dunk, but are at least vast and rigorous, certainly more so than mere speculation alone.</li>
<li>
<a href="https://www.reuters.com/article/us-cryptocurrencies-bitfinex/bitfinex-chief-strategy-officer-departs-idUSKBN1JI2IN">Bitfinex’s Chief Strategy Officer resigned</a> citing concerns that as a US citizen, Tether leaving the US is not aligned with his interests. This despite the fact that Tether recently switched to a bank located in Puerto Rico. More on that below.</li>
<li>Tether issued an additional <strong>$1.3 billion</strong> worth of USDT tokens (including <a href="https://twitter.com/eurtprinter/status/1011257626007625728">$250 million a day before this post was written</a>), bringing the total outstanding issuance (i.e. what they claim to have backed 1:1 in their bank accounts) to over <strong>$3 billion</strong>.</li>
</ul>
<p>Perhaps most notably, however, <a href="https://tether.to/wp-content/uploads/2018/06/FSS1JUN18-Account-Snapshot-Statement-final-15JUN18.pdf">Tether published a new “proof of funds”</a> on their web site, hiring a new law firm to conduct a random spot check of the balances of their two banks. The contents of this latest audit-not-audit, and the people involved, are the primary topics I’ll be covering in this post.</p>
<p>The contents of this new “proof-of-funds” letter were quite similar to the previous “proof-of-funds” letter one originally published by Friedman LLC:</p>
<ul>
<li>The letter is not an audit, but privileged attorney-client correspondence which Tether chose to publish to the world containing a point-in-time balance check. The firm in question “randomly” selected the date of June 1st and obtained balance information from employees of Tether’s banks.</li>
<li>The letter does not cover the source of the alleged funds, i.e. we have no idea if they were legitimately obtained or e.g. the proceeds of crime.</li>
<li>Additionally notable is that Tether’s liabilities are not covered, i.e. we have no idea if this money is merely borrowed and Tether is actually in the red.</li>
</ul>
<p>There are <a href="https://www.reddit.com/r/CryptoCurrency/comments/8si9sy/tether_transparency_update_confirms_11_usd_backing/e107qz4/">several other concerns</a> about this particular letter, but if we do just want to look at it as a point-in-time confirmation of funds, there are reasons why we might give it credence.</p>
<p>The letter was produced by <a href="https://www.freehsporkinsullivan.com/">Freeh Sporkin & Sullivan, LLP</a> (FSS), whose partners are <a href="https://en.wikipedia.org/wiki/Louis_Freeh">former FBI director Louis Freeh</a> and two former federal judges including <a href="https://en.wikipedia.org/wiki/Eugene_R._Sullivan">Eugene R. Sullivan</a>, which the Tether “proof-of-funds” letter cites as their connection for discovering the law firm in the first place.</p>
<p>While there are many reasons to have doubts about Tether’s latest audit-not-audit, the word of a law form consisting former FBI federal director and former federal judge (actually two federal judges, although I’ll only be discussing one of them) is not something to be dismissed lightly.</p>
<p>This post unfortunately contains no hard evidence that what they’re saying is incorrect. I hope it will, however, raise many concerns about this firm’s credibility, including multiple conflicts of interest between them, Tether/Bitfinex, and at least one of Tether’s banks which is likely to have been the subject of the audit-not-audit.</p>
<h2 id="entangling-relationships_2">Entangling Relationships <a class="head_anchor" href="#entangling-relationships_2">#</a>
</h2>
<p><a href="https://svbtleusercontent.com/uWqvrNficqriV7wJKV92me0xspap.png"><img src="https://svbtleusercontent.com/uWqvrNficqriV7wJKV92me0xspap_small.png" alt="wa77nz27bb611.png"></a></p>
<p><em>From <a href="https://diar.co/volume-2-issue-25/">Diar’s “Tether Hits $3Bn Outstanding Following Solvency Report”</a></em></p>
<p>In the week since the FSS “proof-of-funds” has been published, Tether’s skeptics (notably <a href="https://twitter.com/bitfinexed">Bitfinex'ed</a>) have discovered a number of suspicious relationships between individuals involved in the firm, Bitfinex/Tether (they are effectively the same company), one of Tether’s banks, and other things in the cryptocurrency world including the infamous and maligned MtGOX Bitcoin exchange.</p>
<p>The bank which is central to this post is <a href="https://www.noblebankint.com/">Noble Bank International</a>, recently formed in Puerto Rico, which a recent Bloomberg article <a href="https://www.bloomberg.com/news/articles/2018-05-24/bitfinex-said-to-find-bank-in-puerto-rico-after-wells-fargo-exit">highlighted as being Tether’s new bank</a>. As we’ll see, Noble has direct connections to both Tether and FSS.</p>
<p>The rest of this post will highlight each of the notable individuals which play a major part of this story and attempt to convince you there are conflicts-of-interests and a long history of suspicious behavior.</p>
<h2 id="about-eugene-sullivan_2">About Eugene Sullivan <a class="head_anchor" href="#about-eugene-sullivan_2">#</a>
</h2>
<p><a href="https://svbtleusercontent.com/b2Xa9G7YkzxM8S7rpuAfo30xspap.jpg"><img src="https://svbtleusercontent.com/b2Xa9G7YkzxM8S7rpuAfo30xspap_small.jpg" alt="DgVUVYlW0AcRHej.jpg"></a></p>
<p>The above image is taken from <a href="https://twitter.com/Bitfinexed/status/1010306764573757441">Bitfinexed’s Twitter feed</a> and contains an alleged screenshot of a page which briefly appeared on Noble’s web site before being removed. At least as of the time of writing this post the contents of this page still appear in <a href="https://www.google.com/search?q=%22eugene+r.+sullvian%22+%22noble+bank+international%22">Google search results</a> (but unfortunately, it was <a href="https://web.archive.org/web/*/https://www.noblebankint.com/betaadvisors">not completely snapshotted by archive.org</a>):</p>
<p><a href="https://svbtleusercontent.com/wMgdggyCRcGzhpZM4dxQz10xspap.png"><img src="https://svbtleusercontent.com/wMgdggyCRcGzhpZM4dxQz10xspap_small.png" alt="Screen Shot 2018-06-26 at 1.21.03 PM.png"></a></p>
<p>Eugene Sullivan is a retired federal judge who is now a partner at the FSS law firm. He is particularly notable because he’s named directly in the “proof-of-funds” letter:</p>
<blockquote>
<p>Judge Eugene R. Sullivan (Ret.), one of the partners, is a member of the advisory board of one of Tether’s banks. It was through this connection that Tether was introduced to FSS. As well, the firm’s relationship with the bank allowed for the following review to commence in a timely and comprehensive manner, ensuring that no pertinent information was overlooked in the process.</p>
</blockquote>
<p>Based on this admission, the evidence in the Bloomberg article, and the information posted (and removed) from Noble’s web site, it appears there’s a strong case that Noble is one of the two unnamed banks in the report. It also seems like quite a conflict of interest for a law firm to make assertions like this about a bank one of their partners advises, but stay tuned, because there are more COIs with Noble to come.</p>
<p>Are there any other reasons to be suspicious of Judge Eugene? Well, he also happens to be <a href="https://agbrief.com/headline/imperial-pacific-posts-record-interim-results/">non-executive director of Imperial Pacific International Holdings</a>, the holding company for casinos which were recently built in Saipan (a U.S. Pacific island which is home to 50,000 people) called Imperial Pacific Resort (formerly the Grand Mariana) and Best Sunshine Live:</p>
<p><a href="https://svbtleusercontent.com/g1EMwyDrWcKxVQyGaneYLw0xspap.jpg"><img src="https://svbtleusercontent.com/g1EMwyDrWcKxVQyGaneYLw0xspap_small.jpg" alt="01_Front-view_Grand-Mariana-1068x701.jpg"></a></p>
<p>Though not directly connected to the rest of the Tether story (at least as far as I presently know), the casinos are notable for <a href="https://www.bloomberg.com/news/articles/2018-03-14/saipan-casino-operator-is-said-to-be-raided-by-u-s-agents">recently being raided by U.S. agents</a> due to the suspiciously high volumes of money passing through them:</p>
<blockquote>
<p>Imperial Pacific has attracted broad attention in the gaming industry for the volumes being recorded at its Saipan casino, which are far larger on a per-table basis than those at the grandest resorts in Macau.</p>
</blockquote>
<p>A Bloomberg article describes <a href="https://www.bloomberg.com/news/features/2016-11-13/obscure-casino-run-by-a-trump-protege-is-raising-big-questions">Best Sunshine Live as “the most successful casino of all time”</a>. Another Bloomberg article described Imperial Pacific International Holdings as an <a href="https://www.bloomberg.com/news/features/2018-02-15/a-chinese-company-has-conquered-a-piece-of-america">“impossibly lucrative gambling operation” and a “back door to the U.S financial system”</a>.</p>
<h2 id="about-louis-freeh_2">About Louis Freeh <a class="head_anchor" href="#about-louis-freeh_2">#</a>
</h2>
<p><a href="https://svbtleusercontent.com/gjZKYZwExAd1YWJX3RVRVr0xspap.jpg"><img src="https://svbtleusercontent.com/gjZKYZwExAd1YWJX3RVRVr0xspap_small.jpg" alt="imageen.mojahedin.jpg"></a></p>
<p>Louis Freeh has a long and sordid history for which he was practically infamous. During his tenure as FBI director he championed the <a href="https://en.wikipedia.org/wiki/Carnivore_(software)">Carnivore program</a> designed for electronic surveillance, and also pushed for law enforcement access to encryption keys. He also participated in a coverup of the government’s actions during the Waco siege.</p>
<p>For these reasons a BusinessWeek article written in the year 2000 entitled <a href="http://archive.is/eKY5u">“The Case Against Louis Freeh” called for his resignation</a>. <a href="https://www.washingtonpost.com/archive/politics/2001/05/02/louis-freeh-to-resign-as-director-of-the-fbi/106f0bfa-ace7-4c79-baaf-c758e98f0252/?utm_term=.40fd79aad277">In 2001 he resigned in disgrace amidst multiple scandals</a>, including the <a href="https://www.fbi.gov/history/famous-cases/robert-hanssen">espionage scandal surrounding Robert Hanssen</a>.</p>
<p>A web site (and corresponding <a href="https://twitter.com/factfreeh">Twitter account</a>) called <a href="https://factfreeh.wordpress.com/">Fact Freeh chronicles his past including his many exploits since entering the private sector</a>. There’s far too much for me to cover here, but I encourage you to check it out.</p>
<p>There is a story you might be familiar with which I’d like to highlight in particular though: Freeh is connected to two companies involved in purchasing MtGOX after it was hacked: <a href="https://www.reuters.com/article/us-bitcoin-mtgox-investorgroup/investor-group-offers-to-take-over-revive-mt-gox-idUSBREA3922Q20140411">Freeh Group International Solutions and Sunlot Holdings</a>. The latter would eventually take over the beleaguered exchange, and was founded by another person of interest: Brock Pierce.</p>
<h2 id="about-brock-pierce_2">About Brock Pierce <a class="head_anchor" href="#about-brock-pierce_2">#</a>
</h2>
<p><a href="https://svbtleusercontent.com/wuiFiTiszWhhsLofKBu5UR0xspap.jpeg"><img src="https://svbtleusercontent.com/wuiFiTiszWhhsLofKBu5UR0xspap_small.jpeg" alt="lolwtfbrock.jpeg"></a></p>
<p>Brock Pierce is quite the guy, and I mean that in the most disrespectful way possible. You might (extremely hazily) remember him from <a href="https://www.imdb.com/name/nm0682302/">The Mighty Ducks, where in a flashback he plays a younger version of the coach of the eponymous team</a>.</p>
<p>In 1999 in the midst of the Dot Com bubble at the age of 17, he cofounded <a href="https://en.wikipedia.org/wiki/Digital_Entertainment_Network">Digital Entertainment Network</a>, an online media startup promising to deliver original video content in the age of dialup.</p>
<p>After taking $26 million in funding and preparing for a $75 million IPO, the company collapsed amidst allegations that <a href="https://web.archive.org/web/20080117004139/https://radaronline.com/from-the-magazine/2007/11/den_chads_world_marc_collins_rector_1.php">he and his co-founder Marc Collins-Rector raped young boys who were supposed to appear in the company’s videos after threatening them at gunpoint</a>. Brock and his other cofounders fled to Spain to escape prosecution.</p>
<p><a href="https://arstechnica.com/information-technology/2014/05/some-in-bitcoin-group-resign-over-new-board-members-link-to-sex-abuse/">The allegations resurfaced in 2014 when he was appointed to the board of the Bitcoin foundation</a>, in addition to <a href="https://www.ccn.com/brock-pierce-defends-letter-foundation/">allegations that child pornography was found in a Spanish residence in which he and his partners were staying</a>. Brock denies any wrongdoing.</p>
<p>In regard to this particular story, Brock Pierce is notable for two reasons:</p>
<ul>
<li>He is the co-founder of Tether (as can be verified by his <a href="https://www.linkedin.com/in/brockpierce">LinkedIn profile</a> and many other public sources including his <a href="https://twitter.com/brockpierce?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor">Twitter profile</a>)</li>
<li>He is the <a href="http://www.coinfox.info/news/company/1703-nasdaq-will-provide-trading-system-to-a-new-bitcoin-marketplace">co-founder of Noble Markets</a>. Noble Markets is the parent company of Noble Bank International, as <a href="https://www.noblebankint.com/">mentioned on Noble Bank’s web site</a>.</li>
</ul>
<p>In other words: Tether’s alleged bank Noble is owned by a company that Tether’s co-founder also co-founded.</p>
<p>Curiously, FSS claims Tether discovered them because Eugene Sullivan sits on their board. But it seems they weren’t entirely forthcoming about all of the relationships involved.</p>
<h2 id="conclusion_2">Conclusion <a class="head_anchor" href="#conclusion_2">#</a>
</h2>
<p>I wish there were a smoking gun which could conclusively demonstrate wrongdoing on Tether’s part. We don’t have that, and at face value we have the word of a law firm run by a former FBI director and two retired federal judges. Descriptions like that gloss over the many details of the individuals involved though.</p>
<p>I have seen many people concluding from the latest audit-not-audit and the authority of an FBI director and retired federal judges that Tether is on the up-and-up. I think such conclusions are premature. The FOIA request the CFTC denied seems to point to an active and ongoing investigation, and there are far too many conflicts of interest happening here to take the “proof-of-funds” at face value.</p>
<p>As far as I’m concerned, this whole thing stinks to high heaven. But only time will tell.</p>
tag:tonyarcieri.com,2014:Post/the-tether-conundrum2018-01-19T09:30:03-08:002018-01-19T09:30:03-08:00The Tether Conundrum: A Quick Backstory<p><em>NOTE: This post is fourth in a series I’ve written about Bitcoin, including <a href="https://tonyarcieri.com/the-death-of-bitcoin">The Death of Bitcoin</a>, <a href="https://tonyarcieri.com/on-the-dangers-of-a-blockchain-monoculture">On the dangers of a blockchain monoculture</a>, and <a href="https://tonyarcieri.com/a-tale-of-two-cryptocurrencies">A tale of two cryptocurrencies: Ethereum and Bitcoin’s ongoing challenges</a>. I am a former <a href="http://mashable.com/2018/01/17/hodl-cryptocurrency-lingo-beginners-guide-bitcoin-crash/#1rUj3JFO6mqw">HODLer</a>, present <a href="https://www.urbandictionary.com/define.php?term=nocoiner">nocoiner</a>, and perpetual Bitcoin bear, so take that for what you will. Well, the nocoiner part is a bit of a lie: I have a <a href="https://github.com/jpcoin/jpcoin">JPCoin</a>.</em></p>
<p><a href="https://svbtleusercontent.com/ap9qcwiapjgavq.png"><img src="https://svbtleusercontent.com/ap9qcwiapjgavq_small.png" alt="WhichIsTheCause.png"></a></p>
<div style="text-align:center;width:27.5%;margin:0 auto;">Figure 1: If we correlate the declining murder rate with web browser usage, we can deduce that <a href="https://gizmodo.com/5977989/internet-explorer-vs-murder-rate-will-be-your-favorite-chart-today">Internet Explorer was the major causal factor in homicide rates</a>. Thank you Mozilla and Chrome Team, you are doing God’s work.</div>
<p>Bitcoin has certainly been on a roller coaster ride over the past several months, with several people questioning <a href="https://www.express.co.uk/finance/city/906772/Bitcoin-price-news-why-BTC-going-up-cryptocurrency-value-should-buy">“Bitcoin? Is it good? Should I buy it? The price keeps going up like magic and never crashes, at least for long anyway. Buy the dip, and HODL!”</a>. Where Bitcoin bulls say <a href="https://www.cnbc.com/2018/01/16/bitcoin-headed-to-100000-in-2018-analyst-who-forecast-2017-price-move.html">it’s on track to hit $100,000 in 2018</a>, I kind of look at the whole thing as if it’s teetering on the brink of disaster.</p>
<p>For some context, we can look to this quote:</p>
<blockquote>
<p>“The blockchain has a certain stark logical completeness, but it doesn’t address all of the actual human uses required of it. And so it has become encrusted with other human institutions. And those institutions turn out to be unsurprisingly human.”</p>
</blockquote>
<p>(From <a href="https://www.bloomberg.com/view/articles/2017-08-02/bitcoin-exchange-had-too-many-bitcoins">Matt Levine’s “Bitcoin Exchange Had Too Many Bitcoins”</a>, a story about a cryptocurrency exchange which is also central to this post: Bitfinex)</p>
<p>Bitcoin’s maximalists claim it will save us from the tyranny of the printing press of central banks. But what if, in some gesture of supreme irony, Bitcoin’s undoing was actually the very same problem it set out to solve: namely the fiat currency “printing press”, except this time it was being used to prop up the Bitcoin bubble from its inevitable collapse?</p>
<p>Will this be the trigger event that causes the house of cards to come crashing down? I don’t have a crystal ball that can tell me that, but I can tell you this much:</p>
<p><strong>I, and many others, suspect Tether is being used to effectively counterfeit hundreds of millions of dollars of perceived value, which are being immediately reinvested into Bitcoin to keep it from collapsing.</strong></p>
<p>But to explain this, let’s first start at the beginning: what is Tether?</p>
<h1 id="what-is-tether_1">What is Tether? <a class="head_anchor" href="#what-is-tether_1">#</a>
</h1>
<p>The alleged goal of <a href="https://tether.to/">Tether</a> is to create a <a href="https://thecontrol.co/stablecoins-a-holy-grail-in-digital-currency-b64f3371e111">stablecoin</a> which can be traded like a cryptocurrency but whose value is pegged to the United States dollar:</p>
<p><a href="https://svbtleusercontent.com/mukkhwpg3wxea.png"><img src="https://svbtleusercontent.com/mukkhwpg3wxea_small.png" alt="Tethered.png"></a></p>
<p>Well let’s hold up for a bit. <a href="https://i.imgur.com/I2Rt4fQ.gifv">This isn’t decentralized! This isn’t what users crave!</a> It’s a centralized system, traded like a cryptocurrency, whose value is allegedly backed by equivalent “legacy inflationary fiat” US dollar value sitting in a bank account somewhere. From the Tether web site:</p>
<p><a href="https://svbtleusercontent.com/8x8r9uluhofcw.png"><img src="https://svbtleusercontent.com/8x8r9uluhofcw_small.png" alt="1-to-1"></a></p>
<p>Okay, great, so at least according to Tether every one of their tokens, or “USDT”, is allegedly backed by actual US dollars sitting in “reserves” somewhere, or so the story goes. Can we trust someone else besides Tether to back up that claim? Tether says yes, so surely we can trust Tether that someone is keeping tabs on Tether’s “reserves”, right?</p>
<p><a href="https://svbtleusercontent.com/sf1jhvxyistw.png"><img src="https://svbtleusercontent.com/sf1jhvxyistw_small.png" alt="Screen Shot 2018-01-18 at 10.51.25 PM.png"></a></p>
<p>Frequent professional audits! Well as long as those are actually happening, surely everything will work out okay. If anything is amiss, the auditor will certainly alert us to problems, because Tether says they will. Great!</p>
<p>So why might we actually want to use Tether?</p>
<p><a href="https://svbtleusercontent.com/gxvvegvnkw9oq.png"><img src="https://svbtleusercontent.com/gxvvegvnkw9oq_small.png" alt="Screen Shot 2018-01-18 at 10.56.32 PM.png"></a></p>
<p>Now you can use a centralized fiat cryptocurrency to “Be the custodian of your own funds and eliminate exchange custodial risk”. Instead of FDIC insured funds backed by the United States government (but subject to those pesky <a href="https://en.wikipedia.org/wiki/Know_your_customer">AML/KYC laws</a>), you can instead use… a cryptocurrency run by people who totally swear every $1 USDT of funny money has a corresponding $1 USD of real money sitting in “reserves” somewhere, and auditors are totally keeping tabs on things. Allegedly! According to Tether!</p>
<p>Well, the <a href="https://medium.com/@bitfinexed/the-so-called-tether-audit-that-isnt-an-audit-at-all-5a40cfcc2a75">Tether audits tell a different story</a> (you can read the <a href="https://tether.to/wp-content/uploads/2017/09/Final-Tether-Consulting-Report-9-15-17_Redacted.pdf">redacted report directly here</a>), and despite claims they’re being audited regularly I can find only one (heavily disputed) audit report, but I digress. They also got <a href="https://www.coindesk.com/tether-claims-30-million-stable-token-stolen-attacker/">hacked a few months ago and allegedly lost $30 million worth of USDT</a> but that is chump change compared to the massive volumes of Tethers they have printed in the past few days.</p>
<p>So this whole thing sounds super shady. Why would anyone take it seriously in the first place? How can we actually tie its value to fiat currency and ensure the price remains stable?</p>
<p>There are a lot of ways you could possibly accomplish this. Some of them potentially have merit. <a href="https://paulbutler.org/archives/stop-dragging-hayek-into-bitcoin/">Hayek had thoughts on how this could be done right</a>. However, if you’re new to Tether and looking to give it the benefit of the doubt, let me take you through Tether from its inception and then compare it to how it exists today.</p>
<h2 id="tether-the-dream_2">Tether: The Dream <a class="head_anchor" href="#tether-the-dream_2">#</a>
</h2>
<p>Let’s say Tether has actually built something which is truly intended to hold its value at least as well as the US dollar (please hold your guffaws regarding USD’s sinking value and please direct your energies towards America’s Shithole President). There are a lot of good ways we could do this, and then there’s the way Tether originally did it, which is not how Tether works today. So how did Tether originally work?</p>
<p>Initially, you could use a Legacy Bank technology called a “wire” which provides a faster-than-ACH bank-to-bank transfer mechanism. You could buy Tethers and take advantage of their remarkable ability to “bypass financial institutions” and their pesky AML/KYC regulations by buying “USDT” funny money with cold hard cash. Originally Tether would also send outbound wires too, so if you held USDT you could, in theory, cash it out as actual USD at any time.</p>
<p>By bridging the gap between legacy systems and fancy new cryptocurrencies, you can have the best of both worlds: a self-sovereign crypto-asset whose value is “tethered” to the value of the dollar, redeemable at any time for the equivalent in cold hard cash. Awesome! Tether is setting our USD free of the evil legacy financial system.</p>
<p><a href="https://www.cnbc.com/2018/01/16/cryptocurrency-sell-off-tether-faring-better-than-rest.html">A few days ago CNBC noted that in an almost universal downturn across cryptocurrency, Tether was the only thing that was holding value</a>. Great! Pump $USDT, surely it is the future of cryptocurrency.</p>
<h2 id="tether-the-reality_2">Tether: The Reality <a class="head_anchor" href="#tether-the-reality_2">#</a>
</h2>
<p>Soooo… there’s this funny little notice hiding on the Tether home page, ostensibly there since April 2017:</p>
<p><a href="https://svbtleusercontent.com/d1hhqflytz4fg.png"><img src="https://svbtleusercontent.com/d1hhqflytz4fg_small.png" alt="Screen Shot 2018-01-18 at 8.07.40 PM.png"></a></p>
<p>I like news! What’s hiding behind there? Let’s have a look-see:</p>
<p><a href="https://svbtleusercontent.com/ocllxyti4uapfg.png"><img src="https://svbtleusercontent.com/ocllxyti4uapfg_small.png" alt="Screen Shot 2018-01-19 at 12.24.16 AM.png"></a></p>
<p>(from <a href="https://tether.to/announcement/">https://tether.to/announcement/</a>)</p>
<p>That’s a lot of words. What’s the tl;dr? As of approximately April 2017, you could not buy or sell USDT for USD. It’s as if it became… un-Tethered. Whatever legitimate uses of Tether as some sort of cryptocurrency proxy for USD ceased to exist in any form that day, and Tether became “a cryptocurrency we believe holds a dollar value because Tether claims they have the dollars in their bank account”.</p>
<p>At the time this announcement was originally made, <a href="https://coinmarketcap.com/currencies/tether/historical-data/?start=20170119&end=20180119">Tether’s market cap was approximately $50 million. Its current market cap is $1.7 billion</a>.</p>
<p>Why does anyone bother using Tether anyway then? Well, one angle is it’s a scam to confuse users of exchanges, primary, most obviously, and most notoriously <a href="https://www.bitfinex.com/">Bitfinex</a> (with excellent from-the-trenches twitporting by the <a href="https://twitter.com/Bitfinexed">@Bitfinexed</a> Twitter account). The ties between Bitfinex and Tether run deep, with Bitfinex refusing to do actual USD exchanges, but instead settling in “USDT”, ostensibly in hopes of duping naive users and forcing them into a sort of “Tether trap”:</p>
<p><a href="https://svbtleusercontent.com/m3qjyw6zydwhpq.jpg"><img src="https://svbtleusercontent.com/m3qjyw6zydwhpq_small.jpg" alt="RuhRoh.jpg"></a></p>
<p>(from the ever-reputable <a href="https://www.reddit.com/r/Buttcoin/comments/7r712n/so_who_buys_tether/">/r/buttcoin</a>)</p>
<p>Okay, so perhaps Tether is a scamcoin used to deceive users into cashing out into it instead of real money. Still, things could be worse, right?</p>
<p>As of January 19th, 2018, <a href="https://coinmarketcap.com/currencies/tether/">Tether’s daily volume is $3.9 billion USD</a>. That’s right, <strong>Tethers are moving nearly $4 billion worth of value each day</strong>. These are allegedly mostly inter-exchange transfers between cryptocurrency exchanges, which if you buy Tether’s own marketing material are deliberately intended to skirt AML/KYC laws. But have we hit rock bottom yet? Oh my, no.</p>
<h2 id="tether-the-present_2">Tether: The Present <a class="head_anchor" href="#tether-the-present_2">#</a>
</h2>
<p>True to at least some of their words, Tether does provide <a href="https://wallet.tether.to/transparency">transparency into their USDT issuances</a>. But if we actually scrutinize them, we find a sort of transparency which is no better than Donald Trump Jr tweeting incriminating emails. Sure, it’s great if they publicize their suspicious behaviors for all to see, but that should not squelch your alarm bells!</p>
<p>So how many USDT tokens has Tether issued lately? Let’s check out the <a href="https://twitter.com/tetherprinter">@tetherprinter Twitter account</a>:</p>
<p><a href="https://svbtleusercontent.com/eygjeigfnjomw.png"><img src="https://svbtleusercontent.com/eygjeigfnjomw_small.png" alt="TetherTetherTether.png"></a></p>
<p>That’s $450 million dollars in a little less than a week, highly correlated to a <a href="https://www.marketwatch.com/story/a-bitcoin-bloodbath-highlights-these-defensive-cryptocurrency-strategies-2018-01-16">“Bitcoin bloodbath”</a> where the cryptocurrency lost nearly 50% of its value since December. $450 million is a roughly a 40% increase in Tether’s previous $1.2 billion market cap, bringing the total to approximately $1.7 billion USD worth of “USDT”. Now let’s keep in mind that Tether insists that they have this much in equivalent USD value their “reserves”, and that they are “subject to frequent professional audits”. But is any of that actually true? I can’t tell you, but if it is I’m curious where they got nearly half a billion dollars in cash this week and would be very interested in the results of a new Tether audit.</p>
<p>I can’t tell you why the price went down… some people <a href="https://www.bloomberg.com/news/articles/2018-01-17/bitcoin-watchers-running-out-of-explanations-blame-slump-on-moon">apparently suspect the Moon is the culprit</a>, but in examining how the price got back up from sub-$10,000 levels to, as of the current time of writing, the $11,000 range, a point it has somewhat curiously fluctuated around, with just enough headroom to make you think >$10,000 Bitcoins are here to stay.</p>
<p>Over the course of the past few days, when Bitcoin appeared as if it was almost about to crash but at the last minute recovered, we saw an awful lot of buys that look like this, many of which are suspiciously timed shortly after those $100 million Tether issuances:</p>
<p><a href="https://svbtleusercontent.com/veeiiehxmmp5yg.jpg"><img src="https://svbtleusercontent.com/veeiiehxmmp5yg_small.jpg" alt="DT46KLHW0AYrw0S.jpg"></a></p>
<p>– “No fraudulent activity going on here.” <a href="https://twitter.com/Bitfinexed/status/954264130835369984">via @Bitfinexed</a></p>
<p>In the middle of a massive selloff, suddenly out of nowhere we see buys for $500 above market value. And this isn’t just an isolated incident, but <a href="https://twitter.com/Bitfinexed/status/953509726402367488">part of an ongoing pattern where a downward trend in Bitcoin’s price mysteriously reverses shortly after new Tether issuances</a>.</p>
<p>I can’t prove this sort of buying pattern is directly related to Tether issuances. I have no idea who is making these buys. But I can say this absolutely reeks of price manipulation, with timing highly correlated to Tether issuances.</p>
<h2 id="where-do-we-go-from-here_2">Where do we go from here? <a class="head_anchor" href="#where-do-we-go-from-here_2">#</a>
</h2>
<p><a href="https://svbtleusercontent.com/0gkpricidqing.jpg"><img src="https://svbtleusercontent.com/0gkpricidqing_small.jpg" alt="DT1sgZKVMAAepPM.jpg"></a></p>
<p>I have “cried wolf” about Bitcoin crashes several times in the past, and been wrong. Perhaps I don’t know what I’m talking about. Or perhaps the price was being manipulated.</p>
<p>I have no concrete evidence of anything I am saying beyond what is easily reachable via my web browser. This is all speculation. Conjecture. <a href="https://www.youtube.com/watch?v=zN-GGeNPQEg">I could be wrong… I could be right.</a></p>
<p>I still think the cryptocurrency ecosystem has tremendous potential value for society, but I look at what’s happening and see a cesspool of corruption worse than anything Bitcoin hoped to replace. Bitcoin’s attempts to “drain the swamp” seem to be working out about as well as Trump’s.</p>
<p>I dream of a cryptocurrency ecosystem that’s truly a moral good and delivers real value to humanity. Instead I feel we have something where “Ponzi scheme” wasn’t to far off: it appears we may have found Bitcoin’s Ponzi and its name is… Tether (or perhaps Bitfinex).</p>
<p>To anyone at any sort of regulatory body who happens to be reading this: I implore you: this space needs much more regulation. I think it is ripe for collapse and a lot of people are going to lose a lot of money, which from the sound of things is often times borrowed. Long after the “crypto-rush”, people are going to be paying credit card bills for digital assets which are now worthless. Fortunately <a href="https://www.sec.gov/news/public-statement/joint-statement-sec-and-cftc-enforcement-directors">it looks like some regulatory bodies are making moves in the right direction</a>.</p>
<p>To any reporters: please look into this! There is a very bad smell here, but no smoking gun… yet. There are questions that demand answers though: who is executing those Bitcoin buys, and what is their relationship to Tether? And finally the billion dollar question: per Tether’s own claims, their reserves should now number $1.7 billion in USD, sitting in a bank account or some other form of generally recognized liquid currency. Do they?</p>
<p>In either case, I would love to hear from either reporters or regulators on this matter.</p>
<p>If this whole thing leaves you wondering if you should sell your Bitcoins, well I would need to consult my non-existent crystal ball. That said, don’t take this as investment advice, but I don’t expect this will end well.</p>
<p>However, there is one bit of precedent for a massive Bitcoin price increase followed by a massive crash: <a href="https://www.wired.com/2014/03/bitcoin-exchange/">the original 2013 Bitcoin bubble and subsequent collapse which occurred as a result of the collapse of the Mt. Gox exchange</a>. But where Mt. Gox was billed at the time as a “$460 million disaster”, well, that’s worth just about as much as the Tethers printed in the last few days. Is this a multibillion dollar cryptocurrency exchange disaster in the making?</p>
<p><em>NOTE: This blog post series continues with <a href="https://tonyarcieri.com/preview/dXZCFKzZKZH68aUQis8unu/">The Tether Conundrum Part 2: The Plot Thickens</a>.</em></p>
tag:tonyarcieri.com,2014:Post/a-short-statement-on-ashley-williams2018-01-08T07:07:08-08:002018-01-08T07:07:08-08:00A short statement regarding Ashley Williams' abusers<p>It seems there is some <a href="https://www.reddit.com/r/rustjerk/comments/7oonhj/lesscensored_version_of_the_ashley_williams_thread/">drama in the Rust community</a> regarding Ashley Williams’ recent appointment to the Rust Community Team.</p>
<p>Did Ashley violate the Node.js Code of Conduct, and is there an active censorship effort underway by the Rust Core Team to silence any dissenting voices and appoint Ashley as evil demon goddess overlord for life? It’s a Rust Conspiracy! <a href="https://www.reddit.com/r/rustjerk/comments/7oonhj/lesscensored_version_of_the_ashley_williams_thread/dsbjekv/">The accusers are applauding those who have a “skeptical” view about this situation: The sheeple are woke!</a>.</p>
<p>I see things a bit differently. The people cheering on abusers and perpetuating a cycle of abuse.</p>
<p>“Maybe if we follow her around the Internet endlessly harassing her, it will prove our point how that one thing she tweeted one time could possibly be in violation of a Code of Conduct.” There’s a word for people like that: Gamergaters. In my book, following someone around the Internet engaging in a harassment campaign forfeits your right to invoke a Code of Conduct.</p>
<p>This swarm of locusts of this nature descended upon a Reddit thread, and a Rust internals thread, and were all over my Twitter… for days! What is the right thing to do? Not delete their comments? Here’s an alternative hypothesis for you: this is Gamergate-esque abuse against a woman, and they’re trying to infect the Rust community with it.</p>
<p>I am sure there will be many rando dudebros in the comments defending other rando dudebros. Cue the “Gosh, I’ve never opined on a thread about Rust in my life, but I read your response and I don’t think you gave Ashley’s harassers a fair shake. I certainly don’t see any evidence of an ongoing targeted harassment campaign. How dare you call them a swarm of locusts. I understand there’s a woman being harassed here, but what you may not have considered is that she deserves it. You may not realize, but the Internet appointed me to make sure she never lives that tweet down. Gosh, how dare you class me as a repeat offender, I was only engaging in casual harassment. This is censorship, and it violates my First Amendment right to be an asshole!”</p>
<p>This is atrocious behavior, and these comments deserve to be deleted. The people making them need to go take a long hard look in the mirror and think about, if in fact, they are the baddies.</p>
<p>To those of you who are showing up to Rust threads just to complain about Ashley: GET OUT.</p>
tag:tonyarcieri.com,2014:Post/introducing-miscreant-a-multi-language-misuse-resistant-encryption-library2017-10-17T09:00:08-07:002017-10-17T09:00:08-07:00Introducing Miscreant: a multi-language misuse resistant encryption library<p>For the past several months I have been hacking on not just one, but five encryption libraries in five different languages (Go, Python, Ruby, Rust, and TypeScript). Tall order, I know. And worse, these libraries implement what I believe is a novel cryptographic construction. Are you terrified yet? Yes, I’m implementing novel cryptography, in several languages at that, but I’d like to convince you it’s not as scary as it sounds.</p>
<p>Why? Because I’m implementing algorithms originally created by a cryptographer, Phil Rogaway, whose decades of work is <a href="http://www.stanforddaily.com/2016/01/06/cryptographers-honored-with-levchin-prize-at-real-world-cryptography-conference/">just starting to receive the mainstream attention it deserves</a>. These algorithms are extremely simple and designed to be easy-to-implement. Finally, they’re all built atop otherwise standard AES modes.</p>
<p>The library is Miscreant, and it’s available on GitHub in the form of a multi-language monorepo at:</p>
<p><a href="https://github.com/miscreant/miscreant">https://github.com/miscreant/miscreant</a></p>
<h2 id="the-problem-nonce-reuse_2">The Problem: Nonce Reuse <a class="head_anchor" href="#the-problem-nonce-reuse_2">#</a>
</h2>
<p>The recent <a href="https://www.krackattacks.com/">KRACK Attacks</a> have demonstrated a problem in practice which Miscreant aims to solve strategically: all of the cryptographic ciphers we use in practice today fail catastrophically when nonces are reused.</p>
<p>If you didn’t happen to hear about the KRACK Attacks, they are nonce reuse attacks against WPA2, a protocol which is considered the state-of-the-art in WiFi encryption. Depending on the exact WPA2 configuration used, and with an attacker in physical proximity, KRACK can recover plaintext traffic, or for ciphers which fail particularly catastrophically (i.e. AES-GCM), KRACK can enable traffic forgeries, i.e. enabling the attacker impersonating an access point.</p>
<p>Is Miscreant a magical solution to the KRACKs of the world? Probably not. Cryptography is complicated, and rarely is a new cryptography library (or in Miscreant’s case, set of libraries) a magical panacea for a problem that occurs around the time it’s announced. Miscreant’s solution to this particular problem comes at a price: a performance penalty, at least for encryption. Does it make sense for WiFi? Only time will tell… we’ll see but perhaps not.</p>
<p>That said, nonce reuse abounds elsewhere beyond KRACK, particularly anywhere users of cryptography libraries are asked to choose their own nonces. Miscreant can provide a defense in these cases: data-at-rest encryption, encrypting keys with other encryption keys, large file encryption, and even online protocols.</p>
<p>How you ask? Let’s look at what Miscreant provides.</p>
<h2 id="the-solution-misuse-resistance_2">The Solution: Misuse Resistance <a class="head_anchor" href="#the-solution-misuse-resistance_2">#</a>
</h2>
<p>Miscreant implements two modes of AES which provide a unique property called “nonce reuse misuse resistance”. But to understand what that is, I first need to describe a nonce.</p>
<p>A nonce (which is somewhat analogous to an initialization vector, if you’re familiar with those) is a single-use value which enables securely encrypting multiple messages under the same key. Nonces need not be random: we can use a counter, so long as the values are never repeated under the same key.</p>
<p>Repeated use of the same nonce under the same key causes most ciphers to fail catastrophically, as we just saw with the recent KRACK attack. If you repeat a nonce under AES-GCM, possibly the most popular authenticated mode of AES and otherwise a fairly good mode, you can XOR together two ciphertexts produced under the same key/nonce and the keystream will cancel out, leaving the XOR of the two plaintexts.</p>
<p>But even worse, repeating a nonce under AES-GCM leaks the cipher’s authentication key, allowing an attacker to perpetrate <a href="https://en.wikipedia.org/wiki/Chosen-ciphertext_attack">chosen ciphertext attacks</a> including message forgeries and even potentially full plaintext recovery. The XSalsa20Poly1305 and ChaCha20Poly1305 constructions, found in NaCl-family libraries such as libsodium, fail in a similarly spectacular way (despite these libraries often being described as “misuse resistant”).</p>
<p>As an author of <a href="https://github.com/cryptosphere/rbnacl">one of the earliest bindings to NaCl/libsodium</a>, I’ve long felt unsatisfied with this state of affairs. As I authored documentation for that library, I included the following rather unsatisfying caveats on the library’s usage:</p>
<ul>
<li>
<strong>What the algorithm does for you</strong>: ensures data is kept confidential and that it cannot be undetectably modified by an attacker</li>
<li>
<strong>What the algorithm expects from you</strong>: a secret key which must be kept confidential, and a unique (“nonce”) value each time the SecretBox function is used. The SecretBox function must never ever be called with the same key/nonce pair!</li>
<li>
<strong>What happens if you reuse a nonce: ALL IS LOST!</strong> complete loss of the confidentiality of your data (provided nonces are reused with the same key). Do <strong>NOT</strong> let this happen or you are breaking the security of your system.</li>
</ul>
<p>I really wished the underlying algorithm was more robust so this entire category of cryptographic failure simply wasn’t possible.</p>
<p>I’m certainly not alone: nonce reuse misuse resistance has been an unspoken theme of the <a href="https://competitions.cr.yp.to/caesar.html">CAESAR competition</a> to develop next-generation authenticated encryption ciphers. Google has also been working on standardizing a very fast algorithm called AES-GCM-SIV with slightly weaker security properties (self-described by Adam Langley as “<a href="https://mailarchive.ietf.org/arch/msg/cfrg/CVfp8_sy2sNIj_LXCwQ1vbdhpqQ">occasional nonce duplication tolerant</a>”) through the IRTF.</p>
<p>If others, including renowned cryptographers, are working on this same problem, why Miscreant then? I have been primarily attracted to the simplicity of the algorithms it implements. The entries to the CAESAR competition, along with Google’s proposed standard (AES-GCM-SIV), are all quite fast but rather complicated.</p>
<p>For example, during last call AES-GCM-SIV has received two proposals for major changes (one to <a href="https://mailarchive.ietf.org/arch/msg/cfrg/bJv0BHbp0O5LbYE1HjxUKRUBhdg">replace the custom “POLYVAL” function with the “GHASH” function used by AES-GCM</a>, and another to <a href="https://mailarchive.ietf.org/arch/msg/cfrg/stuWAXaqRUj4t2Q8ECzhalldkQg">redo the key derivation scheme yet again for better performance</a>). This was all after a previous round of fixes after the <a href="https://mailarchive.ietf.org/arch/msg/cfrg/k2mpWgod4mbdOxsvN6EtXHb0BAg">NSA discovered multiple key recovery attacks</a>. I’m sure a finalized AES-GCM-SIV will wind up being quite fast, and with good security bounds, but with a complicated specification and implementation which is taking quite awhile to get right.</p>
<p>AES-GCM-SIV relies on special CPU instructions, such as the CLMUL instruction, to accelerate the POLYVAL calculation, making it rather slow in environments that do not have similar hardware acceleration for this algorithm, such as mobile clients or embedded/IoT devices. AES-GCM mode is likewise unpopular in the embedded/IoT space for these same reasons, with modes like AES-CCM (also implicated in the recent “KRACK” attacks) generally preferred instead. It sure would be nice if we had a misuse resistant cipher mode which is fast both on servers and on embedded/IoT devices.</p>
<p>Miscreant implements two algorithms, one of which has been sitting nearly unused on the shelf despite having a nearly decade old RFC, and another that’s a new combination of cryptographic primitives with interesting properties. These algorithms are fast anywhere AES is fast, and do not rely on Intel-specific instructions for performance, making them much more applicable to things like the embedded/IoT space than AES-GCM-SIV. Furthermore, the code size is much smaller, and among the smallest you will find for an algorithm with misuse resistance.</p>
<p>These algorithms are <strong>AES-SIV</strong> and <strong>AES-PMAC-SIV</strong>, and I will describe how they work below. For more on nonce reuse misuse resistance, please see the blog post “<a href="https://www.lvh.io/posts/nonce-misuse-resistance-101.html">Nonce misuse resistance 101</a>” by Laurens Van Houtven (lvh).</p>
<h2 id="aessiv-aes-with-a-synthetic-initialization-ve_2">AES-SIV: AES with a synthetic initialization vector <a class="head_anchor" href="#aessiv-aes-with-a-synthetic-initialization-ve_2">#</a>
</h2>
<p><img src="https://miscreant.io/images/siv-encrypt.svg" width="410px" height="300px"></p>
<div style="text-align:center;font-size:0.8em;">The AES-SIV construction</div>
<p>Synthetic Initialization Vector, or SIV, is a block cipher mode of operation that derives the initialization vector by computing a Message Authentication Code, or MAC, of the plaintext, along with a set of optional message headers. While this might sound suspiciously like the “MAC-then-encrypt” construction we’ve been warned not to use due to potential padding oracles, the AES-SIV construction uses CTR mode which is not subject to padding oracles, and by encrypting data under the MAC ensures there is a strong cryptographic binding between the MAC of the plaintext and the ciphertext.</p>
<p>AES-SIV was (along with the rest of the algorithms implemented in Miscreant) designed by cryptographer <a href="http://web.cs.ucdavis.edu/%7Erogaway/">Phil Rogaway</a> and described in <a href="https://tools.ietf.org/html/rfc5297">RFC 5297</a> in 2008. Though originally intended to solve the key wrapping problem, i.e. encrypting a key under another key (my original motivation for Miscreant), the SIV construction is designed to support a set of message headers, or “associated data”, one of which can be a nonce. This makes SIV modes generally useful for nonce-based encryption, where deriving an initialization vector from the combination of a nonce a message, and optional associated data provides the nonce reuse misuse resistance property.</p>
<p>AES-SIV was originally specified using CMAC (and therefore might be more aptly described as AES-CMAC-SIV), a function which turns a block cipher i.e. AES into a message authentication code. CMAC is a secure pseudorandom function (i.e. a keyed hash), and fixes many of the problems of its similar predecessors like CBC-MAC, but it has one particularly notable drawback: it breaks the message into blocks and, much like AES-CBC mode, uses the output ciphertext of the previous block as an input to the next block. This makes CMAC inherently sequential, whereas modes like AES-GCM (as well as AES-GCM-SIV) are fully parallelizable.</p>
<p>This is fine for AES-SIV’s original intended use case, which is the “key wrap” problem of encrypting other encryption keys. However, it makes AES-SIV comparatively slow on modern Intel CPUs when encrypting longer messages. These CPUs are capable of pipelining AES operations, executing as many as 8 of them in parallel, which provides the effective throughput of one AES round per cycle (with an 8 cycle latency, or I believe 4 cycle latency on newer CPUs):</p>
<p><a href="https://svbtleusercontent.com/vwuvsub4gi0g.png"><img src="https://svbtleusercontent.com/vwuvsub4gi0g_small.png" alt="Screen Shot 2017-10-03 at 8.05.34 AM.png"></a></p>
<div style="text-align:center;font-size:0.8em;">An illustration of AES-NI assembly instructions executing in parallel</div>
<p>Executing AES as a sort of multi-block batch operation like this is not limited exclusively to Intel CPUs. One of the IoT devices I have been playing with, a <a href="https://www.tockos.org/blog/2017/introducing-hail/">Hail</a>, also provides a batch API for performing AES.</p>
<p>Therefore, it’s important for modern schemes to execute in parallel. Fortunately, there is a parallelizable construction quite similar to CMAC which provides the same security properties which we can use instead: PMAC.</p>
<h2 id="aespmacsiv-aessiv-with-a-parallelizable-mac_2">AES-PMAC-SIV: AES-SIV with a Parallelizable MAC <a class="head_anchor" href="#aespmacsiv-aessiv-with-a-parallelizable-mac_2">#</a>
</h2>
<p><a href="https://svbtleusercontent.com/1oqwowow9wrfg.png"><img src="https://svbtleusercontent.com/1oqwowow9wrfg_small.png" alt="Screen Shot 2017-10-04 at 9.37.18 AM.png"></a></p>
<div style="text-align:center;font-size:0.8em;">Illustration of PMAC from the <a href="http://web.cs.ucdavis.edu/~rogaway/ocb/pmac.pdf">PMAC Paper</a>
</div>
<p>PMAC is a simple cipher-based MAC which can operate in parallel, because unlike CMAC’s sequential chaining of outputs to inputs the MAC is computed by XORing together encryptions of the message, then encrypting the resulting block. It’s a bit trickier than that though: if we did this naively, then if any two blocks ever repeated they’d cancel each other out when XORed together. So PMAC includes a special tweak/offset to each block: a value derived from the key which is XORed into the input block before encrypting, and is never repeated. This ensures all blocks will be unique, even if the ciphertext is repeated.</p>
<p>PMAC, when implemented in conjunction with the AES-NI parallelization technique described above, gets great performance in comparison to CMAC. The chart below is a rough benchmark of the relative performance of Miscreant’s Rust implementation of AES-CMAC and AES-PMAC (more on how these are implemented below):</p>
<p><a href="https://svbtleusercontent.com/7x19ensfmic2g.png"><img src="https://svbtleusercontent.com/7x19ensfmic2g_small.png" alt="cmac_vs_pmac.png"></a></p>
<p>Both of these algorithms exhibit higher performance with larger messages, but it’s interesting to note that the PMAC implementation beats CMAC across the board, regardless of message size.</p>
<p>If PMAC is so great, why is it so obscure? One reason may be patents. PMAC was originally patented by its creator Phil Rogaway, but he has since abandoned his patent filings and states that he knows of no other patents besides his which are applicable to PMAC.</p>
<p>For a full AES-PMAC-SIV construction, we swap CMAC for PMAC, leaving the rest of the construction otherwise unchanged. Is it okay to do to this? Are CMAC and PMAC effectively fungible? I asked Phil Rogaway if this is the case and of all of the proofs from the original AES-SIV paper still hold with PMAC in CMAC’s place. This is what he had to say:</p>
<blockquote>
<p>Yes. The proof in the SIV paper uses generic properties of the SIV construction: you can stick in any provably-sound PRF. Quantitative results will depend on the quality of the PRF, but in the case of CMAC and PMAC, the ‘basic’ bounds are the same (within a small constant). I remember there being somewhat improved bounds for PMAC, like [Nandi, Mandal 2007], but by the time you throw in CTR, it probably doesn’t help. So, yes, effectively equivalent, as far as I know.</p>
</blockquote>
<p>Here is a chart of a run of “cargo bench” showing how the pure Rust implementations of AES-SIV and AES-PMAC-SIV compared to an optimized assembly implementation of AES-GCM found in <em><a href="https://github.com/briansmith/ring">ring</a></em>, a Rust crypto library with cipher implementations from BoringSSL (itself an OpenSSL fork):</p>
<p><a href="https://svbtleusercontent.com/9ghmduw1xdcx0w.png"><img src="https://svbtleusercontent.com/9ghmduw1xdcx0w_small.png" alt="siv_vs_pmac_siv_vs_gcm.png"></a></p>
<p>This is a bit of an apples-to-oranges comparison that works against my favor: I’m comparing a highly optimized implementation of AES-GCM to barely optimized safe Rust code. However, there’s still a lot of low-hanging fruit for future optimizations while keeping the code otherwise pure safe Rust. I am presently looking at moving to a faster Rust-based AES-CTR implementation based off of Intel’s implementation, and there is still considerable low hanging fruit for optimization in the AES-PMAC implementation.</p>
<p>What would the performance of an assembly-optimized AES-PMAC-SIV look like? Rogaway estimates it would perform at ~1.25 cycles/byte, or roughly half the speed of AES-GCM. On my 3.1GHz i7, these numbers translate to AES-PMAC-SIV topping out at ~1.3 GB/sec on a single CPU core, which I think is sufficiently fast to keep encryption from being the bottleneck in most cases.</p>
<p>I wanted to include AES-GCM-SIV for comparison in these charts, unfortunately it’s a moving target as the draft evolves and <a href="https://github.com/search?q=aes-gcm-siv">implementations are hard to come by</a> (all of these are out of date per the latest specs). According to Adam Langley the encryption performance of AES-GCM-SIV is roughly 80% that of AES-GCM (which ballparks at approximately 0.8 cycles/byte), with zero-cost decryption versus AES-GCM. If that’s actually the case, an optimized AES-PMAC-SIV should be ~65% slower than AES-GCM-SIV on modern Intel CPUs. However, AES-PMAC-SIV should be much faster (over twice as fast, as a handwaving, hardware-independent guestimate) on embedded/IoT devices that lack something like a CLMUL instruction.</p>
<h2 id="a-word-of-warning_2">A Word of Warning <a class="head_anchor" href="#a-word-of-warning_2">#</a>
</h2>
<p>To my knowledge I’m the only person who has ever implemented the AES-PMAC-SIV construction, and that’s a bit scary (if you know of prior art, please get in touch with me as I’d love to hear about it). I have had to generate my own test vectors which now match across five implementations in five different languages.</p>
<p>All of the Miscreant libraries are presently at version 0.2. I would consider this a pretty early version number for a cryptography library, and one which should give you pause to consider whether these implementations are mature enough to be using yet.</p>
<p>Parts of Miscreant, including some simple finite field arithmetic, are implemented directly in the respective languages supported by the libraries. Depending on which particular language you’re talking about and exactly how it implements integers, there can be concerns around whether it is even possible to implement constant time code in those languages, which is necessary for secure cryptography.</p>
<p>Some next steps are to make the Rust backend accessible in all languages, using bindings like <a href="https://www.neon-bindings.com/">Neon</a> and <a href="https://usehelix.com/">Helix</a>, but right now the Rust backend requires the nightly compiler and only works on x86.</p>
<h2 id="the-key-wrap-problem-and-the-quotkey-rapquot_2">The Key Wrap Problem (and the “Key Rap”) <a class="head_anchor" href="#the-key-wrap-problem-and-the-quotkey-rapquot_2">#</a>
</h2>
<p>When I started Miscreant it was to solve a very particular problem known as key wrapping. The idea of key wrapping is pretty simple: we just want to encrypt an encryption key with another encryption key. This same idea can be applied in some other contexts besides just keys: credentials, particularly ones that include some sort of random identifier, are a great use case for key wrapping.</p>
<p>SIV modes let us do something pretty neat when encrypting another encryption key: we can omit the nonce entirely. Encrypting an AES-128 key (or 128-bit random identifier ala a UUID) can be done safely using a 32-byte (i.e. 256-bit) message. This is the smallest message payload achievable for this use case while still providing authenticated encryption.</p>
<p>I think key wrapping and credential encryption are two particularly interesting use cases for Miscreant, and something I’d like to explore in the future is using Miscreant to encrypt Rails session cookies. Some other interesting use cases are encrypting API tokens, passwords, database URLs, or other sensitive application configuration parameters, while still keeping the encrypted message size small.</p>
<p>Last but not least, I leave you with the “Key Rap” (a play on “key wrap”) from the very end of the original <a href="http://web.cs.ucdavis.edu/%7Erogaway/papers/keywrap.pdf">AES-SIV paper</a>, which I’d like to dedicate all the chronic IV misusing miscreants in the land:</p>
<blockquote>
<p>Yo! We’z gonna’ take them keys an’ whatever you pleaze<br>
We gonna’ wrap ’em all up looks like some ran’om gup<br>
Make somethin’ gnarly and funky won’t fool no half-wit junkie<br>
So the game’s like AE but there’s one major hitch<br>
No coins can be pitched there’s no state to enrich<br>
the IV’s in a ditch dead drunk on cheap wine<br>
Now NIST and X9 and their friends at the fort<br>
suggest that you stick it in a six-layer torte<br>
S/MIME has a scheme there’s even one more<br>
So many ways that it’s hard to keep score<br>
And maybe they work and maybe they’re fine<br>
but I want some proofs for spendin’ my time<br>
After wrappin’ them keys gonna’ help out some losers<br>
chronic IV abusers don’t read no directions<br>
risk a deadly infection If a rusty IV’s drippin’ into yo’ veins<br>
and ya never do manage to get it exchanged<br>
Then we got ya somethin’ and it comes at low cost<br>
When you screw up again not all ’ill be lost</p>
</blockquote>tag:tonyarcieri.com,2014:Post/it-s-time-for-a-memory-safety-intervention2017-03-28T08:45:22-07:002017-03-28T08:45:22-07:00It's time for a memory safety intervention<p><a href="https://svbtleusercontent.com/oguaxiry7ktbng.png"><img src="https://svbtleusercontent.com/oguaxiry7ktbng_small.png" alt="thisisfine.png"></a></p>
<p>Memory safety won’t fix shell escaping bugs. Memory safety won’t fix logic bugs. Memory safety will not prevent an attacker who has obtained your HMAC key from forging a malicious credential that, when deserialized, can call arbitrary Ruby methods (yes, this was a real vulnerability in older versions of Rails). Memory safety will not prevent a federated identity system which uses XML-based credentials from accidentally running attacker controlled commands due to external entity processing (yes, this was a real vulnerability in certain implementations of SAML). A language which provides a memory safe model but binds to unsafe code is still vulnerable when calling into unsafe code.</p>
<p>Nobody disputes these things. Now that we have that out of the way…</p>
<p>Programming in C means you are using an unsafe memory model 100% of the time. It is the programming equivalent of trying to walk a tightrope over a lake full of alligators while trying to avoid getting electrocuted by dangling power lines. The slightest mistake in your arithmetic at any one place in the code can be the difference between a perfectly safe program and remote code execution.</p>
<p><a href="http://hakipedia.com/index.php/Poison_Null_Byte">Even if you’re off by just one byte</a>.</p>
<p><a href="https://www.blackhat.com/docs/us-14/materials/us-14-Rosenberg-Reflections-on-Trusting-TrustZone.pdf">Even if you allow an integer to inadvertently overflow</a>.</p>
<p>It’s okay to program in C. <a href="https://daniel.haxx.se/blog/2017/03/27/curl-is-c/">It’s not okay to be a developer of a an infrastructural piece of software like curl and blow off memory safety as if it doesn’t matter</a>.</p>
<p>Quoting the blog post:</p>
<blockquote>
<h1 id="c-is-not-the-primary-reason-for-our-past-vuln_1">C is not the primary reason for our past vulnerabilities <a class="head_anchor" href="#c-is-not-the-primary-reason-for-our-past-vuln_1">#</a>
</h1>
<p>There. The simple fact is that most of our past vulnerabilities happened <br>
because of logical mistakes in the code. Logical mistakes that aren’t really language bound and they would not be fixed simply by changing language.</p>
<p>Of course that leaves a share of problems that could’ve been avoided if <br>
we used another language. Buffer overflows, double frees and out of <br>
boundary reads etc, but the bulk of our security problems has not <br>
happened due to curl being written in C.</p>
</blockquote>
<p>A bold claim! And also an incredibly disingenuous one. Vulnerabilities aren’t some sort of fungible commodity: they have varying severities. I’m sure curl has had plenty of low-severity logic bugs! <a href="http://seclists.org/bugtraq/2016/Dec/32">And while it’s completely true that curl has had critical vulnerabilities that would be possible in any language</a>, this doesn’t change that fact that memory safety vulnerabilities in curl are among the most severe.</p>
<p>How long has it been since there was an announcement of a severe memory safety vulnerability in curl? Zero. (Well, more by the time you read this).</p>
<p>It’s 12:18AM. I wanted to start writing this post about an hour ago, but I couldn’t because I got home late and the first thing I did was patch OS X on my home laptop. Why? For among other reasons, this:</p>
<pre><code class="prettyprint">curl
Available for: macOS Sierra 10.12.3
Impact: Maliciously crafted user input to libcurl API may allow
arbitrary code execution
Description: A buffer overflow was addressed through improved bounds
checking.
CVE-2016-9586: Daniel Stenberg of Mozilla
</code></pre>
<p><small>Note: this vulnerability was <a href="https://curl.haxx.se/docs/adv_20161221A.html">originally announced last December</a>, but was not patched in OS X until yesterday.</small></p>
<p>curl is certainly not the only problem though. From APPLE-SA-2017-03-27-3:</p>
<pre><code class="prettyprint">AppleGraphicsPowerManagement
Available for: macOS Sierra 10.12.3
Impact: A malicious application may be able to execute arbitrary code
with kernel privileges
Description: A race condition was addressed through improved memory
handling.
CVE-2017-2421: @cocoahuke
AppleRAID
Available for: macOS Sierra 10.12.3
Impact: A malicious application may be able to execute arbitrary code
with kernel privileges
Description: A use after free issue was addressed through improved
memory management.
CVE-2017-2438: sss and Axis of 360Nirvanteam
Audio
Available for: macOS Sierra 10.12.3
Impact: Processing a maliciously crafted audio file may lead to
arbitrary code execution
Description: A memory corruption issue was addressed through improved
input validation.
CVE-2017-2430: an anonymous researcher working with Trend Micro’s
Zero Day Initiative
CVE-2017-2462: an anonymous researcher working with Trend Micro’s
Zero Day Initiative
Bluetooth
Available for: macOS Sierra 10.12.3
Impact: An application may be able to execute arbitrary code with
kernel privileges
Description: A memory corruption issue was addressed through improved
memory handling.
CVE-2017-2420: Pekka Oikarainen, Matias Karhumaa and Marko Laakso of
Synopsys Software Integrity Group
Bluetooth
Available for: macOS Sierra 10.12.3
Impact: A malicious application may be able to execute arbitrary code
with kernel privileges
Description: A memory corruption issue was addressed through improved
memory handling.
CVE-2017-2427: Axis and sss of Qihoo 360 Nirvan Team
Bluetooth
Available for: macOS Sierra 10.12.3
Impact: An application may be able to execute arbitrary code with
kernel privileges
Description: A use after free issue was addressed through improved
memory management.
CVE-2017-2449: sss and Axis from 360NirvanTeam
Carbon
Available for: macOS Sierra 10.12.3
Impact: Processing a maliciously crafted .dfont file may lead to
arbitrary code execution
Description: A buffer overflow existed in the handling of font files.
This issue was addressed through improved bounds checking.
CVE-2017-2379: riusksk (泉哥) of Tencent Security Platform
Department, John Villamil, Doyensec
CoreGraphics
Available for: macOS Sierra 10.12.3
Impact: Processing a maliciously crafted image may lead to a denial
of service
Description: An infinite recursion was addressed through improved
state management.
CVE-2017-2417: riusksk (泉哥) of Tencent Security Platform
Department
CoreMedia
Available for: macOS Sierra 10.12.3
Impact: Processing a maliciously crafted .mov file may lead to
arbitrary code execution
Description: A memory corruption issue existed in the handling of
.mov files. This issue was addressed through improved memory
management.
CVE-2017-2431: kimyok of Tencent Security Platform Department
CoreText
Available for: macOS Sierra 10.12.3
Impact: Processing a maliciously crafted font file may lead to
arbitrary code execution
Description: A memory corruption issue was addressed through improved
input validation.
CVE-2017-2435: John Villamil, Doyensec
</code></pre>
<p>This may look like a lot of vulnerabilities, but security announcements from Apple packed with remote code execution vulnerabilities and kernel-level compromises like this, for both OS X and iOS, are routine. As a security practitioner who reads these things regularly, seeing announcements with so many severe bugs can give one a fatalistic outlook, Nevertheless, many security experts including myself still point in particular to iOS as one of the most secure platforms.</p>
<p>How is this possible? The unfortunate answer is the whole apparatus of modern computing is built on a foundation that is fundamentally unsafe, and when we heap praise on iOS it’s only because it’s slightly less terrible than everything else.</p>
<p>Is it possible to write truly “safe” C code? Many C programmers will talk about all the tactics they employ: static analysis tools, valgrind, Coverity, etc. However, programmers utilizing state-of-the-art C tooling and best practices still constantly produce programs riddled with severe memory safety vulnerabilities. For all these tactics they are losing the memory safety war, because even with great tactics you can’t win a war with a bad strategy.</p>
<p>Flash back to 1995-2005 and C was my favorite language. As recently as 2013 I was shipping major projects in C (projects I was very happy and comforted to see rewritten in a memory safe language). I still think C has merit in certain applications.</p>
<p>But something needs to happen. For one, we can’t have developers of infrastructural projects like curl telling people that memory safety doesn’t really matter. Software like curl is installed on countless computer systems critical to people’s livelihoods. That’s reality. If you’re a developer of software like that, claiming memory safety doesn’t matter is <em>willful negligence</em>. It’s the programming equivalent of saying seat belts don’t save lives.</p>
<p>We need to collectively admit memory safety is a problem.</p>
<p>Is there a strategic solution to writing C we can definitively claim is memory safe? Yes, but it involves doing things like writing Haskell first, then writing C, and proving the C is equivalent to the Haskell. This was accomplished by the developers of the <a href="https://sel4.systems/">Security Enhanced L4 (seL4)</a> kernel, who also proved the Haskell correct under a formal model (and vicariously, the C). <a href="http://cs.unc.edu/%7Ecsturton/courses/verifiedsec/lecturenotessp17/lecture_0214_wang.pdf">How hard was this</a>?</p>
<blockquote class="short">
<p>The overall size of the proof, including framework, libraries, and generated proofs (not shown in the table) is 200,000 lines of Isabelle script</p>
</blockquote>
<p>Formal verification is not a tractable strategy for pretty much any day-to-day C development.</p>
<p>Where does that leave us? Well, for starters, with an awful lot of software written in C, and it’s not going away any time soon, despite how much some of us would like for everyone to just <a href="https://transitiontech.ca/random/RIIR">rewrite it in Rust</a>.</p>
<p>Though I evoked the first step of the Twelve Step program, I’m not really a fan of it. So here’s my two step program for C programmers:</p>
<p><strong>Step One:</strong> Admit memory safety is an intractable problem and even if you write a blog post boasting about how you have a zero Coverity problems policy a memory safety vulnerability in your software will literally be announced the same day.</p>
<p><strong>Step Two:</strong> Investigate alternatives to C. This isn’t my typical “how to pick a programming language” advice, but I hope these are options C programmers may have considered:</p>
<ul>
<li>C++ isn’t memory safe, but used with modern conventions at least provides a safer programming model, along with abstraction features that make programming less error-prone (watch out for those UAFs though).</li>
<li>Go is winning hearts and minds.</li>
<li>Rust is the new kid on the block. It supports a wide variety of platforms, and might even run on that microcontroller you think can’t run anything but C.</li>
<li>Edit: I almost forgot about Swift! It’s neat too.</li>
</ul>
<p>I’m not saying you have to throw away all your C code and start over from scratch. But if you find, say, Rust to be useful, you can start writing new portions/subsystems of a program in Rust (which provides a zero-overhead FFI to/from C, as it uses the C calling model, and even has a neat <a href="https://github.com/servo/rust-bindgen">binding generator</a>). If you can even pull that off, your program has moved from being 100% unsafe to >0% safe, and in my book, that’s an accomplishment.</p>
<p>We need more memory safe code in the world, and we especially need it at infrastructural levels such a curl-like core utilities and operating systems.</p>
tag:tonyarcieri.com,2014:Post/key-rotation-user-experience-crypto-reporting2017-01-14T09:30:03-08:002017-01-14T09:30:03-08:00Key rotation, user experience, and crypto reporting<p><a href="https://svbtleusercontent.com/q1clf9lae1cnnq.png"><img src="https://svbtleusercontent.com/q1clf9lae1cnnq_small.png" alt="Screen Shot 2017-01-13 at 10.57.32 PM.png"></a></p>
<p>WhatsApp was the subject of a recent <a href="https://www.theguardian.com/technology/2017/jan/13/whatsapp-backdoor-allows-snooping-on-encrypted-messages">Guardian article making claims of a “backdoor”</a> stemming from a “bug” in the way WhatsApp handles key rotations for users. The problem? WhatsApp will automatically transmit messages after the recipient’s key has changed without first asking the sender to confirm the new key is genuine.</p>
<p>Far from being a “bug” or “backdoor” (a claim so wrong I am sure hoping the original author of the story <a href="https://twitter.com/SamuelGibbs">Samuel Gibbs</a> will issue a retraction), handling key rotation seamlessly is a difficult problem with a long-storied history, along with many attempts to surface such information to the user in order to ask them to make a security decision, such as in the SSH screenshot above.</p>
<p>Clearly an in-person exchange of key fingerprints is the most secure option for establishing a secure channel, but is inconvenient, often impractical, and doesn’t provide a good means for handling key rotation.</p>
<p>There are two basic models to solving the key distribution and rotation problem in an automated manner that don’t require in-person confirmations with PGP-like “key signing parties” (and I’m sorry to say it PGP/GPG fans but <a href="https://blog.filippo.io/giving-up-on-long-term-pgp/">I too don’t think GPG is the future</a>). These methods to determine trusted keys are as follows:</p>
<ul>
<li>
<strong>The SSL/TLS PKI Way</strong> (used by every https:// web site): Farm trust out to central services (i.e. certificate authorities). Keys can be rotated without your consent or knowledge. Your browser will seamlessly make the same request to a server even if its key has just changed. In messaging apps this generally involves placing a lot of trust in the key servers for the messaging app operators, although systems like <a href="https://coniks.cs.princeton.edu/">CONIKS</a> and Google’s recently announced <a href="https://github.com/google/key-transparency">Key Transparency</a> provide tools for helping keep key servers honest.</li>
<li>
<strong>The SSH Way</strong> (a.k.a. “trust on first use”): record the last known key for an SSH server. If it changes, print a big scary warning (see above). Only after accepting the new key can you connect to the remote host, possibly sending a sensitive credential like a password. (Note that SSH also supports a certificate authority-based PKI)</li>
</ul>
<p>The main difference between these two approaches is an automated decision as to whether we should automatically trust keys after they have been rotated. In the PKI approach used by the web the new key is automatically trusted, whereas in the SSH TOFU model the user is presented a warning upon any key change.</p>
<p>At first glance the SSH approach might seem to offer some better security properties: the user is notified whenever a key changes, allowing them to do some due diligence on whether the key change was expected or if a man-in-the-middle attack is underway.</p>
<p>Unfortunately, as any of you who are familiar with the warning at the top of the post are familiar with, most key change events are mundane and not an attack. This makes prompting the user for every key change a bit “boy who cried wolf”: the encrypted client is asking the user to make a security decision, but typically the user isn’t under attack (or can’t tell they are) and will therefore almost always accept the new key.</p>
<p><a href="https://svbtleusercontent.com/vuhs2i1swvfxka.png"><img src="https://svbtleusercontent.com/vuhs2i1swvfxka_small.png" alt="Screen Shot 2017-01-13 at 11.57.54 PM.png"></a><br>
<small>Image originally from Peter Gutmann’s unreleased book <i>Engineering Security</i></small></p>
<p>Is prompting the user a good idea in this case? Can the user be expected to make a reasonable security decision? Will the constant prompts whenever someone gets a new phone be too annoying? These are difficult questions. </p>
<p>Open Whisper Systems, the non-profit behind WhatsApp’s encrypted messaging protocol, has taken both approaches: WhatsApp does not prompt users by default, relegating such prompts to an optional setting that WhatsApp users must opt into. However, its in-house messaging app Signal does prompt by default.</p>
<p>Why take two approaches? Isn’t one of them “right”? There is definitely a vocal tinfoil hat crowd who would consider anything less than the Signal approach by default as unacceptable. And while this crowd may generally do a good job reasoning about secure systems as technologies, they have generally lacked the empathy to consider such factors as “Can a normal person actually use this? Will they make a reasonable security decision?”</p>
<p>The net result of optimizing for a pathological threat model at the cost of user experience has had a pretty clear security outcome: users would choose insecure tools over the “secure” ones, because the “secure” ones are impossible to use. It takes a lot of empathy to consider whether prompting someone who does not understand what the concept of a cryptographic key even is about how they want to handle a key rotation event will actually have a net positive security outcome.</p>
<p>There aren’t any right answers here: only tradeoffs.</p>
<p>Signal targets a different audience than WhatsApp: they assume out of the gate that you want a more secure, encrypted messenger. WhatsApp, on the other hand, shipped encryption-by-default that the end user doesn’t even have to be aware of. Where Signal targets an audience of millions, WhatsApp is targeting an audience of billions. The (adjustable) defaults in WhatsApp are designed so encryption can be on-by-default at no cost to the user experience, but still allow those who would like to receive security notifications to receive them by opting in.</p>
<p>Consider what web browsers would be like if they prompted a user to make a security decision whenever the key for a site changed:</p>
<p><a href="https://svbtleusercontent.com/sidrq2fctwjlna.png"><img src="https://svbtleusercontent.com/sidrq2fctwjlna_small.png" alt="keychangeui.png"></a></p>
<p>I do not think asking users to make decisions like this would tangibly improve the security of the web. However, I do think it would scare people away from visiting sites in the first place.</p>
<p>Now I’d like to take a bit to talk about crypto reporting…</p>
<p>If there were a backdoor in a popular encrypted messaging app, that is big news, and it should be reported on.</p>
<p>This was not a backdoor. I think, had this story been run by a few security experts in advance, most would’ve told you that it is not a backdoor. First, let me say that if you are a reporter sitting on a story like this and are looking for opinions in advance before a release, I’d be happy to offer mine or find an interested cryptographer to put you in touch with. I would really love to help close the gap between reporters and security experts.</p>
<p>But second, there is a recent story about encrypted messaging apps worth reporting on:</p>
<p><a href="https://svbtleusercontent.com/hnpkv3zydw9ylq.jpg"><img src="https://svbtleusercontent.com/hnpkv3zydw9ylq_small.jpg" alt="C12OTePUkAARrBM.jpg"></a></p>
<p>The recently released, highly disputed, but <a href="http://www.businessinsider.com/trump-dossier-looking-increasingly-credible-2017-1">increasingly credible looking</a> Trump dossier contained this tidbit about the messaging app Telegram.</p>
<p>Though Telegram is often described by the media as “<a href="https://www.wired.com/2016/08/hack-brief-hackers-breach-ultra-secure-messaging-app-telegram-iran/">ultra-secure</a>”, it has default settings considerably worse than anything in WhatsApp: end-to-end encryption in Telegram is <em>off</em> by default, whereas most major messaging apps (including Apple’s iMessage) always encrypt end-to-end.</p>
<p>It’s unclear exactly what security problem with Telegram the dossier above is referring to. Telegram has a history of being <a href="http://news.softpedia.com/news/ss7-attack-leaves-whatsapp-and-telegram-encryption-useless-503894.shtml">exploited through the SS7 network</a> (an attack which works equally well against WhatsApp). But with end-to-end encryption off by default, and <a href="https://cs.au.dk/%7Ejakjak/master-thesis.pdf">the poor quality of the cryptographic design even when end-to-end encryption is on</a>, Telegram leaves a lot of opportunity for novel attacks, especially the kind perpetrated by nation state agencies.</p>
<p>I’m sure there’s an interesting story here, and one far closer to a legitimate security problem in a major messaging app than how WhatsApp handles key rotation.</p>
tag:tonyarcieri.com,2014:Post/4-fatal-flaws-in-deterministic-password-managers2016-11-22T08:00:11-08:002016-11-22T08:00:11-08:004 fatal flaws in deterministic password managers<p>Wouldn’t it be nice if your password manager didn’t need a database? Instead of synchronizing a password vault between your devices, you could use the magic of cryptography to magically transform a master password into a unique password for each site.</p>
<p>There have been a <a href="https://lesspass.com/">numerous</a> <a href="https://libeclipse.me/visionary/">and</a> <a href="https://forgiva.com/">ever</a> <a href="http://masterpasswordapp.com/">growing</a> <a href="https://github.com/appnician/pastor">implementations</a> <a href="http://www.nullpass.org/">of this</a> <a href="https://github.com/pibara/ZeroVault">idea</a>. Much of the marketing material for these tools talks about how using a deterministic scheme allows “sync-free” operation, is “more secure” than a password vault, and often that it’s a newer idea than encrypted password vaults.</p>
<p>In this post, I will argue that you can’t practically provide “sync-free” operation without making your password manager unusable, how using a deterministic scheme harms security, and how it’s actually an old idea which never caught on for good reasons.</p>
<p>Deterministic password derivation schemes date back to at least 2003: <a href="https://www.pwdhash.com/">Stanford PwdHash</a> and <a href="https://chriszarate.github.io/supergenpass/">SuperGenPass</a> are two early examples (If you know of earlier ones, I’d love to hear about them). These tools pre-date most modern password vaults, but never evolved into the sort of practical tools that the latter did.</p>
<p><strong>Edit:</strong> I have been informed of at least one scheme that predates Stanford PwdHash and SuperGenPass: Alan Karp has informed me his <a href="https://sitepassword.alanhkarp.com/">Site Password</a> scheme dates back to 2002. More information on it is available in the <a href="http://www.hpl.hp.com/techreports/2002/HPL-2002-39R1.pdf">Site-Specific Passwords HP Tech Report</a>.</p>
<p>Why is that? Let’s take a look at their flaws.</p>
<h2 id="deterministic-password-generators-cannot-acco_2">Deterministic password generators cannot accommodate varying password policies without keeping state <a class="head_anchor" href="#deterministic-password-generators-cannot-acco_2">#</a>
</h2>
<p>One of the big selling points of deterministic password generators is that they enable “sync-free” operation: you can derive all the passwords for the sites you use from the master password alone, so there’s no state that needs to be synchronized between devices.</p>
<p>Unfortunately, sites have wildly varying and often conflicting password requirements: non-alphanumeric symbols are mandatory! Passwords must be alphanumeric only! Capital letters required! Passwords must be lower-case only! Passwords must be at least 12 characters long! Passwords must be at most 8 characters long! </p>
<p>While many of these requirements seem silly and in a perfect world all sites would adopt the new <a href="https://nakedsecurity.sophos.com/2016/08/18/nists-new-password-rules-what-you-need-to-know/">NIST password guidelines</a>, reality is messy and there is no single deterministic password generation scheme which can accommodate the password policies of all sites.</p>
<p>The most common solution to this problem is to abandon “sync-free” operation and keep some state about tweaks to the password generation scheme which produce passwords that meet the policy for a specific site. Unfortunately, by doing so we lose the #1 selling point for these schemes, and now have state that needs to be synced between devices.</p>
<p>There are other solutions which do remain stateless at the cost of user experience: users can be asked to tweak the generated password to meet the policy every time they log into a site, and hope they can remember the exact settings. I consider this unacceptable from a user experience perspective.</p>
<p>Some authors of deterministic password generators argue that synchronizing “tweak” data is superior to synchronizing the ciphertext of a password vault, because the ciphertext is somehow “more sensitive”. If you agree, please keep reading, because at the end of this post I’ll argue the opposite.</p>
<p>The main takeaway of this point: we can’t practically get away with “sync-free” operation without a terrible user experience. The user experience burdens outweigh the benefits.</p>
<h2 id="deterministic-password-generators-cannot-hand_2">Deterministic password generators cannot handle revocation of exposed passwords without keeping state <a class="head_anchor" href="#deterministic-password-generators-cannot-hand_2">#</a>
</h2>
<p>Strike two against “sync-free” operation deals with revocation: There are many ways we could accidentally expose the password for a specific site: accidentally copying and pasting it into an email, IM, or social media. This is a fairly routine occurrence that happens to the best of us. Another possibility is a security incident that exposes unhashed passwords: this can be anything from a “Heartbleed”-style vulnerability to a site accidentally logging unhashed passwords.</p>
<p>Solving this problem with a stateful password manager is easy: if we’re using a typical encrypted password vault, we just make a new password. If we’re using a stateful deterministic scheme, we can bump a counter associated with the site, and include that counter’s value as part of the derivation scheme.</p>
<p>There’s nothing we can do in a stateless scheme without harming user experience. We could ask the user to remember the site-by-site counter and input the correct counter value to derive the correct password. But I think this is silly.</p>
<p>So once again, we’re stuck with some state to synchronize, and the “sync-free” dream is simply impractical.</p>
<h2 id="deterministic-password-managers-can39t-store_2">Deterministic password managers can’t store existing secrets <a class="head_anchor" href="#deterministic-password-managers-can39t-store_2">#</a>
</h2>
<p>Even if we abandon “sync-free” operation, authors of stateful deterministic password tools argue they’re still beneficial because there’s less data to sync, so therefore they’re faster and more web scale or something like that.</p>
<p>I have a 1Password vault containing thousands of passwords and cryptographic keys. It weighs in at 512kB. I do not see the size of password vaults as a pressing concern.</p>
<p>But beyond that, a deterministic scheme cuts you out of the ability to store data which is not derived deterministically in a password vault.</p>
<p>You can’t store credit card numbers or bank account numbers in such a vault.</p>
<p>You can’t put arbitrary cryptographic keys in such a vault.</p>
<p>You can’t store <a href="https://twitter.com/bcrypt/status/698261778539483136">randomly selected answers to security questions</a> in such a vault.</p>
<p>I consider this to be part of the basic functionality of a password manager. Leaving it out is unacceptable to me.</p>
<h2 id="exposure-of-the-master-password-alone-exposes_2">Exposure of the master password alone exposes all of your site passwords <a class="head_anchor" href="#exposure-of-the-master-password-alone-exposes_2">#</a>
</h2>
<p>The sales material for deterministic password managers often includes some specious reasoning about how they make you more secure than an encrypted password vault, typically making the argument that having any ciphertext at all is some sort of security liability (it’s not; that’s the point of cryptography). But the opposite is true: deterministic schemes have <em>worse</em> security properties.</p>
<p>With a traditional encrypted password vault scheme, we need two things to obtain site-specific passwords: the ciphertext of the password vault, and the master password.</p>
<p>With a deterministic scheme, as an attacker we only need one: the master password.</p>
<p>Under these schemes all it takes to expose the passwords to every site you use is the exposure of your master password. If you accidentally type or paste your master password into email, IM, or social media, an attacker can leverage that alone to derive all of your site-specific passwords.</p>
<p>With a system that uses an encrypted password vault, this is insufficient: an attacker must also obtain the ciphertext of your password vault. This ciphertext acts as a sort of insurance policy: so long as the attacker can’t get to it they can’t leverage your master password to attack you, and getting rid of it is therefore a huge drawback.</p>
<p>Now granted, in either case you’ll probably want to rotate all of your passwords in the event your master password is exposed, just in case. But an encrypted password vault will keep you safer: If you can rotate the master password on your vault, and delete all previous backups and copies of your vault, you’ll be in a lot better shape than if you accidentally expose your master password for a deterministic scheme.</p>
<p>Nevertheless, the authors of these tools use specious reasoning argue they offer better security properties, despite making useless tradeoffs that weaken security. In cryptographic circles, there’s a term for this: snake oil.</p>
<p>A password manager is one of the most important security tools available today: it literally holds the keys to your online existence. For the record I use <a href="https://1password.com/">1Password</a> as my password manager. I hope if you were considering switching from a traditional password vault to one of these deterministic password managers, you now see the flaws and will avoid them in the future.</p>
<h2 id="followup-bolstering-deterministic-password-sc_2">Follow-up: Bolstering deterministic password schemes with a pre-shared secret; offline attacks against encrypted password vaults <a class="head_anchor" href="#followup-bolstering-deterministic-password-sc_2">#</a>
</h2>
<p><u>NOTE: This section was added after the original publication of this post.</u></p>
<p>Following the publication of this post, several people suggested sharing a secret included as an input to the password hashing function as a way to bolster the security of deterministic password generation schemes. This sort of additional input to a password hashing function is typically called a “salt”, and although salts aren’t always “full entropy” (i.e. 128-bit uniformly random value) or kept secret, I would agree that a “full entropy” secret salt value shared once between devices should be sufficient to prevent accidental disclosure of the master password from being immediately catastrophic. Unfortunately none of the people making this suggestion could point to an app that actually does this. If you know of one, I’d love to hear about it!</p>
<p>Others pointed out that password vaults are vulnerable to offline attacks if they are e.g. obtained from a cloud service in encrypted form, and cite this as a disadvantage of using an encrypted vault as opposed to a deterministic password generator. This is also true: encrypted vaults are vulnerable to offline attacks. Ideally <a href="https://keybase.io/warp/">modern password hashing algorithms shouldn’t be vulnerable to offline attacks</a> (though I would suggest using scrypt or Argon2 rather than the cited “WarpWallet” construction, which I view as needlessly complex and redundant), especially if you choose a strong master password, but offline attacks are still worth considering.</p>
<p>Both these claims were the most common responses to my post, often made by the same people, but for whatever reason nobody put the two of them together and suggested using a full-entropy secret salt stored out-of-band from the encrypted password vault to guard against offline attacks. So I’ll go ahead and suggest that: using this sort of out-of-band, sync-once style secret improves the security of either deterministic password managers or encrypted password vaults.</p>