This post expands on an algorithm shown in the book I wrote on floating-point math. It is very common in computing to want to do $e^x$ very quickly and not care very much about how accurately you computed it. This is increasingly true in ML and AI algorithms, which can be very tolerant to noise from numerical error and …

When I recently looked at the state of the art in floating point random number generation, I was surprised to see a common procedure in many programming languages and libraries that is not really a floating-point algorithm: Generate a random integer with bits chosen based on the precision of the format. Convert to …

Twas about 4-6 weeks before Christmas, and all through the math department, not a creature was stirring, not even a plucky young undergrad. Cryptography professors Alice and Bob sat at the elliptically-curved conference table to plan the department's secret Santa. Mallory, the department secretary, had been given the …

I often like to have streams of trials playing in the background when I am working, and the recent trial of interest was the trial of Karen Read, who was accused of murder. One issue in this case was a conflict between two timestamps logged by different apps on a potential suspect's phone. Apple health had logged the …

A guilty pleasure of mine is the pursuit of perfection. It is certainly a vice in most contexts, but there are some problems whose solutions demand a measure of perfection. These are problems that I will refer to as "5-9 problems": problems whose solutions need five 9's (or more) in some dimension. Usually, those nines …

Over the last year, as a person with a hardware background, I have heard a lot of complaints about Nvidia's dominance of the machine learning market and whether I can build chips to make the situation better. While the amount of money I would expect it to take is less than $7 trillion, hardware accelerating this wave …

This account comes from several publicly available sources as well as accounts from insiders who worked at Knight Capital Group at the time of the issue. I am telling it second- or third-hand. On August 1, 2012, Knight Capital fell on its sword. It experienced a software glitch that literally bankrupted the company. …

As you build a computer system, little things start to show up: maybe that database query is awkward for the feature you are building, or you find your server getting bogged down transferring gigabytes of data in hexadecimal ASCII, or your app translates itself to Japanese on the fly for hundreds of thousands of …

I have recently been immersed in the theory and practice of random number generation while working on Arbitrand, a new high-quality true random number generation service hosted in AWS. Because of that, I am starting a sequence of blog posts on randomness and random number generators. This post is the first of the …

A modern CPU is an incredible machine. It can execute many instructions at the same time, it can re-order instructions to ensure that memory accesses and dependency chains don't impact performance too much, it contains hundreds of registers, and it has huge areas of silicon devoted to predicting which branches your …