Category Archives: Uncategorized

Two Theories of Japanese Culture

Two favorite theories from Japan and the Shackles of the Past:

  1. Japan never had a true Axial Age moment – the moment where Athenian philosophy, Buddhism, and Judaic monotheism (among others!) all began to separate the material world we live in from the “spiritual world” above. All human culture to that point was animistic – where spirituality and real gods imbued every aspect of the world and to talk of a separate spirit world didn’t make much sense at all. When Buddhism arrived in Japan it never fully co-opted the native Shinto animism. And it is precisely this “spirituality in everything” approach that characterizes the Japanese reverence for small design details, ceremonial acts, etc.
  2. Feudal Japan under the Tokugawa Shogunate (1600-1868) is widely known for the influence of the samurai warrior class and their spartan ethic of bushido. However, this was a peaceful period! In fact, the samurai were essentially useless at all for 250 years and they responded in two ways. First, they focused more than ever on an extreme version of bushido, seeking to one-up each other with asceticism and military technique. Second, though, they became decadent – noted for their patronage of everything from burlesque to prostitution — which developed customs of extravagant costume and theatrical presentation. It is precisely this dual nature that shocks one about Japan today – the ultra-serious business ethic coexisting easily with the hypersexualized otaku videogame culture, but it has a long history.

These are obviously oversimplifications and I barely know anything about Japan, certainly not enough to evaluate these two theories – but I enjoyed thinking about them.

Where are you from?

I’ve been thinking a lot lately about the question “where are you from”. It rankled me for a long time – in San Francisco when I first arrived it felt like a provocation, a way of checking if you were a native or a gentrifier. Then I made a friend who has spent a lot of time in Navajo nation building relationships with folks there. They ask the same question, apparently, because their relationship to the land itself is so much of a part of who they are, that knowing you grew up on a particular mountain or stream tells them about you in ways that go beyond “local culture”. So I started thinking about which parts of your background are important to get to know you? Which fields have you studied, who are your intellectual and ethical ancestors? Is there a “small talk” question that gets at that? “What did you major in” is probably close, for more intellectual folks who are still in or near college, but is still a pretty weak shadow of what I’m trying to get at. 
And this becomes even more relevant when you think about statelessness and movement as political issues. The 21st century doesn’t have a monopoly on this – human history is full of forced and voluntary migrations – but there are particular new spins on it. Cosmopolites (which I just read and is very good) talks about bidoons in the UAE, whose ancestors were nomadic desert people just like everyone else, but missed the citizenship initiatives of when those states were forming, and are now stateless – and the UAE’s efforts to buy them citizenships in Comoros, a poor island in the Indian Ocean. The most marginalized want citizenship as a way to secure their rights, to be legible to government and justice. Then of course the ultra rich are stateless in a different way – trying specifically to escape from government responsibilities like paying taxes, they collect residences and passports in the most convenient nations. Climate refugees may be one of the key stories of the 21st century and this question of “where are you from” may have a completely different meaning for them.


A very thoughtful recent blog post makes the point that institutions that seem “decentralized” or claim that as a value often exhibit centralizing tendencies over time. Some recommendations:

  • Be specific about what things you want to decentralize, and why. Regard decentralization as an ongoing process that can never be complete.
  • Find checks and balances, so that it is harder for any set of actors to achieve centralizing power. Use multiple forms of decentralization and participation.
  • Consider accountability: often what we really care about is accountability, and a centralized but accountable entity (such as a government antitrust enforcer) can limit the centralized and unaccountable power accumulation that we fear.

Pluribus Skepticism

Is Facebook’s new poker AI really the best in the world?

Facebook released a paper and blog post about a new AI called Pluribus that can beat human pros. The paper title (in Science!) calls it “superhuman”, and the popular media is using words like “unbeatable”.

But I think this is overblown.

If you look at the confidence intervals in the FB blog post above, you’ll see that while Pluribus was definitely better against the human pros on average, Linus Loeliger “was down 0.5 bb/100 (standard error of 1.0 bb/100).” The post also mentions that “Loeliger is considered by many to be the best player in the world at six-player no-limit Hold’em cash games.” Given that prior, and the data, I’d assign something like a 65-75% probability that Pluribus is actually better than Loeliger. That’s certainly impressive. But it’s not “superhuman”.

I don’t know enough about poker or the AIVAT technique they used for variation reduction to get much deeper into this. How do people quantify the skill difference across the pros now?

I’m also a bit skeptical about the compensation scheme that was adopted – if the human players were compensated for anything other than the exact inverse of the outcome metric they’re using, I’d find that shady – but the paper didn’t include those details.


Defensive Randomization

Machine learning is common and its use is growing. As time goes on, most of the options that you face in your life will be chosen by opaque algorithms that are optimizing for corporate profits. For example, the prices you see will be the highest price under which you’ll buy, as based on an enormous amount of data about you and your past decisions.

To counter these tendencies, I expect people to begin adopting “defensive randomization”, introducing noise into your decision-making and forcing corporate algorithms to experiment more broadly with the options they introduce to you. You could do this by simple coin flip, or introduce your own bots that make random (or targeted exploratory) decisions on your behalf. For example, you could have a bot log in to your Netflix account and search for a bunch of movies that are far away from Netflix’s recommendations for you.

One possible future is for these bots to share data between themselves — a guerilla network of computation that is reverse-engineering corporate algorithms and feeding them the information that will make your life more humane.

This is related to:

[mildly inspired by Maximilian Kasy’s Politics of Machine Learning]


Police Science

Very much enjoying Jackie Wang’s Carceral Capitalism.

Especially liked this thought in “This Is A Story About Nerds and Cops“:

Given that critics of the police associate law enforcement with the arbitrary use of force, racial domination, and the discretionary power to make decisions about who will live and who will die, the rebranding of policing in a way that foregrounds statistical impersonality and symbolically removes the agency of individual officers is a clever way to cast police activity as neutral, unbiased, and rational.

Contributing to pandas

Very proud to announce today that I had a pull request merged into the pandas library. In version 0.21, pandas will have a new feature: a way to read in line-delimited JSON in small pieces, which can be useful when working with large files or streams.

This is a fairly small change, technically, but a big deal for me.  Pandas is one of the most commonly used tools in the data science world. When I started at TrueAccord they bought me the book on pandas (Volume 2 coming out next month!). This was my first introduction to any programming language other than Stata, an odd proprietary language that languishes on among economists and epidemiologists. Now, writing software is a core part of my career.

Related, I highly recommend  The Success of Open Source, in which Steven Weber outlines the varied ways in which open source communities elicit and channel cooperation, and explores the complex set of motivations that leads people to contribute to open source.

What I’ve been reading lately

Rebecca Solnit, River of ShadowsSolnit is a marvelous thinker and historian who moves smoothly between well-researched historical fact and philosophical reverie. Here she traces the life of Edward Muybridge whose motion studies of animals are still familiar today. Muybridge was a first-class photographer, a true artist who also made many technical innovations. Solnit takes his collaboration with Leland Stanford as the jumping-off point for an exploration of the way technology has annihilated time and space, and develops a genealogy from those two to the California of today, dominated by Hollywood and Silicon Valley. In her telling, these two industries named for physical places are at the center of a world that, in large part because of their doing, is increasingly disconnected from the world itself.

Mary Robison, Why Did I EverA few years back I made a note to myself to read this novel. I can’t recall why, or at whose urging, but I’m glad I did. Told in over 500 short fragments, Robison is funny and poignant. I was sad to have finished this book.

Diane Coyle, GDP: A Brief but Affectionate HistoryI’ve been meaning to read this for a while, but I am, so far, disappointed. GDP is the single measure that people associate with economic health and growth, to the extent that people say “the economy grew” when they mean “GDP grew”. How the economy is measured could not be more important and Coyle lays out some of the history of how GDP developed, and some of the ways in which it is flawed. This wasn’t the right level of depth for me — took some things for granted and was disappointingly shallow elsewhere — but seems like a good starting point for a deeper read into these ideas.

Nitt Witt Ridge

Art Beal spent 61 years building a house out of found materials at Nitt Witt Ridge in Cambria, CA. He served for a time  as the town garbageman, dumping his truck directly into his own backyard and rummaging for salvageable building supplies with which he slowly built a house in the shape of his own mind. There is now little trace of the 20 feet of landfill underneath the hill. where his house rests.

Beal, born in Oakland, was a celebrated long-distance swimmer in his youth but decamped in his 20’s to Cambria, 200 miles south along the California coast. He built a small house and lived in it with “Gloria” whose life is otherwise lost to history. At some point she disappeared. He abandoned that house and began constructing his masterwork, the unfinished project of the rest of his life.

There is no place in our world for some men. Through accident of birth some men are born different and they accumulate injuries in the world as they repeatedly are rammed through holes of the wrong shape. Beal was lucky. He found a place for his energy, found a way to preserve himself in a world that has no room for difference of mind.

USAFacts, Corporate Hagiography and Historical Ignorance

This morning my circles are talking about Steve Ballmer’s new government data initiative USAFacts as reported in this NYT article.

It’s an interesting project, and I am glad that this is how Ballmer is spending his dotage! It’s a lot better than going into VC as a lot of other tech execs seem to do as they age. I wish him the best.


This is not the first time someone has worked on making government data more accessible. I wish that Ballmer and the media coverage around this launch spent any time at all discussing the many other similar initiatives and how this fits into the ecosystem.

For example, the mission of “a comprehensive summary” is interesting and different, but represents a tradeoff compared to deep contextual understanding. Contrast with the “Scarsdale” series by Thomas Levine!/scarsdale/, for example. Also, this is a classic example of the “How Standards Proliferate” process. Everyone who comes along thinks: “If only there were one canonical home for all government data!” And then you end up with 15 different portals.

I think most notably, USAFacts doesn’t actually make their data open, they just publish reports. That’s a major departure from what a lot of other players are doing, and I wish there was any discussion about why they made that choice. Are there legal requirements connected to some of the data? Surely at least some of it could be open. Is it a desire to keep a “moat”? Who knows!

The tone around this launch irks me in the same way most tech coverage irks me. Ballmer is not the first to think of it, not by a long shot. And his effort to understand what was already out there seems….cursory, at best. Googling “open government data” would have been a very good start.

Why was this published in NYT’s DealBook section? It’s not business reporting at all. DealBook seems to exist as a WSJ competitor so the Times can attract the crowd that just wants corporate hagiography. Related:

If you are interested in learning more about different open datasets, this may be a good start:!/open-data/better-datasets-about-open-data/

Edit: There are two comment threads on HN (1 2) about this, the discussion is pretty good so far. Fave comments: