USAFacts, Corporate Hagiography and Historical Ignorance

This morning my circles are talking about Steve Ballmer’s new government data initiative USAFacts as reported in this NYT article.

It’s an interesting project, and I am glad that this is how Ballmer is spending his dotage! It’s a lot better than going into VC as a lot of other tech execs seem to do as they age. I wish him the best.


This is not the first time someone has worked on making government data more accessible. I wish that Ballmer and the media coverage around this launch spent any time at all discussing the many other similar initiatives and how this fits into the ecosystem.

For example, the mission of “a comprehensive summary” is interesting and different, but represents a tradeoff compared to deep contextual understanding. Contrast with the “Scarsdale” series by Thomas Levine!/scarsdale/, for example. Also, this is a classic example of the “How Standards Proliferate” process. Everyone who comes along thinks: “If only there were one canonical home for all government data!” And then you end up with 15 different portals.

I think most notably, USAFacts doesn’t actually make their data open, they just publish reports. That’s a major departure from what a lot of other players are doing, and I wish there was any discussion about why they made that choice. Are there legal requirements connected to some of the data? Surely at least some of it could be open. Is it a desire to keep a “moat”? Who knows!

The tone around this launch irks me in the same way most tech coverage irks me. Ballmer is not the first to think of it, not by a long shot. And his effort to understand what was already out there seems….cursory, at best. Googling “open government data” would have been a very good start.

Why was this published in NYT’s DealBook section? It’s not business reporting at all. DealBook seems to exist as a WSJ competitor so the Times can attract the crowd that just wants corporate hagiography. Related:

If you are interested in learning more about different open datasets, this may be a good start:!/open-data/better-datasets-about-open-data/

Edit: There are two comment threads on HN (1 2) about this, the discussion is pretty good so far. Fave comments:

Python tip: Inspect function signature at runtime


I have a list of functions with different signatures. There is some set of possible parameters, and I want to call all these functions with the “appropriate” argument for each parameter.

This is a little hand-wavy, let’s look at an example:

def half(a):
    return a / 2

def twice(a):
    return 2 * a

def addition(a, b):
    return a + b

def subtraction(a, b):
    return a - b

functions = [half, twice, addition, subtraction]
a = get_a()
b = get_b()

Desired outcome:

[half(a), twice(a), addition(a,b), subtraction(a, b)]

And we want to do this without making our function definitions too ugly.

Solution 1:

One option is to `get_b()` within the functions that need them. This is not ideal, suppose `get_b` is not a pure function (e.g. a network call), we would want to pass `b` into scope instead of getting it from elsewhere every time it’s needed.

Solution 2:

We could change the signature to accept arbitrary kwargs and then pass a dict of args, for example:

def half(**kwargs):
    return kwargs['a'] / 2

def twice(**kwargs):
    return 2 * kwargs['a']

def addition(**kwargs):
    return kwargs['a'] + kwargs['b']

def subtraction(**kwargs):
    return kwargs['a'] - kwargs['b']

functions = [half, twice, addition, subtraction]
payload = {'a': get_a(), 'b': get_b()}
results = [f(**payload) for f in functions]

This works, but makes each of our function definitions uglier.

Solution 3:

Allow each function to have a different signature, inspect the signature at runtime and pass what is needed.
(Adapted from

import inspect

def half(a):
    return a / 2

def twice(a):
    return 2 * a

def addition(a, b):
    return a + b

def subtraction(a, b):
    return a - b

# Wrapper which:
# * accepts a dict of all possible kwargs and their names
# * inspects the signature of the function
# * calls that function with the correct args
def call_func_with_correct_args(f, possible_args):
    func_args = inspect.getargspec(f).args
    args_to_pass = {k: possible_args[k] for k in func_args}
    return f(**args_to_pass)

functions = [half, twice, addition, subtraction]
a = get_a()
b = get_b()
full_payload = {'a': a, 'b': b}

results = [call_func_with_correct_args(f, full_payload) for f in functions]

So this is nice and clever, but we need to be careful that our function parameters are named correctly and consistently. Essentially we are passing the burden to the function definitions.


I don’t know what a great solution to this might look like. Is there a better way to do this? If all the parameters are different types, Python3’s type hinting might provide another option. What does this look like in other languages?

Book Review | Earthseed Series | Octavia Butler

Parable of the Sower and Parable of the Talents by Octavia Butler

I first heard about Octavia Butler in early 2014 on Twitter, I think originally from Danilo and/or Holly. I saw allusions to something called Earthseed, to humanity’s destiny in the stars, to the idea that “God Is Change”.

Two years later, with fascism on the rise and afrofuturism enjoying a moment of popularity, I went to a Fusion-backed symposium about Butler’s work. ((I learned about this from Alexis Madrigal’s newsletter Five Intriguing things which I’ve previously pluggedAlexis, formerly at the Atlantic and now Editor in Chief at Fusion, is one of the most interesting writers I follow.))  At this event, I was delighted by the idea that all progressives, all activists, are engaged in acts of science fiction; they imagine alternate worlds that could branch off from this one in a plausible way, societies like ours except governed by different principles of the physical or psychological universe.

Sower and Talents, published in 1993 and 1998 respectively, look more prescient by the day. Butler saw the future with great clarity and with a sense of resignation to the hate, destruction and degradation our world would suffer. In the Parable series, environmental catastrophe and economic inequality have created a desperate underclass driven to violence and drugs, whose life is of no value to a police force interested in protecting the property of the rich. In this fertile ground a white supremacist Christian paramilitary organization flourishes with the winking support of Presidential candidate Andrew Steele Jarrett, whose ascendance tears apart the vanishing middle class between liberal values and a frantic need to protect their families and communities from the predations of those even a little less fortunate. Kashmir Hill has already written about the uncanny similarity of this campaign to Trump’s. By the late 80s our future was not murky  to a thinker of Butler’s diagnostic precision.

The series follows Lauren Oya Olamina, a teenage girl who shows us the imagination and empathy and ambition that we will need to survive this bleak world. As a teenager in a middle-class enclave in southern California, Olamina begins to develop a practice called Earthseed, rooted in strong communities, individual self-sufficiency and an embrace of the universe’s ever-changing nature. Earthseed demands resilience and adaptability, with a sort of scientific and moral pragmatism, and points humanity towards the stars for its own survival. As she develops her philosophy it is eventually collected into The Book of the Living, which is “excerpted” heavily in the two books.

In these two books we don’t see anyone leave Earth — we are not given the pleasure of Butler articulating what it would be like for a whole society to live by these principles. We see small communities struggle to adopt these practices. We see them try to integrate new members who are grateful for food and shelter and company but skeptical of any indoctrination. We see major setbacks and minor accomplishments.

When we are defeated by Moloch our devastation is global and absolute and permanent. Our victories are usually are messy and local and temporary, a momentary respite from an ancient foe that is only getting stronger. If we are to survive, we must connect our small patches of humanity into a resilient and adaptable network. Our power is weak and our time is short, but our destiny is in the stars.


Book Review | The Whale | Philip Hoare

The Whale: In Search of the Giants of the Sea by Philip Hoare

A lot like Helen MacDonald’s H is for Hawk. An ode to an animal, to the wild side of ourselves, to our own voyages of discovery from sheltered indoor existences to the rambling untamed spaces that exist outside us, between us, within us. Hawk pays extensive homage to TH White’s The Once and Future King, and seeks communion with White, a loner on a similar journey seeking salvation in an uncompromising beast, in a wild eye that returns a gaze fiercer than we are accustomed to seeing. Hoare does the same with Melville’s Moby Dick. Unfortunately his gloss on Melville only made me pine for the real thing. No convict should ever be punished with the task of trying to out-describe, out-reflect, out-philosophize old Herman, and it is a fool’s errand for a free man.

But the bits of The Whale that are not just shadows of Melville are quite good. Excellent illustrations and photographs liven up personal memories, a sprightly history of 19th century naturalism, and fascinating snapshots of civic and economic history as they tangle with the history of whaling. In the 20th century the US and UK virtually gave up on whaling: the last whaling ship departed New Bedford in 1924. But Japan, Norway and Russia continued, spurred on by economic interests and national pride. Hoare cites a 1997 interview with a Japanese expert in the New York Times:  “It’s not because Japanese want to eat whale meat. It’s because they don’t like being told not to eat it by foreigners.”

Like MacDonald, like Rebecca Solnit, like Melville himself, Hoare understands that the world out there is only comprehensible with respect to and in contrast with the world within, that our idiosyncratic human interiority plays out in large and sometimes unpredictable ways on a global scale. The natural world acquires its beauty as it enters our retina or impinges on our fingertips or sprays its blowhole on our faces. But it becomes sublime as we recognize that whales are not just humans in thick blubber or symbols for sermons: rather we are cousins, together making up a small part of the unity of all living things.

You Must Be Present To Win

“Radical politics is bodies in places.”

I’ve thought about this line a lot this week. It’s from a Rebecca Solnit essay that I cannot find anywhere online, but there’s a version of it here:

It’s focused mainly on the Bay Area and the ways that public space and dissent are threatened by Silicon Valley, but it has a lot of relevance for all of us this week.

Take a few minutes and read it. A good excerpt:

We think of democracy as an ethereal thing, an abstract one, an intellectual one, but bodies coexisting in space is the direct experience of democracy where civil society finds itself and its power. That power can be taken away by direct means — that’s what totalitarian regimes that ban gatherings and crack down on dissent do — but also by indirect ones, such as the design of auto-based, private-space suburbias, or the technologically driven retreat from the public sphere and from embodiment.

What Exactly Are You Protesting

Every day for the last five days, Americans in every major city have organized and sustained protest marches against Donald Trump’s fascism and bigotry. I have spent most of that time trying to muster the courage and energy to join these marches, trying to overcome the gloom and devastation and hopelessness that I felt.

On Friday I started calling my friends across the country to talk about the election results, and it helped. It was good to know that I am not alone in these feelings, to clarify what I feel and why, and to get angry when I hear my friends trying to normalize or diminish what happened.

On Saturday I finally went to march. Shouting and marching in public felt good, felt like the only right thing to do or at least the first step. But the march was so, so small. In a city of 750,000 only 150 people came to protest and almost everyone was under 30. So I texted a bunch of my friends inviting them to join me at the next one. A few are coming, a few said no thanks, and some didn’t answer. And I got one really interesting response:

What exactly are you protesting? Genuinely curious, not being an asshole.

It’s a great question, and I think the answer deserves more length than I was able to give it over text message.

There are two reasons that I think motivated people to show up and protest all week

First, there’s a policy-level protest against essentially DJT’s whole platform. A small sampling from the ACLU’s website, he proposed to:

amass a deportation force to remove 11 million undocumented immigrants; ban the entry of Muslims into our country and aggressively surveil them; punish women for accessing abortion; reauthorize waterboarding and other forms of torture; and change our nation’s libel laws and restrict freedom of expression.

Each of these alone is worth protesting and taken all together they represent a catastrophic direction for our country to take.

Second, this protesting expresses a frustration with the electoral system overall in this country, from the gerrymandering of electoral districts to the systematic stripping away of voting rights from minority groups. Ten percent of Florida’s adults (mostly African American men) can’t vote. This is not an accident, or a tough-but-fair situation. Republican lawmakers across the country have spent decades disenfranchising minorities thereby reshaping our entire political landscape. Now, with the hard-right in control of almost every lever of government, this will only get worse, only make it harder for us to scramble out of this ditch. (Some people today also started chants to remove the electoral college. I don’t agree with that opinion, but movements are broad and you have to accept that you don’t get line-item veto on other people’s chants.)

Even beyond these two, protesting has powerful ramifications.

Protesting normalizes dissent and progressivism. Having an opinion silently is the same as not having it at all. But protesting loudly changes the range of opinions that are visible and seen as popular. Every fight that we will have in this country in the coming years will be easier to win if there is a progressive movement at the forefront of the national consciousness.

This is exactly the mirror image of the vigilante fascism that has sprouted this week. When people see racist bullshit all over their TV, they feel empowered to go yell at a woman in a hijab at Safeway. But when people see your protest (or you tell them about it!), you embolden them, you give them faith that they are not alone, and you inspire them to do their own activism work. This builds on itself, creating networks of activists that sustain and support each other.

After all, this week’s protests would not have been possible without the social movements of the last decade from Occupy to Black Lives Matter. The fight did not begin this week. If you weren’t fighting before — and I was not! — you are joining a fight that has been going on for years. We must learn from our brothers and sisters who have bravely been fighting without us, and build structures and develop tactics for the next people who will join the fight. The coming years will require deep networks and organization, expertise in dissent and the muscle memory to fight back. Let’s build those now before it’s too late.


(Yes indeed, today we’re talking race.)

Yesterday I saw someone online describe herself as “half Canadian”, which has to be the lamest self-proclaimed heritage I’ve ever seen. We all know White People who self-identify as “one quarter Irish” or “one third German” or whatever.

Hello, folks: you are WHITE.

If you are of European ancestry, and most of your family has been in the US more than 2 generations you are white.  Go celebrate Casimir Pulaski Day and wear green in March all you want, you are white. End of story.

I got an amazing response that made me think about this more deeply:

Johann makes a great point: by identifying themselves with a specific national heritage, white people free themselves from the implications of Whiteness — free themselves from having to think about race. Because when white people are just French/Irish/German/Swedish/whatever, race doesn’t matter and there’s no point in talking about it. “We all have our identities, we’re all something-Americans, and everything is great.” But if you are white you have to grapple with what that *means*, what kind of treatment you get in the world, what your individual culpability is.

A similar point is made at greater length in the paper that Johann linked to. There’s a bundle of worldview that white people have which allows them to exist in a de-racialized world — and they/we react negatively to attempts to re-racialize the way they see things (the paper calls this “white fragility”). Because whiteness is unmarked in our society, white people walk around with a view that “I’m just a person, other people have races.” Or people appeal to universalism — “we’re all people” — or individualism — “we’re all our own individuals” — which is closest to the heritage approach.

Of course, turning an eye inwards, I see myself doing the same thing. I think of myself primarily as Jewish, not as white. My dad is an immigrant, my grandparents were survivors of genocide and then displaced persons and then immigrants twice over. But of course I **read** as white in the world, which for many intents and purposes is what whiteness is all about.

:siren: Inner conflict alert :siren:


Anyway, what’s the right name for white people being a little too proud of their ethnic/national heritage?

Seven Year Exercise Windows

Most startup option grants come with a 90-day exercise window when an employee leaves the company (voluntarily or not). This is standard practice. Essentially the problem is that this can force employees to take a major personal financial hit to exercise their options when they leave or are fired, and perversely this is worse the more the company has grown during their time there. (Exercising options comes with a tax hit proportional to the amount the stock has increased in value since the option was granted.)

There’s an easy solution: when an employee leaves, convert their ISOs to NSOs and extend the exercise window.

Today my employer TrueAccord extended our exercise window to seven years for people with two years or more at the company. I’m proud of what this means — this is a progressive move and it shows a dedication to the employees of the company. In general I’m skeptical about equity as a carrot for startup employees (see here for an extremely skeptical view), but this is the right way to do it.

For more, see
and point (2) in

AI Clears New Hurdle; beats European Go champion 5-0

Today Google’s DeepMind team announced that they built a Go-playing AI which beat the European Go champion 5 games to 0. This has been a long time coming! Look at timeline of games conquered by computers:

tic tac toe: 1952
checkers: 1994
chess: 1997
go: 2016

Let’s make a simplistic assumption that the game-playing ability of computers varies as Moore’s Law, which means that it doubles every 18 months. Software has improved as well, so this is a lower bound

So beating checkers required a computer 2^63 times as powerful as beating tic-tac-toe. And chess required a computer 2^4.5 times as powerful as checkers, a factor of 22.6. But the Go program was 379625062 (or 2^28.5) times more powerful than Deep Blue.

These differences are shockingly large, and it turns out that checkers and chess are virtually indistinguishable on the spectrum of human cognition. Both are about 70% of the way between tic tac toe and Go — but who knows how far the spectrum goes?