Author Archives: Louis Potok

Cognition Is Social

Oddly enough, this stylized fact bears fruit on several different levels.

E pluribus unum

Intelligence emerges non-additively from the interaction of many less-intelligent agents. A group can have intelligence greater than any of its individual parts, or even their sum. Of course, the opposite can be true as well.

This phenomenon has many forms and names:

  1. Brian Eno’s idea of scenius: “an ecology of talent…supporting each other, looking at each other’s work, copying from each other, stealing ideas and contributing ideas.” (see takes from Kevin Kelly and Austin Kleon).
  2. Marvin Minsky’s Society of Mind modeled human intelligence as a society of agents competing for resources and power, and collaborating to achieve shared goals.
  3. Hayek’s point that computation is difficult and information scattered, so to solve the social optimization problem, our only hope is to use markets to aggregate many local decisions.
  4. The hive mind of twitter (the cool kids are calling it an “egregore”), the world of memes, of the tyranny of ideas.
  5. Individual minds often work better in a group setting. We respond to rewards, social rewards are powerful, and if you are rewarded for “good” thoughts then you generate more of them.

Counterpoint: “nine women can’t make a baby in one month.”

E unibus pluram

The human intelligence explosion can’t be explained by normal processes of natural selection. What new threats or opportunities emerged in the savannah to create such an intense selection pressure? Answer: other humans.

Runaway evolutionary processes are usually explained by an arms race of some sort. Humans are intensely social creatures, and in our evolutionary environment the most interesting thing is other people. Being able to anticipate what other people will do and incorporate it into your actions is an immense edge. All of our social actions (flattery, threats, flirtation, requests, manipulation) rely on having accurate mental models of other people. So success depends on two traits: the ease with which you can simulate other people, and the difficulty of being simulated. The arms race goes exponential because of the “coincidence” that these two are both grounded in “intelligence”.

You can tap into this with a simple technique: Decide who you would ask for help, and imagine what they’d say.

[epistemic status for this section: shakier than I’d like. I didn’t invent this idea, but I haven’t done the work to firmly ground it in the originating research.]


Find a scene, think in public, work with the garage door up, use your social imagination productively.

Stability and Play

I often think about how heavy my head is. When I let it rest on my hand, or lie down, I feel its weight and think about how much work my neck is doing to keep it upright throughout the day. The neck is, generally, an incredible system which keeps the head stable and allows for fully three-dimensional movement (pitch, roll, yaw) through a fairly wide range.

When I’m stressed or unhappy I usually feel that emotion to be physically located in the back of my neck, and sometimes treating the physical symptoms can alleviate the emotional distress. Which is to say in the normal course of events my neck is working hard.

The other day I was doing some yoga with a youtube video, and I got to a part of the session where I was told to move my head in circles to stretch out my neck. The instructor said something like “remember to keep your core activated”, I tightened up, and felt something completely new.

I became aware of a freedom in my neck that I had never felt before, a looseness that let me stretch farther, with less risk and worry, than I ever had. The muscles relaxed, the low-grade worry about keeping my head from toppling over dissipated, and I could playfully explore all kinds of new movements.

This all came about, of course, because I had given myself more physical stability with the rest of my body. This is a general pattern: there is a certain amount of stability necessary across every system, and stability in one place can ripple out, allowing other parts of the system to relax and play freely. Stability here is not static, but a control loop of sensing the outside world and making microadjustments to maintain whatever equilibrium is important.

Note that this isn’t a “nerve center” controlling the reaction of different components: stability within one component can trickle to the rest without direct control.

There are other kinds of stability we can think about, too.

Corporate research labs are kind of like this: Xerox PARC, Bell Labs, Google X. Only a company that is absolutely printing money, whose core business is rock-solid, can allow for such an unstructured approach to R&D. Here stability is basically cashflow; play is expensive.

Another is the advice given to startups that they shouldn’t try to innovate on corporate structure, because their core mission is already so difficult. The scarce resources here are time and attention for the people involved, but also some sense of risk. If your central idea is already likely to fail, you want as little risk in the rest of the system as possible.

Similarly, young writers often try to live bohemian lives, but are usually advised to build routine in their lives so they can be reckless in their art. There’s been an analogous idea floating around the techno-rationalist worlds that getting married early frees you to do greater career work, and I think the hypothesis is similar. (The econ blogosphere has been talking for a while about the marriage premium but the emphasis there is slightly different.)

Stability Mapping

When you are trying to understand, explore, or improve a system, why not try “stability mapping”? Which parts of the system generate stability, and which use it up or rely on it? Which kinds of stability are in surplus, and which are in short supply? How does stability percolate from one part of the system to the next?

Right now, organizations of every sort are reeling with VUCA (new term for me, see slide 14 here) . This analysis suggests one promising approach: identify the sources of stability that still remain and build around the kinds of stability they give you.

Fear of Criticism

When I was younger I hated to read any comments that teachers made on my work. It was manageable for math and science, where there was usually a right answer, but for any kind of free-form writing it was absolutely terrifying. When I got an essay back I would quickly look at the grade. If it was good enough, I would skim the written notes, scanning for praise, and trying to ignore as much as possible any ideas for improvements or notes on weaknesses. If it was bad, I would shove it into my backpack and never look at it again.

In school these assignments would be structured with submissions for intermediate outlines and drafts. And the purpose of this structure is to teach you, correctly, that even your best first effort can always be improved. But because I hated the process so much I would subvert it. I would usually just ignore any comments on the draft and rewrite it myself, or just re-submit the first draft as the final assignment.

Why was this so hard?

All my self esteem came from seeing myself as “smart”. I saw being smart as something static: you have it to a certain level, and it’s always the same. If I tried hard at something, and it wasn’t perfect, that deviation was because I wasn’t smart enough. Therefore the only meaningful output of doing well was praise, validating that I was smart, and the risk was discovering that I wasn’t as smart as I hoped or believed. This was terrifying, because it was the only self-worth I had, and I couldn’t imagine any other source of it.

I love, unironically, the Mos Def song Fear Not Of Man where he says (paraphrasing):

“You’re valuable, and not because you have a lot of money, or because someone thinks you’re sexy. You’re valuable because you were created by God.”

I don’t think you need to be religious to have a strong sense of self-worth, but the problem here is real and can really mess you up. It’s a bit hard to admit that this line is so important to me, because the underlying sentiment sounds so earnest and dumb. But in general, the most obvious truths sound stupid and are sometimes for that reason inaccessible to smart people.

Some helpful ideas

I’ve come a long way since I was younger, but this is still sometimes a challenge and I haven’t found a silver bullet.

First, a few quick tactics. These help me disengage from feedback, and allow me to see it as improving the work rather than reflecting on me as a human being:

  • Explicitly emphasize, to yourself and others, both before creating something and when asking for review, that done is better than perfect and you’ve prioritized getting something out there over getting it exactly right.
  • Use language like “I’m still tightening it up / playing with the structure / figuring out the right ordering”. Sometimes that’s even true.
  • Separate out the parts you want feedback on. “The language is still rough, I’m more looking for feedback on whether the core ideas make sense.”

One big change has come from seeing other people fail in the same way. In the workplace, I’ve come across a number of coworkers who send around a perfectly formatted, delightfully designed piece of work with fundamental errors in logic that several other people would have caught immediately in a draft or outline. So you can think of incorporating feedback early as a way to avoid wasting your own time.

This also became much easier to me when the feedback came from “the outside world” rather than “other people”. Working on my first startup, it became obvious that all the finely-crafted arguments in the world wouldn’t be as helpful as just running an experiment and looking at the data. Internalizing how useful that process was, and considering personal feedback to be an extension of it, went a long way. However, this situation is still in some ways closer to math homework than writing an essay. If I can put metrics on something, and think of it the feedback as “objective”, it’s easier to receive. Not everything in the world is amenable to this approach.

More broadly, it was motivating to adopt the frame of curiosity about the world rather than authoritative judgment on me and my work. Our language around this is very unhelpful. We use the term “feedback” to mean both “new information about an idea” and “another person’s judgment of whether your personality traits are helpful to your goals”. These two things are orders of magnitude apart in how much maturity they take to process, and usually you are dealing with the easier kind.

Another idea I’m exploring is that when you go to other people for feedback you are building a relationship with them. It’s like a grooming ritual for primates: by going to someone for help, you are building a stronger bond with them. You can then see their feedback as “an expression of care for you”. This can help disengage from the content of the feedback for long enough to read it and internalize it. This requires a relatively high level of trust and safety to work.

Unfortunately, caring a lot about the quality of a piece of work isn’t enough to break through this barrier. It can actually make it harder to receive feedback. The more tied up you are in whether something succeeds the harder it is to deal with critique, which might actually make you less likely to believe it will be successful. Alienation from your work can be helpful, if you can flip it back off when you need to. I suppose the ideal is to be 100% tied up in your work when you’re working, and then switch to an outside view where you and others think together about how it can be improved.

A Taxonomy of Slowness

You’re on a car camping trip with some friends. After a long night of sitting around the fire and looking at the beautiful stars, the morning light filters into your tent and wakes you up slowly. You’re well-rested and your hair smells like a campfire, and it’s time to make breakfast. You and your friends eat and clean up together. Now it’s time for everyone to break camp and head home. You shove your sleeping bag back into its sack, pack up your tent, and look around in the grass to make sure you haven’t forgotten anything.

Then you see that none of your friends are even close to finished. You didn’t particularly hurry, so why were you the first one to finish? You load up your car and decide to walk around the campsites to see what’s happening.

Just not working

Sarah is playing chess on her phone. She wins her game, puts her phone in her pocket, and thinks for a minute. Is she going to start packing up? No, she decides to call her mom to catch up. Whether she’s procrastinating or just enjoying the slow pace of life, it’s going to be a while before she’s ready to leave.

I can’t start yet

Sarah’s boyfriend Dave wanders over to you and grins. “Sarah’s on the phone again… we’re never going to get out of here.”

You look at him, and then over at their tent, and then back at him. “Oh,” he says. “I’m waiting for her to get off the phone. I can’t take down the tent until she packs up her clothes. Might as well wait before I pack anything up myself.”

Does Dave really believe there’s nothing he can do? Has he considered getting started anyway, or asking Sarah to get off the phone, or just packing her things up himself?

Analysis paralysis/yak shaving/tool obsession

You continue on your tour, and see your friend Paul sitting in front of his tent with his laptop out. He’s got his headphones in and he’s typing furiously.

“Hey Paul, what are you up to?” He looks up and takes one headphone out.

“Oh, well, I started to pack up, and then I thought that it would be nice if I could pack up my tent even smaller. You know, I never really learned how to pack up a tent – I just figured it out for myself. There might be much better ways to do it that I never learned about!”

“So you were watching a video on how to pack a tent?”

“Oh, no. I opened a few tabs with those videos. But I realized that I didn’t want to watch one video and then forget how to do it, so I started taking notes. But then I wasn’t sure if I’d be able to find the notes again next time, so I started reading blog posts and watching videos on different note organization methods.”

You leave, quickly, before you have to hear him try to pronounce zettelkasten.

Energetically doing the wrong things

At the next campsite, Jamie is vigorously shaking out her car’s floor mats. You ask her why.

“I went to a productivity retreat once, and the success guru there told me that the first step to succeed in anything is to visualize yourself having already succeeded. When I pictured myself leaving the campsite, my car was clean. So I decided to clean my car before packing up.”

“But you could leave even if your car wasn’t clean, right?”

“Oh, I guess that’s true.”

It seems like Jamie is productive by some definition, but if all she cares about is driving home, she definitely doesn’t need to clean her car right now.

Playing hero

Ashley’s campsite is next, but she’s not there. You pull up a chair and wait for her. That cool morning air, you notice, is turning into a hot afternoon, and you think about traffic beginning to coagulate on the highways home.

Ashley walks up to you and passes right by without saying a word. She goes to the picnic table where you all had breakfast, picks up a trashbag, and turns around back in your direction.

You had completely forgotten, but she’s right. You need to take all your trash to the dumpster at the end of the road before you break camp. You feel bad for not helping out.

“Ashley, why didn’t you ask for help?”

“Oh, I just figured I’d do it myself. You know how slow our friends are, if they have to do anything besides pack up their own things we’ll never get out of here.”

She continues on her way, crunching gravel underfoot. You’re grateful that she’s cleaning up for everyone, but you can’t help wondering what if would take for her to actually rely on someone else to help.

Doing the right things, but slowly

The last campsite you visit is Matt. He’s facing away from you, and is in the middle of pulling the poles out of his tent, so you just watch him. He pulls the tent pole another foot out of the sleeve, and then pauses. He tilts his head to the side, thinking. Then he moves his hand, painstakingly, and pulls the tent pole even further out.

Why is he moving so slowly? Maybe he’s unsure whether he’s doing it right, so he needs to consciously reflect on every step. Maybe he feels at peace in that deliberative energy. Maybe he’s really just not in a hurry. What would make him go faster?

As you walk back to your car you start thinking. You finished first, but was it really a competition? Your sleeping bag is going to be really wrinkled next time, and actually you might have put your clean clothes and dirty laundry in the same section of your backpack… Oh, it would be good to take out some snacks for the drive home, did you put those somewhere organized or just jam them into some other bag?

What you did isn’t perfect, but you’re finished. It’s time to go home.

Two theories of driving

Two thoughts from wandering around Phnom Penh by foot, moto and tuktuk.

Being a good driver

What does it mean when you call someone a good driver? There are really three components to being “good”:

  1. Values: You are making an appropriate tradeoff between speed and safety.
  2. Ability: a package of skills that expand your speed-safety possibility frontier (more safety at the same speed, or more speed for the same safety).
  3. Adaptation: How well can you predict what other drivers will do? How well can they predict what you will do?

We talk about being a good driver as though it’s just #2, but usually in the US I suspect it’s about #1. Or, maybe one of the skills in #2 is “accurately assessing risks” and when someone is a bad driver they may not realize accurately what tradeoffs they’re actually making.

When travelling, people often say things like “the drivers here are terrible.” And there may be geographic variation in #2, but more of it is probably #3. Being a good driver is tied up in your deep knowledge of local driving behavior, and how well you match other drivers’ mental models so they can anticipate what you will do and act accordingly.

This framework applies to many things in life.


As a cyclist in the US I noticed that I was often frustrated with cars but never had a problem with buses. Buses were so big and slow-moving that they were easy to circumvent and avoid. Like in the natural world, things that are too removed from you in scale are just not direct competitors. They occupy different niches.

Here in Phnom Penh things are similar. There are three common vehicle classes: motorcycles/scooters, tuk-tuks, and cars/SUVs. Cars are so big and bulky that it’s easy to get around them on a moto, but tuk-tuks are just the right size to always be in your way whether you’re in a car or on two wheels.

This is similar to the social phenomenon of “the narcissism of small differences”: the closer someone is to you, the more they compete for the same resources, and the more threatening they become.

Civilizational Memory and Resilient Knowledge

[Update: this is a subconscious paraphrase, or at least extension, of Jonathan Blow’s excellent talk Preventing the Collapse of Civilization which I watched a few months ago. Thanks JP for the reminder.]

The US government used to make a substance called Fogbank, a critical component in fusion bombs. Then we stopped making it for a while, and “forgot” how to make it.

Actually, it turns out we never really understood how to make it at all. When we wanted to make more of it, we created a new facility based on the historical records of the first time around. That plan didn’t work. It turns out that what made Fogbank work were certain impurities in the final product, traceable to chemical impurities in the input. The modern inputs were purified better than the historical inputs, so the impurities disappeared, and Fogbank 2 didn’t work until this was all tracked down and figured out. No one knew this the first time around. (See page 20 here, via a comment on MR.)

This is story is both terrifying and comforting. It’s good that we were able to eventually figure out the problem. But fusion bombs are one of the most technically sophisticated artifacts of modern civilization, produced by the wealthiest country in world history. Massive resources have been put into their research, engineering, and the institutions responsible for them. What else could fail similarly? What would it cost to fix? What would the ramifications of temporary failure be?

Most of our modern technosphere relies on extreme specialization in complex engineering disciplines. However, much of this knowledge is implicit, rather than explicit.1 A very short version might say: explicit knowledge is what you can write down, implicit knowledge is what you can’t. “Carbon has 6 protons” is explicit knowledge, “how to evaluate a novel” is implicit. Of course you could write a guide to evaluating a novel, but reading that guide would not lead people to perform the same evaluation that you do. Another example: most recipes are actually a blend of explicit and implicit knowledge. The amount of the ingredients and the order of adding them is usually explicit, but knowing when something is done, or when it has been suitably mixed, or small adjustments based on variable ingredients, are all implicit.

Worse, often these processes are highly context-dependent, and the people performing them don’t know what elements of the context really matter. This is the case for Fogbank – the nuclear physicists didn’t know that the impurity was relevant.

This implicit/explicit divide isn’t just on the level of individuals, but also institutions. Codifying process is virtually impossible, and any system with humans in it is organic and adaptive, so defined processes become obsolete immediately. If an institution (a research lab, a company, a division) dies, that knowledge doesn’t live on in any one individual: it dies as well. Even explicit knowledge is often under-documented in organizations. Most broadly, there is an intelligence in systems, and we often don’t know how to recreate it.

Markets can probably ameliorate some of these concerns. If components are truly critical, there should be strong incentives for firms to maintain these systems of knowledge. And one would hope that for critical components, the market incentives are such that things could be rediscovered quickly. But firms can go out of business for unrelated reasons, and much of our critical infrastructure is highly concentrated or actually quasi-governmental, so markets cannot be the only solution.

I’d like to read more about this – is there a good literature out there? What would the field be called – knowledge resilience?

Some related links and ideas:

  1. I don’t have a good citation for this – I really only know this dichotomy in an informal sense. Some googling suggests that it traces back to the work of Michael Polanyi but please chime in if you know more! []

Two Theories of Japanese Culture

Two favorite theories from Japan and the Shackles of the Past:

  1. Japan never had a true Axial Age moment – the moment where Athenian philosophy, Buddhism, and Judaic monotheism (among others!) all began to separate the material world we live in from the “spiritual world” above. All human culture to that point was animistic – where spirituality and real gods imbued every aspect of the world and to talk of a separate spirit world didn’t make much sense at all. When Buddhism arrived in Japan it never fully co-opted the native Shinto animism. And it is precisely this “spirituality in everything” approach that characterizes the Japanese reverence for small design details, ceremonial acts, etc.
  2. Feudal Japan under the Tokugawa Shogunate (1600-1868) is widely known for the influence of the samurai warrior class and their spartan ethic of bushido. However, this was a peaceful period! In fact, the samurai were essentially useless at all for 250 years and they responded in two ways. First, they focused more than ever on an extreme version of bushido, seeking to one-up each other with asceticism and military technique. Second, though, they became decadent – noted for their patronage of everything from burlesque to prostitution — which developed customs of extravagant costume and theatrical presentation. It is precisely this dual nature that shocks one about Japan today – the ultra-serious business ethic coexisting easily with the hypersexualized otaku videogame culture, but it has a long history.

These are obviously oversimplifications and I barely know anything about Japan, certainly not enough to evaluate these two theories – but I enjoyed thinking about them.

Where are you from?

I’ve been thinking a lot lately about the question “where are you from”. It rankled me for a long time – in San Francisco when I first arrived it felt like a provocation, a way of checking if you were a native or a gentrifier. Then I made a friend who has spent a lot of time in Navajo nation building relationships with folks there. They ask the same question, apparently, because their relationship to the land itself is so much of a part of who they are, that knowing you grew up on a particular mountain or stream tells them about you in ways that go beyond “local culture”. So I started thinking about which parts of your background are important to get to know you? Which fields have you studied, who are your intellectual and ethical ancestors? Is there a “small talk” question that gets at that? “What did you major in” is probably close, for more intellectual folks who are still in or near college, but is still a pretty weak shadow of what I’m trying to get at. 
And this becomes even more relevant when you think about statelessness and movement as political issues. The 21st century doesn’t have a monopoly on this – human history is full of forced and voluntary migrations – but there are particular new spins on it. Cosmopolites (which I just read and is very good) talks about bidoons in the UAE, whose ancestors were nomadic desert people just like everyone else, but missed the citizenship initiatives of when those states were forming, and are now stateless – and the UAE’s efforts to buy them citizenships in Comoros, a poor island in the Indian Ocean. The most marginalized want citizenship as a way to secure their rights, to be legible to government and justice. Then of course the ultra rich are stateless in a different way – trying specifically to escape from government responsibilities like paying taxes, they collect residences and passports in the most convenient nations. Climate refugees may be one of the key stories of the 21st century and this question of “where are you from” may have a completely different meaning for them.

Oh, is that all?

How did the ancient Egyptians build those giant pyramids? Did they have access to some secret technique that we don’t know about? Well, yes and no.

We have some hints about the process, and in 1997 a team of researchers tried to replicate this as best they could. They failed. In How To Build a Pyramid, Jimmy Maher quotes team leader Mark Lehner:

Although we failed to match the best efforts of the ancient builders it was abundantly clear that their expertise was the result not of some mysterious technology or secret sophistication, but generations of practice and experience.

Oh, just generations of practice.

There is a fundamental confusion here [1] What could be more mysterious and secret than a technique that takes hundreds of years of experience to develop, and that afterwards cannot be adequately communicated? Or a level of sophistication which can barely be sketched out before achieving it? To think otherwise is to fetishize knowledge while holding contempt for the process of acquiring it.

The process itself is the amazing thing. Amazing that humans are so good at it by default (it may be the secret to our success) and also that we have developed in the scientific method a version that is in some ways many times more powerful. Can you imagine Galileo seeing a space shuttle and thinking “I don’t know how they did it, but I’m sure that if I did this for several hundred years, I’d be as good as them.” It’s true, of course, but it really misses the point. If this is your mindset, what sort of technology or sophistication would impress you? What method of acquiring that knowledge would have looked like anything other than “generations of practice and experience”.

James Scott’s Seeing Like A State makes the forceful point that this buildup of tradition often creates implicit knowledge (metis) that cannot be gained in other ways (as far as we know) and we dismiss it at our peril.

  1. I don’t know if Lehner himself is confused, or if he’s imagining his audience confused. []


A very thoughtful recent blog post makes the point that institutions that seem “decentralized” or claim that as a value often exhibit centralizing tendencies over time. Some recommendations:

  • Be specific about what things you want to decentralize, and why. Regard decentralization as an ongoing process that can never be complete.
  • Find checks and balances, so that it is harder for any set of actors to achieve centralizing power. Use multiple forms of decentralization and participation.
  • Consider accountability: often what we really care about is accountability, and a centralized but accountable entity (such as a government antitrust enforcer) can limit the centralized and unaccountable power accumulation that we fear.