New Media, Old Patterns

I’m reading through Marshall McLuhan’s Understanding Media, and I’m struck by his prescience when it comes to electronic technology. This book was published in 1966 and he predicts what electronic technology would do over the next 50 years: create an organic, interconnected web for collective consciousness.

But old media never really leaves, it has stained our subconscious, and any new media will necessarily have citations of the old media embedded within. You can see this clearly in programming paradigms: object-oriented programming is the manifestation of our industrial conception of processes as fragmented and specialized. We even resort to factory-terminology when the dearth of vocabulary strikes, and it’s common to personify programs as little people doing their share of the work and passing it on to the next person down the line. An assembly line of data.

Of course, no way McLuhan could have predicted that programming paradigms themselves would reflect our nostalgia for a mechanical world. But what he did predict is that computers would allow us to think globally and complexly about the interconnectedness of our problems. And I claim we’re getting closer. You can see it in newer programming paradigms: service-oriented architecture, functional languages, and neural networks. In service-oriented architecture, we see the blueprint of the network of simultaneous dependencies. In functional languages, we say functions take the state of the world and return a brand new world. And in neural networks, we try explicitly to emulate the interweaving neuronal connections of the mind. It becomes harder to describe how data flows, but the results are better and more organic.

In functional programming in particular, parallels to category theory clearly illustrate that that old fragmented set theory has become pasé. Functions used to be defined as subsets of the power set, with clearly identifiable elements that are transmogrified by functional application. In category theory, functions are simply arrows, with the details abstracted behind properties and interrelations.

The trend is clear: we’re moving to a more complete understanding of reality, and this will be reflected in the way we talk to computers. So learn how to get hip with it, or else be stuck writing code that might as well be widget assembly instructions for factory workers.

Architecture and Design

I was setting up an open-source library called Collectd last week, and I was having some serious problems understanding how to get it to work. I figured out the central gap in my understanding: I didn’t know what the library was created for. According to the documentation, Collectd is a “system statistics daemon”, but the way it’s architected is incredibly modular, so in order to give an accurate picture of what the library can do, the documentation meanders around different possible features you can configure and optimize.

But that’s not what I wanted to know. I just needed a quick demonstration of what the library does so that I could compare its function to another library. Because I had to install plugins to get even basic functionality from the library, and the documentation was written in such a way to give equal weight to all the possible plugin options, there was no clear path to functionality.

From an architecture perspective, Collectd seems to be very maintainable, extensible, and flexible. You can pipe statistics from just about any conceivable protocol, host the library anywhere, and output the results to a handful of different formats. I’m sure it rarely crashes and delivers advertised functionality to many different types of consumers. 

But from a design perspective, it’s not clear at first what it actually can do. It’s a swiss army knife that doesn’t come with any tools. It’s not easy to see how it can fit into my code. And the documentation isn’t opinionated enough to pick a preferred configuration.

As a case study, this library is an example of good engineering to the detriment of marketing and design. Even as an engineer, I find it hard to use because of the lack of opinionated design and targeting marketing that explains how the library solves my problem. 

Cognitive Dissonance is an Enemy of Progress, Creativity, and the American Way

Not every noun can be paired with every verb. Or at least that’s what my mind tells me. I come up with names in order to simplify my picture of what can and can’t happen. Cognitive dissonance is what happens when a person observes that the nouns they assign to themselves for whatever reason can’t be matched up with the verbs they actually do. It’s a base reaction to abstractions threatening to collapse and render the world a whole lot more complex than it was 10 minutes ago.

But cognitive dissonance so often stands in the way of people and what they could be. “I’m not an athlete, why would I use barbell training?”. “I’m not a computer programmer, why would I learn how the internet works?”. I’ve heard all kinds of these excuses from myself and others, I’ve noticed this pattern of ego protection is anathema to personal growth.

In fact, the most fascinating people I know would never refer to themselves as artists, engineers, or athletes. They just did what those nouns do. They were defined by verbs, not nouns. I might be inclined to even call it a rejection of object-oriented princples, if I were feeling particularly rhetorically clever.

So I’ve resolved to think of myself differently. Not as an object with strict behavior, whose action requires consistency, but rather as an amalgam of what I do every day. This cuts both ways, in that it allows me to do things that seem unlike me, but it also calls into question beliefs I’ve held about myself that are no longer true. I call myself a musician, for instance, but the last time I practiced an instrument was months ago. So I am what I do, and I can do anything. Now I’m one step closer to a writer, look at that!

Can You Hear the Shape of a Bubble?

I was reading this article about Hitler’s final days when it reminded me of common failure patterns I’ve observed in large organizations. Not that Nazi government would’ve been great had it not had these issues, but instead I find it interesting that even an organization as illustrious as the Third Reich had systematic issues analogous to those that might plague, say, Facebook.

Lies Upon Lies! Or Shall We Call It “Upward Management”?

The first thing about the article that struck me is that Hitler is quoted as saying “everyone has lied to me, everyone has deceived me”. This is most likely the classic case of an information bubble: no one wants to be the one that tells Hitler his regime is failing, and so none do, resulting in an acute lack of accurate information in the top tiers of management (I use the term purposefully to highlight the analogy with corporations). I remember reading in high school an article in Foreign Policy magazine about how Saddam Hussein believed he had WMD because of the frightening efficiency with which faulty information propagated through his regime. I don’t know if this is true, but it certainly is believable; I mean, why would he incite the strongest military in the world without a trump card? The thing that amazes me is how this efficiency at propagating rosy noise is present in almost every large organization.

As a mathematician, I like to think of social networks in terms of graph theory, and in particular of signals propagating through those networks in terms of how it propagates through edges in the graph. One explanation for why particularly rosy information travels so well in political organization is that very well connected people, are people through which most information must be passed, because of their privileged position in the network. But this role in the network is precisely the stake they hold in not letting damaging information through – they stand to lose their jobs (or in Hitler’s case, their lives) if they let the wrong information through. Thus, the highly connected nodes in a network (usually the most valuable nodes in a social network) become bottlenecks for information and thus stall the information flow.

Employees? Who Needs ‘Em?!

Any manager must at some time come to the conclusion that it was not mismanagement, but rather the managed, that caused the fire currently consuming them alive. In Hitler’s case, he blames the employees of the state: the German people! “[They] deserve to perish” are his words. Now of course, those privileged nodes I was speaking about before are to blame for their refusal to propagated information fairly upwards. But equal blame must go to the power structure itself for failing to incentivize accurate information instead of just pleasing information. It’s like a body trying to do something complex, like play a piano, with its senses providing increasingly inaccurate information as the piece becomes harder, further estranging the player from a good performance.

Identify the Politicians, Grill Until Truthful (or Golden-Brown)

So how do we solve this problem in a real organization? The knee-jerk reaction of hiring honest people is naive, as even in the most grounded of companies, people rarely reach those privileged social positions without some degree of politics. Well, how about probing the network at various sections at random to be able to hear the signal through the noise? I’ll call this the “Undercover Boss” solution: listen when they think no one is listening.

But this course of action is inefficient. More efficient is identifying those bottlenecks of information (perhaps through some test information you plant yourself at the bottom of the social ladder), and then grilling them one-on-one about the status of the company. This solution is efficient, but relies on the assumption that most of the communication structure is efficient at passing messages, except for a few nodes. I propose that this method is optimal in this case, what do you think?