Author Archives → Limbic

Algorithm wars and ultrafast machine ecologies

In an amazing 2011 TED talk, Kevin Slavin explained how “algorithms shape our world”.

Its well worth a watch to prepare you for the the main part of this post, an analysis of Black Swan events  in financial markets caused by exactly the sort of algorithms Slavin describes.

From “Financial black swans driven by ultrafast machine ecology”:

Society’s drive toward ever faster socio-technical systems, means that there is an urgent need to understand the threat from ‘black swan’ extreme events that might emerge. On 6 May 2010, it took just five minutes for a spontaneous mix of human and machine interactions in the global trading cyberspace to generate an unprecedented system-wide Flash Crash. However, little is known about what lies ahead in the crucial sub-second regime where humans become unable to respond or intervene sufficiently quickly. Here we analyze a set of 18,520 ultrafast black swan events that we have uncovered in stock-price movements between 2006 and 2011. We provide empirical evidence for, and an accompanying theory of, an abrupt system-wide transition from a mixed human-machine phase to a new all-machine phase characterized by frequent black swan events with ultrafast durations (<650ms for crashes, <950ms for spikes). Our theory quantifies the systemic fluctuations in these two distinct phases in terms of the diversity of the system’s internal ecology and the amount of global information being processed. Our finding that the ten most susceptible entities are major international banks, hints at a hidden relationship between these ultrafast ‘fractures’ and the slow ‘breaking’ of the global financial system post-2006. More generally, our work provides tools to help predict and mitigate the systemic risk developing in any complex socio-technical system that attempts to operate at, or beyond, the limits of human response times.

As a great friend of mine, who happens to be a banker who is very senior in the markets, once commented to me by mail, talking about how trading was accelerating into millisecond cascades as described by this paper:

slow the fck down everyone

Brainstorming Doesn’t Really Work

“The first empirical test of Osborn’s brainstorming technique was performed at Yale University, in 1958. Forty-eight male undergraduates were divided into twelve groups and given a series of creative puzzles. The groups were instructed to follow Osborn’s guidelines. As a control sample, the scientists gave the same puzzles to forty-eight students working by themselves. The results were a sobering refutation of Osborn. The solo students came up with roughly twice as many solutions as the brainstorming groups, and a panel of judges deemed their solutions more “feasible” and “effective.” Brainstorming didn’t unleash the potential of the group, but rather made each individual less creative. Although the findings did nothing to hurt brainstorming’s popularity, numerous follow-up studies have come to the same conclusion. Keith Sawyer, a psychologist at Washington University, has summarized the science: “Decades of research have consistently shown that brainstorming groups think of far fewer ideas than the same number of people who work alone and later pool their ideas.”

Brainstorming Doesn’t Really Work : The New Yorker. via Simoleonsense

Collective intelligence and the “genetic” structure of groups

A very interesting piece from MIT on Collective intelligence and the “genetic” structure of groups:

First is the question of whether general cognitive ability — what we think of, when it comes to individuals, as “intelligence” — actually exists for groups. (Spoiler: it does.)

And what they found is telling. “The average intelligence of the people in the group and the maximum intelligence of the people in the group doesn’t predict group intelligence,” Malone said. Which is to say: “Just getting a lot of smart people in a group does not necessarily make a smart group.” Furthermore, the researchers found, group intelligence is also only moderately correlated with qualities you’d think would be pretty crucial when it comes to group dynamics — things like group cohesion, satisfaction, “psychological safety,” and motivation. It’s not just that a happy group or a close-knit group or an enthusiastic group doesn’t necessarily equal a smart group; it’s also that those psychological elements have only some effect on groups’ ability to solve problems together.

So how do you engineer groups that can problem-solve effectively? First of all, seed them with, basically, caring people. Group intelligence is correlated, Malone and his colleagues found, with the average social sensitivity — the openness, and receptiveness, to others — of a group’s constituents. The emotional intelligence of group members, in other words, serves the cognitive intelligence of the group overall. And this means that — wait for it — groups with more women tend to be smarter than groups with more men. (As Malone put it: “More females, more intelligence.”) That’s largely mediated by the researchers’ social sensitivity findings: Women tend to be more socially sensitive than men — per Science ! — which means that, overall, more women = more emotional intelligence = more group intelligence .

But where Professor Malone’s ideas get especially interesting from the Lab’s perspective is in another aspect of his work: the notion that groups have, in their structural elements, a kind of dynamic DNA. Malone and his colleagues — in this case, Robert Laubacher and Chrysanthos Dellarocas — are essentially trying to map the genome of human collectivity , the underlying structure that determines groups’ outcomes. The researchers break the “genes” of groups down to interactions among four basic (and familiar) categories: what, who, why, and how. Or, put another way: what the project is, who’s working to enact it, why they’re working to enact it, and what methods they’re using to enact it.

…Group intelligence, though, Malone’s findings suggest, can be manipulated — and so, if you understand what makes groups smart, you can adjust their factors to make them even smarter. The age-old question in sociology is whether groups are somehow different, and greater, than the sum of their parts. And the answer, based on Malone’s and other findings, seems to be “yes.” The trick now is figuring out why that’s so, and how the mechanics of the collective may be put to productive use. Measuring group intelligence, in other words, is the first step in increasing group intelligence.

Malone and his colleagues have identified 16 “genes” so far, as expressed in groups like Wikipedia contributors, YouTube uploaders, and eBay auctioneers. “We don’t believe this is the end, by any means, but we think it’s a start,” he said — a way to rethink, and perhaps even revolutionize, the design of groups. Organizational design theory in the 20th century, he noted, generally focused on traditional, hierarchical corporations. But as digital tools give way to new kinds of collectives, “it seems to me,” the professor said, that “it’s time to update organizational design theory for these new organizations.”

MIT management professor Tom Malone on collective intelligence and the “genetic” structure of groups » Nieman Journalism Lab.