{"id":5408,"date":"2008-11-29T18:06:15","date_gmt":"2008-11-29T17:06:15","guid":{"rendered":"http:\/\/www.limbicnutrition.com\/blog\/analysis-second-order-effects-and-black-swans\/"},"modified":"2020-09-26T21:55:25","modified_gmt":"2020-09-26T19:55:25","slug":"analysis-second-order-effects-and-black-swans","status":"publish","type":"post","link":"https:\/\/www.limbicnutrition.com\/blog\/analysis-second-order-effects-and-black-swans\/","title":{"rendered":"Analysis, Second Order Effects and Black Swans"},"content":{"rendered":"
\"\"<\/a>
Kevin Kelly has a super interesting section of his upcoming book “The Technium” devoted to what he calls “The Pro-Actionary Principle”:<\/span><\/figcaption><\/figure>\n
\n

The current default algorithm for testing new technologies is the Precautionary Principle. There are several formulas of the Precautionary Principle but all variations of this heuristic hold this in common: a technology must be shown to do no harm before it is embraced<\/strong>. It must be proven to be safe before it is disseminated. If it cannot be proven safe, it should be prohibited, curtailed, modified, junked, or ignored. In other words, the first response to a new idea should be inaction until its safety is established. When an innovation appears, we should pause. The second step is to test it offline, in a model, or in any non-critical, safe, lowest-risk manner.\u00a0 Only after is has been deemed okay should we try to live with it.<\/p>\n

Unfortunately the Precautionary Principle doesn\u2019t work as a reliable safeguard. Because of the inherent uncertainties in any model, laboratory, simulation, or test, the only reliable way to assess a new technology is to let it run in place. It has to be exercised sufficiently that it can begin to express secondary effects. When a technology is first cautiously tested\u00a0 soon after its birth only its primary effects are being examined. But in most cases it is the unintended second-order effects of technologies that are usually the root of most problems. Second order effects often require a certain density, a semi-ubiquity, to reveal themselves. The main concern of the first automobiles was for the occupants — that the gas engines didn\u2019t blow up, or that the brakes don\u2019t fail. But the real threat of autos was to society en masse — the accumulated exposure to their minute pollutants and ability to kill others at high speeds, not to mention the disruptions of suburbs, and long commutes \u2013 all second order effects.<\/p>\n

Second order effects \u2013 the ones that usually overtake society \u2013 are rarely captured by forecasts, lab experiments, or white papers.<\/p>\n

…The absences of second-order effects in small precise experiments, and our collective impulse to adapt technology as we use it, make reliable models of advance technological innovations impossible. An emerging technology must be tested in action, and evaluated in real time. In other words the risks of a particular technology have to be determined by trial and error in real life. We can think of this vetting-by-action algorithm as the Proactionary Principle. <\/strong><\/p>\n

[The Pro-Actionary Principle by Kevin Kelly<\/a>]
<\/strong><\/p>\n<\/blockquote>\n

Nassim Nicholas Taleb, author of The Black Swan<\/a>, agrees:<\/p>\n

\n

Taleb believes in tinkering \u2013 it was to be the title of his next book. Trial and error will save us from ourselves because they capture benign black swans.<\/strong> Look at the three big inventions of our time: lasers, computers and the internet. They were all produced by tinkering and none of them ended up doing what their inventors intended them to do. All were black swans<\/a>. The big hope for the world is that, as we tinker, we have a capacity for choosing the best outcomes.<\/p>\n

\u201cWe have the ability to identify our mistakes eventually better than average; that\u2019s what saves us.<\/strong>\u201d We choose the iPod over the Walkman. Medicine improved exponentially when the tinkering barber surgeons took over from the high theorists. They just went with what worked, irrespective of why it worked. Our sense of the good tinker is not infallible, but it might be just enough to turn away from the apocalypse that now threatens Extremistan.<\/p>\n

[Times Online, 1st June 2008<\/a>]<\/p>\n<\/blockquote>\n

There seems to be some sort of backlash against the deluge of “Analysis”, especially “Risk Analysis”. Right now the signal to noise ration in public discussion – especially about futurity, policy and risks – is now heavily dominated by noise. As Charlie Edwards from Global Dashboard put it recently”<\/p>\n

\n

Do we need to call \u2018time out\u2019 on global risk analysis?\u00a0 The NIC report on global trends 2025 is one of a plethora of recent publications on global risks and security challenges from think tanks, Government departments, the defence community, NGOs, business, academia, and the media. Do we really need any more?<\/p>\n

3 questions spring to mind:<\/p>\n

1. Are we suffocating under the weight of all this analysis?
2. Should we consider having a period of consolidation and reflection?
3. Do we need a transformational shift from analysis to action?<\/p>\n

[The Seduction of Analysis, Global Dashboard, 25th November 2008<\/a>]<\/p>\n<\/blockquote>\n

This is a theme explored by sociologist and skeptic Frank Furedi<\/a> writing in the Times Higher Education:<\/p>\n

\n

As someone devoted to academic research, I feel increasingly embarrassed when I encounter the words “research shows” in a newspaper article. The status of research is not only exploited to prove the obvious, but also to validate the researcher’s political beliefs, lifestyle and prejudice.<\/p>\n

…advocacy research has now acquired an unprecedented significance in Western culture. One important driver of its expansion is the growing significance that people attach to their lifestyles. The very subjects that advocacy research addresses suggest that lifestyle issues such as emotional orientation, parenting styles and the management of relations have become increasingly politicised.<\/p>\n

In a world where lifestyle has unprecedented significance, people seek to endow it with moral worth. So it matters when a study concludes that children of gay parents “do just fine” or that single mothers’ sons can succeed at school, or that marriage protects elderly adults from mental illness.<\/p>\n

Naturally, academics also take their lifestyles very seriously. But it is important that we resist the temptation to discover the moral worth of our lifestyle through our research. And maybe we should take the lead in informing the public that when they see the words “research shows”, they should assume the role of a sceptic.<\/strong><\/p>\n

[The Times Higher Education, 20th November 2008<\/a>]<\/p>\n<\/blockquote>\n

I see some themes developing here: advocacy research, journalism of attachment, flat earth news and cognitive biases all mutating and amplifying in recursive reinforcing feedback loops. It is some sort of incestuous emergence that generates confusion and entropy. These confusions and false choices are paralysing us, all of us, at precisely the time when urgent action is required in multiple domains.<\/p>\n

Kevin Kelly again:<\/p>\n

\n

Technologies must be evaluated in action, by action. We test them in labs, we try them out in prototypes, we use them in pilot programs, we adapt our expectations, we monitor their alterations, we redefine their aims as they are modified, we retest them given actual behavior, we re-direct them to new jobs when we are not happy with their outcomes.<\/p>\n

Of course we should forecast, anticipate and minimize known problems from the start.<\/p>\n

All technologies will generate problems. None are problem free. All have social costs. And all technologies will cause disruptions to other technologies around them and may diminish technological benefits elsewhere. The problems of a new technology have to be weighed, balanced, and minimized but they cannot be fully eliminated.<\/p>\n

Furthermore the costs of inaction (the default response called for by the Precautionary Principle), have to be weighed together with the costs of action. Inaction will also generate problems and unintended effects.\u00a0 In a very fast changing environment the status quo has hidden substantial penalties that might only become visible over time.\u00a0 These costs of inaction need to be added into the equations of evaluation.<\/p>\n<\/blockquote>\n

Kelly then goes on to list the 5 Pro-actions that for the basis of the Pro-Actionary Principle (which in turn is a revision of Max More’s original<\/a>):<\/p>\n

\n

1. Anticipation<\/strong>All tools of anticipation are valid. The more techniques we use the better because different techniques fit different technologies. Scenarios, forecasts and outright science fiction can give partial pictures. Objective scientific measurement of models, simulations, and controlled experiments should carry greater weight, but these too are only partial. The process should try to imagine as many horrors as glories, and if possible to anticipate ubiquity; what happens if everyone has this for free? Anticipation should not a judgment. Rather the purpose of anticipation is to prepare a base for the next four steps. It is a way to rehearse future actions.<\/p>\n

2. Continuous assessment<\/strong><\/p>\n

We have increasing means to quantifiably test everything we use all the time. By means of embedded technology we can turn daily use of technologies into large scale experiments. No matter how much a new technology is tested at first, it should be constantly retest in real time. We also have more precise means of niche-testing, so we can focus on susceptible neighborhoods, subcultures, gene pools, use patterns, etc. Testing should also be continuous, 24\/7 rather than the traditional batch mode. Further, new technology allows citizen-driven concerns to surface into verifiable science by means of self-organized assessments. Testing is active and not passive. Constant vigilance is baked into the system.<\/p>\n

3. Prioritize risks, including natural ones<\/strong><\/p>\n

Risks are real, but endless. Not all risks are equal. They must be weighted and prioritized. Known and proven threats to human and environmental health are given precedence over hypothetical risks.<\/p>\n

Furthermore the risks of inaction and the risks of natural systems must be treated symmetrically. In More\u2019s words: \u201cTreat technological risks on the same basis as natural risks; avoid underweighting natural risks and overweighting human-technological risks.\u201d<\/p>\n

4. Rapid restitution of harm<\/strong><\/p>\n

When things go wrong \u2013 and they always will \u2013 harm should be compensated quickly in proportion to actual damages. Penalizing for hypothetical harm or even potential harm demeans justice and weakens the system, reducing honesty and penalizing those who act in good faith. Mechanisms for actively fixing harms of current technologies indirectly aid future technologies, because it permits errors to be corrected quicker. The expectation that any given\u00a0 technology will create harms of some sort (not unlike bugs) that must be remedied should be part of technology creation.<\/p>\n

5. Redirection rather than prohibition<\/strong><\/p>\n

Prohibition does not work with technology. Absolute prohibition produces absolute outlaws. In a review of past attempts to ban technology, I discovered that most technologies could only be temporarily displaced. Either they moved to somewhere else on the planet, or they moved into a different niche. The contemporary ban on nuclear weapons has not eliminated them from the planet at all. Bans of genetically modified foods have only displaced these crops to other continents. Bans on hand guns may succeed for citizens but not soldiers or cops. From technology\u2019s point of view, bans only change their address, not their identity. In fact what we want to do with technologies that produce more harm than good is not to ban them but to find them new jobs. We want to move DDT from an insecticide aerial-sprayed on crops to a household malaria remedy. Society becomes a parent for our technological children, constantly hunting for the right mix of beneficial technological friends in which cultivates the best side of each new invention. Often times the first job we assign to a technological is not at all ideal, and we may take many tries, many jobs, before we find a great role for a given technology.<\/p>\n

People sometimes ask what possible role of humans might play in a world of extremely smart autonomous technology? I think the answer is we\u2019ll play parents; redirecting active technologies into healthy jobs, good friends, and instilling positive values.<\/p>\n

If so, we should be looking for highly evolved tools that assist our pro-actions. On our list should be better tools for anticipation, better tools for ceaseless monitoring and testing, better tools for determining and ranking risks, better tools for remediation of harm done, and better tools and techniques for redirecting technologies as they grow.<\/p>\n<\/blockquote>\n

\n

[The Pro-Actionary Principle by Kevin Kelly<\/a>]
<\/strong><\/p>\n<\/blockquote>\n","protected":false},"excerpt":{"rendered":"

The current default algorithm for testing new technologies is the Precautionary Principle. There are several formulas of the Precautionary Principle but all variations of this heuristic hold this in common: a technology must be shown to do no harm before it is embraced. It must be proven to be safe before it is disseminated. If … Read more<\/a><\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[27,32,33,34,5,14,15,42,17,19],"tags":[200,300,296],"class_list":["post-5408","post","type-post","status-publish","format-standard","hentry","category-anthropology","category-business","category-computing","category-contemporary-culture","category-humanities","category-philosophy","category-politics","category-psychology","category-science","category-sociology-social-sciences","tag-img","tag-processed","tag-published"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/prY0k-1pe","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":32812,"url":"https:\/\/www.limbicnutrition.com\/blog\/the-shirky-principle-2\/","url_meta":{"origin":5408,"position":0},"title":"The Shirky Principle","author":"Limbic","date":"August 11, 2018","format":false,"excerpt":"\u00a0\"Institutions will try to preserve the problem to which they are the solution\" - Clay Shirky This is also known as Regulatory Capture\" \u201cWhen a regulatory agency, created to act in the public interest, instead advances the commercial or political concerns of special interest groups that dominate the industry or\u2026","rel":"","context":"In "Contemporary Culture"","block_context":{"text":"Contemporary Culture","link":"https:\/\/www.limbicnutrition.com\/blog\/category\/humanities\/contemporary-culture\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":25605,"url":"https:\/\/www.limbicnutrition.com\/blog\/amazons-leadership-principles\/","url_meta":{"origin":5408,"position":1},"title":"Amazon’s Leadership Principles","author":"Limbic","date":"April 16, 2017","format":false,"excerpt":"Jeff Bezos recent shareholder newsletter has received much praise in the tech press. Inc drew special attention to his principle of \"Disagree and commit.\" When I read the article, this principle felt familiar to me, then I remembered where I had seen it before. It is the 13th Amazon Leadership\u2026","rel":"","context":"In "Business Psychology"","block_context":{"text":"Business Psychology","link":"https:\/\/www.limbicnutrition.com\/blog\/category\/business\/business-psychology\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":26985,"url":"https:\/\/www.limbicnutrition.com\/blog\/revealed-preferences\/","url_meta":{"origin":5408,"position":2},"title":"Revealed Preferences","author":"Limbic","date":"June 24, 2017","format":false,"excerpt":"Revealed Preferences is a term from economics that neatly encapsulates \"watch what I do not what I say\".\u00a0 I see this as a good guiding principle in life. Ignore your own stated preferences, goals, pleasures and interest. Analyze your actual behavior to determine your own Revealed Preferences.\u00a0 It also has\u2026","rel":"","context":"In "Miscellaneous"","block_context":{"text":"Miscellaneous","link":"https:\/\/www.limbicnutrition.com\/blog\/category\/miscellaneous\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":17533,"url":"https:\/\/www.limbicnutrition.com\/blog\/the-four-elements-of-leadership\/","url_meta":{"origin":5408,"position":3},"title":"The Four Elements of Leadership","author":"Limbic","date":"March 26, 2016","format":false,"excerpt":"Michael Lombardi, former General Manager of the Cleveland Browns and current member of the coaching staff on the New England Patriots enumerated four elements of leadership: Management of attention, aka plan. Systems and processes are offshoots of management of attention. Management of meaning, aka communication - explain the plan. Management\u2026","rel":"","context":"In "Business Psychology"","block_context":{"text":"Business Psychology","link":"https:\/\/www.limbicnutrition.com\/blog\/category\/business\/business-psychology\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":32856,"url":"https:\/\/www.limbicnutrition.com\/blog\/the-russel-conjugation\/","url_meta":{"origin":5408,"position":4},"title":"The Russel Conjugation","author":"Limbic","date":"November 24, 2018","format":false,"excerpt":"Eric R. Weinstein on The Russel Conjugation: \u201cThe basic principle of Russell Conjugation is that the human mind is constantly looking ahead well beyond what is true or false to ask \u201cWhat is the social consequence of accepting the facts as they are?\u201d\u00a0 While this line of thinking is obviously\u2026","rel":"","context":"In "Anthropology"","block_context":{"text":"Anthropology","link":"https:\/\/www.limbicnutrition.com\/blog\/category\/science\/anthropology\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":8728,"url":"https:\/\/www.limbicnutrition.com\/blog\/four-degrees-of-separation-that-help-simplify-work\/","url_meta":{"origin":5408,"position":5},"title":"Four degrees of separation that help simplify work","author":"Limbic","date":"September 15, 2011","format":false,"excerpt":"From:\u00a0http:\/\/changingminds.org\/blog\/0902blog\/090225blog.htm Much work these days is packaged up as projects, with plans, resources and time-bound deliverables. Managing projects is a skill as the various risks and issues can easily trip you up. In particular the sheer complexity can cause much extra work and conceal important issues. Here, then, are four\u2026","rel":"","context":"In "Consultancy"","block_context":{"text":"Consultancy","link":"https:\/\/www.limbicnutrition.com\/blog\/category\/business\/consultancy\/"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/www.limbicnutrition.com\/blog\/wp-json\/wp\/v2\/posts\/5408","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.limbicnutrition.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.limbicnutrition.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.limbicnutrition.com\/blog\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.limbicnutrition.com\/blog\/wp-json\/wp\/v2\/comments?post=5408"}],"version-history":[{"count":0,"href":"https:\/\/www.limbicnutrition.com\/blog\/wp-json\/wp\/v2\/posts\/5408\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.limbicnutrition.com\/blog\/wp-json\/wp\/v2\/media?parent=5408"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.limbicnutrition.com\/blog\/wp-json\/wp\/v2\/categories?post=5408"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.limbicnutrition.com\/blog\/wp-json\/wp\/v2\/tags?post=5408"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}