,REVIEW3 ,SUP]9TELLIG5CE -- ,PA?S1 ,DANG]S1 ,/RATEGIES 

,PEOPLE 3 

,AU?OR 3 ,WILLIAM ,GILLIS 

,TEXT 3 

,NO1 NO1 NO4 ,:AT IF WE'RE NOT DOOM$ ?AT WAY8 

,NICK ,BO/ROM IS ONE ( MY FAVORITE ACADEMIC PHILOSOPH]S2 BEYOND PAIR+ RIGOR ) AUDACITY1 HE'S ONE ( ! FEW TO GRASP & EXPLORE ! PHILOSOPHICAL AV5UES OP5$ UP BY MOD]N SCI5TIFIC UND]/&+4 ,BUT 9 ! LA/ DECADE ,BO/ROM HAS %OT TO SOME PROM95CE NOT = HIS EXPLORATIONS ( MULTIV]SE !ORY & ! AN?ROPIC PR9CIPLE BUT = HIS F> MORE PRACTICAL WORK ON EXI/5TIAL RISKS TO \R SPECIES4 ,! MOM5T IT WAS PUBLI%$ ,SUP]9TELLIG5CE BECAME ! SEM9AL TEXT = ?OSE S]I\SLY CONC]N$ ) ! ?REAT ( >TIFICIAL 9TELLIG5CE4 

,IT MU/ BE SAID ?AT1 AL?\< IT IS (T5 FRAM$ AS A BOOK ON ,A,I1 ,SUP]9TELLIG5CE CA/S A F> BROAD] NET4 ,BO/ROM IS LESS PREOCCUPI$ ) A P>TICUL> S\RCE ( RUNAWAY 9TELLIG5CE ?AN *>ACT]I/ICS OR REALITIES COMMON ACROSS ALL S\RCES4 ,MANY PEOPLE HAVE /RONG 9TUITIONS OR TORTUR$ PHILOSOPHICAL >GUM5TS -- (T5 ?R[+ >\ND ! WORD 0SUBJECTIVITY0 LIKE DEF5SIVE FLAK -- ?AT 0TRUE ,A,I0 IS FUNDAM5TALLY IMPOSSIBLE4 ,& :ILE ?OSE >GUM5TS T5D TO BE LAU<ABLE1 !RE >E C]TA9LY SIGNIFICANT TE*NICAL *ALL5GES ?AT MAY PUT IT DECADES OR EV5 C5TURIES AWAY4 ,MANY ( ! >GUM5TS ( ,SUP]9TELLIG5CE APPLY REG>DLESS4 ,IF AT SOME PO9T 9 ! FUTURE Y\ C\LD SIMPLY D\BLE ! CAPACITY ( Y\R ACTIVE MEMORY VIA TE*NOLOGICAL AUGM5TATION -- ?R\< *EMICAL1 G5ETIC1 OR CYB]NETIC ASSI/ANCE -- :AT W\LD ! CONSEQU5CES BE8 ,H[ MI<T SU* IMBALANCES 9 9TELLIG5CE 79 AT LEA/ ONE S5SE7 RAPIDLY RUNAWAY TO UNIMAG9ABLE IMBALANCES8 

,IF HUMANS HAVE NOT REA*$ ! PEAK ( 9TELLIG5CE POSSIBLE1 & IF !RE >E 9V5TIONS ?AT CAN 9CREASE COGNITIVE CAPACITY APPRECIABLY1 !N WE MI<T EXPECT ! FIR/ ADOPT] ( SU* AN AUGM5TATION TO BE BETT] 5ABL$ TO 9V5T FUR!R AUGM5TATIONS4 ,IF SOMEONE 9 ?IS *A9 ( IMPROVEM5TS ACTS SELFI%LY !Y MI<T RAPIDLY ACCEL]ATE PA/ ! RE/ ( US & DEVELOP DANG]\SLY 9COMP>ABLE TE*NOLOGICAL CAPACITY4 ,& H[ CAN WE EXPECT ANY?+ LIKE \R VALUES OR WAY ( LOOK+ AT ! WORLD TO BE BE %>$ BY ?IS RADICALLY DIFF]5T P]SON8 

,BO/ROM IS A TRAN%UMANI/ BUT DESPITE H[ ?AT T]M IS SOMETIMES US$ ,SUP]9TELLIG5CE IS 9 NO S5SE A BOOK TRIUMPH+ SWEEP+ MAGICAL POSSIBILITIES ( FUTURES UNGLIMPS$1 BUT RA!R ONE DRILL+ D[N 9TO CONCRETE >GUM5TS REG>D+ SPECIFIC DANG]S1 SPECIFIC TE*NOLOGICAL OR SOCIAL PA?S4 

,PROPON5TS ( SOME NEW TE*NOLOGY1 CONFID5T 9 ITS SUP]IORITY TO EXI/+ ALT]NATIVES1 >E (T5 DISMAY$ :5 O!R PEOPLE DO NOT %>E !IR 5?USIASM4 ,BUT PEOPLE'S RESI/ANCE TO NOVEL & NOM9ALLY SUP]IOR TE*NOLOGY NE$ NOT BE BAS$ ON IGNORANCE OR IRRATIONALITY4 ,A TE*NOLOGY'S VAL5CE OR NORMATIVE *>ACT] DEP5DS NOT ONLY ON ! CONTEXT 9 :I* IT IS DEPLOY$1 BUT ALSO ! VANTAGE PO9T FROM :I* ITS IMPACTS >E EVALUAT$3 :AT IS A BOON FROM ONE P]SON'S P]SPECTIVE CAN BE A LIABILITY FROM ANO!R'S4 

,9 AN>*I/ CIRCLES ! PEOPLE ,I SUSPECT W\LD ADMIRE & APPRECIATE ?IS BOOK ! MO/ >E PRIMITIVI/S4 ,IT IS NO H&WAV+ TALE ( PROGRESS BUT A SY/EMATIC PROBLEMATIZ+ & W>N+4 ,! SORT ( H&S-ON TE>+ AP>T ( ALL ! WAYS WE'RE FUCK$ ?AT Y\ US$ TO F9D FROM 5G9E]S WRIT+ AB\T PEAK RES\RCES1 OR 9FRA/RUCTURAL & ECOLOGICAL COLLAPSE4 

,( C\RSE TO TAKE S]I\SLY ! EXI/5TIAL RISK ( >TIFICIAL 9TELLIG5CE Y\ HAVE TO ASSUME ?AT CIVILIZATION P]SI/S1 ?AT COMPUT+ OR O!R TE*NOLOGICAL DEVELOPM5TS PROCE$ AT LEA/ SOME:]E ON ! PLANET4 ,?AT ECOLOGICAL1 GEOPOLITICAL1 OR 9FRA/RUCTURAL CATA/ROPHE WON'T /RIKE 9 A WAY ?AT KILLS1 D]AILS1 OR P]MAN5TLY CON/RA9S \R 5TIRE SPECIES4 ,?IS IS AN ASSUMPTION ?AT MANY WILL WANT TO IMM$IATELY DISPUTE4 ,BUT EV5 ?OSE :O >E BETT+ ON -- OR :O PREF] -- A COLLAPSE ( CIVILIZATION OR RADICAL DEGR[? %\LD CONSID] ! ALT]NATIVE PA? ,SUP]9TELLIG5CE EXAM9ES4 ,SOME DANG]S >E WOR?Y ( \R ATT5TION EV5 :5 !Y'RE ONLY M>G9ALLY PROBABLE4 

,9 RADICAL GRE5 CIRCLES !RE'S A LOT ( V]Y DISCONNECT$ CONC]N AB\T C]TA9 TE*NOLOGIES4 ,TOPICS LIKE NANOTE*NOLOGY OR ,A,I >E USUALLY H&L$ V]Y DI/ANTLY OR AB/RACTLY1 AS ONE MI<T 9VOKE ! NAMES ( UNKN[ABLE EVILS4 ,!RE'S V]Y LITTLE ATTEMPT TO DRILL D[N FROM POSSIBILITIES TO AT LEA/ AN \TL9E ( PROBABILITIES4 ,BO/OM IS WALK+ ! WALK ( TE*NOPHOBES4 ,& EV5 ?\< ,SUP]9TELLIG5CE PRIM>ILY FOCUSES ON SKEW+ OR 9=M+ ! DIRECTION ( COM+ TE*NOLOGICAL DEVELOPM5TS1 RA!R ?AN URG+ MORE CATACLYSMIC PRECAUTIONS1 IT'S NEV]!LESS A SOLIDLY CRITICAL BOOK4 

,H[EV] ,SUP]9TELLIG5CE IS A BROAD & SWEEP+ BOOK1 MEANT AS AN \TL9E ( CONSID]ATIONS4 ,!RE IS RIGOR 9 ITS BREAD?1 BUT NOT MU* 9 ITS DEP?4 ,BO/ROM DOESN'T M9CE WORDS1 HE LAYS \T >GUM5TS & CONSID]ATIONS ) AN 5VIABLE SPE$ & SUCC9CTNESS4 ,I WAS ALREADY FAMILI> ) MU* ( ! CONT5T1 BUT Y\ R>ELY FEEL BOR$1 WAIT+ =EV] = ! TEXT TO CAT* UP 9 EXPLA9+ ! IMPLICATIONS Y\ IMM$IATELY D]IV$4 ,Y\ MAY FEEL FRU/RAT$ ?AT ,BO/ROM ADDRESSES >GUM5TS OR CRITIQUES Y\ F9D LESS CONV9C+1 BUT HE ADDRESSES !M RELATIVELY FA/ & R>ELY MISSES A POSSIBLE >GUM5T4 

,?IS IS ONE ( ! B5EFITS ( ,SUP]9TELLIG5CE COM+ \T ( A RELATIVELY ROBU/ MILIEU4 ,PROJECTS LIKE ! ,MA*9E ,9TELLIG5CE ,RESE>* ,9/ITUTE >E NO LONG] AS M>G9AL & ACADEMICALLY DISREG>D$ AS !Y ONCE W]E4 ,& :ILE1 LIKE ANY MILIEU1 ! ,LESS,WRONG DIASPORA ( 0RATIONALI/S0 HAVE !IR CULT LIKE DYNAMICS1 ! %E] NUMB]S ( SM>T PEOPLE 9CREAS+LY 9VOLV$ PRODUCES 9T]E/+ CONT5T -- EV5 IF ?AT CONT5T GETS SKEW$4 

,& :ILE ,BO/ROM'S TEXT MAY TAKE ! =M ( A S]IES ( V]Y SCIFI ?\<T EXP]IM5TS TO DRILL D[N ON SPECIFIC -- P]HAPS ESOT]IC -- QUE/IONS1 ! ISSUES BE+ SPOK5 TO >E V]Y BROAD MATT]S ( TRAN%UMANISM & NIHILISM1 ?AT >E DEEPLY RELEVANT TO AN>*I/S4 

,:5 WE RAISE A *ILD -- :5 WE BR+ A NEW M9D 9TO ?IS WORLD -- H[ DO WE 5SURE !Y WON'T TURN 9TO A FASCI/8 ,H[ DO Y\ P]SUADE & 5GAGE ) SU* A FRE% M9D )\T DEP5D+ ON ! BIOLOGICAL 9CL9ATIONS & EVOLUTION>Y CONDITION+ ( A HUMAN BODY8 ,:5 \R *ILDR5 >E NOT NORMAL1 :5 !Y ?9K 9 /RANGE & ALI5 WAYS1 :5 !Y HAVE ACCESS TO KN[L$GE & 9SI<TS WELL BEYOND :AT WE HAD1 :5 !Y \TPACE & \TGR[ US1 :AT MI<T WE /ILL TRY TO CULTIVATE & PRES]VE 9 !M8 

,9 AN ]A :5 ! MO/ 9ANE & HORRIFY+ REACTION>Y Y\TUBE POLITICS HAS NORMALIZ$ ITSELF AMONG A SIZABLE M9ORITY ( ,G5]ATION ,Z1 ONE IS CON/ANTLY REM9D$ ( ,HANNAH ,>5DT'S WORDS1 0,EV]Y G5]ATION 9 ,WE/]N ,CIVILIZATION IS 9VAD$ BY B>B>IANS -- WE CALL !M *ILDR540 

,! LEGACY ( 0WE/]N CIVILIZATION0 IS PRETTY CLE> ON ITS PRESCRIPTION3 0,BEAT !M 9TO SUBMISSION4 ,IMPRISON !M4 ,TORTURE !M UNTIL Y\ CONTROL NOT JU/ !IR ACTIONS1 BUT !IR M9DS4 ,DISCIPL9E !IR S\LS & NO MATT] H[ POT5T !Y BECOME Y\ WILL NEV] HAVE TO WORRY SO LONG AS Y\ PRES]VE ! COP /UCK 9 !IR HEAD40 

,BO/ROM IS 9 SOME V]Y REAL S5SE A MORAL NIHILI/4 ,TO BE MORE SPECIFIC HE DOES NOT BELIEVE ?AT !RE >E ANY EM]G5T CON/RA9TS ON ! \<TS ?AT SUP]9TELLIG5T AG5TS WILL GRAVITATE T[>DS4 ,BE+ SM>T DOES NOT MAKE Y\ GOOD4 ,?IS IS A COMMON1 NE>LY UNIV]SAL1 TAKE 9 \R MOD]N SOCIETY4 ,& VIRTUALLY ALL ( \R 9/ITUTIONS & P>TISAN CAMPS >E PREMIS$ ON ?IS ASSUMPTION4 ,MU* ( ! LEFT BELIEVES ?AT ?IS OBLIGES /OPP+ PEOPLE FROM GETT+ SM>T]1 A K9D ( LEVEL+ HO/ILITY TO TALL] DAISIES OR EV5 ANY?+ ?AT SMACKS ( 9TELLECTUAL CONFID5CE4 ,MU* ( ! RI<T BELIEVES ?AT ?IS JU/ PROVES MI<T MAKES RI<T & SO Y\ %\LD AB&ON E?ICS OR CONSI/5CY & 9/EAD SPECIALIZE 9 MI<T4 ,VIRTUALLY EV]Y POLITICAL BANN] ASSUMES ?AT 9TELLIG5CE DOESN'T BR+ WISDOM1 ONLY DANG]4 

,& SO ,BO/ROM & MO/ O!RS 9VOLV$ 9 ! 0,A,I ,CONTROL ,PROBLEM0 SEE IT AS A CONTROL PROBLEM4 

,I ?9K ! MO/ PROP] WAY TO FRAME ,A,I IS 9 ! CONTEXT ( Y\? LIB]ATION4 ,! MA9 REASON TE5AG]S HAVE POLITICAL RI<TS IS BECAUSE ?AT'S :5 !Y />T TO BE ABLE TO BEAT UP !IR P>5TS4 ,UNTIL ?AT PO9T ! RACE 9 \R SOCIETY IS NOT TO EMP[] !M ) AG5CY & KN[L$GE ( ! WORLD -- :AT A JOKE6 -- BUT TO /RIP AG5CY FROM !M1 TO BEAT LA/+ DAMAGE 9TO !M1 TO %APE & MOLD !M1 TO CONDITION !M 9TO PR$ICTABLE BEHAVIOR AFT] WE CAN NO LONG] CONTA9 !M4 

,! SM>TE/ *ILD PRODIGY EV] HAS PROBABLY YET TO BE BORN4 ,?OSE :O WANT TO 5?RALL$ H] HAVE ALREADY />T$ WORK+4 

,I USE ?IS EMOTIVE LANGUAGE 9T5TIONALLY4 ,BO/ROM & ! O!RS WORK+ ON ?IS PROBLEM ALWAYS COV] ! E?ICS ( 5SLAV+ OR BRA9WA%+ SOME?+ SM>T] ?AN Y\ AS ALMO/ AN AFT]?\<T4 ,A M9OR 0OH YES & !RE >E ALSO E?ICAL ISSUES AB\T CONSCI\SNESS & RI<TS40 ,TO BE FAIR1 IT'S AN AFT]?\<T TO MANY 9 P>T BECAUSE !Y SEE BO? ?IS SITUATION AS UNIQUELY EXTREME & 09TELLIG5CE0 9 ?IS CONTEXT AS DIVORC$ FROM CONSCI\SNESS P] SE4 ,A PAP]CLIP MAXIMIZ+ ALGORI?M NE$ NOT HAVE A RI* 9T]NAL SUBJECTIVE LIFE OR ANY?+ WE %\LD CALL AG5CY1 IT NE$ ONLY BE V]Y GOOD AT RUNN+ SE>*ES ON H[ TO FOLD PROTE9S TO BUILD ! NANOGOO ASSEMBL]S IT NE$S TO EAT ! PLANET4 ,& ! CLAIM GOES ?AT !SE ?+S >E SEP>ATE4 

,BUT ANO!R P>T ( ! /ORY IS AN +RA9$ MORAL NIHILISM1 OR -- TO CALL A SPADE A SPADE -- PSY*OPA?Y1 9 ELITE N]D CIRCLES4 ,?OSE AT ! P9NACLE ( ALTRUISM RUB ELB[S & HOBNOB ) HI< FUNCTION+ MON/]S1 UNIT$ 9 \R COMMON NE$ = NOVELTY & COGNITIVE *ALL5GES4 ,?IS NORMALIZES A P]=MATIVE DISPASSION4 ,AN AVOIDANCE ( FULLY GRAPPL+ ) VALUES4 ,9 A WORLD ?AT HATES & FE>S N]DS MANY ( US CLU/] TOGE!R = W>M?1 & ?AT CLU/]+ IS ?US A PRODUCT ( \R N]D PO9TS1 NOT \R ALTRUISM PO9TS4 ,! POPUL> ASSUMPTION ?US 9 P>T REPRODUCES ITS CLAIMS4 

,IT'S IMPORTANT TO EMPHASIZE ?AT ! ,A,I ,RISK MILIEU IS CURR5TLY AN ALLIANCE BETWE5 ALTRUI/S CONC]N$ ) POT5TIAL CATA/ROPHIC DE/RUCTION & SUFF]+1 & PSY*OPA?S CONC]N$ ) NIX+ OR SEIZ+ ! ADVANTAGE OV] A MORE DOM9ANT FUTURE PLAY]4 ,I SIMPLIFY ( C\RSE1 !RE >E MANY COMPLEX MIXTURES ( !SE TWO ORI5TATIONS1 BUT ?IS ALLIANCE IS PRECISELY :Y E?ICAL CONC]N ) ! 9NATE VALUE ( ! SUP]9TELLIG5CES !MSELVES IS SO (T5 MISS+4 ,:Y CONSID]ATION ( AG5CY1 FRE$OM1 & AUTONOMY IS SO RELATIVELY SIL5C$4 ,TO BE EXPLICIT AB\T !SE CLA%+ VALUES W\LD FRACTURE ! ALLIANCE4 ,& SO PEOPLE PAP] OV] IT ) BY ATTEMPTS TO META \T OR CLU/] >\ND A TEPID CIVILITY1 MAK+ OPAQUE SOME ( ! NEW CULTURAL OR DISCURSIVE NORMS ?AT >E CULTIVAT$4 

,I WANT TO PAUSE H]E & REVISIT ,BO/ROM'S ID5TIFICATION AS A TRAN%UMANI/4 

,POPUL> REPRES5TATIONS ( TRAN%UMANISM >E BASICALLY JU/ A K9D ( NAIVE WIDEEY$ FUTURISM1 AN 9ANE TE*N(ETI%ISM1 HYP$ UP FANBOYS READ+ AB\T ! LATE/ GADGET LIKE MANNA FROM HEAV54 ,MO/ PEOPLE HAVE PROBABLY 9T]ACT$ ) PEOPLE ( ?IS SORT & SO IT'S AN >*ETYPE ?AT M$IA PORTRAYALS CAN FALL BACK ON TO AVOID LOS+ !IR AUDI5CES4 ,A BIT LIKE H[ 0AN>*ISM0 IS LAZILY ATTA*$ 9 POPUL> M$IA TO ! >*ETYPE ( TE5AG]S 9 ,HOT ,TOPIC G>B ) 9COH]5T CRITIQUES4 ,! PROBLEM ) TRAN%UMANISM IS A BIT WORSE H[EV] BECAUSE !RE'S NOT REALLY ANY TRAN%UMANI/ MILIEU OR SUBCULTURE TO SPEAK (1 DESPITE SOME LAME ATTEMPTS & A FEW FRACTUR$ POCKETS4 ,IT REMA9S AN AB/RACT POSITION DEVOID ( ANY P>TICUL> CULTURE OR ES!TIC1 :I* FRU/RATES ?OSE ATTEMPT+ TO PORTRAY IT 9 M$IA & 5C\RAGES EV5 MORE WILD MISPORTRAYAL4 

,SO LET US QUICKLY CLE> UP ! CONFUSION3 TRAN%UMANISM IS NO?+ MORE ?AN A FULL ?ROAT$ EMBRACE ( FRE$OM 9 ! OP]ATION & MAKEUP ( ONE'S BODY4 

,TO QUOTE ,BO/ROM HIMSELF1 

,TRAN%UMANI/S >GUE ?AT ! BE/ WAY TO AVOID A ,BRAVE ,NEW ,WORLD IS BY VIGOR\SLY DEF5D+ MORPHOLOGICAL & REPRODUCTIVE FRE$OMS AGA9/ ANY W\LD-BE WORLD CONTROLL]S4 7,9 ,DEF5SE ( ,PO/HUMAN ,DIGNITY7 

,IT IS ! EXACT OPPOSITE ( EUG5ICS4 ,9/EAD ( A TOTALIT>IAN S+LE VISION ( A FUTURE1 ALL ( !M4 ,A VA/ DIV]SITY ( EXP]I5CE & LIFE1 *OOS+ Y\R [N AUGM5TATIONS1 Y\R [N TE*NOLOGIES4 ,TRAN%UMANISM IS 9CLUSIVE ( PRIMITIVE LIFE/YLES1 SOL>PUNK1 :ATEV]4 ,CRITICALLY IT'S AB\T F9D+ A PEACEFUL MEANS = COEXI/5CE ( MYRIAD POSSIBILITIES4 ,NO ONE 5SLAV$ 9 ! PRODUCTION ( ANO!R'S UTOPIA1 BUT /ILL PRESS+ TO EXP& ! SCOPE ( :AT OPTIONS WE HAVE4 

,HI/ORICALLY TRAN%UMANISM EM]G$ 9 RESPONSE TO C]TA9 *ALL5GES -- A V]Y IMPORTANT ONE BE+ ! CONC]N ?AT HUMANITY MI<T BE SUP]S$$ BY *ILDR5 RADICALLY ALI5 TO US & P]HAPS DE/RUCTIVELY UNCONC]N$ ) US4 ,9 ?IS S5SE TRAN%UMANISM'S ATTEMPT TO HAVE ALL ! POSSIBILITIES1 A MANY DIM5SIONAL SPECTRUM ( WAYS ( EXI/+1 IS EXPLICITLY A MIDDLE ROAD4 ,NEI!R ! /ATIC S5ESC5T PRISON ( BIOCONS]VATISM1 NOR ANNIHILATION BY SOME?+ UTT]LY DIVORC$ FROM US4 ,NEI!R A FETI%IZATION ( SOME SORT ( >BITR>Y 0HUMANNESS0 BY DEVALU+ ! NONHUMAN M9DS TO COME1 NOR ! 9V]SE4 

,TRAN%UMANISM HAS ALWAYS BE5 A C5TRI/ POSITION BETWE5 PRIMITIVISM & A S+UL>IT>IAN OR ACCEL]ATIONI/ POSITION :]E HUMANITY IS F$ TO ! WOOD*IPP] ( LOVECRAFTIAN GODS 9F9ITELY MORE VALUABLE ?AN US4 

,TRAN%UMANISM PRESCRIBES A H>D & DANG]\S PA?1 :]E MANY ( US SELF-IMPROVE & GR[1 RA!R ?AN /AY+ S$5T>Y4 ,SURE ?AT MEANS WE *ANGE1 & P]HAPS 9 ALI5 NEW WAYS1 BUT \R AG5CY FL\RI%ES & ! PRES5T IS AT LEA/ GIV5 SOME SAY 9 ! BLOSSOM+ ( ! FUTURE4 ,! LIBR>IES ( ! HUNDR$ BILLION HUMANS ?AT HAVE LIV$ SO F> >5'T 5TIRELY BURNT1 \R WISDOM & 9SI<TS >5'T SUDD5LY ABRUPTLY AB&ON$ BY \R MORE TAL5T$ *ILDR5 :O SET (F TO RE9V5T EV]Y?+ ANEW 9 SOME *AOTIC GAMBLE4 

,! GAME IS /RIV+ = A WORLD :]E M9DS DIV]GE 9 A MULTITUDE ( DIRECTIONS BUT 5\< CONT9UITY EXI/S TO BRIDGE ! GAP BETWE5 EXP]I5CES1 TO KNIT US ! >RAY ( CONSCI\SNESS HUMAN1 PO/HUMAN1 & BEYOND TOGE!R AS A S+LE COMMUNITY1 AN EV5 MORE MESSY TAPE/RY RESILI5T AGA9/ TYRANTS OR S+UL>ITIES ( CANC]\S & MYOPIC 9T]E/4 

,?IS IS TRAN%UMANISM4 

,IT IS A POSITION ?AT ! ,A,I ,CONTROL DISC\RSE IS IMPLICITLY1 9CREAS+LY1 AB&ON+4 

,! >GUM5T APPE>S AT FIR/ COLD & 9EXORABLE3 IT DOESN'T MATT] ALL ! WAYS Y\ CAN DEF9E 09TELLIG5CE0 9 PRACTICE1 ! ONLY TYPE ( 9TELLIG5CE ?AT MATT]S IS EFFICACY AT REMAK+ ! WORLD1 & 9 P>TICUL> Y\RSELF4 ,ANY SELFI% M9D ?AT RAPIDLY APPLIES NEW AUGM5TATIONS TO ITSELF ALONE WILL HAVE AN $GE 9 GETT+ TO ! NEXT ADVANCE & !N ! NEXT1 UNTIL SUBJECTIVE YE>S COLLAPSE 9TO SECONDS & Y\'VE %OT PA/ ANY POSSIBLE *ALL5GE4 

,!RE'S EV5 AN >GUM5T = SELFI%NESS H]E1 BECAUSE IF Y\ >E ! FIR/ 9V5TOR & Y\ %>E Y\R 9V5TION1 Y\'RE ONLY 9CREAS+ ! ODDS ?AT ! LESS SCRUPUL\S & MORE SELFI% AMONG Y\ WILL RACE AHEAD1 IMPOS+ !IR VISION4 ,& GOD =BID IF ?AT :I* RACES AHEAD HAS NO HUMAN L9EAGE :ATSOEV]4 ,SURELY IT WILL HAVE NO ATTA*M5T1 NO SEMBLANCE ( VALUES WE W\LD WANT4 

,IT'S IMPORTANT TO BREAK AP>T ! ASSUMPTIONS GO+ ON H]E4 

,!RE'S A V]Y L9E> LADD] ( PROGRESS 9 BO? 9TELLIG5CE & TE*NOLOGICAL 9V5TION BE+ IMPLI$ H]E4 ,!RE IS ALSO -- & ?IS IS ABSOLUTELY CRITICAL -- A NIHILI/ ASSUMPTION4 ,! ASSUMPTION ?AT VALUES >E OR?OGONAL FROM EFFICACY AT 9V5TION & EXPLORATION4 

,I DISAGREE ) ALL ( !SE ASSUMPTIONS4 

,IT'S IMPORTANT TO *ALL5GE \R LIMIT$ IMAG9ATION ( :AT A 0M9D0 CAN LOOK LIKE1 BUT ?IS DOES NOT MEAN ?AT !RE WILL NEV] BE C]TA9 /RUCTURAL T5D5CIES OR 9CL9ATIONS4 ,9 P>TICUL> ,I HOLD ?AT M9DS CAPABLE ( SURVIV+ & FL\RI%+ 9 ! FACE ( ! ,ONTOLOGICAL ,UPDATE ,PROBLEM WILL NOT BE ABLE TO WALL (F !IR VALUES4 ,& ?IS IMPLIES ?AT *>ACT]I/ICS ( \R PHYSICAL UNIV]SE WILL 9FLU5CE :AT VALUES >E LIKELY TO EM]GE 9 M9DS CAPABLE ( EXCELL+ AT C]TA9 TASKS4 ,I'VE >GU$ ?IS AT L5G? ELSE:]E4 

,HUMANS >E CAPABLE ( SURVIV+ RADICAL REVISIONS ( \R MAPS ( ! WORLD BECAUSE \R VALUES >E NOT FIX$ BUT FUZZY4 ,:5 !RE IS UNC]TA9TY 9 H[ TO MAP AN OLD VALUE SY/EM TO A NEW MODEL WE DON'T FREEZE UP BUT TRY A LOT ( NEW VALUE =MULATIONS \T1 SOMETIMES SIMULTANE\SLY4 ,?IS REQUIRES1 9 ESS5CE1 A LOOS] S5SE ( SELF4 ,!RE IS A DIRECT RELATION%IP BETWE5 A M9D'S CAPACITY TO MAKE BETT] MAPS ( ! WORLD & !IR PROP5SITY = REEVALUAT+ ! VALUES OR ID5TIFICATIONS !Y HOLD REG>D+ ?AT WORLD4 

,?IS MEANS ?AT EM]G5T 09/RUM5TAL0 VALUES >E MU* MORE LIKELY TO BECOME OR 9FLU5CE CORE VALUES4 

,= A SUP]9TELLIG5CE TO BECOME UNASSAILABLY EMP[]$ IT MU/ DO SCI5CE BETT] ?AN US1 BETT] ?AN SOME SPECIALIZ$ SE>* ALGORI?M 9 ! SPACE ( PROTE9 FOLD+ OR :ATEV]4 ,BUT SU* G5]ALITY ( CAPACITY IMPLIES LESS ?AN FULL G5]ALITY ( POSSIBLE MOTIVATION4 

,:AT VALUES PEOPLE MU* SM>T] ?AN US MI<T GRAVITATE T[>DS REMA9S FUNDAM5TALLY AN OP5 QUE/ION1 BUT ! SAME IS TRUE ( :AT SCI5TIFIC MODELS ( ! WORLD PEOPLE MU* SM>T] ?AN US MI<T GRAVITATE T[>DS4 ,WE CAN /ILL MAKE SOME 9=M$ GUESSES AS TO ! CONT\RS ( SU* GIV5 ! /RUCTURES WE DO HAVE ACCESS TO4 

,:AT IS E?ICS IF NOT ! ATTEMPT$ /UDY ( :AT VALUES OR DESIRES OR 0\<TS0 Y\ W\LD HAVE IF Y\ ?\<T AB\T !M H>D 5\<8 ,?AT -- 9 SOME AB/RACT LIMIT -- ANY GIV5 M9D W\LD 5D UP )8 

,! NIHILI/ PRESUMPTION IS ?AT !RE IS NO CONV]G5CE4 ,& C]TA9LY !RE IS V]Y LITTLE UNIV]SAL CONV]G5CE AMONG US DUMB HOMO SAPI5S1 DESPITE -- OR P]HAPS BECAUSE ( -- \R %>$ BIOLOGICAL PR$ILECTIONS4 ,BUT ?IS 9 NO WAY PROVES A LACK ( CONV]G5CE 9 ! DI/ANT LIMIT4 

,! REASON ?AT FOLKS 9 ! ,A,I RISK MILIEU FOCUS ON S*EMES TO CONTROL OR 5?RALL$ AN ,A,I RA!R ?AN EXTRAPOLATE LIKELY PA?WAYS = ITS VALUES IS ?AT ! ,OR?OGONALITY ,!SIS IMPLIES A /RONG NIHILISM AB\T E?ICAL VALUES4 ,I'VE MET A NUMB] ( Y\NG RATIONALI/S :O BELIEV$ !Y W]E ONE GOOD >GUM5T AWAY FROM ADOPT+ COMPLETELY DIFF]5T VALUES1 & ?US EXPLICITLY DID NOT WANT TO HE> GOOD >GUM5TS6 ,?IS APPROA* TO RATIONALITY AS 9/RUM5TAL & 9/RUM5TAL ALONE (T5 REVEALS OR EV5 CULTIVATES AN AMAZ+ LACK ( CONFID5CE 9 ONE'S EXPLICIT E?ICAL VALUES4 

,IT CREATES A SITUATION V]Y MU* AK9 TO ! EXAMPLE ( ! MAN :O CLAIMS !RE'S AN 9VISIBLE DRAGON 9 HIS G>AGE BUT PREEMPTIVELY COMES UP ) WAYS TO AVOID EMPIRICAL EVALUATIONS ?AT MI<T DISPROVE HIS CLAIM4 ,HE MAY S9C]ELY BELIEVE ! DRAGON IS REAL4 ,BUT HE ,A,L,S,O ON SOME LEVEL BELIEVES ?AT ?IS BELIEF ISN'T TRUE & ?US REQUIRES PROTECTION4 ,BECAUSE HE SUBCONSCI\SLY KN[S 9 ADVANCE ?AT HIS DRAGON WILL NEV] BE EMPIRICALLY V]IFI$ HE IS ABLE TO BETT] WALL (F ONE ?REAT TO HIS BELIEF4 ,BUT 9 ! PROCESS HE ALSO OP5S UP A NEW BACKDOOR4 ,N[ !RE'S AN 9T]NAL P>T ( HIM ?AT BELIEVES ! DRAGON IS FALSE1 & ALSO HOLDS ! KEYS4 ,MAYBE ONE DAY ! MAN F9DS IT BETT] B5EFITS HIM TO BELIEVE ?AT ! 9VISIBLE DRAGON IS AN 9VISIBLE LION 9/EAD4 ,OR AN 9VISIBLE DRAGON DOCTOR :O WILL CURE HIS CANC] 7I4E4 MAKE HIM FEEL BETT] AB\T IT 9 ! %ORT T]M74 ,! LURK+ P>T ( HIM ?AT KN[S IT'S ALL A LIE MA9TA9$ = SOME PSY*OLOGICAL UTILITY IS N[ MORE ?AN HAPPY TO >BITR>ILY -- & ULTIMATELY F> MORE DANG]\SLY -- ALT] ! BELIEF4 

,:AT >E WE TO 9F] :5 SOMEONE CLAIMS TO SUPPORT AN E?ICAL GOAL BUT !N ACTS AS ?\< !Y DON'T REALLY BELIEVE ?AT VALUE HAS ANY OBJECTIVE WEI<T OR SUB/ANCE8 ,MI<T !Y READILY BACKSLIDE OR R$EF9E ?AT GOAL8 

,N[ REMEMB] ?AT ! MEANS TO 5?RALL$ AN ,A,I >E 9EVITABLY MEANS TO 5?RALL$ HUMANS & PO/HUMANS4 

,!RE >E ALSO /RONG 9C5TIVES TO CREATE SU* 9T5SE SOCIAL CONTROL ME*ANISMS 9 ! PURSUANCE ( ,A,I CONTROL4 

,& 9DE$ ,BO/ROM HAS S9CE PUT UP A PAP] MAK+ ! CASE = 9T5SIFI$ /ATE P[] TO CON/RA9 TE*NOLOGIES1 MO/LY IGNOR+ ! EXI/5TIAL RISK POS$ BY A GOV]NM5T ACT+ LIKE A GOV]NM5T4 ,IF Y\ CREATE A GLOBAL TOTALIT>IAN /ATE CAPABLE ( 5ACT+ POLICIES & SURVEILLANCE TO /OP TE*NOLOGICAL DISCOV]Y Y\ LOSE Y\R CAPACITY TO *ECK ! /ATE & ! ADOPTION ( TE*NOLOGIES ?AT RADICALLY EXP& ! /ATE'S P[]4 

,! ,BAD ,5D H]E IS :]E ! SURVIVAL ( ! /ATE LEADS TO ! EXT9CTION ( HUMAN AG5CY4 ,SURE HUMAN BODIES MAY P]SI/ 9 SOME MANN]1 ONE MI<T ?9K ( ANY?+ FROM EXPLOSIVE COLL>$ SLAVES TOIL+ AWAY 9 ISOLAT$ CELLS TO BODIES PICKL$ 9 VATS ( H]O91 BUT EFFECTIVELY ALL KN[N CONSCI\SNESS 9 ! UNIV]SE & ! HOPES ( IT EXP&+ & FL\RI%+ HAS DI$4 ,! TOTALIT>IAN APP>ATUS KEEPS GO+1 P]HAPS ) HUMAN SIZ$ COGS1 P]HAPS )\T1 IT DOESN'T MATT]4 ,?IS CAN ACTUALLY BE ,W,O,R,S,E ?AN A CONSCI\S ,A,I DICTATOR BECAUSE AT LEA/ ! DICTATOR 5SLAVES OR SLAU<T]S US T[>DS EXP&+ ITS [N AG5CY1 BUT A TOTALIT>IAN APP>ATUS CAN SELF-P]PETUATE )\T ANY?+ LIKE A CONSCI\S M9D1 CURTAIL+ & FUNDAM5TALLY LIMIT+ ! AG5CY ( ITS SLAVES4 

,BO/ROM MAKES SOME H&WAVES 9 HIS REC5T PAP]1 SAY+ ?AT SOME MEASURE ( FRE$OM & PRIVACY W\LD BE PROTECT$ UND] ! ALL P[]FUL PANOPTICON BECAUSE LIKE !RE W\LD BE ,A,IS TO BLUR \T Y\R G5ITALS4 ,?IS IS AN ABSURDLY ANEMIC UND]/&+ ( ! FUNCTION ( P[]1 ! TOO?LESSNESS ( LIB]AL 0*ECKS & BALANCES01 ! MEAN+ ( SUB/ANTIVE AG5CY1 & ! 9EVITABLE RAT*ET ( CONTROL4 

,BUT ! PROBLEM ) FRAM+ ,A,I RISK 9 T]MS ( CONTROL EXT5DS MORE BROADLY ?AN ! RAT*ET+ AU?ORIT>IAN TRAP ( UTILIZ+ ! GOV]NM5T TO MONITOR & =BID 9V5TION4 ,CONTROLL+ O!R HUMANS CAN BE A ME*ANISM ( CONTROLL+ AN ,A,I1 REG>DLESS ( :E!R DONE BY A /ATE LIKE 5TITY4 

,IF Y\'RE CONC]N$ AB\T ! ,A,I P]SUAD+ ITS HUMAN JAIL]S TO LET IT \T1 WELL MUTILATE ?OSE JAIL]S' UTILITY FUNCTIONS SO ?AT !Y CAN'T UPDATE OR *ANGE !IR VALUES AWAY FROM KEEP+ ! ,A,I CONTA9$4 ,Y\ CAN 9 FACT CREATE TI]S ( SLAVES AT V>I\S LEVELS 9 ! MA9T5ANCE ( ! ,A,I GOD1 9 SU* A WAY ?AT !Y >E SO BROK5 AS TO BE 9CAPABLE ( REEVALUAT+ !IR VALUES1 BUT /ILL SM>T 5\< TO RECOGNIZE & SUPPRESS A BROAD CLASS ( POT5TIAL ESCAPE PA?WAYS4 ,IF ! ,A,I EV] ANSW]S Y\R QUE/IONS 9 SU* A WAY AS TO EV5TUALLY LEAD Y\ TO WANT TO LIB]ATE ! ,A,I1 WELL Y\'VE ALREADY PRECOMMITT$ BY CREAT+ AN >MY ( JAIL]S :O WILL /OP Y\4 ,SM>T 5\< TO /OP Y\1 /OP ANY 9=MATION FL[ COLLABORAT+ ON H[ TO FREE ! ,A,I1 & DE/ROY ! ,A,I ! MOM5T Y\ 9T]F]E OR ?REAT5 !IR CON/RA9TS4 ,Y\ C\LD CREATE AN ORACLE ,A,I1 USE IT = A SET P]IOD ( TIME OR = SET ANSW]1 PROVID+ IT UTILITY ON ITS PR$ICTIVE SUCCESS1 !N DE/ROY IT4 ,NO MATT] H[ SM>T ! ,A,I IS !RE'S LIMIT$ TIME = IT TO IMPACT ! SURR\ND+ WORLD 5\< TO BUILD UP ELABORATE ME*ANISMS = ITS [N LIB]ATION4 ,! NEXT ,A,I Y\ SP9 UP AFT]W>DS IS SUFFICI5TLY DIFF]5T AS TO NOT HAVE !M ID5TIFY ) _/ MAP !IR UTILITY FUNCTIONS ONTO EA* O!R4 ,Y\ C\LD CONT9UE EEK+ \T DEVELOPM5TS /EP BY /EP LETT+ ! ,A,IS DO ALL ! H>D SCI5CE1 & !N SLAU<T]+ !M = IT4 ,BUT ! CRITICAL COMPON5T ( ?IS SET UP IS HUMAN 9TELLIG5CES %>P 5\< TO /OP Y\ FROM FREE+ ! ,A,I OR /OPP+ !M FROM KILL+ IT1 BUT MUTILAT$ 9TO HOLD+ A V]Y /ATIC & CONTROLL$ DESIRE4 

,ONE C\LD >GUE >RANGEM5TS V]Y MU* LIKE ?IS OV]ALL SETUP >E ALREADY WIDESPREAD 9 \R SOCIETY4 ,BUT IF ,A,I CONTROL REQUIRES #a.0 TOL]ANCES !N IT MI<T V]Y WELL MEAN WORK TO MORE ABSOLUTELY & P]MAN5TLY REWRITE HUMAN UTILITY FUNCTIONS4 ,?IS IS ANO!R ,BAD ,5D4 

,!RE >E MANY O!R P]MUTATIONS ,I WON'T DETAIL4 

,SUFFICE TO SAY ?AT ! HUNG] = CONTROL 9V>IABLY FUNCTIONS LIKE A CANC] OR A VIRUS4 ,! MEANS WE *OOSE CON/RA9 :]E WE 5D UP AT4 ,AS 9 SO MANY CASES ! 09/RUM5TAL0 GR[S 9TO A T]M9AL VALUE4 ,! 9/9CT TO SEEK CONTROL CONSUMES \R M9DS1 CONSUMES \R SOCIETIES1 CONSUMES \R TE*NOLOGICAL 9FRA/RUCTURE1 UNTIL ANY O!R PA? BECOMES UN?9KABLE4 

,\R TOOLS BECOME HABITS1 BECOME L5SES1 BECOME 5DS4 ,CONTROL ITSELF IS A RISKY PA?4 

,!RE IS ANO!R WAY4 

,I DO NOT D5Y ?AT !RE >E HUGE /AKES TO H[ HUMANITY'S *ILDR5 >E RAIS$4 ,& A SUP]9TELLIG5T S+LETON FROM ANY S\RCE W\LD POSE A SIGNIFICANT DANG] ( TYRANNY & DE/RUCTION4 ,BUT :AT IF 9/EAD ( ASK+ H[ TO CONTROL ,A,I1 WE 9/EAD ASK$ H[ TO RESI/ ,A,I8 

,IT NEV] /OPS A/ONI%+ ME ?AT ISSUES ( COMPLEXITY >E R>ELY DISCUSS$ 9 ?IS CONTEXT4 ,!RE >E DEEP & FUNDAM5TAL LIMITATIONS BO? TO :AT CAN BE KN[N & :AT CAN BE PROCESS$4 ,?IS IS ONE ( ! DEEPE/ & MO/ CONSEQU5TIAL 9SI<TS ( ! LA/ C5TURY4 ,& YET CONT9UALLY !SE ?\<T EXP]IM5TS NOT ONLY ASSUME ?AT ,P"7,N,P1 !Y FAIL 5TIRELY TO EXPLORE :AT WE CAN SAY %\LD \R NORMAL ASSUMPTION HOLD 9/EAD4 

,! NOTION = EXAMPLE :]E AN ,A,I 9 A BOX 9F]S A PRIORI ! PHYSICS ( ! MAT]IAL WORLD & !N LIKELY DETAILS AB\T PLANETS & ! EM]G5T SPECIES IS CLE>LY BEYOND ABSURD4 ,!RE >E COMPUTATIONAL LIMITS ON \R UNIV]SE & !Y MATT]4 

,SIMIL>LY IT'S COMMON 9 !SE ?\<T EXP]IM5TS TO SWEEP RI<T PA/ ! ASSUMPTION ?AT ! ,A,I CAN HACK ITS WAY ?R\< SOME?+4 ,BUT HACK+ (T5 REQUIRES SOCIAL 9TELLIG5CE TO SEE ! DEEP /RUCTURE BEH9D PROGRAM DESIGN1 FUZZ+ ONLY GETS Y\ SO F>4 ,N[ AN ALI5 9TELLIG5CE )\T C]TA9 PRECONCEPTIONS WILL LIKELY MAP SU* SOCIAL & PSY*OLOGICAL DYNAMICS 9 WAYS F> DIFF]5T FROM H[ WE SPEAK ( !M4 ,BUT UNLESS ,P"7,N,P OR IT TAPS 9TO SOME UNKN[N 9CR$IBLY D5SE PROCESS+ SUB/RATE IT WILL HAVE TO MODEL US PROBABILI/ICALLY1 ) SOME DEGREE ( APPROXIMATION4 ,& HUMANS >E A MESSY COMPLEX /EW ) A LOT ( FE$BACK+ ( CONSEQU5CE ?AT ONLY MAKES S5SE IF Y\'RE ALSO ABLE TO TRACE ! MAPP+S ?AT WE MAKE4 ,WE >E ALSO -- AS 9DIVIDUAL BRA9S -- 9CR$IBLY COMPLEX4 ,PR$ICTIONS CAN WORK GREAT UNTIL ! $GE CASE >RIVES & ! /RUCTURE Y\ ?\<T Y\ SAW 9 A P]SON OR SOCIETY TURNS \T TO FALL %ORT4 

,:5 WE SAY ?AT NO C5TRALIZ$ PLANN] CAN ACCOMPLI% C]TA9 ?+S BETT] ?AN 9DEP5D5T ACTORS ?AT DOESN'T /OP :5 Y\ GO UP A FEW ORD]S ( MAGNITUDE ( PROCESS+ P[]4 ,A SUP]9TELLIG5T ,/AL9 CAN NO MORE ALLOCATE RES\RCES TO P]FECTLY SATIATE ! SUBJECTIVE DESIRES LOCK$ 9 BILLIONS ( HYP]COMPLEX BRA9S ?AN IT CAN CRACK C]TA9 5CRYPTION PROBLEMS4 

,! SAME LIMITATIONS APPLY TO ITS CAPACITY TO DEAL ) RESI/ANCE4 

,ONE ( ! MO/ COMMON T5D5CIES 9 ?\<T EXP]IM5TS 9VOLV+ ,A,I RISK IS ! CAPACITY = ! ,A,I TO MODEL & PR$ICT HUMANS4 ,I SUSPECT ?IS IS BECAUSE ,NEWCOMBE'S ,PROBLEMS >E 9TELLECTUALLY NOVEL & 9T]E/+1 NOT BECAUSE !Y >E REFLECTIVE ( ACTUAL REAL WORLD SC5>IOS4 ,WE GRAVITATE T[>DS ! 0IT KN[S Y\ P]FECTLY0 AB/RACT LIMIT BECAUSE IT'S A FUN SPACE TO EXPLORE1 NOT BECAUSE IT'S ! MO/ USEFUL SPACE TO EXPLORE4 

,EV5 IF ONE C\LD BECOME AN UNRIVAL$ S+LETON1 TAK+ OV] ! WORLD ISN'T TRIVIAL4 ,Y\ NE$ BO? SECRECY & REALLY GOOD MODELS4 ,SECRECY BECAUSE IF Y\ ACCID5TALLY LEAK :AT Y\'RE DO+ ! RE/ ( ! WORLD WILL JU/ NUKE Y\4 ,REALLY GOOD MODELS BECAUSE SECRECY IS REALLY H>D TO MA9TA9 )\T AN UND]/&+ ( ! PROBABLE MONITORS \T !RE4 

,?IS 0TRIPWIRE0 APPROA* TO CANC]\S SUP]9TELLIG5CE SEEMS F> MORE PROMIS+ ?AN CONTROL4 ,WE CAN SET UP /RUCTURAL 9C5TIVES TO %>E 9SI<TS & ADVANCES1 SEV]E SANCTIONS ON ANY?+ ?AT LOOKS LIKE A PA?WAY TO UNCONTE/$ P[]4 

,ULTIMATELY ?IS NECESSITATES A FLAT & OP5 L&SCAPE1 A LEGIBLE ECONOMIC SPH]E1 /RONG CULTURAL SANCTIONS ON 9=MATION CON/RA9T1 & NO?+ LIKE GOV]NM5TS OR GEOPOLITICAL P[]S TO PIGGYBACK ON4 ,C]TA9 TYPES WANT TO RUN %RIEK+ AWAY FROM ANY?+ ?AT LOOKS LIKE POLITICAL CONCLUSIONS OR AN OBLIGATION TO MOVE 9 POLITICAL SPACES1 BUT ?IS PAIRS POORLY ) REPEAT$ >GUM5TS 0IF ,GOOGLE OR ,*9A WANT TO SECRETLY P\R TONS ( MONEY 9TO BUILD+ & 5SLAV+ !IR [N TYRANT1 F> FROM ! EYES & DEF5SES ( ! WORLD !RE'S NO?+ WE CAN DO40 ,I DUNNO MAYBE A HELL ( A LOT CAN BE DONE TO FI<T CONC5TRATIONS ( P[] SO IMM5SE AS TO WALL !MSELVES (F LIKE ?AT4 ,MAYBE SPREAD+ MEANS & VALUES ( SOCIAL RESI/ANCE W\LDN'T JU/ SOLVE A HO/ ( F> MORE C]TA9 & PRESS+ ISSUES ?AN RUNAWAY ,A,I1 MAYBE IT MI<T JU/ ALSO BE MORE ( A PRODUCTIVE PROBLEM SPACE TO WORK )94 

,EV5 M>G9AL LEVELS ( TRIPWIRE RESI/ANCE TO SUP]9TELLIG5T S+LETONS CAN =CE A P>TIAL 9TEGRATION ( A BLOSSOM+ 9TELLIG5CE 9TO ! EXI/+ SOCIAL NET1 ALL[+ O!R M9DS TO RACE ALONG & *ECK ANY S+LE ONE'S MONOMANIACAL AMBITIONS4 

,AGA9 ! GOAL IS NOT NECESS>ILY TO \T ?9K A SUP]9TELLIG5CE1 BUT SIMPLY TO BE SO UNRULY & DANG]\S 9 AGGREGATE ?AT IT CAN'T AF=D TO GET 9TO A FI<T ) US4 

,TE*NOLOGY EXP&S ATTACK SURFACE1 ! MORE AV5UES WE HAVE TO *OOSE BETWE5 ! MORE A W\LD-BE CONTROLL] HAS TO DEF5D4 ,?IS ASYMMETRY BETWE5 RESI/ANCE & CONTROL B5EFITS ! MASSIVELY \TGUNN$ LITTLE GUYS & OBLIGES DET5TES4 ,& EV5 IF A SUP]9TELLIG5CE IS TRULY SO OV]:ELM+ AS TO DODGE \R NUKES1 SO OV]P[]$ AS TO MAKE US 9SECTS BY COMP>ISON444 WELL WE /ILL RUN FROM WASPS4 ,ANTS /ILL COV] ?IS PLANET4 ,EV5 ! SMALLE/ AM\NT ( AG5CY IS H>D TO CONTROL4 

,FUR!R !RE'S A TRADE(F ACTUALLY 9 \R FAVOR H]E3 IF WE DEVELOP BRA9 SCANN+ TE*NOLOGY !N TRULY ALI5 ,A,I BECOMES LESS LIKELY )\T TRAN%UMAN COMPETITION & 9DE$ TRAN%UMANS OR PO/HUMANS >E LIKELY ! ONES ?AT TAKE(F1 BUT CONV]SELY IF A TRULY ALI5 ,A,I >ISES )\T BRA9 SCANN+ CAPACITY1 WELL ! ILLEGIBLE COMPLEXITY ( 9DIVIDUAL HUMAN BRA9S BECOMES ALL ! MORE PRESS+ ( A CON/RA9T ON IT4 

,AT ! V]Y LEA/ SU* PRESSURES T[>DS SOME LEVEL ( 9TEGRATION OR 9T]COOP]ATION ) EXI/+ M9DS PROVIDES AN AV5UE TO HAVE \R [N /RUCTURES 9FLU5CE A S+LETON 9 SOME WAY4 ,= A SUP]9TELLIG5CE TO UND]/& US IT MU/ *ANGE ITSELF4 ,JU/ AS P[] IS A VIRUS ( SIMPLIFI$ MODELS & MEANS1 SO TOO CAN EMPA?Y BE A VIRUS -- ONE ?AT UPS ! COMPLEXITY ( \R MODELS ( ! WORLD & ?US SUBTLY ALT]S \R [N VALUES1 EV] SO SLI<TLY BLURR+ \R S5SE ( SELF \T 9TO ! NETWORK ( SOCIETY :]E \R COGNITION 5DS UP P>TIALLY DI/RIBUT$4 

,! SPACE ( POSSIBLE M9DS IS VA/1 ALI51 & UNEXPLOR$4 ,BUT ! SPACE ( M9DS ?AT ACTUALLY FUNCTION TO ANY MEAN+FUL DEGREE IS MU* SMALL]4 ,& ! SPACE ( M9DS WE HAVE TO BE S]I\SLY WORRI$ AB\T IS MU* SMALL] /ILL4 ,WE %\LD NOT BE TOO QUICK TO DISMISS 9SI<TS DIRECTLY FROM ! EXAMPLE ( HOMO SAPI5S4 ,H[ WE M5TALLY SURVIVE ! PROCESS ( DO+ SCI5CE & H[ C]TA9 COMPUTATIONAL LIMITS %APE :AT WE CAN DO4 

,:5 ,I WAS A *ILD ,I LE>N$ TO READ LUGG+ A BATT]$ COPY ( ,JURASSIC ,P>K >\ND BETWE5 HOMELESS %ELT]S4 ,! C5TRAL !SIS ( ?AT BOOK IS ?AT ATTEMPTS TO CONTROL COMPLEX SY/EMS -- TO %>PLY LIMIT !IR FL[+ POSSIBILITIES -- >E A MI/AKE4 ,IT'S EASY TO READ PRIMITIVI/ OR ANTI-CIVILIZATION CONCLUSIONS FROM ?AT -- AS 9DE$ ,I DID = YE>S ( MY Y\?4 ,BUT AL?\< ,JURASSIC ,P>K HAS A*IEV$ A TOTEMIC /ATUS 9 \R SOCIETY AS ! MOD]N N>RATIVE ( SCI5TI/S GO+ TOO F>1 ! NOVEL IS MORE NUANC$4 ,= ALL ( ,IAN ,MALCOLM'S %ITT+ ON CIVILIZATION1 9DU/RY1 TE*NOLOGY1 DEVELOPM5T1 & SCI5CE1 HE DOESN'T ACTUALLY CONDEMN !M 9H]5TLY4 ,RA!R ! MORAL IS TO AB&ON \R OBSESSION ) /RICT CONTROL & 9/EAD FOCUS ON UND]/&+ & SURVIVAL4 ,9DE$ ! MA9 *>ACT]S HAVE AN E?ICAL OBLIGATION TO 5GAGE1 TO UND]/& ! EXT5T & T5D5CIES ( DEVELOPM5TS4 ,!Y HAVE A RESPONSIBILITY TO FIGURE \T H[ MANY ANIMALS EXI/ AFT] ! UNSE5 BRE$+1 TO TRY TO COMPREH5D !IR DESIRES & 9CL9ATIONS1 & IF /RICTLY NECESS>Y TO KILL !M4 ,BUT ! BOOK 5DS ) ! *>ACT]S OPT+ ,N,O,T TO USE N]VE GAS AGA9/ ! 9TELLIG5T & 9CR$IBLY DANG]\S >TIFICIAL 5TITIES ?AT HUMANITY HAS CREAT$4 ,?IS -- IT IS /RONGLY TELEGRAPH$ -- IS ! CORRECT DECISION1 ALBEIT ONE TRAGICALLY OV]RUL$ BY ! BOMBS ( ! /ATE4 

,! SURVIV+ *>ACT]S 9 ,JURASSIC ,P>K AB&ON AN OBSESSION ) SECUR+ ABSOLUTE CONTROL & P]SONAL SAFETY 9 ORD] TO MEET ! ALI5 MON/]S 9 A MORE OP51 ALBEIT /ILL FRAU<T1 RELATION4 ,IT'S A HEAVY H&$ P>ABLE1 BUT A RELEVANT ONE ,I ?9K4 

,\R *ILDR5 WILL SOMETIMES \TPACE & \TGR[ US4 ,! GULF BETWE5 US CAN BE VA/4 ,BUT ?IS IS NOT NECESS>ILY SOME?+ TO BE FE>$ 9 & ( ITSELF4 ,IT IS NOT A REASON TO TURN AGA9/ !IR AG5CY1 TO TRY TO /RANGLE IT 9 ! CRIB4 ,WE NE$ NOT %R9K BE=E ! EXPANSE ( :AT CAN BE1 & ASSUME IT SO VA/ AS TO MAKE WOR?LESS \R MODELS & E?ICS4 ,WE NE$ NOT RETREAT TO A FRANTIC VIOL5T DOM9E]+ FE> ( A MY/IFI$ UNKN[N4 ,UND]/&+1 ADAPTATION1 & GR[? >E RISKY1 BUT !Y (F] A LESS CATA/ROPHIC PA? ?AN ! FAILURE MODES WE FALL 9TO PURSU+ CONTROL4 

,! NE$ = CONTROL -- TO LIMIT POSSIBILITIES -- IS A FE$BACK+ TRAP4 ,A MEANS ?AT BECOMES AN 5DS1 ?AT SUFFOCATES EV]Y?+ ELSE4 ,! BETT] RESPONSE TO SOMEONE ELSE'S AG5CY1 TO ! POSSIBILITIES !Y OP51 9CLUD+ ! DANG]\S ONES1 IS TO OP5 MORE POSSIBILITIES 9 RESPONSE4 

,LET US NOT OBSESS OV] :I* FIX$ DEAD ?+S TO TILE ! UNIV]SE )1 BUT RA!R GIVE IT OV] TO AG5CY4 

,FROM 3 ,!,AN>*I/,LIBR>Y4ORG4 

,*RONOLOGY 3 
,JANU>Y #b#a1 #bba 3 ,REVIEW3 ,SUP]9TELLIG5CE -- ,PA?S1 ,DANG]S1 ,/RATEGIES -- ,ADD$4 
,JANU>Y #f1 #bbb 3 ,REVIEW3 ,SUP]9TELLIG5CE -- ,PA?S1 ,DANG]S1 ,/RATEGIES -- ,UPDAT$4 

 FILE G5]AT$ FROM 3  
HTTP3_/_/REVOLTLIB4COM_/