June 19th, 2024
- #2tweet being a rationalist is self-evidently the right way to be, if you're blocking out a lot of things
- being a rationalist is self-evidently the right way to be, if you're blocking out a lot of things
- some misc notes to self
- samesided = bought in?
- partnerture
- esoterritory
- > Hopefully you see the resemblance between this vision of AI and a genie from folklore. The AI is all-powerful and gives you what you ask for, but interprets everything in a super-literal way that you end up regretting.
- vaguely thinking about genies & AI risk — and it seems like the genie here is very much unchecked mind/left hemisphere
- > The Argument From Complex Motivations
AI alarmists believe in something called the Orthogonality Thesis. This says that even very complex beings can have simple motivations, like the paper-clip maximizer. You can have rewarding, intelligent conversations with it about Shakespeare, but it will still turn your body into paper clips, because you are rich in iron.
There's no way to persuade it to step "outside" its value system, any more than I can persuade you that pain feels good.
I don't buy this argument at all. Complex minds are likely to have complex motivations; that may be part of what it even means to be intelligent.- this is a great point tbh. I hadn't noticed the dimensionality of the orthogonality thesis
- like a sentient alien could have vastly different motivations, but they would also be as complex as its intelligence
- the article has some issues though, eg
- > Comic Book Ethics
These religious convictions lead to a comic-book ethics, where a few lone heroes are charged with saving the world through technology and clever thinking. What's at stake is the very fate of the universe.
As a result, we have an industry full of rich dudes who think they are Batman (though interestingly enough, no one wants to be Robin).- actually there are a bunch of Robins. people who read about AI risk and go "omg how can I be of service I'll do anything." maybe fewer than there used to be? or just further from me
- > Data Hunger
As I mentioned earlier, the most effective way we've found to get interesting behavior out of the AIs we actually build is by pouring data into them.
This creates a dynamic that is socially harmful. We're on the point of introducing Orwellian microphones into everybody's house. All that data is going to be centralized and used to train neural networks that will then become better at listening to what we want to do.
But if you think that the road to AI goes down this pathway, you want to maximize the amount of data being collected, and in as raw a form as possible. - this conflates AI doomers and AI accelerators
- although given what happened with OpenAI ¯\_(ツ)_/¯
- maybe that's kinda reasonable tbh
- AI-obsessed people are closer than people who are elsewhere
- convo today
- friend > god as anthropomorphism of reality
- we sometimes try to negotiate with god
- me > it's not only possible but wise to negotiate with god
- in part because what you're negotiating with is your conception of god
- friend > I don't like the conflation of god and my conception of god
- but... I can't escape the fact that whatever god I'm relating with is always my conception of god
- now I'm naked
- other friend > tracing a self-reference loop
- dereferencing a pointer
- a paradox associated with the fact that you have an icon meant to represent something that [can't be readily referred to]
- the icon is self-dissolving
- so you might as well pick a form of the icon that's easiest for you to interface with
- friend > story of chimps buying food with tokens
- but they can't buy themselves out of the environment with tokens
- helplessness
- me > that situation is contrived
- there's a being that outclasses you, that exists outside your reality, and is controlling your situation in a contrived/arbitrary way
- that has been how many people have conceived of god
- but, it seems to me, is incorrect
- huhhh Uncle God instead of Father God
- avuncular vibes instead of paternalistic vibes
- will generally love you and be there for you, but will let you make your own mistakes rather than trying to force you to do it all right
- friend > I was talking about not negotiating with god in the sense of like
- saying "I won't do my taxes if I don't get the green light"
- it makes no sense to threaten to not cooperate with the supercooperation cluster
- I was missing the positive negotiation part
- mating dance stuff
- we have found the god and he is us