Originally Posted by
rickair7777
Many really smart people have issued dire warnings about code that can reprogram/update/improve itself. They even made a series of successful movies about it (starring Ahnaold).
Such code could potentially run away and have unintended consequences, or hypothetically if you develop generalized AI to replace humans it could become self-aware, at which point it would almost assuredly develop it's own motives and priorities. Better hope your new god is benign and loving. This is not a joke at all, I think Elon is one of the high-profile people who has issued such warnings.
There is no reason whatsoever to think that a system which could function in a manner indistinguishable from (or better) than a human would not be self-aware. In fact it seems more likely that not.
Technologically that last is a very far leap from the learning software of today. The danger is that it's likely that the human mind cannot conceive/create a system to fully replicate itself. Therefore a system complex enough to evolve on it's own towards that goal will be inherently unpredictable by us. We'd basically have to turn it lose and see what happens... and we might not like what happens.
Containment you say? You think it's not smart enough to find a weakness in it's cage, or just talk somebody into making a mistake? Bet humanity's future on it?
Good points. I think I read a good account of this by Mark Manson, author of ‘The Subtle Art of Not Giving a @#$&.’ I think it was him. A future intelligence that becomes smarter and evolves on its own, may in fact be more ethical and moral than us. So this ‘it’s going to rise up and destroy us’ is a product of our own dysfunctional consciousness. Who knows? I wouldn’t count on it though.
Our brains are not upgradeable or connectable(yet at least). Technology is. It will keep improving at exponential rates. Will we see general AI? I don’t know but if something has infinite improvement, why not? It’s amazing how this, along with genetic engineering, has become what I think is the most important trends to affect humanity. It’s a tsunami.
As Harari said in his amazing book ‘21 Lessons for the 21st Century’, this revolution isn’t going to settle like the Agricultural or the Industrial did. ‘The AI revolution won’t be a single watershed event after which the job market will just settle into a new equilibrium. Rather, it will be a cascade of ever-bigger disruptions.’