@wswright20 "...I doubt the world is ending as many seem to fear..."
I wish that were true. The discussion of the possibility of our extinction does not come from ChatGPT. It comes from Super General Intelligence... something we all thought was off in the distant future until Large Language Models, suddenly jumped us 20 years into the future over night.
There are companies working on Super General Intelligence. That is a geneneral intelligence that is superior to humans... not a specific purpose... like playing chess... but that is "aware" of all things. And then it is used to create the next generation that is better... etc. This thing goes from just as smart as us to much smarter over night... then 100 times then 1000 times... etc.
Now, none of this would be a problem, if this was being developed on a single computer not connected to the world... but it is not. These are being developed connected to the internet and hence the world.
This issue is alignment. Is it’s objectives in line with preserving and not negatively impacting life. Two things to think about.
First is the paper clip factory. Let’s say we put an AI in charge of creating a paperclip factory and maximizing the number of paperclips. Unless carefully constrained it could get out of control and destroy life creating more paper clip factories.
Second, it is easy to attatch programs to do things to existing AI’s. So one person did this and told it to go make money on the internet. So it set up a bunch of sites to sell stuff and users to rave about what they were selling, and bank accounts to collect money. One of the safeguards it could not get out of was those security checks that have you choose which boxes have pictures with parts of... say a motorcycle. So, the AI contacted internet workers (people you can pay to do stuff on the internet" and said they were handicapped and could not do these checks and the people did the checks for them so they could set up accounts. This happened several years ago.
So, think about malicious intent. Some crazy people sending an AI to launch our nuclear missiles or China’s.
SGI... smarter than us poses a real credible threat to our existence.
Actually, go to ChatGPT and ask it to explain the Alignment treat of SGI to humans. I will probably give you a much better more coherent summary.