AI Misuse Concerns

Assuming the technology will not be used for nefarious purposes seems incredibly naive

You do know that not every ‘advance’ in technology is always good, right?

Every kind of technology can be abused, doesn't mean progress should stop.

The question isn't "what's wrong with the technology," but rather "what harm can it do in the context of our societal development?"

A harmless example of technology with potentially devastating potential, that it's hard to imagine is not being used to such ends as we speak.

Just because all those other inventions didn't wreck humanity doesn't mean this one won't.

Any technology can be used in ways the inventor had not intended, and possibly would not wish.

Any tech can be harmful, cf. cryptocurrency or nuclear bombs. My comment mostly referred to taking it to such extreme with a particular tech. Some people also tend, while demonizing (the yet to be confirmed to be possible even in theory) AGI and its capabilities, to completely sideline humans deploying it, as if they would be guilt-free.

"so advanced it's a threat to humankind" and "some people might use this in a bad way" are incredibly different.

It seems like you are unfairly singling out one particular advance in technology among many which could be equally responsible for killing or surveillance.Furthermore, since this also opens some very positive perspectives, why not try and work together towards them? Whatever well-intentioned people do, the ill-intentioned would still have researched this and devoted it maybe more exclusively towards evil purposes.