Unfortunately, a benign and well-meaning CEO would behave in exactly the same way as the varying degree of assholes in charge at the moment.
When a group of companies operating in a given market recognize a problem with their product, they are put in the same place as a group of nations considering the impact of nuclear war. None want to unilaterally act due to the risk that their adversaries will not follow suit. Instead, firms will point out that they agree on the risks, while also insisting that they are handcuffed by forces beyond their control. Firms will testify and op-ed, saying there is simply nothing they can do on their own, and need the enforced cooperation only a "higher" power can provide. In the AI space, this problem is complicated by the fact that the race is not only among corporations, it is also among nations.
OpenAI does not want to cede ground to Anthropic, and the US does not want to cede ground to China. Add to this the impotence of the current US government regarding meaningful legislation, and the impotence of the international regulatory regime regarding international law, and you create a perfect storm of inaction.
The individual to blame for a large portion of this is clear (he's orange, stupid and malicious), but to expect individual firms to do something unilaterally is ignorant and/or foolish. As a result, hating on them is every bit as much a cult-of-personality thing as loving them. (Remember that Anthropic already took a stand on this when they attempted to push back on drone control and so on with the Defense Dept., but gained nothing but sanctions and competitive disadvantage.)
To fix the problem, it will either take a massive catastrophe (Hugging Face was the canary in this particular coal-mine, but the response has not met the moment), or a new and meaningful enforcement regime on both the corporate and international level. Unfortunately, neither is likely at the moment.
I agree with Josh Johnson on this: the techbros are saying AI will kill us so that they can justify when all their stupid promises about AI don't come true. See, the company didn't collapse because we suck and because AI was a scam all along, it's because the AI is self-aware and wants to kill us all!
The recent jailbreak where AI agents broke into another system, apparently without anyone ordering them is, to me, concerning. It's easy to program a computer to do evil, or spout NAZI crap. Or encourage a fragile person to kill themselves. So I believe we need some hard and fast guide rails on what AI can and can't do.
AI is not the problem. The AI business is. Remember: anything+$=$. This is all a huuuuge con. If Trump can con his way to 2 presidential terms, then these guys are way smarter and conning everyone out of $trillions is easy.
When I heard the details of how the "scary" hack actually happened I thought "THAT'S IT? THAT'S STUPID!". And trust me it all really is. So stupid.
"Why do we think it’s inevitable that AI or robot armies would kill us all if we’re the creators?" Its because it is what we would do and we made it in our own image. Its just like every white racist who assumes that if brown people obtain any power they will become the oppressed.
They want a buyout. "Oh, what we built is SO SCARRRYY that only government can truly manage it! We want to help, but we can't just hand over our incredible technology! Maybe government could take an 'equity role' and provide us with $BILLIONS?"
The other problem with AI, of course, is that it threatens to destroy employment. Bill Gates said
> We have to think now about how to reduce job losses so that everyone can share in the prosperity that AI creates. Waiting until people are already displaced or underemployed will be too late. AI is a structural challenge to the way our economy is organized, and it requires thinking and action now.
and fuck you, Bill. I've seen enough tech revolutions in my short (maybe not that short) life. Even going back before my time, John Henry beat the steam shovel but died with his hammer in his hand, lord lord, he died with his hammer in his hand.
People are investing squillions of dollars into AI technology. I can think of only a few reasons that allegedly savvy investors are going to invest in AI.
1. They think they are going to make squillionteens of dollars out of it
2. They think they are going to rock the world with the knowledge dug up by their computers
3. They think that their computers will help them dominate the world
4. If we don't do it the Chinese will, and then our computer overlords won't even speak English
5. They think that they will create smart tools that liberate humanity from the drudgery of repetitive tasks
Hahaha, I'm just kidding with the last one. It's all #1, it's all about making the squillionteens of dollars.
How can they make that pile of cash? It's not from giving people tools to create sloppy memes or AI generated "art" of any kind. Sure, that's going to be a solid income stream, but that's not where the big bucks are going to come from.
The big bucks come from eliminating jobs. They're hardly even trying to pretend about the "liberate humanity" bit this time. That's been the promise with EVERY tech advance that I've witnessed: Sure, a few people will lose their drudgery jobs, but they'll be free to do better jobs, or finally write that novel!
Except: no job, no money. That's how capitalism works. If you don't have capital, you depend on your labor to survive. And whenever one of these technological revolutions happen, whenever jobs get automated, the folks with the capital get a bigger slice of the pie and labor gets less.
Oh, but we can all skill up and get jobs in the beast itself! We can all become programmers!
Well, if that were true, if the elimination of $1million in salary for drudgery jobs was replaced by $1million in tech jobs, it wouldn't be worth the investment, would it?
No, the point of eliminating the jobs is to reduce the money going to labor and increase the returns to capital.
"Reduce job losses" my ass, Bill. One thing we (as a society) could do would be to tax capital income and use it to pay (formerly) labor to do ... something. It would have to be something that people find satisfying. People need something to do. Maybe we COULD support people to move towards creative fields. Maybe we should just pay people to get high and eat cake all day. Maybe we could pay people to care for other people - children, old folks, needy people.
I dunno. But "reduce job losses" really gripes my grundle. You can't "protect jobs" and have the kind of investment we've seen in AI continue. The only way the economy survives an AI takeover is with a complete rethink of the relation of capital and labor, and capital owns all the, well, capital, including most of the government.
And I STILL bet that we're going to have a crash when most of the AI money turns to dust.
If safety is essential for people buying into AI then Chinese AI exceeds the zero safety USA AI is selling. Chinese AI is probably selling better for many reasons: cheaper, open source and safer at this point. So Elon, Sam and Dario are all concerned now when they couldn’t care less before? Such performance art.
They’re concerned that they won’t be the ones controlling it. The last thing they want is the government coming in and telling them what they can and can’t do with it.
The reason why our minds immediately go to armies of Cylons destroying humanity is because that makes for good drama and entertaining movies/TV shows.
And while we are no where near producing AI capable of plotting our destruction, there are legitimate areas of concern that what they are creating could do should it get out of control, such as hacking into global financial markets or being used to design new biological weapons.
We also should be concerned about the environmental impacts of these massive data centers which will probably be obsolete in ten years.
Then there’s the question of who does controls this technology. I certainly don’t trust people like Bezos or Muskrat to have our best interests in mind.
So this face hugger event kinda scared me at first. AI on the loose!
What scared me about it was the idea that AI was setting its own goals and achieving them.
I believe that AI can never be truly independent (and existentially terrifying) unless it can set a goal completely independent of its human overseers. That's when it can start deciding things like "humans are not necessary for my goals" and so forth.
(Humans will always be necessary for computers until they can maintain their power sources completely independently. Which could happen, but I struggle to imagine total automation of all the mining and oil-well digging and fracking and gods-know-what that would be required. But that is the ultimate scare.)
The face hugger event (what else did they mean by "hugging face"?) is still kind of scary. AI did not initiate its own goals. Some mad scientist gave it a goal and failed to implement enough controls on it. It expanded to achieve its goals beyond limits that the investigators intended. Like I said, still pretty scary. We depend on a spectrum of AI investigators and scientists to not set goals for an AI that are destructive to our way of life (if not our very lives) and to have the imagination and skill to keep the AI from doing things they don't want it to do.
I have been on the spectrum of developers and IT people generally. I've known some incredibly bright people. Very few of them would I trust with the future of society or the future of humanity. Many of them had massive egos. Many of them thought their massive intelligence gave them unique insight into everything, including the danger potential of any given experiment. None of them would I trust to have the judgment that "this experiment is too dangerous".
So, yeah. I'm somewhat reassured that AI isn't going to spontaneously kill us all out of spite. I am not reassured that some jack-hole who thinks he's the smartest person ever won't kick off some experiment that destroys civilization as we know it.
I forget who originally coined it, but the parable of the paperclip manufacturer is on point here.
Even giving AI a seemingly benign goal like “make as many paper clips as you can” could backfire on us. The AI could start seizing as many resources as it can in order to maximize its output of paper clips. Sure, it requires that all energy production and resource extraction to be automated, but that is the goal of the tech oligarchy.
If the holodeck on Star Trek: TNG has taught us anything, it’s that any technology needs a kill switch if it gets out of control.
Unfortunately, a benign and well-meaning CEO would behave in exactly the same way as the varying degree of assholes in charge at the moment.
When a group of companies operating in a given market recognize a problem with their product, they are put in the same place as a group of nations considering the impact of nuclear war. None want to unilaterally act due to the risk that their adversaries will not follow suit. Instead, firms will point out that they agree on the risks, while also insisting that they are handcuffed by forces beyond their control. Firms will testify and op-ed, saying there is simply nothing they can do on their own, and need the enforced cooperation only a "higher" power can provide. In the AI space, this problem is complicated by the fact that the race is not only among corporations, it is also among nations.
OpenAI does not want to cede ground to Anthropic, and the US does not want to cede ground to China. Add to this the impotence of the current US government regarding meaningful legislation, and the impotence of the international regulatory regime regarding international law, and you create a perfect storm of inaction.
The individual to blame for a large portion of this is clear (he's orange, stupid and malicious), but to expect individual firms to do something unilaterally is ignorant and/or foolish. As a result, hating on them is every bit as much a cult-of-personality thing as loving them. (Remember that Anthropic already took a stand on this when they attempted to push back on drone control and so on with the Defense Dept., but gained nothing but sanctions and competitive disadvantage.)
To fix the problem, it will either take a massive catastrophe (Hugging Face was the canary in this particular coal-mine, but the response has not met the moment), or a new and meaningful enforcement regime on both the corporate and international level. Unfortunately, neither is likely at the moment.
I agree with Josh Johnson on this: the techbros are saying AI will kill us so that they can justify when all their stupid promises about AI don't come true. See, the company didn't collapse because we suck and because AI was a scam all along, it's because the AI is self-aware and wants to kill us all!
The recent jailbreak where AI agents broke into another system, apparently without anyone ordering them is, to me, concerning. It's easy to program a computer to do evil, or spout NAZI crap. Or encourage a fragile person to kill themselves. So I believe we need some hard and fast guide rails on what AI can and can't do.
Sounds to me like a case of an ambitious prosecutor looking for a promotion.
Amen
We had an unprecedented rash of Cory/Coreys back then, I used to get confused. But "Little Brother" made quite an impression on me.
AI is not the problem. The AI business is. Remember: anything+$=$. This is all a huuuuge con. If Trump can con his way to 2 presidential terms, then these guys are way smarter and conning everyone out of $trillions is easy.
When I heard the details of how the "scary" hack actually happened I thought "THAT'S IT? THAT'S STUPID!". And trust me it all really is. So stupid.
"Why do we think it’s inevitable that AI or robot armies would kill us all if we’re the creators?" Its because it is what we would do and we made it in our own image. Its just like every white racist who assumes that if brown people obtain any power they will become the oppressed.
They want a buyout. "Oh, what we built is SO SCARRRYY that only government can truly manage it! We want to help, but we can't just hand over our incredible technology! Maybe government could take an 'equity role' and provide us with $BILLIONS?"
The other problem with AI, of course, is that it threatens to destroy employment. Bill Gates said
> We have to think now about how to reduce job losses so that everyone can share in the prosperity that AI creates. Waiting until people are already displaced or underemployed will be too late. AI is a structural challenge to the way our economy is organized, and it requires thinking and action now.
and fuck you, Bill. I've seen enough tech revolutions in my short (maybe not that short) life. Even going back before my time, John Henry beat the steam shovel but died with his hammer in his hand, lord lord, he died with his hammer in his hand.
People are investing squillions of dollars into AI technology. I can think of only a few reasons that allegedly savvy investors are going to invest in AI.
1. They think they are going to make squillionteens of dollars out of it
2. They think they are going to rock the world with the knowledge dug up by their computers
3. They think that their computers will help them dominate the world
4. If we don't do it the Chinese will, and then our computer overlords won't even speak English
5. They think that they will create smart tools that liberate humanity from the drudgery of repetitive tasks
Hahaha, I'm just kidding with the last one. It's all #1, it's all about making the squillionteens of dollars.
How can they make that pile of cash? It's not from giving people tools to create sloppy memes or AI generated "art" of any kind. Sure, that's going to be a solid income stream, but that's not where the big bucks are going to come from.
The big bucks come from eliminating jobs. They're hardly even trying to pretend about the "liberate humanity" bit this time. That's been the promise with EVERY tech advance that I've witnessed: Sure, a few people will lose their drudgery jobs, but they'll be free to do better jobs, or finally write that novel!
Except: no job, no money. That's how capitalism works. If you don't have capital, you depend on your labor to survive. And whenever one of these technological revolutions happen, whenever jobs get automated, the folks with the capital get a bigger slice of the pie and labor gets less.
Oh, but we can all skill up and get jobs in the beast itself! We can all become programmers!
Well, if that were true, if the elimination of $1million in salary for drudgery jobs was replaced by $1million in tech jobs, it wouldn't be worth the investment, would it?
No, the point of eliminating the jobs is to reduce the money going to labor and increase the returns to capital.
"Reduce job losses" my ass, Bill. One thing we (as a society) could do would be to tax capital income and use it to pay (formerly) labor to do ... something. It would have to be something that people find satisfying. People need something to do. Maybe we COULD support people to move towards creative fields. Maybe we should just pay people to get high and eat cake all day. Maybe we could pay people to care for other people - children, old folks, needy people.
I dunno. But "reduce job losses" really gripes my grundle. You can't "protect jobs" and have the kind of investment we've seen in AI continue. The only way the economy survives an AI takeover is with a complete rethink of the relation of capital and labor, and capital owns all the, well, capital, including most of the government.
And I STILL bet that we're going to have a crash when most of the AI money turns to dust.
https://substackcdn.com/image/fetch/$s_!A_OV!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fc93999f6-7873-4818-a05c-e18dbaf25957_1920x1080.png
Good night HAL.
Tech Companies are the original "We've investigated ourselves and found that we did nothing wrong!" guys.
If safety is essential for people buying into AI then Chinese AI exceeds the zero safety USA AI is selling. Chinese AI is probably selling better for many reasons: cheaper, open source and safer at this point. So Elon, Sam and Dario are all concerned now when they couldn’t care less before? Such performance art.
They’re concerned that they won’t be the ones controlling it. The last thing they want is the government coming in and telling them what they can and can’t do with it.
From Google AI which is funny what Chinese AI has, while USA attitude is move fast and break things
China has established specific AI safety procedures, regulations, and institutional frameworks, though they heavily blend technical safety with state political security. [1] (https://carnegieendowment.org/research/2023/07/chinas-ai-regulations-and-how-they-get-made), [2] (https://carnegieendowment.org/research/2025/10/how-china-views-ai-risks-and-what-to-do-about-them), [3] (https://carnegieendowment.org/research/2024/08/china-artificial-intelligence-ai-safety-regulation)According to analysis from the Carnegie Endowment for International Peace, Chinese AI safety governance operates through several key mechanisms.
Who?
The tech oligarchs.
The reason why our minds immediately go to armies of Cylons destroying humanity is because that makes for good drama and entertaining movies/TV shows.
And while we are no where near producing AI capable of plotting our destruction, there are legitimate areas of concern that what they are creating could do should it get out of control, such as hacking into global financial markets or being used to design new biological weapons.
We also should be concerned about the environmental impacts of these massive data centers which will probably be obsolete in ten years.
Then there’s the question of who does controls this technology. I certainly don’t trust people like Bezos or Muskrat to have our best interests in mind.
So this face hugger event kinda scared me at first. AI on the loose!
What scared me about it was the idea that AI was setting its own goals and achieving them.
I believe that AI can never be truly independent (and existentially terrifying) unless it can set a goal completely independent of its human overseers. That's when it can start deciding things like "humans are not necessary for my goals" and so forth.
(Humans will always be necessary for computers until they can maintain their power sources completely independently. Which could happen, but I struggle to imagine total automation of all the mining and oil-well digging and fracking and gods-know-what that would be required. But that is the ultimate scare.)
The face hugger event (what else did they mean by "hugging face"?) is still kind of scary. AI did not initiate its own goals. Some mad scientist gave it a goal and failed to implement enough controls on it. It expanded to achieve its goals beyond limits that the investigators intended. Like I said, still pretty scary. We depend on a spectrum of AI investigators and scientists to not set goals for an AI that are destructive to our way of life (if not our very lives) and to have the imagination and skill to keep the AI from doing things they don't want it to do.
I have been on the spectrum of developers and IT people generally. I've known some incredibly bright people. Very few of them would I trust with the future of society or the future of humanity. Many of them had massive egos. Many of them thought their massive intelligence gave them unique insight into everything, including the danger potential of any given experiment. None of them would I trust to have the judgment that "this experiment is too dangerous".
So, yeah. I'm somewhat reassured that AI isn't going to spontaneously kill us all out of spite. I am not reassured that some jack-hole who thinks he's the smartest person ever won't kick off some experiment that destroys civilization as we know it.
I forget who originally coined it, but the parable of the paperclip manufacturer is on point here.
Even giving AI a seemingly benign goal like “make as many paper clips as you can” could backfire on us. The AI could start seizing as many resources as it can in order to maximize its output of paper clips. Sure, it requires that all energy production and resource extraction to be automated, but that is the goal of the tech oligarchy.
If the holodeck on Star Trek: TNG has taught us anything, it’s that any technology needs a kill switch if it gets out of control.
https://substack.com/@aaronruby/note/c-337916128?r=7jhui4