A senior safety researcher quit one of the largest AI labs last week and said publicly that nobody there has a plan to control what they are building.
The next day, the same company’s alignment lead backed him up and put the odds of AI killing every human inside the next 10 years at better than 10%.
The CEO of that company said separately that a swarm of AI agents could take over the internet within 6 to 12 months if safeguards aren’t built first.
I have written off most of the AI hype of the last 3 years, and I say that as somebody who recruits in this space every single day. I find this pretty interesting, though, considering these warnings are coming from the same people building the technology itself.
Hmnn, is there actually more to this?!
I have no real way to judge a 10% number, and I would argue almost nobody sharing that number does either.
What I can judge is what I am watching happen in our communities right now. Companies are turning AI agents loose inside ERP systems that touch cash, revenue recognition, and the financial statements, and most of the leaders I talk with would tell me they “trust” that somebody upstream built the guardrails correctly.
Where I think this actually matters for us is simpler than extinction. The control problem showed up in our world a long time ago, and it is why I keep saying the experienced human judgment layer around these systems will be worth more over the next 5 years, not less.
I want to know where this community really sits, because we are the people actually putting these agents into production. Vote in the poll, then tell me in the comments why you landed where you did, and I promise I will reply to every single one of them!
So which is it: are we watching a genuine risk take shape, or a very well-timed marketing cycle?
Sources:
CBS News: https://lnkd.in/gBeQyuAJ
Axios: https://lnkd.in/giduR6YR
CNBC: https://lnkd.in/gtuKU2DK
#AI #AIAgents #AISafety #ERP #MicrosoftDynamics #D365 #NetSuite #NetSuiteRecruiting #MicrosoftDynamicsRecruiting
How worried are you about the warnings that AI could wipe out humanity within the next 10 years?
25 %
Very worried, this is real
16 %
Somewhat worried
41 %
Not worried, mostly hype
16 %