The high risk categories read like a list of successful lobbying groups to me, each of which managed to get AI effectively outlawed for their domain (thus protecting their jobs for some time).
I this assume changing them would entail fighting with the same lobbying groups which put them there in the first place.
(Small note: wouldn't your tool, at least as presented, fall outside of the AI act since it was created with AI -though deniably- but doesn't actively use any AI in it's operation?)
____On the risk of downplaying the example you listed:____
I think what’s missing is an alternative. We shouldn’t copy the classic American way of waiting till the harms and externalities play out until something is done. I think it’s foolish to “just let tech work it out.” You present your teaching tool as a benign example, but I think it’s anything but. I could be missing something but i don’t think it precludes you using your invention yourself or for your children, but the risks associated with using it at scale and thereby shifting the way children interact with education and the world around them does merit careful consideration. if you are unsure of those risks, I have a quick 7-10min video by a software developer who has been working in the area since the 80s-he explained it really well.
____Clarifying the issue at hand:____
what is also not clear is how burdensome following the articles you stated are. it could be that its work but its simple, or it could be a mountain of documents. like i think each of the individual steps are actually reasonable and people should follow them, but maybe it is more complicated than appears at face values, and could be implemented in a way that is over engineered.
____On how iterative governance trumps no governance:____
you astutely point out how technology changes and regulation can’t foresee the future and then, in the blog post, point out how that the EU is already looking to update the regulation. I think you are actually highlighting what the EU is doing right, approaching governance as an iterative process adapting to the moment. We should be starting with something, and going from there. laws and regulations should be living breathing documents, not a task to complete and never revisit again. Now, is the EU prices perfect, likely not, but as i read it these are good indications.
____On AI risks:____
I also think that there the post points out how we can’t predict AI in the future, but when mentioning risks it only mentions CBRN, cyber, and loss of control. Of these 3, only 1 (cyber) is here and now. The others are predictions not yet realized. The risks not mentioned that are here and now, are AIs ability to amplify mis and dis information that can divide and degrade society, the use of AI for coercion, the use of AI for generating CSAM, other human rights violations, impacts on development and learning, impacts on the workforce, ability to inform and enable self harm, ability to inform and enable harm to others (e.g. school shootings), etc. The CBRN risk is largely fantastical, uncharacterized, and largely rooted in information leaks from classified to unclassified spheres that could be rooted out pre-training or via post-training methods.
The high-risk use-case rules are clearly unworkable — you shouldn't need a fundamental rights impact assessment to deploy a math quiz. That argument landed.
From the American side though, the opposite failure mode is just as dangerous. Yesterday the Pentagon blacklisted Anthropic using a designation normally reserved for foreign adversaries — not because the technology failed, but because the company refused to remove two restrictions from its military AI contract: no mass domestic surveillance, no fully autonomous weapons. Hours later, OpenAI took the same deal with the same stated restrictions. The Pentagon accepted from OpenAI what it punished Anthropic for insisting on. There is no US law governing any of this. No legal definition of meaningful human control, no framework for what the military can demand from AI contractors, nothing.
The technology is moving faster than either government can legislate. The EU overcorrected with use-case bureaucracy so granular it catches quiz apps and electrician scheduling tools. The US did nothing, and the vacuum got filled by a government using national security designations to coerce companies into dropping safety features. Both are failing, just in opposite directions. You make a strong case that the GPAI systemic risk rules are worth keeping. The American experience is a pretty good argument for why.
The high risk categories read like a list of successful lobbying groups to me, each of which managed to get AI effectively outlawed for their domain (thus protecting their jobs for some time).
I this assume changing them would entail fighting with the same lobbying groups which put them there in the first place.
(Small note: wouldn't your tool, at least as presented, fall outside of the AI act since it was created with AI -though deniably- but doesn't actively use any AI in it's operation?)
The tool integrates an API call to Claude Haiku!
____On the risk of downplaying the example you listed:____
I think what’s missing is an alternative. We shouldn’t copy the classic American way of waiting till the harms and externalities play out until something is done. I think it’s foolish to “just let tech work it out.” You present your teaching tool as a benign example, but I think it’s anything but. I could be missing something but i don’t think it precludes you using your invention yourself or for your children, but the risks associated with using it at scale and thereby shifting the way children interact with education and the world around them does merit careful consideration. if you are unsure of those risks, I have a quick 7-10min video by a software developer who has been working in the area since the 80s-he explained it really well.
____Clarifying the issue at hand:____
what is also not clear is how burdensome following the articles you stated are. it could be that its work but its simple, or it could be a mountain of documents. like i think each of the individual steps are actually reasonable and people should follow them, but maybe it is more complicated than appears at face values, and could be implemented in a way that is over engineered.
____On how iterative governance trumps no governance:____
you astutely point out how technology changes and regulation can’t foresee the future and then, in the blog post, point out how that the EU is already looking to update the regulation. I think you are actually highlighting what the EU is doing right, approaching governance as an iterative process adapting to the moment. We should be starting with something, and going from there. laws and regulations should be living breathing documents, not a task to complete and never revisit again. Now, is the EU prices perfect, likely not, but as i read it these are good indications.
____On AI risks:____
I also think that there the post points out how we can’t predict AI in the future, but when mentioning risks it only mentions CBRN, cyber, and loss of control. Of these 3, only 1 (cyber) is here and now. The others are predictions not yet realized. The risks not mentioned that are here and now, are AIs ability to amplify mis and dis information that can divide and degrade society, the use of AI for coercion, the use of AI for generating CSAM, other human rights violations, impacts on development and learning, impacts on the workforce, ability to inform and enable self harm, ability to inform and enable harm to others (e.g. school shootings), etc. The CBRN risk is largely fantastical, uncharacterized, and largely rooted in information leaks from classified to unclassified spheres that could be rooted out pre-training or via post-training methods.
The high-risk use-case rules are clearly unworkable — you shouldn't need a fundamental rights impact assessment to deploy a math quiz. That argument landed.
From the American side though, the opposite failure mode is just as dangerous. Yesterday the Pentagon blacklisted Anthropic using a designation normally reserved for foreign adversaries — not because the technology failed, but because the company refused to remove two restrictions from its military AI contract: no mass domestic surveillance, no fully autonomous weapons. Hours later, OpenAI took the same deal with the same stated restrictions. The Pentagon accepted from OpenAI what it punished Anthropic for insisting on. There is no US law governing any of this. No legal definition of meaningful human control, no framework for what the military can demand from AI contractors, nothing.
If you’re interested, I wrote the full timeline here: https://anthonytaglianetti.substack.com/p/the-compliance-test
The technology is moving faster than either government can legislate. The EU overcorrected with use-case bureaucracy so granular it catches quiz apps and electrician scheduling tools. The US did nothing, and the vacuum got filled by a government using national security designations to coerce companies into dropping safety features. Both are failing, just in opposite directions. You make a strong case that the GPAI systemic risk rules are worth keeping. The American experience is a pretty good argument for why.