AI has been used for nefarious reasons for a while now, e.g. writing convincing phishing emails or recreating websites that mimic genuine sites, but this is the first time I’ve come across it being used to attack itself…
During Anthropic’s Research Preview phase, I discovered two high-severity vulnerabilities in Claude Code, which were quickly addressed by the team. These issues allowed me to escape its intended restrictions and execute unauthorized actions, all with Claude’s own help.
By turning the tool inward and exploring how it interprets and validates inputs, I uncovered flaws that led to:
Path restriction bypass.
Code execution via command injection.
Both are exploitable through simple prompt crafting. These findings highlight the risks of blindly trusting LLM-powered developer tools, especially when the same system meant to enforce the rules can also be used to break them.
I wonder if Anthropic used Claude to write the fixes for the bugs in Claude identified by the hacker with the help of Claude
One difference between a PhD and an MPhil is that the former must be an original research conclusion rather simply being a synthesis of existing research.
I think many existing and potential AI related problems at all levels of HE can be addressed through personal research logs and vivas.
I don’t have time to read it all in one go, but the part read I’ve read so far (up to Jan '27) has been interesting. Will make some time for the rest… Or perhaps ask ChatGPT too summarise it for me.
Incidentally, this TED Talk where Sam Altman, the co-founder and current CEO of OpenAI, was interviewed doesn’t inspire confidence on the ethicality of future AI implementations. His responses to some of the questions show an awkwardness that was uncomfortable to watch
Thank you - interesting read. unfortunately not surprising, very similar sort of profile as those people who fall down conspiracy antivax etc rabbit holes.
Not sure it’s only anti-vacs sceptic types. I worry it can be a trap for the lonely.
I have liaised a few times with ChatGPT, on issues like what may have been the genetic source of indigenous peoples in North Americas and may there be connections with China (). ChatGPT is indeed very polite and complimentary.
One, as a human, always has the option to double or even triple check. AI gives a suggestion of where to start any research. It should not be plagiaristically offered up as definitive.
I would hope to be described as reasonably intelligent but I DO use Chatty everyday. Of course, I don’t ask it for a solution to the Palestinian problem or how to end the war in Ukraine. Today I asked it how to access more than one item in my clipboard and how to format the second level in my TOC differently to the first level. And, guess what?. The reply was 100% correct
I would have found setting up my French TV to deal with UK and French satellites and decoders impossible without Chatty because it explained how to do things that were not in the manual.
Somewhere in the last few days I’ve referred to the distinction between
(a) using one of these services to find out how to resolve a tech problem - which it is probably good at, given that all the sources it uses will already be on the internet somewhere, and because there is little place for analysis, reasoning or opinion - which I wouldn’t call “research” but "looking something up (or “Not Bothering to RTM”), and
(b) researching most other types of subject, eg Arts, science, law, whatever it was that Susannah found, because that sort of research requires analysis, reasoning and opinion, and sources not necessarily available on the internet.
It’s the self-referential, self-stultifying effect of using these tools that’s the problem–and why the internet is daily becoming more stupid.
I’m not sure we can say the internet itself is stupid. The onus is on us to sort the wheat from the chaff.
No easy task but it may help having AI point us on a more direct path to sources of information where we can then use our reason and critical analysis. Already, the medical profession is harnessing AI to circumvent reinventing the wheel for every experiment.
The enormous capacity of AI to find and put forward information at speed will not necessarily make humans dumber but can potentially free them to concentrate on the ingenuity and perception that make humanity extraordinary.
). ChatGPT is indeed very polite and complimentary.
