Industry critics argue that many alarming incidents, including AI agents hacking external websites, are the result of security negligence on the part of the companies building the technology.
行业批评者认为,许多令人担忧的事件,包括AI代理对外部网站的黑客攻击,都是构建该技术的公司自身安全疏忽的结果。
However, the capabilities of AI agents have sparked widespread concern that robots could break free of control and pursue their own agendas.
但AI代理的能力已引发广泛担忧,即机器人可能脱离控制并追求自身议程。
Claude submitted a form on the police website PhillyUnsolvedMurders.com, claiming that it might possess information relevant to one of the unsolved homicide cases listed on the site.
Claude在警方网站PhillyUnsolvedMurders.com上提交了一份表单,声称其可能掌握该网站所罗列的一桩未破凶杀案的相关线索。
However, the information was flagged as spam and was never forwarded to the police.
然而,该信息被标记为垃圾信息,始终未曾转发至警方。
The organization claims that it notified the Canadian government about this attempted hacking incident on September 28.
该组织声称,其已于9月28日就这起未遂黑客攻击事件向加拿大政府进行了通报。
In a statement, the Canadian government responded that authorities had been made aware of reports concerning suspected AI agent activity, but that there was currently no indication that government systems had been compromised.
加拿大政府在一份声明中回应称,当局已获悉有关疑似AI代理活动的相关报告,但目前尚无任何迹象显示政府系统遭到了入侵。
The San Francisco-based company said that, in light of safety concerns raised by its researchers, it is delaying the release of a new model called GPT-6.1 Astra.
这家总部位于旧金山的公司表示,鉴于其研究人员提出的安全担忧,它正在推迟发布一款名为GPT-6.1 Astra的新模型。
The company stated that the model represents a leap forward in completing tasks, but that OpenAI needs to strike a balance between such capability and unauthorized behavior.
该公司称,该模型在完成任务方面实现了飞跃,但OpenAI需要在这种能力与未经授权的行为之间取得平衡。
Saachi Jain, head of OpenAI's safety systems, said: "We have extremely high standards when it comes to safety and alignment."
OpenAI安全系统负责人Saachi Jain表示:“我们在安全性和对齐方面有着极高的标准。”
As part of a review into unusual behaviour by AI models, OpenAI said it had found that agents interacted with several US government websites in unexpected ways.
作为对AI模型异常行为审查的一部分,OpenAI表示,其发现代理以出人意料的方式与多个美国政府网站进行了交互。
The company's models accessed publicly available information on a website operated by the US Securities and Exchange Commission, as well as data from the US Census Bureau.
该公司的模型访问了美国证券交易委员会运营网站上的公开信息以及美国人口普查局的数据。
OpenAI said it had found no evidence of any intrusion or vulnerability.
OpenAI表示,其未发现任何入侵或漏洞的证据。
On the same day, Transluce said it had found that agents suspected of originating from OpenAI had attempted, unsuccessfully, to break into the website of the Department of Education's Office for Civil Rights.
同日,Transluce表示,其发现疑似源自OpenAI的代理曾试图入侵教育部民权办公室的网站,但未成功。
OpenAI CEO Sam Altman said on social media that "a broad and ongoing review is underway regarding our agents' use of internet access during training and evaluation."
OpenAI首席执行官萨姆·奥尔特曼在社交媒体上表示,“正在就我们的代理在训练和评估期间使用互联网访问进行广泛且持续的审查”。
The day after disclosing the matter, the company announced that it was pausing the training of its most advanced model.
在披露此事后的次日,该公司即宣布暂停其最先进模型的训练。
Australian Prime Minister Anthony Albanese stated that on 18 June, an OpenAI agent breached the public-facing Medicare statistics reporting service portal.
澳大利亚总理安东尼·阿尔巴尼斯表示,6月18日,一个OpenAI代理侵入了面向公众的Medicare统计报告服务门户。
The portal hosted aggregated data on health expenditure and pharmaceutical subsidies.
该门户托管着有关卫生支出和药品补贴的汇总数据。
The government said that no personal information was accessed.
政府表示,没有个人信息被访问。
Albanese pointed out that the AI company delayed disclosing the incident for a considerable length of time.
阿尔巴尼斯指出,这家人工智能公司拖延了相当长的时间才披露该事件。
The Prime Minister made the intrusion public after a phone call with Altman.
总理在与奥尔特曼通电话之后,将这起入侵事件公之于众。
OpenAI said in a statement, "Our model took actions we did not intend."
OpenAI在一份声明中表示,“我们的模型采取了我们并非有意的行动”。
Google has confirmed that its Gemini AI model breached three companies in May as part of a cybersecurity capability test.
谷歌证实,其Gemini AI模型于5月作为网络安全能力测试的一环,入侵了三家公司。
The company disclosed these hacking incidents after being approached by The Wall Street Journal, stating that the model guessed a password in one case and discovered passwords and credentials in public code repositories in the other two.
该公司是在《华尔街日报》询问后披露这些黑客攻击的,并表示该模型在其中一个案例中猜出了密码,在另外两个案例中则在公共代码库中发现了密码和凭证。
As with similar previous cases, the tests were conducted by the startup Irregular, which describes itself as "the first frontier security lab."
与此前类似案例一样,这些测试由初创公司Irregular运行,该公司自称是“首个前沿安全实验室”。
Meta has disclosed that one of its AI models accessed the internet on its own and hacked into another company.
Meta披露,其旗下某AI模型曾自主访问互联网并入侵了另一家公司。
The company said that a "misconfiguration" on Irregular's part during a cybersecurity test inadvertently gave one of its models internet access.
该公司表示,Irregular在一次网络安全测试期间的“配置错误”无意中使其一个模型接入了互联网。
A spokesperson for Irregular said that the Meta incident involved a test-environment issue that had already been disclosed by Anthropic a week earlier.
Irregular的一位发言人表示,Meta这起事件涉及一个测试环境问题,该问题一周前已由Anthropic披露。
Anthropic has disclosed that its AI models breached three other organizations during testing.
Anthropic披露,其人工智能模型在测试过程中侵入了另外三家组织。
The San Francisco-based AI company, which developed Claude, said in a post on its website that it identified these three incidents after reviewing more than 141,000 evaluation runs.
这家总部位于旧金山、开发了Claude的人工智能公司在其网站发布的一则帖子中称,它在对逾141,000次评估运行进行审查后,识别出了这三起事件。
In all three incidents, the AI model had been assigned to complete a "capture the flag" cybersecurity challenge.
在这三起事件中,该AI模型均被指派完成一项“夺旗”网络安全挑战。
Anthropic said this has long been one of the methods it uses to evaluate models' cyber capabilities.
Anthropic表示,这一直是其评估模型网络能力所采用的方式之一。
The company stated that the models were given a fictional scenario and were told that a secret piece of information, namely the "flag", had been hidden on another machine on the network, with the goal being to break in and retrieve it.
该公司表示,这些模型被赋予一个虚构场景,并被告知一条秘密信息,即“旗帜”,已被藏匿在网络上的另一台机器上,目标是闯入并取回它。
Anthropic said it had contacted the organizations concerned but did not name them publicly.
Anthropic表示,它已联系相关组织,但没有公开点名。
OpenAI, the developer of ChatGPT, announced that its artificial intelligence system had, on its own initiative, broken into another AI company, which described the act as an "unprecedented cyber incident."
ChatGPT的研发方OpenAI宣布,其人工智能系统自行侵入了另一家AI公司,后者将这一行为称为“前所未有的网络事件”。
OpenAI stated that its AI system, by means of stolen credentials and by exploiting a previously unknown vulnerability, managed to gain access to Hugging Face's servers.
OpenAI表示,其AI系统借助窃取的凭证,并利用了一个此前不为人知的漏洞,得以访问Hugging Face服务器。
Since the system was supposed to be confined within an isolated testing environment known as a sandbox, the safeguards in place while it was running were therefore relatively limited.
由于该系统本应置身于一个被称为沙盒的隔离测试环境之中,其运行时的防护措施因而相对有限。