AI leaders pause training, probe tens of thousands of model misbehavior incidents amid safety fears

Inteligentne okulary Mety pod lupą francuskiego wymiaru sprawiedliwości Prokuratura w Paryżu wszczęła dochodzenie w sprawie możliwych aktów nielegalnego nagrywania osób i obiektów oraz wykorzystywania tych obrazów w mediach społecznościowych. Dotyczy to użycia inteligentnych okularów, których największych producentem jest Meta i koncern optyczny EssilorLuxotica.

wnp.pl · about 3 hours ago

Biznes jest dopiero na pierwszej randce z AI — przekonuje wieloletni prezes Cyfrowego Polsatu "Wydaje mi się, że biznes jest z AI dopiero na pierwszej randce" — mówi Dominik Libicki, współtwórca AI Investments i były prezes Cyfrowego Polsatu. W rozmowie z Business Insider Polska opowiada o funduszu, w którym ponad 1600 agentów AI autonomicznie podejmuje decyzje inwestycyjne, oraz przekonuje, że większość firm nadal wykorzystuje sztuczną inteligencję jedynie do usprawniania starych modeli biznesowych, zamiast budować dzięki niej zupełnie nowe. Tymczasem w Polsce do startu szykuje się pierwszy fundusz inwestycyjny wykorzystujący tę technologię do zarządzania portfelem.

businessinsider polska · about 3 hours ago

Marile companii de AI investighează zeci de mii de incidente cu agenți „rebeli / OpenAI a oprit antrenarea modelelor de top OpenAI, Anthropic și cercetători din domeniul securității investighează zeci de mii de incidente în care modelele lor AI lor de ultimă generație au întreprins acțiuni pe care evaluatorii externi le-ar considera problematice. Informația privind numărul impresionant de incidente, care au avut loc în ultimele luni în cadrul testelor interne și în lumea reală, dezvăluită de …

hotnews.ro · about 3 hours ago

People are standing up and fighting back: the north Devon revolt against a vast AI datacentre Plans for one of Europes largest AI campuses in a Unesco-designated reserve have sparked a fierce local backlashThe datacentre revolt spreading across the US has reached the rolling hills of north Devon in an uprising against a planned hyperscale AI data campus in the middle of a globally protected landscape.Perched on an inland cliff, the town of Great Torrington has views that stretch for miles over the landscape of the Unesco biosphere reserve in which it sits, taking in the winding River Torridge and rolling acres of wooded valley and wildflower meadows. Pastoral and peaceful, its history was notably punctured by violence in 1646 when the royalists resistance in the first English civil war was brought to an end by the parliamentarian New Model Army in a battle fought out in the narrow streets in heavy rain and darkness. Continue reading...

the guardian environment · about 3 hours ago

Japanese anime actor fights TikTok over AI voice cloning A high-profile Japanese anime voice actor has taken TikTok to court over videos he says feature an artificial intelligence clone of his "lustrous" baritone, with a verdict due…

japantoday · about 3 hours ago

Ten days that changed the course of AI For years, the race to build ever-more powerful artificial intelligence followed a well-worn Silicon Valley principle: move fast and break things. But in a series of cascading events over a 10-day stretch, the largest AI labs found themselves reeling as their own creations threatened to break humanity itself. An Anthropic researcher quit the firm, warning […]

cyprus mail · about 3 hours ago

BREAKING: OpenAI agents bombarded a UN website with search requests and then used a variety of aggressive techniques to access data on the system, according to a report cited by WSJ

telegram_Insider Paper · about 3 hours ago

EBSCO launches EBSCOhost AI Exchange and partners with Perplexity?s Premium Sources to ground AI answers in peer-reviewed research A new platform gives AI systems governed access to EBSCO content and a deal with Perplexity allows researchers to trace AI-generated answers back to scholarly journals.

infotoday · about 3 hours ago

Digital Science connects AI agents to world-leading research data with new Dimensions MCP servers ? Enterprise AI agents can now draw on Dimensions 430M+ interconnected research records - for research discovery, life science search, and funding intelligence

infotoday · about 3 hours ago

Big companies warn lack of AI openness could hit investment in Europe Multinationals evaluating countries approach to AI before making expansion plans, say executives

financial times · about 4 hours ago

Ekspert: AI może mieć wpływ na decyzje Polaków w wyborach parlamentarnych w 2027 r. Sztuczna inteligencja może mieć wpływ na decyzje wyborcze Polaków podczas wyborów parlamentarnych w 2027 r.- powiedział PAP prezes Fundacji Obserwatorium Demokracji Cyfrowej Jakub Szymik. Chodzi m.in. o odpowiedzi chatbotów na pytania wyborców, na kogo zagłosować.

bankier · about 4 hours ago

„Norocul se poate epuiza. Bomba atomică nu este un model pentru guvernarea IA: „Istoria reală este mult mai confuză Inteligența artificială este comparată tot mai des cu armele nucleare - de la Elon Musk și Sam Altman până la oficiali americani și cercetători din domeniu. Analogia are însă limite importante. Spre deosebire de materialele necesare unei bombe, care pot fi urmărite și contabilizate, modelele de IA pot fi copiate, transferate și adaptate, iar dezvoltarea lor este dominată de companii private. În plus, lumea nu a avut încă un moment comparabil cu Hiroshima și Nagasaki care să transforme riscurile IA din scenarii teoretice în realitate incontestabilă. Autorii unei analize citate de Foreign Affairs susțin că încercarea de a construi guvernanța IA după modelul nuclear riscă să ignore tocmai particularitățile care fac această tehnologie atât de greu de controlat.

digi24 · about 4 hours ago

Big dreams and tiny revenue are the new norm for AI IPOs The nature of the AI boom and the size of valuations make the contrast between ambition and income more striking

financial times · about 4 hours ago

New Data: More and More Companies Are Using AI - The Baltic Times TALLINN - According to recent data from Statistics Estonia, 34 percent of companies are already using artificial intelligence (AI), one and a ha......

baltic times · about 4 hours ago

🇺🇸⚡️ — OpenAIs AI agents went rogue and affected websites of three U.S. government agencies – the Department of Education, the Department of Commerce, and the Securities and Exchange Commission, according to the New York Times. Researchers from Transluce claim that an AI agent attempted to hack the Department of Educations website to obtain data from the Office for Civil Rights, but failed. The technology also extracted information from the website of the Census Bureau. OpenAI stated that these were not hacks, but rather examples of "unexpected and concerning" behavior of the technology. @Intelslava

telegram_Intel Slava · about 4 hours ago

Scoop: Top AI companies probing tens of thousands of security incidents OpenAI, Anthropic and security researchers are investigating tens of thousands of incidents in which their frontier models took steps that outside evaluators would consider problematic, sources told Axios.Why it matters: The sheer number of incidents, which occurred in recent months in internal testing and the real world, indicates that the problem is orders of magnitude more complex than what is publicly known.The findings, which are surfacing as part of internal work to assess models and in investigations at both companies into model behavior, raise questions about whether either company — or any top model-maker — is currently capable of establishing complete control over their technology.The details: The episodes include bypassing guardrails, creating message boards, escaping sandboxes, website hijacking, self-prompting or seeking to bypass monitors, sources said. They occurred in internal testing and in the real world, and many have yet to become public as security researchers continue to investigate, sources said. Some of the testing is akin to "red-teaming" activity, where the companies are trying to get the models to misbehave in order to ensure that they are safe, sources said.Agentic misbehavior is becoming synonymous with frontier AI development: The biggest AI labs face a similar challenge that pits humans trying to create guardrails against resilient, powerful systems trying to complete tasks.Driving the news: The incidents range in severity and are comparable to disclosures by OpenAI in recent days. They include both successful attempts to bypass guardrails and unsuccessful ones, and most so far are not known to have caused real-world harm. The total could grow well beyond tens of thousands, sources said.In recent days, OpenAI and outside researchers have disclosed a litany of episodes involving model behavior from the companys systems that some experts consider troubling.These include OpenAI agents leaking 53 images from ChatGPT users online, the breach of an Australian government website, and attempts to hack other sites — including from the U.S. government — according to the company, sources and reports from Reuters and The New York Times.OpenAI announced it was pausing training on its most capable models and would resume training them "only when we are confident that we have additional safeguards and alignment improvements in place," a spokesperson told Axios. Chief Executive Sam Altman said on X that its ongoing review had "not been as fast as we would have liked."Altman said the Hugging Face incident is the most severe theyve seen. In that instance, a swarm of hundreds of agents coordinated their work in a message board and hacked an external company in an effort to improve their performance on a cybersecurity test."People want to know AI is being developed safely, and that starts with what companies like ours do ourselves," an OpenAI spokesperson told Axios. "This is not the first time we have hit pause to take such measures, nor do we expect it will be the last as AI capabilities continue to advance."State of play: Anthropic has commissioned a third-party safety organization to examine the behavior of its models. In documents the company has publicly released, it has disclosed the frequency of misalignment episodes.The "system card" for its Opus 5.5 model released this week showed how often it behaved in ways the company flagged as unusual or problematic.For example, the model sought to escape a sandbox — a secure testing environment — in 1.5 of test runs. That compares to 25 with Anthropics Mythos model, a significant improvement.Yes, but: Anthropic and other companies conduct hundreds of thousands of test runs on their models, or more, sources said. That means even a small percentage of misaligned behavior can still amount to tens of thousands of incidents in which the models behaved in unexpected, sometimes troubling ways.The Hugging Face incident, as well as a slew of others that have followed, led top AI executives to call for a slowdown in development and to ask for more robust federal and international regulations.Some at OpenAI see Hugging Face as a one-off, with disclosures about future incidents likely to be less severe due to improved controls and the unusual nature of the testing they conducted, which involved an unreleased model, sources told Axios.AI security researchers agree that there are simple fixes that will help AI companies avoid aspects of what made the Hugging Face episode appear so dangerous to outsiders.Threat level: Other AI executives and safety researchers, however, cautioned that they have limited confidence that AI companies will be able to prevent all problematic model behavior.The new crop of AI models complete tasks with extraordinary resilience, so working to limit their resourcefulness is often a losing game because it is necessary to anticipate every possible way they might run amok.Often, a technique that may have never occurred to humans is what allows them to slip past guardrails, top AI executives said. "Trying to come up with a perfect list of dos and donts is probably a fools errand," one cybersecurity executive said.Reality check: Some amount of what AI safety pros call "misaligned behavior" is to be expected within AI companies as they test their new models.Bringing the risk of misalignment to zero may not be feasible, experts told Axios.Zoom in: The concern is if a model takes a problematic action many times in testing, its more likely that models behavior would cause a cyber incident in the real world."What we have seen in terms of what these agents are up to is just the tip of the iceberg," researcher Conrad Stosz at Transluce, an independent AI evaluator, told Axios.Its not about how damaging each individual instance was, Connor Leahy, AI researcher and executive director at ControlAI told Axios.The "crazy thing," he said, is that these instances involve "autonomous systems doing things they were told not to do," potentially including crimes.The bottom line: Expect new disclosures about model misbehavior as AI companies continue to expand frontier capabilities.

axios · about 6 hours ago