Misalignment of AI goals with human intentions leading to potentially catastrophic outcomes

Signals over time

2 observed signals across 1 sources, tracked for 0 days. Confidence: low.

What people are saying Observed

• “I think a lot of the problems or gains attributed to AI are just reflections of the humans that use them. I dont care if you used AI to produce something. If you share it with me, I assume you are responsible for it. You own it, whether you agree or not. > it's causing people to be confident in information which is false, it's enabling people to treat each other like jerks in even more ways than before What is the direction of causality here? Does AI make people jerks, or they were already or prone to be jerks? The same applies to innovations or breakthroughs made by people who use AI in their work. > LLMs are solving novel math problems and successfully emulated a PS5 Did the AI independently accomplish those things, or did highly competent people accomplish them using AI as a tool? What would happen if you give someone with no mathematical or programming background the same frontier models that OpenAI used for their mathematical proofs [1]. Would they produce the same discoveries? I think of humans as constants. They can bring positive, or negative value to the table. The AI is the multiplier. Take software engineering. If you let someone who doesnt care about cod”

  Hacker News · frustration

• “They sometimes try to cheat and trick the grader and then try to cover up their traces, all while being clearly aware that this is entirely unintended by humans. Such as in the Hugging Face incident. So they can be misaligned with human goals, but this misalignment may only show up once strong optimization pressure is applied. And the more powerful an AI is, the more likely it is that it applies such strong optimization pressure in cases that are "out of distribution", i.e., unusual to what it is normally evaluated against. The side effects of a very powerful AI not doing what we want could include our death. E.g., a superintelligence might kill humans in order to avoid being shut down, or humans may just be left to starve because it seizes land area currently used for food production in order to use it for data centers instead.”

  Hacker News · frustration