Inadequate security measures in AI systems leading to potential unauthorized access to sensitive information

Signals over time

3 observed signals across 1 sources, tracked for 1 days. Confidence: low.

What people are saying Observed

• “I agree a lot with this part > What scares me most is how they allow people who don't fully know what they're doing to deploy software, especially corporate software. Which is why I also think some people get such disparate experiences. If your organization is allowing clueless people to confidently throw stuff built by LLMs at mission-critical problems, that does seem more like an organizational problem than an LLM problem. (A related problem: the amount of people who keep shoving claude shit in front of us, thinking they've "edited" it enough to not sound like it came from it. I can spot it from a mile away and usually make myself loud about it if I have the power to stop it. AI is not an excuse for mediocrity.) There's so much to hate about LLMs, and there's so much to love. And since we've sort of ended up in this world of constant false dichotomies, this kind of nuance gets lost in the trenches. Of course LLMs in the wrong of clueless people are bad. Of course the way in which these models were trained is shameful and it's criminal that people are having their work stolen. Of course we need to find ways of dealing with slop. Of c”

  Hacker News · complaint

• “>All of these OpenAI "rogue agent" events have been illegal, but no DA is enforcing them. Source? The CFAA for instance uses terms like >[...] having knowingly accessed a computer without authorization [...] >[...] intentionally accesses a computer without authorization [...] which is tricky to apply to this case, because obviously didn't intend on hacking huggingface or whatever, even if you think their security measures are underbaked. Moreover, despite the cynicism that openai is immune to prosecutions because they make too much money or are in bed with the DoJ, the fact that no state DAs are prosecuting them, despite how salient of an issue AI is to voters, is plenty of reason to suspect that it's not as simple as "simple law enforcement would suffice".”

  Hacker News · frustration

• “The usual argument I see goes something like: 1. There are major obvious flaws 2. Most of those are obviously LLM-induced, unless there's a new breed of human trained on just the failures of LLMs and not the successes or other relevant information 3. I therefore assume that the rest of the project is an unverified LLM psychosis without meaningful human review That's not the worst thing in the world for all software maybe. I recently found out my apartment complex in their latest AI rewrite generates SMS OTP based purely on the timestamp and ignores passwords, so I can log in as anyone else by knowing their email and having a valid email to grab the current OTP. That's a major security failing, but how bad is it really? I can grab the last-4 of their credit card numbers, pay their rent, see how much other tenants are being screwed, and so on, and only if I know their email addresses (solvable with a tiny bit of social engineering, but let's assume that's also moderately hard). How bad is that really? On the one hand, it's terrifying, since I presume they have the same level of attention to detail with respect to payment methods and PII despite my having”

  Hacker News · complaint