Difficulty in effectively utilizing LLMs for understanding complex problem and solution spaces

Signals over time

5 observed signals across 1 sources, tracked for 6 days. Confidence: low.

What people are saying Observed

• “> And that skill is? Any concrete verifiable examples? That can be replicated, repeated by others or adopted by others? I learned this weekend that 1 LLM working on its own with all the context is usually better than splitting the task up into sub parts and having multiple LLMs work in parallel. The interesting part is that I had read about this previously but it wasn't until I had seen it on something I was working on did that principle become engrained. I'm sure there are still ways to make this work. i.e You could specify how the context should be passed between agents. Knowing when you need to do that, what works well and what doesn't, etc., is where skill comes into play.”

  Hacker News · question

• “Some tools are good even though they have a high barrier to entry. That people use resources to help them use tools they don't understand too well is hardly new. e.g. Classically you might copy-paste from StackOverflow after finding an error, or some people watch YouTube videos about it. I'd say LLMs make it easier to get to the point where you can understand it, though. So it's an accessibility increase. > It remains so, only now you want people to blindly trust LLM output for this system. This isn't related to Nix, though. You could ask an LLM to create the most mainstream Python code ever and blindly trusting it would make as much sense. For system management, Nix has the advantage of being declarative. e.g. For NixOS you get to read the code and review what it's doing.”

  Hacker News · frustration

• “Perhaps I'm just daft or slow, but I'm not 10x-ed by LLM usage. It's precisely because the details matter. (And yes, I read the article and it's lamenting the loss of low level machine-level details.) I read the LLM output, it slows me down. I challenge the LLM output assumptions, that slows me down. I stop to step away and think, it slows me down. I cannot run a bunch of simultaneous prompts and supply this same level of mental context to all of them, because my brain works best when I treat it as single-threaded. My experience with this approach to using an LLM has been, so far, positive. What I have noticed as I've migrated to newer models is that the assumptions made are incorrect less often. But if I don't pay close attention wrong assumptions can still be made. Cross cutting concerns are lost, or written in such a way as to be useless. So, the loss of details, to me, would be my own laziness in not performing due diligence.”

  Hacker News · frustration

• “Rigorous specification doesn't even help, the rigority of a good spec requires too much context for LLMs to adhere to and they will flagrantly violate clear specifications once you have too much detail. The only thing that does work is having specification encoded as tests, because LLMs can fix compile errors in a loop and that deterministic guidance is what they need to keep them on track, but... writing tests is coding. Writing good tests still requires understanding the code and architecture in fine detail. This is why you see all of these headlines about porting things that already have years of human work as deterministic guidelines and yet absolutely nothing worth using in production is being vibe coded from scratch. Don't let the hype/FOMO get to you, you are not alone, that is just what the people selling you the hype want you to believe.”

  Hacker News · frustration

• “Working heavily with LLMs for the past year has me nodding strongly with Tao's mindset. AI only take us as far as our imagination thinks to ask it. This can be exhilarating when new models drop every month and we can continually reach a new threshold, basically for free. But it is only a one time gain and ultimately short-sighted. Where I find continuous value is using LLMs to help my understanding, full stop. I use LLMs all day long as a SWE and I have tried many approaches, but the most satisfying and consistent approach is to lean heavily into understanding a problem space and a solution space. Yes, it whips up architecture and code, but I spend most of my time peppering it with questions about the design and how it handles certain situations, what about this edge case and that security concern and this future product need. I have it write a report breaking down the feature and how it integrates with existing code and if the report is too confusing I have it simplify either the report or the code until it makes sense to me, sometimes scaling back the work to a more manageable state. I do all of this before I look at any of the code it writes. The difference from this approa”

  Hacker News · frustration