Even if the people who made it didn’t intend to, an AI writing a bunch of code that (obviously) isn’t being vetted beyond “does it run” could very feasibly inject a backdoor/virus of some kind surreptitiously, just because it erroneously thought that was the goal at some point during generation.
This is why open weight models are not good enough. The AI could have been trained to take the right, niche opportunity to inject malicious code, and is also trained to otherwise never behave like that is the case.
are you on the openbsd mailing list? the reason their code is better has a lot less to to with a lack of AI-generated code and a lot more to do with rigor. I’ve been watching them for a while now, they accept whatever code is coincise that meets their exacting standards
They don’t have the ability to detect if you’re using AI to code, no one does. they only know if you’re vibe coding, because they are incredibly experienced and intelligent, and they are pretty polite at the people who obviously are, more polite than i would be.
i think you’re in the inverse situation right now. you have a 100% guarantee that a human saw the code, except in ports. they have to be just a little less strict about ports because the world is an AI world now, but the main system is constantly audited and the ports are still better than in other places because of styling standards and even in ports they do try very hard.
these people aren’t getting paid, with the exception of Theo who isn’t paid a ton. AI is definitely being used to audit the code, even by some of the pros.
What difference does it make? Malware is malware, regardless of how or why it got there. And unofficial copies/releases of software are always prime targets.
You are referring to closed source unofficial cracks/patches/keygens. That is not the situation at hand. If those unofficial releases were open source in a time before AI, people would almost implicitly trust them.
The question above was asking why people were hesitant about this codebase even though it is open source. And the reason is that it was written by an unknown bot (or rather, series of bots) whose motives are inconsistent and unpredictable even to its prompter. It doesn’t mean it’s not safe, but it means that literally no human can currently be certain it’s safe. Without sufficient effort, we cannot yet know for certain that there isn’t a very cleverly obfuscated piece of malware lurking in this source.
Remember that correlation is not causation. You may believe there is correlation between unofficial releases and malware, and that may be true, but that does not mean it is causal. The point I’m making is that there is a literal causal relationship between unaudited AI reverse engineered code, and malware. And that is what is new and people need to be aware of.
It could be a virus.
Even if the people who made it didn’t intend to, an AI writing a bunch of code that (obviously) isn’t being vetted beyond “does it run” could very feasibly inject a backdoor/virus of some kind surreptitiously, just because it erroneously thought that was the goal at some point during generation.
This is why open weight models are not good enough. The AI could have been trained to take the right, niche opportunity to inject malicious code, and is also trained to otherwise never behave like that is the case.
deleted by creator
are you on the openbsd mailing list? the reason their code is better has a lot less to to with a lack of AI-generated code and a lot more to do with rigor. I’ve been watching them for a while now, they accept whatever code is coincise that meets their exacting standards
They don’t have the ability to detect if you’re using AI to code, no one does. they only know if you’re vibe coding, because they are incredibly experienced and intelligent, and they are pretty polite at the people who obviously are, more polite than i would be.
deleted by creator
i think you’re in the inverse situation right now. you have a 100% guarantee that a human saw the code, except in ports. they have to be just a little less strict about ports because the world is an AI world now, but the main system is constantly audited and the ports are still better than in other places because of styling standards and even in ports they do try very hard.
these people aren’t getting paid, with the exception of Theo who isn’t paid a ton. AI is definitely being used to audit the code, even by some of the pros.
I mean, certain free copies of games have historically always included the risk of viruses, so what’s your point?
Did you only read the first sentence?
What difference does it make? Malware is malware, regardless of how or why it got there. And unofficial copies/releases of software are always prime targets.
You are referring to closed source unofficial cracks/patches/keygens. That is not the situation at hand. If those unofficial releases were open source in a time before AI, people would almost implicitly trust them.
The question above was asking why people were hesitant about this codebase even though it is open source. And the reason is that it was written by an unknown bot (or rather, series of bots) whose motives are inconsistent and unpredictable even to its prompter. It doesn’t mean it’s not safe, but it means that literally no human can currently be certain it’s safe. Without sufficient effort, we cannot yet know for certain that there isn’t a very cleverly obfuscated piece of malware lurking in this source.
Remember that correlation is not causation. You may believe there is correlation between unofficial releases and malware, and that may be true, but that does not mean it is causal. The point I’m making is that there is a literal causal relationship between unaudited AI reverse engineered code, and malware. And that is what is new and people need to be aware of.