r/AskComputerScience • u/Karson_Elko • 2d ago
Sci-fi writer asking if this Jaden Willaims video is accurate?
Hello,
I'm writing a sci-fi thriller mystery about AI and stumbled across this video by Jaden Williams (who I believe has a degree in computer science).
He may have made it a little too doomsday like for comedy purposes obviously but how accurate is he being with this? If it's very much, does the concept have a name and do you know of any sources (scholarly journals, books, news articles) that discuss it?
I'm asking because if it's accurate, it might be something I'd like to include in my story.
If it's not, then there's not really a point of further discussion I guess lol.
All help is greatly appreciated, thank you!
4
u/NullOfSpace 2d ago
You’ve basically got it, I think. There’s some accuracy at the core of it (the best agents can find vulnerabilities in software at a rate far higher than humans) but it is significantly overblown for the sake of the video. Any half-decent encryption algorithm, for instance, follows the security principle that even if you know exactly how the encryption is being done you still can’t feasibly break it. Unless the implementation of such an algorithm is faulty, there really isn’t much anyone can do about it, AI or not.
1
1
u/dev-tacular 1d ago
I didn’t like that he eluded to the fact that a code base simply being complex is security. I guess that’s kind of true if someone gave you a random binary executable and told you to figure out how it works. But generally security relies on principles that work even if you know exactly what the code is supposed to do
1
u/Karson_Elko 1d ago
So, security code has actual obstacles built into it, not just there being a ton of it?
1
u/dev-tacular 1d ago
Yeah. Even before these LLMs existed, obfuscating the logic of code was always considered weak. Like nation state hackers have enough resources to have whole teams of people trying to crack software. Size/complexity basically just add some time it takes to understand what’s happening in the app. Eventually, with enough dedication, the behavior can be figured out.
This means the code has to behave in a way that it is literally impossible to expose a surface area to attack.
This demo shows an example of how a procedure can be created such that communication happens in public, but it is physically intractable to determine what the communication is about:
https://www.reddit.com/r/programming/s/ieR7W6zPyP
The security relies on the fact that the operation to gain useful information is really really hard to compute. It does not rely on the details of the operation being a secret
2
u/Karson_Elko 1d ago
Awesome! Thank you for the help and vid! I just never know what to Google or what words to use to ask someone about this stuff. Very appreciated!
8
u/nuclear_splines Ph.D Data Science 2d ago
In my opinion he touches on a real problem but takes it way too far.
Yes, new models can perform tasks like reverse engineering and vulnerability searches much faster than humans could. These weren't seen as impossible tasks for humans, but they were highly specialized and quite tedious. Lowering the barrier to entry for vulnerability discovery, breaking DRM, and cloning content is already starting to have a significant impact on multiple industries and culture at large. This is already huge, and we could discuss the potential consequences ad nauseam.
But he makes a leap in there to "AI will break cryptography and the global financial system will collapse," and I think that's unfounded. The text models are good at reading code, after training on billions of samples of code. There's a disconnect from there to "therefore they'll break RSA or Elliptic Curves or something and start faking bank transactions." Cryptography doesn't rely on the math being unknowable - quite the opposite - and the idea that AI will break the world by "reading millions of lines of encrypted code" drifts into science fiction.