For sure, but just like that whole “poison our pictures” from artists thing, the people building these models (be it a company or researchers or even hobbyists) are going to start modifying the training process so that the AI model can recognize bad code. And that’s assuming it doesn’t already, I think without that capability from the getgo the current models would be a lot worse at what they generate than they are as is lmao