ls AI Code Actually BAD?
https://www.youtube.com/watch?v=s0oN6Tx5Xxo&t=285s
无作者
Is code written by AI actually bad or is that just something that people like to say? What, is up guys? My name is Aerno. Welcome back to Cherno talking about AI for the 10th time this, week. Jokes aside though, it is obviously a bit of a hot topic. It’s the largest shift in software, engineering since, well, I don’t know, software engineering. And if there’s one thing that I hate, more than anything else, it’s lies. Lies and misrepresentation. I like the truth. I like, finding the truth. and then making a video about it. So, here we are. And if you’re thinking, how, on earth are we going to answer such a question? Well, I don’t really know, but we’re going to work, it out. And just to be clear, this whole AI thing, am I a fan of the way that software engineering, and the world has taken a turn into artificial intelligence? No comment. But it’s just the way, that it is. And I think we have to accept these things. I mean, Lionus, the creator of Linux,
has accepted things as well. or is it Lenos? And actually, one thing that I like that he wrote was, that now, like this year, at this point in time, even one year ago, it wasn’t really anywhere near, what it is today. And I think that dramatic shift is something that’s worth realizing and I guess, adapting to. Anyway, I’ve made a ton of videos on AI. I’ll have them linked up there. Just like, flick through some of them that will probably give you a reasonable understanding of my thoughts.I don’t think they’ve changed too much since me making those videos. But today, how do we actually, go about discussing this question of whether or not AI code is is bad. The reality is that it, can be very very bad to the point that I’m like, what’s happening? But then on the other side, it, can be very very very good. So it seems like the secret source here is somehow getting AI to write, code that is good. And I think that at this point in time, at this point in history, the correlation.
between AI and quality code seems to also be kind of consistent with the quality of engineer. If I’m, reviewing two pull requests and they’ve been written by engineers of various skill levels,but predominantly by AI, you can still see that quality difference, right? which is interesting, because AI wrote the code and yet one one of the pull requests is like really good and definitely, no comment. I mean maybe there’s like some subtle things like code conventions or whatever that that, doesn’t really matter too much but the actual engineering behind it is solid versus the other, PR. I don’t know man like maybe we should rethink the approach to this entire task. And that’s sort, of what I’ve been saying, you know, this whole time as I’ve been making these AI videos, that, at this point in time, it’s definitely not good enough to just launch and have it make all the, decisions and just do all of the implementations with no human input. Since we are seeing such a.
large quality difference and the person using it is the kind of variable here that sort of confirms, the situation. I guess you could dumb it down and say that like does Visual Studio the IDE produce, bad code? Like, you know, that would be ridiculous to say, wouldn’t it? Because you would be like,well, it’s kind of about the person writing the code. Obviously, it’s just a tool that lets you, write code. Clearly, there’s a difference between an IDE and an AI agent. But you can kind of see, where I’m going with this, hopefully. Depending on what it is you’re working on and how complex that, project or task might be, you will need to roll up your sleeves and do some actual engineering.But again, context is everything. I’m sure there are a lot of things that you really don’t have to, be a programmer at all and you can just ask it to make you a little HTML website and everything will, be fine versus, you know, things that actually require real engineering. Again, sorry uh,
you know, web dev people. I don’t I don’t know if it’s worth apologizing, but I feel like I keep, accidentally dissing these people without meaning it. Again, sincerely accept my apology. So, since, my software engineering context is typically game engines, there’s actually a lot of considerations, when it comes to game engines and using AI to sort of help you make game engines. The first, thing is game engines are quite complicated and they involve lots and lots of different systems.These systems need to most of the time work together and not just work together but work, together well. That implies a lot of context. In the real world at like AAA grade scale, you would, have entire teams of engineers sort of working together to make sure that these two systems that, they are responsible for actually coexist and work really well. Because it’s not really just about, do they work together. That requires some design obviously, but not maybe a huge amount depending.
on what it is. It’s do they work together well? Are they efficient? Are they well optimized? What, is the data pipeline between them look like? Are they going to run well on a variety of different, hardware and platforms? This context and that sort of, you know, high-end state-of-the-art scale is, notoriously difficult for AI to deal with because, again, it’s a lot of context. It’s a lot of things, for it to, you know, consider and actually realize, but then also, it’s not exactly trained, on all of these proprietary, state-of-the-art, you know, bleeding edge technologies. It’s trained on, open-source GitHub repositories mostly. Does that mean that it produces bad code if you’re working, on a game engine? Well, I mean, yes and no. It certainly can, but I think that that’s the kind, of project where you just dial up the engineering involvement from a human engineer. You also need, to set like a bunch of quality standards and make sure that it understands how this engine works.
at a high level and how you would write code for it. And then things like security are also, very important to get right for a game engine, especially if it makes multiplayer games, which, might take users personal information, but then also if it’s a multiplayer game, it’s got that, connection to a player’s device. So, we certainly need to be sensitive with that and also prevent, cheaters. But again, humans also make mistakes and compromise security. We have various tools like, sanitizers and static analyzers to try and detect some of these and they certainly exist in the AI, world as well. One of the best ways to ensure that your project remains secure on all levels, is by using AURI by Endor Labs, the sponsor of this video. AI of course rapidly decreases development, time. Lots and lots of teams are shipping faster than ever before. But that can also come with, a significant downside. They could be shipping vulnerabilities faster than ever before. Did you.
know that only 10% of AI generated code is both functionally correct and secure? Not surprising, to be honest. You and I have both seen some of the code that it writes. AURI is one of the best ways to, prevent risk at scale without slowing you down. To write secure code, LLMs need high quality context.Endor Labs feeds tools and data into purpose-built agents that reason like engineers. AURI does it all, for you: catches vulnerabilities, blocks malicious packages, and actually writes fixes right inside, Claude Code, Cursor or VS Code. AURI is free for individual developers and super easy to try.Check out the link in the description and give it a go today. Thank you Endor Labs for sponsoring, this video. So, is AI code actually bad? Well, why don’t we just give it a go and have a look. I’ve, been talking about this game engine series that I wanted to make using AI. I thought that would be, a really good way to demonstrate me actually working on a somewhat serious semi-long-term.
project because that would actually show it like in the real world instead of some I don’t know, carefully crafted example that is probably very simple. But then I noticed a comment that someone, left on my most recent video. Then why hasn’t Hazel been released yet, mate? And like yeah,like I I guess I kind of see the point. I mean, the reason why Hazel still hasn’t been released, is because I’m not really working on Hazel. I’m working on Lucky Robots and Lucky Engine, which, is this thing, which by the way has actually been released. You can download it and give it a go.I haven’t made videos about it yet, but I will in the future. We did release like the first version, of that back in June, and it’s basically got like all of the features of Hazel in it. But yeah,that comment got me thinking that like, okay, maybe before I start this game engine series,why don’t I just release some form of Hazel because I’ve been talking about that for a while,
it would be a shame to kind of leave that unreleased. And also, realistically,the amount of work remaining to sort of make it releasable, at least in its current state,is not really that much. And especially with AI, it should be a piece of cake, right? So,basically, that’s the plan. Let’s finish and release Hazel and then we’ll start the new, game engine series project. And I like that plan for two reasons. First of all, we get Hazel out,which, you know, I kind of want to do, but then also from the video/AI point of view, we actually, use AI in a real project that’s not that small. If I start making a new game engine from scratch,that shows you how to start a project with AI and, you know, kind of use it at a smaller scale,at least initially. But here we’re actually diving into a real project, a a game engine, not a huge, game engine or anything, but like a game engine still. And then we can see what it’s actually like.
and we can see if the code that it writes is bad, is actually bad. And I think this is nice because, we finally get to actually do something instead of just talking about stuff. So first and foremost,before we dive right into this, because that’s exactly what we’re doing, let’s have a look at, the state of things like where is Hazel at the moment. So on hazeln.com if we go to millanote, board we can see the initial road map to release. This is what it was like before the whole lucky, robots thing happened and I’m now not building a game engine but I’m building a I don’t know, industry defining robotics simulator. So this was the road map to release. Let’s talk about, it. So render 2025 this is done. I think this was like the last thing that happened. If we look at, fork yeah merged from render 2025 into the dev branch. This is what happened all the way back in, January this year. So, Render 2025 has actually been done and it’s in dev. I guess we can Am I.
logged in here? Let’s uh copy this and wow, this is exciting. Look at this, guys. We are developing, Hazel. We’re actually doing some work. I can’t select this bloody orange thing, though. I guess, I actually have to redraw that. That’s annoying. This is going so well. Done. So, this is done.for Android 20.5 is done. Basic multiplayer. I believe this is done as well. So this is was, well this was in the net branch and I think the net branch was merged into dev. If I just look at, this quickly. So that was like when was this net merged dev into net? I mean 0x looks like he’s, done some stuff on this after I merged it to dev I guess because net has been in dev for ages. So,as far as I’m as far as I’m aware, this is basically done. We might want to look at some, of the stuff Zerox has done in case there’s some fixes there that I wasn’t aware of. But I’m pretty, sure that Genesis 3D was actually um cuz I was I noticed also that Genesis 3D had a bunch of stuff.
done by 0x as well. And some of it was like, you know, not even that long ago compared to like when, I last touched Genesis 3D. So, interestingly, um I think the reason for that is he’s just been using, it to sort of test out animation because this was like a good sort of skeletal animation project.But anyway, I think we we use this as our test case obviously for Net. And as far as I’m aware,Net has kind of been done, but it might need some refinement. We want to obviously make, sure that when we ship Hazel, you can make a multiplayer game with it. That was the whole, point of this. So, I think we will have to revisit this and make sure that it’s actually, done. This definitely is done because it’s literally in dev, but this might be in dev,but might not work and might have other fixes. Prefabs 2.0. Okay, so the thing with prefabs 2.0, is that so when I started working at Lucky Robots on Lucky Engine, the first engineer that I hired,
I was like, we got to fix this prefab system. Uh, and he did and he did a really good job at it. So,this is actually done inside Lucky Engine. And so, part of me is like, well, it’s not in a, great place inside Hazel at the moment. And yes, there are lots of bugs. Um, however, it’s kind of, already done in Lucky Engine. So, I I don’t know if I want to redo that work. I think I might just, like backport that to upstream cuz obviously Lucky Engine is forked from Hazel and my plan was sort, of to contribute back nonrootics features into Hazel. But we’ll see. Like the the way the prefabs, work work at the moment might be okay for this release so that we don’t get hung up with doing a, lot of work that might be redundant in the future and also has already been done. So it’s not that, fun. Now this is kind of fun. Like I would much rather work on the GPU particle system which we, don’t actually have in Lucky Engine yet either. Um and this kind of already exists. So this is.
actually interesting right because this already exists. I sort of made this outside of Hazel.uh just like it’s a like a pure C++ Valkcon project. Um and it works decently well. It’s, not like fully feature complete, but it it works and AI would probably have an absolute field day, simply reimplementing this inside Hazel because it’s got the whole reference code, right? It’s, not trying to just come up with a GPU particle system. It’s got one. It just needs to obviously, adapt that to Hazel and use Hazel’s rendering API instead of pure Vulcan. Uh there are some sort of, harder things to do with this. Like for example, the GPU parallel sort is using an AMD. It’s like, using one of AMD’s GPU sorting libraries, which is can be a bit of a pain to integrate. Um especially, with like Hazel’s opinionated sort of the way that it deals with like shaders and render, passes and stuff. So we’ll see. But um I don’t know, maybe maybe if everything’s going really.
well and we do everything else, we’re like, “Yeah, let’s just slap in a little GB particle system.”, But we’ll see because you can kind of depending on what kind of particles you want. Obviously,it’s nowhere near ideal, but you can actually just use C to spawn like a bunch of entities that, you then reuse. So, you have like a pool that you just reuse, like a thousand of them or whatever,and then you can just use Hazel’s instance rendering to get pretty good performance if you’re, just drawing quads or cubes or some kind of simple particles. Uh, obviously nowhere near as optimal, as a GPU particle system, but definitely possible. And you can just trigger that through C#. That’s, how we’ve done it for all of our past games. So, it’s definitely not like a vital feature. So,we’ll we’ll we’ll see how we go with that. Editor 2.0. So, it needs Yeah. Okay. So, this is like, one of those things that’s sort of half done in Lucky Engine. Uh we’ve definitely improved the.
undo and redo system a lot, but we haven’t really introduced this action system that I wanted to.Um we are going to probably do that soon in Lucky Engine. I don’t know if this is worth.Maybe we can just kind of clean up the undo redo system a little bit. Um, actually that’ll be an, interesting thing to sort of get AI to do and see like what it thinks and what kind of quality it, can produce here. Animation 2025. So, um, no idea where this is at cuz obviously Jim was working on, this inside Hazel. It kind of looks like he did everything except for IK. IK definitely exists, inside Lucky Engine cuz it’s kind of a a really important thing for robotics, but we don’t have, that in Hazel. I think that’s okay though. I think we just don’t don’t have that for now and that’s, fine. Asset workflow refinement. I think Hazel’s asset system probably fine. Um like I think that, uh this actually works. We haven’t really it’s not like we’ve rewritten it in Lucky Engine, but.
we have I guess just fixed some of the edge cases and like I don’t know hot reloading works really, well. I was just working in a scene in Blender and I could just reexport it and it was instantly, reloaded in Lucky Engine. So, that was really um good and I’m sure we could probably get Hazel to, that. I mean, it should already work in Hazel, but it just might be slightly broken, especially with, the new renderer. So, I’m sure we could get it to that place pretty easily before release if we need, to. So, we’ll just kind of revisit that a bit. And then C# API finalization. Yeah, I mean that’s just, kind of going over the C# API, making sure it’s sort of solid to so that we avoid API breaking, changes, but that’s fine. Um, QA and polish, that’s obviously happened for Lucky Engine. Uh,not here. Um, would probably be good to sort of introduce some testing as well because we don’t, have any automated testing inside Hazel at all and we do inside Lucky Engine and that obviously.
helps a lot. So, I might um kind of do that. But I didn’t do that initially for Hazel because I, sort of wanted to keep it more fun to work on and in kind of a smaller scale. But obviously if it’s, like, you know, going to be released and used by people uh and especially in the context of Lucky, Engine, you sort of need that. But also, I will say AI helps tremendously with testing because not, only can it just write a whole bunch of unit tests and other tests that you might, you know, kind of, be too lazy to write or not really want to write, it can also do automated testing for you. It can, come up with scenarios that it just tests and then it just keeps running that and it gives you a, report at the end that’s like these things failed and these are my recommended fixes. Like it’s, actually really good for testing. Should probably make a video about that as well. But yeah,so this isn’t actually too far off. I think let’s look at Render 2025 today. This video is probably.
dragging on, but we’ll see. If you guys enjoy this series and you want to see more of this,then I uh we’ll try and make this more often. I mean, it’s not I think realistically I should, be able to squeeze it into my schedule cuz it’s kind of it’s just a bit of like chill programming, really chill prompting. And then maybe next time we look at multiplayer, I’d like to like spin up, a server and see to what degree this works. That would be nice. Um and then yeah, the rest we’ll, worry about later. So I am on the latest commit of the dev branch. This is not expected to be stable.Um 2024.1 is like our latest release that is sort of stable, but dev is not stable. So I, actually don’t know what uh state this is in, but of course if you guys want access to this,you can go to patreon.com/thecho. Apart from getting access to the private Hazel repository,it does also help support the development of Hazel. Obviously my priorities are elsewhere.
at the moment. I’m focusing on Lucky Engine, Lucky Robots. So if I was you, I would I would not sign, up on Patreon if you’re actually expecting to have a wellsupported engine that you can use.But if you do want access to the code, you want to follow along with some of the stuff that we’re, doing here and you want to support me then of course you’re welcome to do that. I have over here, the yeah the dev branch of Hazel which I have not launched in a very long time. So I’ve I’ve given, it a build and actually while I was building cuz I did a build all I immediately noticed issues. So, um I don’t know if we like get AI to fix this stuff but um basically the actual engine and, Hazelnut compiles but there’s two errors. So, if you clone Hazel right now on dev branch, you will, get these errors. And these are they’re not like critical. You might think that the build failed,but it didn’t really because all the projects worked except for one, which is Hazel’s server.
And that’s just because as part of the new renderer, we also updated the Imui version.And a lot of things inside were deprecated and removed because I think we were on like a two or, threeyear-old version of Imui of DMGUI before renderer 2025 happened. So, I must have done, the updates elsewhere, but not for this. You know what? This this I wasn’t planning to start here,but this might actually be like a really easy first task for it. Okay, so let’s try and do, that. Now, let me quickly say that I have no idea how this is going to go because this like this is, the Hazel repository. You can see there’s no like Claude or anything here. Like I’ve never used, AI myself inside this repository ever. There’s nothing here that has anything to do with AI. This, is the first time I’m using AI on Hazel outside of like Lucky Robots, Lucky Engine, obviously. Okay,so if we open up uh a terminal, um man, this is a moment, eh? I’m defiling Hazel. Um let me quickly.
say that I’m going to use Claude code here just in the CLI. Uh, I am not being paid by Anthropic,unfortunately. I guess they have a lot of money. So, bad joke. Sorry. I don’t really care what you, use. Uh, to be honest, I’ve kind of I kind of want want to try Kurszer again because I haven’t used, it in like 6 months and I just want to give it a go. Kursa lets you use a bunch of different, models, which could be helpful. Codeex as well, I’ve never really used. And also, Kursza has a CLI, now, which is nice. I like CLI. Just a nice quick workflow. But anyway, um I don’t know. We’re just, going to use Claude. Um a lot of people actually have also been been very curious about the usage,like you know, show us your usage at the end of this. Uh the usage is fine, guys. Like I’m not, exceeding my like monthly allocation of usage on my account, like basically ever. Sometimes I’ll, hit the like 5 hour limit and I’ll have to wait, but like I’ve never exceeded the weekly limit.
This is my uh work account that I use for work as well. Um this is my current week. 2% used.Like is this the right account? Yeah. I mean like here are my stats, you know, to be transparent.I got this particular account I guess sometime in June. Longest session 26 days. Damn. 13.6 million, tokens. Longest streak 6 days. Most active day June 21. And then models. So yeah, a bit of Opus, 4.7. I guess a lot of Opus 4.8 and then yeah, I tried Fable for a little bit, but you can see I, kind of switched back to Opus 4.8 because I don’t know, I wasn’t having the biggest uh success with, Fable. Now, the thing is these models, they also take a long time to do things. That’s why I sort, of want to try Kurszer again. Maybe get some much much much faster iteration time on easier tasks on, like trivial tasks. But yeah, like I I’m happy to share my usage. But again, this is the same, account that I use for work. So, it’ll be mixed in with that. But considering I don’t exceed there,
and I actually am doing quite a bit of engineering at work. Uh, yeah, but this is the max account.This is the this is the 100 per month, account, I think, but then I upgraded because I did run out of tokens. So, this gets you, I think,5x usage. This gets you 20x usage. I might be wrong, but I think that’s what it is. And this has, been like pretty good for me and I’ve never had to top up. So, but again, I’m also not just letting, this run 24/7 as you saw from my usage. I’m using it as a tool to help me do things. So, for that, use case, it seems to be fine. Okay. So, um this is actually going to be interesting because I,uh, you know, the way I would I would normally do this, this is a brand new project. I just showed, you that like this is I mean it’s not it’s a brand new project for Claude. It’s an entire engine,but it knows nothing about this. I would be like, basically what I did in my previous video.
called how to use AI for programming. I would be like, how do I use you for this project? Like,let’s set up like a clawed MD file. Let’s set up some skills. Honestly, I think the first thing I, would do is tell it to do a deep analysis of this repository and then like document its findings for, reference. Um, and what you can do that’s quite useful as well is that you can, you know, you, don’t obviously want it to always have the entire context of this project loaded because that’s just, too much context and it might not need a lot of it. So you can do something like create an index, that has the important things that it might want to reference and then allow it to sort of read, other files that contain a lot more about that subject so that if it’s working on that it can go, to the file and expand its context when necessary rather than having to like you know grab this, entire thing and then deal with it all the time whereas it it might not need to. So that’s that’s.
probably what I would do, but let’s not do that. And you know, we’re trying to sort of evaluate the, quality of the code, right? So I’m just going to tell it that I’m getting this error. Um I guess, what I’ll do is I want it to know I mean I guess it’s clear that it’s Hazel server, but I was going, to say I’m going to select everything. I guess I’ll just I’ll be a I’ll be a beginner. I’ll, select the whole thing. Copy. And then I’m going to be like I’m getting this build error. Um, and, let’s see what happens. I’ve got auto mode on. Uh, which is fun. This error is a dear muy API change.Correct. It said item overlay was deprecated and removed. That’s what I said. Oh, I almost forgot.So, this is Opus 4.8 with X high effort. So, we’re not on like max effort or anything. It’s X high.I tend to use X high which you know probably not needed for this and it would be faster if I used, something a bit less intense but in terms of like usage and exceeding usage like this is what I use.
a lot of the time and then if this is failing or we’re going around in circles I’ll try and go up, to max codebase already migrated everywhere to set next item allow overlap. Yes, that’s true.I remember that I did that manually before before AI. Let me look at the direct Analog and hazelnuts, editor layer. So, it’s doing some discovery here. And then look at that. So, it’s basically just, changing that invisible. Okay. Yeah, it’s just adding the set next item overlap before it does, the actual invisible button. And as a result, it’s removed this set item allow overlap because it’s, already done it here. Both edits are done. Cool. And then look at this. It’s given me a nice uh, summary. So, server layer.cp CPP hit a dear immi API break set item allow overlap called after an, item was obsoleted in 1.89.7 and finally removed from the header in 1. Okay, so we’re on I think, 1.92.6. So I guess we just kind of caught this or maybe not. It says event is actually on this.
Okay, the rest of the codebase already migrated to set next item overlap which is called before cuz, you can see this is called after an item and this is now called before. So server layer was just the, uh the one that I neglected and then it made the same two changes that editor layer already has. So, it even referenced that and then also told me that there’s one uh duplicate as well and that launcher, layer over here has the identical obsolete set item allow overlap call. it wasn’t in your build, out but because it’s guided behind if menu bar which is just like me probably just disabling, that that code temporarily or something kind of like an if zero but I just like to actually write, something here that’s readable so it only breaks when that macro is enabled which it never would, be and the launcher project is built so this is cool right so apart from and if we go to fork I, guess and we look at the diff yeah uh it’s made this change so it’s just removed it from here.
and it’s added it here and I think it also added oh yes allow overlap which is what you have to do, and again I did do that in the past inside editor layer and the thing is like I mean obviously this, is like a very simple example I’m not trying to be like oh look AI is so wonderful but this is, exactly what I would have done like exactly like this is the this is what you do you have to set, the appropriate attribute the appropriate flag over here to allow overlap you remove it from, here cuz that function doesn’t even work anymore and And it’s supposed to be now done before the, button, not after the button. That’s it. So like this is the this is like the objective fix for, this. This is just what you need to do, right? So in this case, even though I didn’t tell it a, thing about anything, it still got this right. So yeah, so uh in conclusion with my extremely deep, testing, uh AI actually seems to write pretty damn good code. So don’t forget to like and subscribe.
Okay, so that’s great. Um, I guess we can commit that, right? We can just be like, uh, should we, fix the launcher? Nah, I think um I think that code that’s behind if menu bar will be stripped.And let me point out, this is another thing that I just would not have noticed ever. Like it actually, noticed uh code that was disabled from compilation that also was using this old pattern. That is that, is one thing I like about AI that you know it does miss things obviously, but you know, humans miss, things. I clearly would have missed this, but it didn’t. So, it’s just nice that it’s like, “Oh,by the way, there’s this thing you probably forgot about, and that could also be an issue.” Like,that’s really nice. Thank you, Claude. I guess before I commit it, though, I should test it. Um,I mean, I I was pretty confident in that fix because I I’ve seen it before. I made, the same fix inside editor layer, but um yeah, let’s give it a compile. Let’s give it a test.
Let’s see what happens. And actually, I guess uh this is the server. So, let’s um set the server, as a startup project and launch the server. We’ll do a lot more of this. Oh, it doesn’t work. Okay,that looks like a shader library issue. Can’t find a shader. This will probably be the working, directory is wrong. Why does it even I guess it needs the shaders. Okay, let’s have a quick look.Okay, so server app actually is trying to read a shader pack, which that’s probably the problem.Let’s have a look because that’s what you would have done in a disc build, but I don’t know if I, have a shader pack loaded and it’s probably better for us to just read the shaders raw. And we have, a break point here. Let’s skip past that and then Okay, so it’s still not working. It’s still trying, to load an asset pack though. I guess that’s how the server works. Like the server just has, to load a built project. So, we can’t really even run it because I don’t even know what it would be.
trying to load considering this is just a check out of dev and we haven’t really built a game.But you can see it’s trying to load server asset pack. So, maybe let’s leave that for now. We’ll, get back to that and maybe revive Genesis 3D in a multiplayer fashion with the server and the, client and everything when we have that like uh net episode of this new series. I guess a fixed, breaking in guey API change in server layer and we can commit and push. Check that out.I have committed and pushed to Hazel for the first time in like 7 months. Crazy. Okay, so let’s, try and launch the editor and see if that works. Okay, so here we have the editor. Um, I clicked on, something and it immediately crashed. So this is a good start. I think I saw some Vulcan validation, errors as well. Okay, so this is the editor. Um, if I right click to move the camera, it’s not, working. Uh, I guess I have to click first to hide to make the viewport focused and then right click,
which is like really annoying. So, that’s something that we can ask AI to fix. Otherwise,I guess it works. Let’s hit play. Okay, we are Oh, we just crashed in Jolt physics. Some sort of it’s, doing some SSC stuff without just some matrix multiplication. It looks like a MAT 4 times a, MAT 4 and then it’s it’s just crashing inside the SSC code. Okay, that’s that’s weird. I guess we, have some sort of invalid memory. Maybe actually don’t know what this is to be honest. Well, you, know what to do with that. So So let’s try and get AI to fix that. So okay, so how would I how would, I realistically deal with this? I guess I would just This is important, right? Because we have, this text here that says floating point invalid operation. So if I copy this and I’m just going to, say now I’m getting a runtime crash when playing my scene. This is interesting because like it has, no idea what any of this is. It’s going to have to do a lot of investigating because obviously it.
doesn’t know. I don’t even know if it knows this is a game engine. I guess it’s done like a little, bit of work, but like knows that like hazelnuts a thing and whatever, but like yeah, we don’t have, any context at all. It’s going to have to like discover that itself, which is cool. After running, for a few seconds, I get this crash. And then I’m going to paste in that. Did that not work?Copy. Paste. Maybe it is. It’s just not showing up for some reason. And then I’m going to also copy, the call stack. Now, this is happening on another thread. So, that’s one thing I noticed. Right. So,we’re on this like worker five threads. This is just like one one of the Jolt worker threads. So,if we go back to the main thread, the main thread is um just yeah, it’s just synchronizing with all, these worker threads that Jolt has dispatched in order to do the physics. So, that’s obviously, important to include. So, I’m going to probably go from this isn’t a big deal. I could just arr c and.
then v. So that worked. So can I not copy this or something? No, I can. That’s really weird. Yeah.Okay, we’ll just do that. So let me just quickly see. So getting around time uh crash after running, for a few seconds, we get an unhandle exception over here. So I actually don’t really I feel, like I’ve I feel like I remember this crash. I feel like I’ve seen it before, but I don’t, really remember what it is. It could actually be an invalid. Um, this is a physics blow up in Jolt,not an MUI issue. Yeah, correct. I probably should have done SLC clear, by the way, to just clear the, context because obviously it doesn’t really need the MUI stuff. The reason why I didn’t do that is, because this was like its only understanding of the engine so far. And I was like, yeah,maybe it saw some things that were sort of uh gave it at least some information on what this project, is versus just completely cold. So, yeah. Okay, so it’s saying that the crash is a nan in the body’s.
center of mass transform. Huh. Okay. Okay, I wonder how it got that nan. So it multiplies that, transform and the scalar just one by the shapes rotation translation matrix and the FP fault, means the transform already contains. Yeah. So we yeah we do need to find out what the what’s kind, of injecting the nan. So confirmed release defines floating point exceptions enabled only disc emits, it. So in your release build jolt unmasks the invalid operation FP exception at the top of, physics system update. That’s why an NAND turns into a hard fault instead of propagating silently.It’s a diagnostic that’s pointing you at the exact step where a body’s transform goes bad. Okay. So,uh after cooking for 3 minutes and 27 seconds, it I guess it’s not 100% sure,but it’s sort of trying to guess, right? So, I mean this is true. This is what we saw. But, then two independent facts combined to make it a hard crash. N Yeah. Yeah. You build on M floating.
point exception. Okay. A body with degenerate mass property sits quietly until a contact of, constraint actually exercises bad. Yep, that’s that’s definitely plausible where the nanora, certainly comes from a jolt shapes computes collide density with an unguarded division.Yeah. Okay. So two degenerate but easy to hit inputs each poison the mass properties. So if, we have a zero scale component obviously that will give you a nan because we are dividing by, volume over here to work out the density. Right? So that’s that’s just flipped the equation over, here or the mass is zero. So the scale is zero basically which makes the volume zero or the mass, is zero. Wonder what it is though. I wonder what in that scene is setting that though. Okay. So, basically the fix that that it wants us to do is just to guard the density mass computation at the, source. It’s interesting that it hasn’t just gone ahead and done it. It’s asking me first. It’s like.
it it knows that it’s that I’m making a video and it’s like I just want to I just want to make sure, either that or because it has so little context it just wants to make sure that it’s like it’s, very cautious. I don’t know. Basically the whole idea here is that we are setting a minimum volume, to a very very small number here and then we are warning if that is wrong and we’re resetting it to, the minimum so that it’s not zero. And we’re doing the same thing with mass over here. Right? So, we’re making sure that we don’t go below 0.1 for the mass. Interesting that it that it chose this, value for the minimum volume but this value for the minimum mass. And also it made a constant. In, fact, it made a constant expert out of the volume, but it just chucked in the mass. So, it’s a little, bit weird. See, this is like one of those things that I would be like, let’s talk about some code, conventions and maybe how you write code instead of just doing it randomly. But this is this is an.
interesting theory. So, what I would do here is um I think this is like a I I think this is a a, good solution. Like I I think that um if this is stuff that is set by the user, which it can be,and it’s not sanitized beforehand somewhere, if it reaches this stage and it gets used in these, equations, like it can obviously cause a crash. If it doesn’t cause a crash, it will it could cause, really weird physics behavior, which is also bad. However, um because I’m particularly interested, in what is causing this, one thing I could do is I could also grab the scene. So I think hazelnut, sandbox project assets scenes animation test right I think this is this is the scene file.So what in my scene could be causing this now um I could just say at animation test I guess that’s, easierh scene right I can just tag the file or you can just drag it in like that as you can see. So,let’s do that cuz I’m kind of interested if it’s going to be like, oh yeah, this entity in your.
scene has a scale of zero or has a mass of zero. But if there’s nothing in here that contains an, explicit zero, then maybe a default somewhere is zero or like, you know what I mean? Like I I could, just accept this fix and this fix would work and is probably a safe thing to have in the engine.But you can see that I’m sort of being a bit more curious about this because I suspect that maybe, something else is wrong. Like we we can’t just be getting the zero from thin air. Like where is that, zero volume or zero mass actually coming from? The other path I could have taken is just completely, doubting AI and just being like, well actually maybe this has nothing to do with the mass of the, volume. It might be wrong, but this is obviously very plausible. So I’m entertaining this because, this yeah could well be the issue. Okay. Okay, so see this is what I’m talking about. No collider, has a zero or tiny scale axis that rules out the degenerate volume theory for this scene. This is.
why it’s important to investigate and use your brain. Okay, so this is the smoking gun, no pun, intended, for a shooter scene. The bullet prefab spawns two dynamic bodies and they have exactly, the profile that generates ns. Now, I am pretty sure I did not. I mean, I definitely didn’t fire, my gun, but also in that scene, you have to first get ammo before you can fire. So, I don’t think, any bullet prefabs would have been instantiated at the time of this crash occurring. So, I think, this is wrong. Okay, so it audited every physics active entity. So, that took 6 minutes, a decent, amount of time. The prime suspect is the bullet prefab. I’m pretty sure uh you’re wrong though,buddy. I didn’t shoot, just walked forward and it crashed after a few seconds. Also, to shoot,I think I first have to pick up ammo, which I did not. I think if I was actually working on this,I probably would have given it a little bit more information, such as, okay, if you’re.
sure it’s not the scene, we need to trace the actual physics API implementation or something.Maybe we’re injecting some sort of default values that are set to zero for objects that don’t define, certain parameters and we shouldn’t be. We should be setting something a bit more sensible. But, let’s see what it decides to do next based off of this. While it’s thinking though, I do also, want to point out that like realistically um you know, if I was working on this,I I guess I would be curious enough depending on if there was a deadline to fix this bug,but I would be probably curious enough to try and find the actual source of this happening. Maybe I, should try to debug this at the same time as as Claude. But the fix that it suggested is like, this is obviously just like a defensive patch that will prevent this crash probably from from taking, place. Th this is also definitely a viable thing that a human engineer would probably do. Like the.
title of this video is is AI code actually bad? And is this bad code? No. I I I I guess like again, this this weird magic number constant and then an actual const expert for the volume but not, for the mass. Like that’s a bit weird. I would ask it to fix that up. Uh also definitely something, that could come up in a code review if someone wrote this. I’d just be like yeah, you know,maybe make that a constant or something or even set it to like the same the same the same value.I don’t know why they have to be different values cuz again we’re just trying to prevent a zero here, really. But yeah. Okay. So that rules out bullets entirely. And you can see that now it’s verifying, how Hazel handles the many colliders that have no rigid body components. Kind of digging into the, actual sort of physics implementation, which is good. Okay, it reckons it’s one of exactly three, bodies. So, the crash stack only touches active and dynamic bodies. Your scene has exactly three,
and all three start airborne or embedded and settle the moment you hit play. the pistol and, two ammo boxes ranked by how likely each is to blow up on settling. This is f why. Okay,I think this needs some steering. So yeah, so so it reckons that definitely um a specific object, is like causing the crash, which which makes sense. Um but I I don’t know. It’s like it’s, recommending what to implement two safety nets or fix the actual content which it it again is just, guessing that it’s pistol and the point is to out of the floor so nothing spawns into penetrating, that shouldn’t cause a crash though. So I’m going to do some steering. Can you instead add some, logging so we can determine which entity causes the crash? So, this will probably be a little bit, better because it it did kind of suggest that I do that myself because it’s saying that this get, user data is the entity UU ID and so I can read the culprit’s identity at the crash. But I don’t.
really want to do that via the debugger cuz I probably have to do a debug build so I can inspect, all those variables and not have them be optimized away. So adding some logging is just going to be, faster and then that way obviously I can just feed the log back to it and it can have a look. Good, call. Logging that survives a crash is the right move. Okay. So, it’s getting a lot darker uh here,but but um it’s added quite a quite a lot of sort of diagnostic logging. Like, it’s written quite, a lot of code here. We can do a little bit of a code review in a minute. It doesn’t really matter, though. This is just diagnostic code. So, it’s not going to really stay here. But you can see it, is struggling um cuz it’s trying to it’s saying there are like clanging errors and there’s all, cuz like none of this is set up for it to sort of loop over. So, if we actually were setting, this up as a as a project for Claude to work with or for agents to work with, we would want to set.
up tools for it to be able to, you know, try and build and potentially even run the project to see,you know, what’s going on and if it’s actually getting things right because we don’t want to have, to build it ourselves and then paste in a build error and then have it fix that. Like, it should, be able to to do that so that we can work on other things. So, I was complaining about like analyzer, noise and all this stuff that that doesn’t really matter. So, I might even just stop it right here, or just tell it not to um you know, not to try and build anything because we haven’t given it the, the tools to be able to actually do that properly. But the code itself, so it’s a pre-simulation body, sanity scan is what it came up with because it’s a little bit hard to to determine in the actual, uh you know crash site what the entity is by that point in time. So, it’s got a couple static bulls, up here. Where where is this? Like what is the context of this? It’s in jolt scene.cpp. Yeah.
So, it’s it’s just doing like a scan over like literally all of the active bodies that it has,which we get over here. It’s got a lock here so that it can read them properly. And then it’s just, yeah, checking to see if they’re valid basically. I wonder if this will turn up anything. Okay,so it’s telling me to rebuild and then reproduce the crash. All right, let’s do it. Let’s see what, happens. Okay, so let’s hit play. And yeah, see, I don’t even have to walk. It just crashes. So,let’s see in that log. Okay. Did I Oh, okay. Here we go. Um, is non-finite and will crash the next, physics step. Okay, so I guess that worked. Look at all this stuff, though. This would be, like every frame as well, so it’s probably just full of that. Okay, so at this point, I guess I, just stop. Um, I won’t worry about pasting the log because I’ll just say like I was suggesting, um what to write here. But is interact is the kinematic body interact. Okay. I think that’s.
like some trigger that allows me to interact with things that is kind of attached to the player. So,as I walk around, if that trigger goes over a gun, I’m allowed to pick it up. I don’t know. Jim, put this scene together, so I’m not sure exactly how. But brash reproduced read log. Uh, you know,don’t have to really paste anything in. I could have pasted that in. Maybe that would have been a, little bit faster, but we’re going to we’re going to pick the path of least resistance here and have, it look at all of the logs so we can see. Okay, cool. So, it’s unambiguous. It’s not the props, themselves. It’s two kinematic interact trigger spheres so that they’re on the props. I thought, they were on the player, but they’re actually on the actual uh two objects that you can interact, with, which is the ammo box and the pistol. So, their velocities explode exponentially within, a second. Wow. Until they go non-finite and fault jolt. Oh, okay. That’s interesting. So,
it’s I guess I guess that Okay, this is actually very interesting because I guess this wouldn’t, have fixed that because it thought that it was I mean it it could have also been the cause of it, though. But if it’s if it’s something going wrong with their their velocities and they’re becoming, non-finite. Non-finite. What does non-finite mean? Infinite. Non-finite. That’s weird. This, is actually uh you know coincidentally becoming hopefully a decent example that’s showing you, why you kind of still need to think about things and not just like slap in this fix. Again, this, may not have even fixed it, but like why you need to actually maybe do some investigation and, you, know, think about things and use Claude as like a tool to to do all of this debugging, which I, guess I could have done manually, but it probably would have taken me longer to be honest. This is, taking quite a while. like these things are like I don’t know last time it was like 6 minutes or.
something. This this is taking me you can probably see how dark it’s getting around me. Um you know, it’s it’s it’s the evening now so this is taking a bit of time. I don’t know if I could have done, it faster potentially but I’m using it as a tool and it’s obviously uh you know it kind of helps to, almost rubber duck debug this as well but it does think that it’s sort of found this. So let’s see, what it decides to come up with. I really want to try some of the faster models as well. Like I know, there’s a new version of Grock that’s supposed to be like Grock 4.5 that’s supposed to be like super, fast and that’s like I guess what SpaceX have been using. Okay. So what does this say? Root, cause an engine bug not bad scene data. So each frame for a kinematic body Hazel does velocity, equals target minus current divided by TS. And TS is wall clock frame time. So that’s real world, time for how much time has passed between last frame and this frame. It’s our time step. It’s our.
delta time. So then jolt integrates that velocity for this which uses the fixed time step not ts.So the body actually moves that amount. Okay. So physics should be using its own fixed time, step because I mean it is using it in most places but we were still passing in the time step which, happened to be faster than 60 Hz because we were just running faster than 60 Hz. VSync probably, wasn’t wasn’t on even if it was like this is a 144 Hz monitor. The point is we shouldn’t be tying, uh time step here which is our actual frame delta time. We should be dealing with like this fixed, time step. So it’s got collision steps times fixed time step and that’s our integration time, that it’s passing in to move kinematic bodies and that’s why it was specifically targeting kinematic, bodies. So see this is probably the real fix. This this is definitely a lot more plausible because, it actually found a bug in the engine. So yes, let’s go ahead and implement that and we’ll see.
if that fixes it. I will say though like while it’s implementing this fix um one thing that is, a bit weird is that like so obviously this scene used to run fine uh and then we just changed the, renderer to like render 2025 um and that’s what I merged into dev and I I feel like since then it, hasn’t been working. So basically what I’m getting at is why why is it now a problem? Is it because, I mean one theory could be the new renderer is so much faster that it’s running this scene is just, running faster now and that is what’s causing it to crash because we were using time step instead, of fixed time step. That’s a theory but yeah I mean if we have a look at four quickly just to see, what it did. Um, so it’s added this thing. It’s added this thing. Set linear velocity clamped.Clamp. I wonder if we can just do part of this to see if it was that. There’s that diagnostic that, we can revert. And then this is the actual meat of it. So this is now using integration time instead.
of time step, which is what it was talking about. And that of course is our collision steps times, our fixed time step. It’s added some fat comments to explain everything. It’s kept the note from 0x, here. And yeah, that’s that’s all that’s done. That was my thing earlier. And then yeah, this, is just diagnostics. So we look at the context of this. This is inside move kinematic. So it just, makes sure that delta seconds that are coming in is move kinematic what we call from here. Yeah. So, body move kinematic comes in now with integration time. So that means that this parameter over here, float delta seconds is that fixed time step now our integration time. But then it’s still, guarding it to make sure that if it’s less than or equal to zero, we return. Wonder why. Okay,this is just to avoid a divide by zero. It seems not that it really would be zero, I guess. I mean,since it’s not the fixed time step, it’s collision steps. What if collision steps are zero? Oh, well,
yeah, we we guard against that here. Oh, that’s new. Yeah, see like that that kind of makes it, unless your time step is zero, you’re not going to get a zero there. But it’s just being extra safe,which is fine. Okay, let’s let’s launch this and see if that has indeed fixed the bug. But yeah,if I had to do a quick code review over this code, like it’s I don’t know. It’s it’s fine, to be honest. There’s a bit of a um a weird like I guess it’s tried to keep stuff consistent maybe,but maybe not. Like I just noticed, for example, here it used a static cast, but then here it, didn’t. It just used a C style cast. And I’m not sure if it did this because the code around it was, C style, but then here maybe it knew that this is temporary code, so it doesn’t really matter, how it’s written and it just preferred to use a static cast. I don’t know. But if we hit play,I guess we’re not crashing. Look at that. Let’s play this whole scene and see if we.
can still interact with like the objects and stuff. I mean, we would we would be able to.All it’s done is changed the uh the the time step to be correct. So, E to pick up. I can’t, pick this up anymore. Oh, I can’t pick up my gun. Okay, so maybe it is actually broken. Oh,there’s a second ammo box over there. I only ever saw this one. That’s weird. Okay, so I can’t pick, up this object, so that’s a bit weird. But it is not crashing anymore. Um I wonder if Oh boy. Okay,let’s You guys can’t see what I’m doing, but I’m just Oh, I’m causing a crash. It turns out that, crash was probably because uh this Oh, no. It’s just resizing crashes. I didn’t even open up. I, thought it was like the Muewi viewports thing, but maybe not. Okay, there’s going to be a few bugs, to to fix here, I guess. Have a look at physics colliders. I’m just Oh, does that not work? Uh all, yeah, here we go. So yeah, so it’s this sphere. So if I just hit simulate. Yeah. So it’s it’s here.
You can see it kind of goes down, but I think this sphere is uh interact. There it is. Rigid, body. And there’s a sphere collider is this thing. You can see I can make it bigger and smaller. And, control Z just keeps Okay. Yep. So if I hit stop for a second. So, why am I not able to? Cuz that, that seems like it should work if it’s in that spot. Oh, and actually, it brings up the text. Oh,there we go. I can pick it up now. So, why was it not working? Why was it not working before?That’s really weird. So, I’ve picked it up. Let me do physics colliders. Yeah, it’s definitely there.So, I’m definitely in there. And and to be honest, what’s weird as well, this scene might just be, broken. I think the engine bug is fixed because the the press E to pick up is is showing up,right? So, the fact that it says press E to pick up, I’m pretty sure that only appears, you can, see it over here, the text that I’m pretty sure only appears if you’re nearby. So, we we must be.
colliding. Let me just stop this for a sec. So, if we go over here again. Yeah, the text. Yeah. So,the fact that I can’t pick it up anymore and I could just then I think might actually not be the, colliders. It might be something is wrong with the scene or I don’t know something has changed. So,we can probably debug that next time. But the crash is obviously fixed. Um,that’s the important thing. There’s so many other issues though with this with this engine. So,it’s going to be it’s going to be tough to ship this thing. But anyway, it is now dark. So, let’s, wrap up this video. Hope you guys enjoyed. Let me know what you thought in the comments section, below. Don’t forget to hit the like button if you want to see more content like this and if you want, me to keep fixing Hazel and ship it. I honestly didn’t really know how this video would turn, out in terms of is AI going to write bad code or good code. That’s the thing. I sort of wanted to.
be realistic. I didn’t want to script this video and then have it do exactly what I wanted it to, do and then try and make some kind of point. I’m not really here to make a point. I’m here to just, share, I guess, my life and my experiences with you without some sort of bias. Hopefully, I mean,of course, there are going to be implicit biases. We all have them. We just have opinions. We’re, we’re human. But from this example, hopefully it was clear that it’s definitely capable of sort, of doing something and it’s capable of reasoning about this engine. Obviously, it fixed the imge, gooey bug very, very quickly whilst knowing nothing about any of this technology or this, codebase. But then with something a lot more complicated, it took a while and it really, required me sort of guiding it and even helping it trying to, you know, add diagnostics so it could, then read them again and work out what to do. So yeah, your mileage may vary and you still have.
to think about how to do things and give a lot of instruction to it in some cases, especially, if you’re working on more complicated things. That’s to be expected, but it is a tool and it, successfully helped us fix this problem obviously and I think it was good. I appreciated having the, tool versus not having the tool. Thank you guys for watching. I will see you next time. Goodbye.
--- 由 vCaptions 生成 ---