Podcast: Creating 3D Objects With AI

In this episode, we talk about the initiative from PolySpectra to generate 3D CAD models from text using AI.

EPISODE NOTES

(0:40) - Teaching AI CAD Is Hard

Check out nething.xyz to try out the tool we discussed in today’s episode!

Transcript

What's going on, folks? Welcome back to the Next Byte podcast. And in this one, we're talking about how you can use AI to create literally any object that you want in the blink of an eye. If that's got you excited, then buckle up and let's get into it.

I'm Daniel, and I'm Farbod. And this is the NextByte Podcast. Every week, we explore interesting and impactful tech and engineering content from Wevolver.com and deliver it to you in bite sized episodes that are easy to understand, regardless of your background. 

Farbod: All right, people, as you heard, we're bringing together the worlds of AI and CAD in this episode, which is really like special for us because, you know, CAD, mechanical engineering, we study mechanical engineering. We love CAD-ing in our free time and AI, we've been having a lot of fun using AI tools in the past year or so. So, bringing these two worlds together is pretty interesting.

Daniel: Well, and before we jump too far into it, I think we should define CAD in case people don't know what it is, right? Computer Aided Drawing. It's basically the way by which the vast majority of engineers designing things in the physical realm, right? So, think even electronics engineers, mechanical engineers, et cetera. They use a computer to help them with their design drawings. And in many cases, it's not just creating a 2d print that you send to a supplier and helps you create, oftentimes you're trying to create a 3d model that's representative of the thing you want to design, create, you can do simulations in CAD, it's pretty awesome, but it's pretty fundamental basis for a lot of physical engineering is designing and creating these 3D models on a computer.

Farbod: For sure. And it's even more general than that because you have, like you said, if you look around your house right now, the items like computers, TVs, some CAD software was used to develop those. That's an easy one. But then when you think about video games, like creating some of those components, they also use some sort of computer aided design tool to do that and accomplish that as well. Traditionally, CAD tools require, like there's a learning curve, whether you're using it for engineering, there's, I would say like what, three or four main programs, some of them great for big assemblies for like making cars, others great for hobbyists, et cetera, et cetera. And then on the design side for like the artistic creations, they have their own tools as well. So, you have to kind of develop that skillset, that expertise for all these different tools, which means it's not very beginner friendly. Like there's a barrier to entry there. So, what happened here is that these folks at PolySpectra, well if I'm not mistaken, their expertise is actually in resin for additive manufacturing.

Daniel: Yeah. They make materials for 3D printing.

Farbod: Right. They were like, huh, we have AI, and AI is pretty good at understanding like just raw text. And then there's these tools that allow us to generate code that creates CAD models. There's nothing out there right now that brings these two worlds together, even though there's a lot of AI tools for generating images and generating minute long videos, shout out to open AI doing that recently. So, they were like, what if, what if we could like kind of crack this code and make that a possibility?

Daniel: Well, and it's like, if you're someone who's inventing something or designing something, you have a big idea. Let's say you don't know CAD or you don't want to spend all this time going through all the intricacies of creating this base model that maybe you do some fine tuning and then you're ready to go. It's relatively easy right now in the space of video generation, image generation, text generation to go to something like open AI and use one of their tools that are open source and, you know, pretty widely accessible. One of the fastest growing consumer applications ever. You can go in there and type like, oh, I want to make, I don't know, a lamp with a dog on top of it. You can type that into Open AI's, chatGPT or their DALL-E tool or their Sora tool. It'll create a video about it. It'll create pictures of it. It'll create texts to help you sell it. But when you actually want to create and do the 3d model that you need to help engineer and manufacture and design something, there's nothing in the world that helps bridge that gap yet. And that's where this team from PolySpectra came in and said, hey, let's make this thing that can help you design anything, which is cool. Cause it's the name of the tech, the name of the tool is called neThing. How do you design anything?

Farbod: That XYZ, the three dimensions.

Daniel: Yeah, come on, man. You just start with text, and you can describe what you want, and then it helps translate it into code that a machine, computers can understand, which then is easily translatable into a 3D model.

Farbod: Yeah, yeah, and like I kind of said at the beginning, there's really no one else that's doing anything quite like this, like raw text, straight up CAD design. But there are other, there's competition out there that's doing text to image to then a 2D or 3D. Now, apparently this just does not work quite that well, which is why these folks wanted to try this idea. And yeah, what they've done, like you said, is that they've built this AI model, which is able to take that raw text and generate that output design that you want. Now, this is one of those rare topics that we talk about where you can actually, like as our listener, you can go to this article and you can access the link where you can work with this model. Try it out for free. We did it. Both of us use this product, because again, we're interested in CAD and we're doing the research for the episode. But it's rare that I feel like you get to experience one of the products that we're talking about on this podcast. But I would definitely urge you guys to try it out, because it is pretty interesting. And I have some feedback. So, I will say, before I get into my feedback, these folks have kind of highlighted some of the challenges for building an AI model like this. They made a point of saying, hey, look, the AI is kind of blind. It has no context. It can't visualize what you want to make from just the description. Yeah. Right? It needs to turn that into these commands. There's that layer in between. And by the way, on top of that, to get an AI model, you need data to train it on. And it's really difficult, because this code to CAD design, that's kind of like a niche thing, where there's only a couple experts that are working on that don't mean there's not examples to train on.

Daniel: There’s not billions and billions of web articles out there for this AI to consume and train and learn on. That's part of the large part of large language models, that requires a lot of training material to be proficient at it.

Farbod: The large is kind of the main driver there.

Daniel: There's not a large amount of data for this AI engine to kind of consume and train itself and understand.

Farbod: And then the last thing that they put in there, which is personally I don't think it's that big of a problem, at least not at this stage, is that there's no one right way to make a design in CAD. So, there's like CAD competitions that happen every year, specifically on the engineering design software, like the Autodesk Design, SolidWorks, whatever, where they show you a picture of a thing, and then you and I, for example, would try to make it as fast as possible, as accurately as possible.

Daniel: And I took a CAD class in college, which was similar, like we'd all get a physical assembly to go measure on the table. We could take a bunch of pictures of it, take our calipers, take a bunch of measurements. And then it's like, all right, break, go to your desk and start designing this in CAD. That was interesting. And like you said, there was no one right way to do it. A number of people could arrive at the correct answer, right? Which is an appropriately representative 3d model on the computer that matches the physical object that we were looking at and I could have started by. If we're designing this teacup on the table, I could have started by designing the cup part and you could have started by designing the handle. And maybe I did a sketch of the 2D profile of that cup, then I extruded it around in a cylinder, and maybe you started by creating the cylinder and then put it in a hole in it. You extruded out the inside, right? There's a number of different steps to get to the correct answer. And maybe some of them are less computationally intensive, maybe some are less memory intensive, but it's all basically based off of the intuition of the designer. There's no correct procedure for AIs to start to really master the correct steps on modeling something.

Farbod: For sure. So those are like the challenges that they're already facing, which is going to translate to issues on the outcome that you're going to get from this AI, but what something that stood out to me as a benefit of the approach that they've taken is that the backend for the generation of the object. There's like three different models you can use to generate these objects, but the best one, from I think both of our experiences.

Daniel: We were like, oh, code 123D.

Farbod: Yeah, build123D is the best one. And build123D is an open-source Python library made by enthusiasts, like just out in the world, where you can write lines of Python code, which by the way, Python is one of the, I think most widely used programming language in the world at this point. Very easy to pick up on. That's what's like when the AI, when you say make a teacup, for example, it then goes to build 123D's like, okay, so I want you to draw a line with this command, make a cylinder with this command, so on and so forth. So that's what's doing the magic on the backend.

Daniel: And a lot of the challenges we talked about, right, are about trying to bridge the gap between large language models, which are good at language, and then into 3D space, right? Bridging that gap is really challenging. What Ray and PolySpectra did here is they're like, all right, let's use this library that's already widely known, already really reputable, already used by a lot of people out there. That covers the part, it takes code and then it generates it into 3d text. All you're trying to do is then, the core innovation here, the secret sauce is training AI on generating code in that language. And I say language very specifically because LLMs, large language models are really, really good at generating language from language. So, you take written English language or written Spanish language or, pick, pick your language of choice, but take, take our human communicative language. LLMs are getting really, really good at doing code generation, turning that into a language that computers can understand. So really that's where they focused here is they've trained the AI to generate code that translates directly into a model like build 123D that is easily generatable into a 3D CAD model.

Farbod: For sure. And now I want to jump into a little bit about my experiences of using this tool. Yeah. So, some things I noticed is that it tends to do pretty well with specific, like very specific step-by-step instruction. So, something I tried out was for example, create a four by four-by-four cube, did that really well and I was like, okay, now I want you to put a chamfer on the edges, chamfers like a flat edge on all sides. At first it struggled, it crashed, but then it was able to like recover and do it. And then I was able to ask for like some more slight modifications. However, if I set something like make around a cube, or even in the article, there was a, they were like, this was the output we saw from making a pipe with flanges. Even that input, it struggled with, it kept loading, and then it crashed on me. And I think even this is expected, because at the end of this article, I forgot who wrote it, but whoever's writing it is like, hey, we know that this is kind of in a rough state, and we're actually like hoping you guys will use this tool. And the data that it gets from all these people's inputs will be used to train it and provide that missing data that we need to make this like this reliable, robust tool that can actually be used to take someone that has no idea how CAD works and allow them to create these 3D objects to launch a business or launch their creative endeavor.

Daniel: Well, and let me just say, you said, you don't know who wrote it. It was Ray. He's the CEO of PolySpectra.

Farbod: Gotcha.

Daniel: Wrote this article. He actually reached out to me on Twitter and he's like, yo, check this out. Is this tool cool? And that's when I texted you, I was like, dude for both, we got to cover this. And then he's like, oh, by the way, I wrote an article about it on Wevolver. I was like, cool, we can absolutely cover it on the podcast.

Farbod: Well, Ray, if you're listening, one thing I like to call out whenever it's there, the formatting of this article was great. The flow of information, chef's kiss.

Daniel: Yeah, it makes it really, really easy for us to communicate technology well when the article that we're basing it off of is really, really good at communicating technology well.

Farbod: For sure.

Daniel: I will say, Ray deserves a shout out here. He's the founder of PolySpectra and CEO. Like we said, they make high performance resins for 3D printing. He's been working at the intersection of material science and additive manufacturing for a while. And really one of the core mainstays of additive manufacturing is doing 3D CAD design like this. So, he's an expert here. He also was on the Forbes 30 under 30 list. Nice. So, a little bit of a killer, uh, love it Ray, but I obviously appreciate you reaching out to us on Twitter, being open to feedback about this. He's been super open, like saying, hey, show me where it breaks. Show me where it works, works well. Show me where it doesn't. And it kind of adopts this mentality you were saying where community involvement is one of the most significant parts of this. He knows that it's early. He knows that it's just a fledgling model starting to learn how to walk. But if our contributions help to refine and improve the AI's capabilities, this thing could become super dangerous in the future where it's like when you type in pipe with flange, it loads a hundred percent of the time and you can start modifying that into something more complex, like I want a liquid natural gas pipeline with 12 holes like one of the standard designs in an industry. And the reason I say that is a lot of engineers at really, really well-established companies, I'm thinking of like big three automotive OEMs, when they're going to design something new, they have a catalog that they can pull from. For sure. When they say, oh, I want to design a seat, they just go through their catalog and they can pull the previous seat and then they just make some minor modifications on that. That's what makes it really, really easy and really, really fun for those people to do the engineering part, because they don't have to start from scratch. I'd like to think that this tool could democratize that headstart to everyone trying to do design, whether that's professional engineers, designers, artists, students, people who are just trying it for the first time for fun, making it more approachable to where they can literally just type in an idea of what their 3d model is and basically get that headstart that a lot of professional engineers at these well-established companies get, kind of removing the barrier to entry to being able to do successful CAD.

Farbod: So, I agree with you there. And I want to extend that even a little bit further. One of the things I remember my first time using, I think solid works. One of the things I loved was the hole wizard, like making a hole seems like such a trivial thing, but oh my God, is it challenging?

Daniel: Well, especially with threads.

Farbod: Exactly. With threads. And it's, it's just such a pain to like go from scratch, make that, but this wizard is like, Oh, Oh, you want to make this hole? What's the entrance? What kind of thread? Like everything. And it just makes it for you. Right? And as an engineer, you get to focus on, like you said, the fun parts, like creating the rest of the components that you need in the assembly and making sure everything's working right. So, when I was experiencing this AI model, the only thing that came to my mind was like, once this reaches a point where it's mature enough to give me just, not even like full-on assembly, just one component the way I wanted to, like let's say 99% of the time, this would be a killer plugin. Where instead of a whole wizard, it's just part wizard. And I can go in and say, give me a two by four that's like, you know, to this cut with these angles or whatever. Or like give me a bowl that's like this. That would be so cool and so applicable across the board. Like it wouldn't even have to be program native. So that's what came to my mind. And honestly, I think my mind might've just kind of gone to that because Visual Studio Code has copilot for programmers where you get to write a comment that says spit out the code to parse this text for this thing and it just gives you that block of code. But as the programmer, you have full control over the rest of the program.

Daniel: Well, and I'll say like personally, as someone who doesn't have a lot of coding experience, being able to use a tool like that, or I sometimes type my queries in the chat, it makes it less intimidating. It makes it more attainable for me to make something that works in the real world. I like in this to like another intellectual revolution maybe where more people are gonna be able to have all sorts of technology and all sorts of tools available to help them turn their ideas into real life. Obviously, this is probably many, many, many, many code generations in the future for this tool, anything, but it's really interesting for us to see like, obviously we both tried it, we're sending each other screenshots like, oh, look what I made, look what I made. It is a little buggy right now. It is really, really early and Ray knows that. And, but it's cool for us to see like the domain that this is centered in. It's poised to make a huge difference. If not just for hobbyists and designers and engineers for like a lot of people in the world who have an idea and want to be able to turn that into something real. I'm going to get really corny here, but they're, uh, PolySpectra's motto is make it real. And I feel like this is like right up their alley, right? They're letting people turn things from their imagination into real things and kind of giving them a head start on that through the most challenging part of it, which is getting something, an idea from your head or a sketch on the whiteboard into like a 3d CAD model that you can actually edit and send to a supplier and say, go make this.

Farbod: Yeah, for sure. And there's another thing I want to talk about before we like move on from this, an integration and the analogy between co-pilot. When I'm programming and there's something that I'm not like, I don't want to waste my time too much on, on like researching and this AI spits out the answer for it. I find myself like looking through its logic and kind of building that knowledge over time of like, Oh, that's how that worked. So, when I'm thinking about like the utilization of this tool or the use case of this tool, imagine like getting a part that is you as a beginner, you didn't know how to make, but this thing spits it out for you. And in addition to that, because they were talking about how there's multiple ways to get to the end point, right? You also get the history of how that part was made. So, if it's a cylinder that has a hole in it, it's like, oh, we first made the sketch, we extruded it and then we put the hole in. And you're like, oh, cool, that's how I can make it on my own if I ever wanted to.

Daniel: Well, yeah, I was gonna mention that too. I feel like even after using this, I used all my free credits. So, I did 30 of these, burned right through them right away, Ray, so I'll let you know I loved it. But you're doing those 30 free generations. I saw the build 123D code that was generated as a part of that. I'm like, I think I'm starting to learn this. Like, this would be a really interesting way for me to do CAD. It makes me think of when I worked at Tesla, we were like stuck in a really challenging design problem and they brought in a bunch of like really expert CAD engineers. They're contractors that came in for a couple of weeks to help us like do a bunch of CAD work. And I noticed these guys, they barely even touched their mouse. Right. They're doing almost all of the, almost all the designing just by typing commands into the computer and they're like, this is the most efficient way. Then you can use the mouse to help pan around and like build assemblies and stuff like that after. But I thought that was super interesting because now I started looking at that build 123D code. And I'm like, maybe this is attainable for me. And obviously I don't think I'm there yet, but after, yeah, after using a tool like this for a little while, it definitely helped me to understand like how that this Python code build 123D it could be super useful and made it feel like it's not out of my own reach to learn that, even though I've literally zero professional coding experience.

Farbod: Absolutely. I totally agree with you. And in terms of our experience using it, one thing I wanted to point out, one of the prompts I gave it was like, make me an L bracket. And what came out was definitely not an L bracket. I think you had said it to you. So, I was thinking like, they were talking about how other attempts at creating something like this was taking text to image and then the 2D, 3D object to be created. And obviously what they're doing different, what we've been hammering on this entire episode is take raw text that comes up with the code that then creates the object. I think it could be very interesting to have a step that's kind of like a closed feedback loop that says, now that the object is created, right? Take an image of what it looks like and then do a query of what I asked for, an L bracket. Do those two match? If they don't, reattempt until you get closer to like the desired input.

Daniel: No, I agree, man. Like one of the things I did to kind of gut check myself when you create, when you said L bracket, I was like, is this like a terminology that only you and I know? And I went and looked in Google and it's like L bracket. Oh, that's exactly what I wanted.

Farbod: Exactly what I want.

Daniel: So, I do think that'd be interesting to kind of use that again. I think the approach here is correct, which is instead of trying to use one of these really powerful AI models to generate an image and then try and convert that image into a 3d CAD model. That's really, really challenging. I think they've got the correct approach here, which is like language to other language and this other language happens to be this programming language that is used to render 3d CAD images. Right? So, you go language to language to CAD. I think that's the right step.

Farbod: I totally agree.

Daniel: The first step there, the language, you know, human language to computer language translation, they could absolutely use images of things, basically to improve that translation. So that when I say pipe with flange that worked well, but when I said elbow pipe with flange, it didn't work well that you can start to train this model to understand what an elbow pipe looks like. So, then it becomes much more robust for generating all these different types of geometries and shapes.

Farbod: I totally agree. And I think that's that there's a, I don't want to get too much into it. Couple. No, last year we talked about MIT, AI efforts to allow LLMs to better understand context, because when it came to image generation, they were struggling with the placement of different items. I think this could be like what we're talking about. The close feedback loop could be like a little bandaid, but seeing those two come together and getting the context relationship embedded into this AI could make this user experience much better. All that to say, I actually ended up enjoying myself using this product more because it wasn't working, because I felt more like a contributor to this effort that I am so interested in. Again, because I love CAD, I would love to see everyone like, my mom being able to use AI to generate cat files for whatever she wants. So, I'm pretty invested in what the future of this looks like.

Daniel: Yeah, I will say it's, it's really cool. It's designed to be shared, designed to be collaborated with. Even like when I create a model, I can just copy paste the URL and share it with you and you get the same model. I think it's really, really interesting. It's designed so that there's this community aspect of it, which is absolutely required because folks were telling you go toy around and play with it tell the PolySpectra team what works and what doesn't, because that will help the entire community as a whole be able to improve this thing and make it to a point where like we're saying, it's like this awesome wizard tool that you can just go in your CAD tool and type, hey, I want this, I want this exact geometry and it's gonna spit it out for you.

Farbod: Yeah, no, the future looks pretty exciting for all things computer aided design and I'm here for it.

Daniel: Yeah, me too, man.

Farbod: All right, I think, is that it? Should we do a quick summary?

Daniel: Yeah, let's wrap it up.

Farbod: All right, folks, computer aided design, it's very commonly used by engineers to create physical products, but artists still also use it to create models for video games and whatnot. Typically, it requires a lot of domain expertise. You have to have the skillset to know how to use programs X, Y, and Z to create whatever products you want. Well, in the world of AI, we've quickly seen domain expertise kind of go away, or at least remove the barrier to entry to creatives, for example, creating images, creating short-term videos, or even text. So why can't we do that for computer aided design models? Well, the folks at PolySpectra, they wanna tackle this problem. They wanna take raw text, spoken English language or any language really, and convert it to code language that can create any object that we want. In fact, their tool is readily available now. You can go look it up, nething.xyz to try and see what you end up getting. But I really wanna emphasize, you being part of this community and trying this tool out is actually what's gonna lead to its success. So, please, please go and try it out.

Daniel: Yeah, awesome. Yeah, like you said, love the wrap up there. Tools called nething.xyz. We're gonna link that in the show notes.

Farbod: Should we spell that out?

Daniel: Yeah, N-E, the letters. The letters N-E-T-I, N-E-T-H-I-N-G.xyz. Pronounce anything because you can, in theory, design anything. We're gonna link that in the show notes. We're also gonna link an episode, or link an article that's written about it, kind of lets you understand the challenges they went through trying to teach AI how to CAD, which is, it was a challenging journey, but it looks like we're on the first step there.

Farbod: For sure. All right, everyone. Thanks for listening. As always, catch you in the next one.

Daniel: Peace.

As always, you can find these and other interesting & impactful engineering articles on Wevolver.com.

To learn more about this show, please visit our shows page. By following the page, you will get automatic updates by email when a new show is published. Be sure to give us a follow and review on Apple podcasts, Spotify, and most of your favorite podcast platforms!

--

The Next Byte: We're two engineers on a mission to simplify complex science & technology, making it easy to understand. In each episode of our show, we dive into world-changing tech (such as AI, robotics, 3D printing, IoT, & much more), all while keeping it entertaining & engaging along the way.