Research in Shape grammars, computational design, computer ad design and design theory. Is the director of the Shape Computation Lab, a research group that explores how the visual nature of Shape can be formally implemented with new technologies to enable new paradigms in computer ad design. Design automation, visual scripting, and creative design. I'm really excited about this speaker today. He both in the College of Computing and in the School of Design. So let's welcome Dr. Things Sec 1212, can you hear me well, Hopefully the action will not be too strong. Hopefully at least as strong as the name. Okay, delighted. Let me check here my time. Delighted that I'm here with you Today I will talk about the project that I think you will hopefully find it fascinating. It is the work of the last, definitely ten years of work that you do at the lab. But in many ways, it's really the whole life sort of worth of work to make it happen. It is not easy what you will be seeing today. The presentation is called Shape Machine. From software to practice, I will outline the problem, the software they have really put together to tackle these issues. And then the exciting things that are happening here at Georgia Tech, from architecture, from design students and computing students that they take this on. Let's see how do that. Yeah, sometimes analogies are good to start a topic. Of course, they fail if you push them too much, but they're good to start a visual analogy. You're all very familiar, of course, with word processing. The idea of finding things that you want in a text because you want to find them, the rest of the document because you want to change them in whatever you want to do with that, But once you circle it in a PDF or a Word or anything, because there is the database behind, they can find all these instances. It's not anymore the thing that you did on the screen, but it's really the underlying representation that allows it to be found everywhere in the document and to the Internet. In the Internet for that matter. So that sort of structure of letters to words, to sentences to what have you has revolutionized word processing and made a big thing, I don't know, 50, 60 years ago from various professions. Note that notaries were the first profession that actually got the word processing because they were able to change contracts within minutes. They would just change some names, addresses, and they would make some good money for that. So what would that mean? That's the analogy really, if we had the same kind of thing with the characters and all, and the words and everything. But what would it mean if we had other media, other symbols? If we had a plan, for example, Could we circuit details of that? Could we find instances of that? Could we change things like that in architectural design or basically in every profession, every discipline that uses sakes as a major venue for recording and communicating ideas. What if it is chemical engineer? What is electrical engineer? Circuits whatever it is with the same simplicity and eagerness and that you can do with word processing. Here is mechanical engineering. Finding relationships of finding parts of things. Anything really that stands out to you, not to the, not to the database, but really to your own eyes, that sort of thing. Let me see if this will work right there. All right? Okay. That idea that I just explained to you when about four years ago we went around and said, we have a solution around this topic. Cdl, Creative Destruction Lab, the Create here at Georgia Tech. The Georgia, the GRA, they had an interest in the work that we did. We tried to put together a video that communicate these ideas. You see here, for example, that drawings in the PDF are entirely useless. You cannot really, you cannot really query them in any way, nonsevere. We're claiming back then that was really our prototype, that we can actually have ways of making them work in the very same way that word processing works. So you see here, for example, the user selects a particular thing. This is a column, and the design tells you therefore. And the software says that there are four instances or it's a special relationship between a column and the wall. And the software would be clever enough to tell you that there are two instances of this particular relationship of the column to the wall. So that's the first question of the day. Why is that that defined in the replace operations that are so essentially in Word or Excel have yet to be implemented in Cad. Why am I here and sort telling you this staff, what's going on here? It seems very important, don't you think, right? Why? So that's really the first topic. Well, there's a good answer to that, because shapes have no unique representation. It sounds bizarre just to say that. What do you mean? Like we have all these incredible drawings, models. We have Autocad, we have all this stuff. What do you mean they have no unique representations? Well, in order to demonstrate that those of you that you work with any sort of digital graphics, you know exactly what I'm talking about. But in general, we made, again, this video to demonstrate the ridiculousness and the complexity of this argument. Here we pretend we have a user that wants to find a vector file to work with. So rather than, you know, sort of downloads the document from the web, tries to open it, it looks like a plan, it's fantastic. But then when she tries to work in the instances you see something that perhaps some of you recognize broken lines. It's a mess. It's an absolute mess. Most of the time, the architects, designers, they make a big cup of coffee and they clean up the drawing. That's what they spent a good time in the office, taking a thing entry, working it entirely. And here you see when you rotate the drawing, you might even have even bigger problems like asymptotes, like, you know, mistakes of the translational schemes between the vector graphics, or mistakes of the users, or who knows whose mistakes. It's an absolute mess, right? Imagine now here, once you go through this particular sort of codification, what you see as an arc is an arc. What you see as a line is an R and all that. Again, this was our attempt back then to like three years ago to show what does that mean? And of course, once you assume that there is something like that, the drawings, they can have a unique representation, then you can actually quit them. And this is really where the biggest problem exists. Unique safe representation does not mean structure. This is really a mindset that you have to change your work about your world, about how you look at stuff. You do not need to have well defined structures to search things. In fact, that what makes safe say what makes designers different. And you all know from your friends, you know if you are designers, you are slightly different from the engineers. And all related and different here you see how we assume that you can select parts in design, you can you can edit these parts. And the interface here in Rhino in three D, modeling, sort of nerves modeler and now the machine would look at all the instances. Right? You see the instances and before we clean them up and look now all the intersections are cleaned up. The very same way that you would use a character in Word and you would change your name for another name. Here you use that and of course, once you have a drawing, you don't need to look up a details. But what makes this drawing a drawing or a plan? Because we're, they're standing at their walls, their staircases, their windows. Well, if you understand them, the computer can understand them too. That's really the idea without having them restructured in advance. I guess there's so many different ways from talking about that in my classes, but one way to think about it is that you have put the carriage before the horse, so to speak, in computer graphics, and then we're trying to come up with solutions around it. Is this something new? Again, the oddity of me coming here in 2023 and telling you about instances and drawings seems very essential to be on the side. Here is an excerpt I really like from Malcolm Call, Executive Director in Auto Desk 2006, writing in one of the most sort of popular magazines, Architectural Design, Architectural Design, AD. And talking back then about the incredible input, influx of programming in design. And he says in 2006, he describes his experience 20 years before that. In 86, when he was at UCLA working with this guy, George Stein, at UCLA, he was his student, and I was also George student. So he says back then, you know, there was something interesting. It was called Safe Grammars. A desk. We knew about them, and we knew that there was something wrong with the way the whole system has been set up because it creates these black boxes. However, down there, the keys of death, the better formal structures had no radio application, no need to use them. That lack of interest, because it's complicated. Is it because the whole industry went on the pattern recognition on the image, on the pixel based stuff. Who knows? A variety of reasons why this thing happened. What he was referring to was this is the milestone of all the work that we will be presenting today. This is the 1975 books of the two dissertations that came out, one from George Tiny and the other from James Gibbs. Very close friends in their '20s, They wrote this. They got the Best Paper award for their idea. It was a popularization of the Chomsky's sort of work, but using directly saves as opposed to symbols. Since then they keep on reworking. Yeah, this idea here, I like that this is really the first paragraph, the absolute first paragraph, of James Gibbs book on the safe grammars and their uses. Safe grammars provide a means for the recursive specification of saves. The formalist for safe grammars is designed to be easily usable and understandable by people at the same time to be adaptable for using computer programs. A big stake, really a big claim, right? Understandably, those of you that you have an issue with programming or he has taken you some time, perhaps you sympathize with this sort of view. This is from 2015, about 40 years since. The original book that's George Stein himself. He's still around his at Mit. He still works. He had a book last year, highly recommended. You will see the featuring in the rest of the presentation. This is by making grammars, and George still uses his rules. The left hand side, right hand side by a pencil. So what is the basic core of the argument we have? Save every pair of shapes have a rule. Now, how can I use a rule? When do I use the use in a better sense? When do I use a rule if there is a transformation that makes the rule the same. But I'm actually looking for part of the design itself. It doesn't need to be there before. If it is for me, if I search to find it, then that's all I need. If in fact there is a Sarsation matches that, subtract that transformation and add the corresponding right hand side. It straightforward. Let's look at it pectoral, it always makes more sense. You see here the rule says, if you find a lower case K, connect the endpoints with this funny sort of sap and polyline and apply to the two squares. Mind you, there is no lower case in these two squares. I drew two squares. That's the whole idea, right? I don't need to anticipate who will look at what for, what they would look in the model if there is a shape and yes, for example, it is why it is because I can embed it into that. I can embed it, I sort of move it around. You can do it with a trace paper or you can do it with computers and you can embed it. And if you can embed it, you subtract it. And then you add the next piece of the line notes here that all of that stuff, you don't need to specify them in advance. And then of course, then in the end use them and make a new shape like that. Now you would expect that this would happen for all instances in the design. And here you see, for example, because the symmetry of two squares is eight, for those of you that like and enjoy symmetry theory, there are eight different partitions. So the computer or the human might find eight matches and might have eight possible designs to work with. What that thing is is that it is the typical use, especially all of you, that you do computer graphics at very well. If I have a shape for example, and apply transformation, it's straightforward. That's really what we do. Computer graphics, I have a shape, I transform it, and I have something exciting to work with. Here is the inverse of it. I have two shapes and I try to see whether there is a transformation that matches the one to the other. It is not always the case. It might be that there is no transformation that can do that. If, for example, you look at this K and I put this kin to square, so you can actually see that a bit better. You can switch your eyes back and forth between the outline and the shape in between. If you start with a K, there are this one shows here. There are 18 different kinds of Ks that we can actually talk about. Now if you look only about the outside the boundary, the original square, it can be translated, can be rotated. It can be scaled. All of these are squares. Or it can be stretched, become rectangle, can become bus. It become, become a parallelogram. Can become a trapezoid or a quadrilateral. And all of them underneath the reflections, we can be freakingly precise about what we mean. Find x. There are 18 different things that we can find on this particular case, and for all of them, of course. Zillions of transformation that we need to match. Now, the actual algorithm can be there forever. Those of you lab geometry, you know it in fact, that this is really something quite standard to see how can you calculate transformations from the one to the B here is from 1986. Not a very particularly great one, but actually let's show that this was published and all that. If I want to see if that what is the transformation that makes that right? Then first you translate it. You translate the large one there. In this particular algorithm, you move everything to the Cartesian origin. Then they should have a rotation that they don't have, and then a scale. And then you move it back to show that this shape, it is part of that, right? That's really the algorithm. The algorithm is there. What we have done here, I mean, the work that I'm presenting today is really the work with my Phd students and primarily with this phenomenal guy, Kurt Hong, who finished his Phd 32 years ago, and now he's the University of Kansas. We have this presentation that says how many this is really gained. Straightforward, but it's nice to see it in a table. What kind of information do you need to determine whether two shapes are related? You need basically one point to determine whether there is an isometric. Imagine you have a tiling. You want to find the tile, you want to put it together. All you need is one point of the tile, and you just move it around. You move the whole thing by one point. You need two points for similarities, three points for affinities, and four points for linearities. These are different types of informations. All I'm trying to say here is that this is the bare minimum. Of course, in everything out there can be a thousands of points. You have to check now. Yeah, all this about transformations, but we're missing something very important. Tapes are unruly, they are behaving, they are different from everything else. Yes, of course you hear transformations that make them exciting and all this stuff, but in themselves they are unruly, deeply unruly, so much deep. That is an incredible discussion and historicity on the relationship of symbolic media, text, numbers, music and notes to the actual shape out of lines and planes and everything. So here is again from George Stine, his new book, Shapes of Imagination, Highly recommended, fantastic reading to those of you that you write. You enjoy this discussion today so that if you have two Ls and you have structured them as such, see what happens in the flickering of the eye in the moment once you start relating them in particular ways. Here is, for example, when you start bringing the two Ls together, we have a square in between that pops out. If you shift them even more like along the diagonal, we have this really bizarre situation with three squares. Some eyes. I see your eyes. Actually, your eyebrows go like what. But now I hope you can see that. Then if you really make it even more, suddenly becomes a gestalt square with a cross or four squares or five squares. You start counting squares here. Here you have like other shapes, you have the two small squares, four reticle around the sever in between. Here's a Greek cross. You see what I'm trying to say here? This is really a metaphor for creativity in design. Those of you do design, you understand when you fool around with the stuff, you know by the moment you sift things around, new ideas come either you can see them yourself by rotating the paper around, or your instructor can see that, or your friends can see that or enjoying your favorite beverage. You can change your mindset and you can see them in different ways. But that's the whole point, changing representations like that. There are two problems with this thing. Everybody enjoys it, everybody talks about it. How do you go about it? That's another story. That's the thing. This is where the autocad put like the kiss of death says we cannot do that. The first one is that this is computationally very difficult if you really want to find this peer, I mean this is the outline of a, per, a large column, so to speak, in a classical plan by Palladium. Here you see the actual shape that you're searching here is the underlying infinite construction lines, or the equation lines or descriptors that the shape is embedded. If you want to find out how many ways you can do that here, if you look only for concrete copies, there is 1,000 searches you need to make. Why? Because this shape here consists out of 240 lines, 1234567. It consists out of 70 lines that they go like this and consists out of 1,125 points of intersection. That's really what the database gets. Points, infant lines and lines that you see. If you want to find out this 1234 line shape here, if it's exactly the same concrete copy, you need 1,000 if you want to find scale one. 1 million searches in the computer. If you want to find under affinities stretches, you need 1 billion calculations. And under affinities 1 trillion calculation, it's ridiculous. Really says that's something wrong with this idea. Because also this is a ridiculously simple drawing. Those of you that you have friends or you are designers, you know that the design specifications can be very, very complex. The other problem, of course, is the aspect of precision. Precision meaning that you have two lines. You think these two lines are the same. But if you really zoom, you realize that these two lines perhaps you have different ***gths. The details are not exactly the same. Of course, then you say fuzz it. The problem is that once you do it second, third, fourth time that you do this fuzziness, the calculation fails and the computer stops and cannot have any answer to. That is all this happens because of the floating points and all that. Again, these two problems are the keys of death of the auto described several efforts here we have documented 70. I show up here on the screen with 2013 with our first effort in doing a graph representation with my former Phd team. Because we did it with that because it thought like it cannot be done. So let's use graphs because somehow they are more available and commercial and robust. We abandoned that and here we have 2020. We were able to solve through the engineering of the same machine, through the work of Kurt Hong here some interfaces of things of the past that they have been familiar in everyone who does this sort of work. Here we are, we announced here that, in fact we do have a software that tackles the two ridiculous impossibilities that I just told you about the lines, the complexity and the imprecision. There's a new computational technology that brings say based search and say, base replace to the Cam world. How do we do it by addressing these two things that I told you and basically taking into heart what the formalism talks about. The book that I recommended, that again, I will recommend to look at Georgia's new book, George Stein's book. It talks about this embed, embed, fuse, embed fuse cycle once you do art and we do exactly the same in the underlying representation of the safe machine. Let's see how we do that. First of all with the complexity, we have a nice formalism there. It is called save signature that recognizes that not all points are the same. If you realize that you can have different conditions around points, you can actually see the design in profound ways. You can see that when we have this point signature, we have some remarkable improvement. That's one thing that we do. This is the most important thing is this. That says every time we apply the rule, the design changes, the actual representation changes in front of view is exactly the same, but underneath it changes so that you can never have a hiccup in the calculation of the shifting representations. This, once we put it together, we did two things to basically announce to the academic community that this is not the Holy grall has been actually figured out for at least straight lines and arcs. And now we have to climb this incredible mountain up. We start recreating some of the classic calculations that have been proposed forever in the field. And so this miraculous calculation we saves here is says if you see a K shape, put that end like that, this bracket, make this U shape. And therefore you go from the two squares, you go to this octagon and the pentagons around it. Pentagons, yeah. But then again, without really caring about any database constraints, you can always impose your own structure. We're saying, look at this shape here. This smaller K, K, or I don't know, arrow and the line. Put it in any way you like it here. The machine calculates. We actually ask it now to be under similar transmission. How large you want it, how small. All these end points are not in the shape itself. They are just matched upon it. You don't need to have them like registered within that. Here, again, just to show this incredible flexibility with the shapes, imagine that we say you find that K shape. Now make this, it's very strange to actually describe that. I mean, what is this two lines floating? That they connect but once you do it. We have the remarkable, again, fluctuation of a design in front of your eye. And this is the eureka moment that every designer feels once they start drawing. And with a pencil and so forth. Not with Rhino, not with auto desk, not with autograd, not with beam, nothing but with the pencil. That's the idea behind that. And on the other hand, we wanted to show as well one of the other, again, flags on the field to see how you can do design, that can be more meaningful. Imagine now left hand side, right hand side. These are actions. If we're trying to create an Irish here, you want to verbally discuss, describe this rule if you see an arc and another arc, but extend it in a way so that the two end points are perpendicular one to the other. How much, how long? It's impossible. Designers usually, so most of the times actually say, I don't know, I just did it. Or if they know, they just the point, they point to drawings, they point to the stuff behind them that's here. There is no need to describe that with words. You just draw them. You model them, and you say you ask the machine to work them out. This is two different examples, one for changing representations easily on the fly. The other taking curves and everything else in creating something very specific. That's the question one like demonstrating the field tutors, where do you move on? Question number two, the day, once you have something like that, what happened to go from a file that looks word processing to actually Rhinoscript to plus like using characters to use a programming language. Is that possible to take the shapes and make a programming language out of it by introducing a logical processing framework with stage loops, jumps, conditionals, and all that. Is it possible actually to program shapes where the line is not really a line, but it's actually a line of code but really a line specification. It makes your mind go around trying to sort out these conditions. We have also good news on this topic too. So far I've been talking about these two things. We have I guess three ways of working with the safe machine or basically 21 is the manual and the other is the automated. The manual. You can define the rules from scratch exactly as you've seen so far. We actually saw the derivation when you pick up a from the model itself. These are the manuals. You select a rule and you see what you want to do with that. The other one is the automated. We put conditionals, jumps, looks, and we put them in a particular framework. And we call it draw script as the DS stuff like we'll talk about it later on. The discussion here is some of the basic frames to see how we went around it. This is from the work of one, does his Phd in here in CS, but he also enjoys very much the work with the same grammar. So we work together with Professor Varma on that has put a whole manual of starting to work with draws. Now you see for example, who rules. There are three rules here. Every rule is in a particular window, like windows in computing. And every window has inside some information. It tells you under which transformation. The left hand side will go to the right hand side. Whether you will look it, whether you will apply it serially or in a parallel sort of fashion. What does this say? If I see a square, add another square in, and make the exterior square green. If I see the same thing, the counter clockwise, the other clockwise, if I see a green thing, make it black. And apply the stuff and apply it four times. Applied four times, and apply to once here. Applied it for all green things because it's really parallel. This is serial one by one. The actual calculation is exactly what you would expect. It recognizes the square and moves and puts another 1 second third here, one to three. And then it goes the other way around to complete, and then in the end becomes black once you start. Then having a framework of calculations. You can see where this thing can go. The rules, now you can put them in their own world. You can name them the framework that is very rudimentary, Just number them. This is the rule number one. This is rule number two. This says when you start jump to number one, calculate three times go back, then two from the two jump to two, go four times, go back. And then it's the same thing that you saw before. But now introducing the jumps from block to block. We can have conditionals when the window now becomes green or it has weight. If you start to the one, if you see a square inside, go to two and apply that three times, go back again if you don't see anything, which means see nothing. If you don't see anything, yeah, yeah. That means you see something then you go here, you apply that three times, go back, change the colors and all of them, the same thing. We keep on working on this fashion. This is like the work now, Van Cluster, he does his Phd in music Technology. He got fascinated with our work as well. He introduced the pseudorandom generators basically to make it really to have the rules firing ad hoc in any way. Very much like a programming language. If you see a line here with two endpoints, Jagt it and make one more point and make a deflection like that. And then in the end, all the lines and then you start applying very much like the Lindy stuff that you can do, Lindy, but do them like this. I don't know if you've done a liter my or stuff like that. It would be nice to sow the actual code next to it is what it takes to program with the my three and what it takes to do it in shape machine with draw script. This is just a question here, just to be funny, you understand that what I'm talking about here, hopefully you see it. Profound is really very new, like it is off the beaten path in computing and design.