This in mind, how can we actually implement this sort of representation? So here's the solution that we came up with. So we're given some time series data. It can either be one long-time series are several sets of shorter time series. And we assume that the data lie on or near a manifold. And then we follow a three-step procedure. In the first step we cluster the data into overlapping patches. Okay? So you can imagine paints and these data points performance some clustering algorithm on, clustering algorithm on them. In our case, we'd use k-means. And when you do this, you sort of implicitly get these overlapping co-ordinate domains. So, so this gives us our patches and then we also follow another procedure to make sure that the patches overlap with each other. Then, then the next step we basically learn how to flatten each of these patches. So what we do is that for each patch, we approximate this coordinate map phi that takes us from the high dimensional space, the low dimensional Euclidean space. What the neural network. So you don't have to use a neural network necessarily. But this is just sort of a convenient choice that we made because neural networks pensive do well in approximating arbitrary functions. Now simultaneously we also learned by its inverse or an approximate inverse of this map phi. And so this gives us a way to map back and forth between this high dimensional representation and this low rep, low-dimensional representation. In a way that's at least on paper, is lossless and information. For those of you who have some experience with machine learning, you might recognize this sort of structure as an autoencoder. The finally and the last step, what we do is each patch. Now that we have time series data in the reduced space, we will learn the dynamics from this lower-dimensional time series. And it's important to note that the dynamics in this reduced representation are still nonlinear. And so again, we use neural networks to learn the dynamics. And so you can either use a neural network to learn some set of ordinary differential equations. Or what we decided to do in this case is to learn time delta t, Delta t flow maps. Okay? And so this is our entire framework here of how we're translating this atlas of charts representation to something that we can actually implement computer. You'll do a heartbreak and assume at this point, this is all blind, symmetrically on there. Yes, what I'm showing here is blind, but I'll show, I'll show one example later where we do Bacon a continuous. I have a question here. For, for, for, for number 2. When you're using to encode them in that way, you are already determining the dimensionality or fewer small authors like low-dimensional space, you are actually supplying Android know. Like, like how do you specify that? Yes. So the way that we do this is basically, you can imagine following an iterative procedure where you might pick some large value of n at first or anxious, sorry, let me go back. So what we've been doing in our group is picking very low values of n and thens increasing, slowly increasing value of N by one and monitoring the reconstruction error of the data. Now at some point, once n becomes large enough, we see that the reconstruction error plateaus at a very small value. And that's what we consider to be the true dimension of the manifold. And in the last example, I'll show that more explicitly. Okay, Thank you. Yeah. And one thing to note is it I think that is an important point that we have to. We pick the value of n beforehand. But one of the nice things here is that because we're doing, we're breaking things up into patches. You can determine the dimension on just a single patch. And then because the dimension is a global property, you can then apply that same dimension to all the other patches. But in any case, let's, let's continue. And like I said, I'll show that more explicitly in the last example. Okay, So, so that's our framework. And so now I want to show it on six examples that are increasing complexity. So I'll start with something very simple, just really simple periodic dynamics. And then I'll end with a PD back is interment dynamics. Though, the first example we're going to look at it as just a particle that's moving around a circle. So super simple it, this is normally a two-dimensional system, right? You have x and y coordinates, but of course we know that the circle is a one-dimensional object. And our data consists of 40 points that are equally spaced on the circle and correspond to one revolution around the circle. Okay? And now we just follow our procedure step-by-step. So in the first step we form these overlapping patches that you can see that in this case we've chosen to use three patches that are clusters. So we just directly perform the k-means algorithm on this data and it gives us the following three clusters. Then, as I mentioned before, we have an additional procedure that makes the patches overlap with each other and that's, that's represented by these open circles. Let's insert a picture that as, as these patches expanding into their neighboring patches. Yeah, So now we'll deal a heuristic for how you, how you decide how many regions to split it up into. We don't, we've been thinking about this a little bit. We, the thing if, if there might be some sort of topological data analysis that might know, might tell us if if patches have holes or something like that. And then if you have, if you have a path that has a whole, then you might want to split it up into, into more patches. But we haven't really, we haven't really done any serious thinking about that. That was just one idea. What happens. We'll try and do one, like to obviously to feel that what happens to the procedure, though if you do one, the procedure doesn't break. It's just that you won't be able to reduce to as, as low of a dimension as if you had proper number of patches. Though. So if we just use one patch here, then we can now reduce this to a one-dimensional system. Whereas if we use more than one patch, once I go through the example, you'll see that we end up with a one-dimensional system. Though not nothing breaks, but you just can't get the same level of reduction that sure, Thanks. We use in she materially reduction when we have continuous symmetry or the US charts to reduce continuous symmetry. Then there is a definition of the border of the patch. There is the way the symmetries reduce that you have to make sure that your flow is transferred to the chart. Because it's to get a Poincare section and the edges when the flow is actually in the chart. So you lose it. So it's well-defined in that case, I'm wondering where you could do something like that. But there's some border but you shouldn't really cross. Cuz you know, yeah, I will fall and you're trying to protect the two planes. So obviously if you go to more than a half of a sphere in three dimensions, then it starts being double. Protection. They are not unique map that wanted a speech or with trusts herself is the same, right? So you have the same boundary. So what you're describing with the sphere, I guess, that uses a particular choice for the protection. But there are definitely other protections that still work in that case. Yeah. I mean, just a very simple format in a sphere projected government plan. Yeah. Mostly. I'm just saying maybe you have some criterion for borders of your practice. But it's where your charts is not doing well. In our groups. Whose got the strophic, you're just okay. Yeah. And so that's that's a good point and certainly worth thinking about. Like I said, we haven't really thought about it in any serious kind of way, but it's certainly worthwhile. Okay, so, so let's keep going. So though we have the circle now and we've broken it up into three patches, three overlapping patches. And here's, here's each, each of those patches. And so now in the next step, what we do is we just train are to find produced representations for each of these. So basically, we're training these neural networks to flatten out each of these pieces just to this into these one-dimensional line segments brand we can also map back from the one-dimensional line segments to the x and y coordinates for each batch. And then finally, within each of these loaded with lower dimensional representations, we can then train our neural networks to learn the dynamics. And that's just represented by a F1, F2, and F3 here. And so in the end, what we have is this collection of one-dimensional dynamical models that we can then sew together to form a global finance, the whole model. And if we take an initial condition and evolve it forward, forward in time using our patchwork of dynamical models. Then we get the colored points that are shown here. So I'm showing the, the first and the 25th period of motion. And if you can see the solid gray lines on top of those points, those correspond to the true dynamics. And so we're, we're getting dynamics that are quite faithful to the true dynamics here. And what I really want to point out here is that we have nice smooth transitions between the charts. So as we go from one color to another color, It's, it's quite smooth. So I guess my question is the same but delicate thing is do cooling the chart and borders on top of each other. And you have that told me control my program. Yeah. And maybe you actually remind me about that at the end. So that is something that I think is worth talking about at the end about that, please remind me. Okay, so that was the very simplest example that we could think of. And so now let's go to something that's just a slight bit more challenging though. It's basically the same kind of thing. We accept in three dimensions instead of two. We have a particle that's moving around the surface of a taurus and it's still a periodic, periodic orbit. So again, we just follow the same procedure. We break up the data into three chunks, make them overlap with each other. Then we take each of those patches. We learn, produce representations are then so again, we can reduce it to a one-dimensional representation. And then when we learn the dynamics in these lower dimensional coordinates. Finally, we can though everything together to come up with a global dynamical model. And again, if we take an initial condition of all it forward under the dynamics, we see that it's very faithful to the true dynamics of the system. And again, we have these nice smooth transitions between the patches. Uga flipped chairs chart. So very non-uniform distribution. Most adults over time, new federals. Sigma African thought. Oh, okay, so in the definition of a manifold, there's actually, you know, it's, it's sort of a good thing and a bad thing, but there's essentially no restriction on the coordinate maps that take you from the manifold to a Euclidean space. So the good thing about that is that it's easier to deal with when you're having to prove theorems and do analysis. But of course, you could imagine that from a practical point of view. You might want the score to maps have some sort of a structure that Mike might make the learning procedure easier or might make the dynamics. These are to learn. So for this example, we have not imposed any structure on our neural networks in size your remarked, right? If you look at the middle of the red set of points in panel C, right? The points are very dense in the middle, a much further spaced apart at the ends. Whereas perhaps what you might want to see is that they're evenly spaced in the representation. So at least theoretically there is no advantage or disadvantage to how this thing is sort of stretched in, brush differently in different parts of both. But what we've seen in some later examples is that this does make a practical difference. So later on I'll show an example where we do impose some sort of structure when we're trying to learn these coordinate maps. And it ended up helping us from a practical point of view. Daniel, can you actually explain it on under under sea? What's F1, F2, F3, me. What the density of the points represents. What is the horizontal direction? The direction is the coordinates are Euclidean space. Yes. So the, the arrow is just pointing in the direction in which the dynamic smooth. So, so if we go back to the very far left, we're looking at the torus. We're moving around in the positive direction. And so the arrows in panel C, you just step. This corresponds to that direction, the direction in which the dynamics are, are pushing you. And so what F1, F2, and F3 represent are just the time delta T flow map. So the map that takes you from 1 to the next point to its neighbor. So that's what we're, in this case, that's what we're actually trying to learn. The flow maps in separate rooms. Got it. Okay. Any other questions while we're here? Should I move on? Is the interpretation for fluid or mappings like fly, want to fly inverse. So one way you could think of it is Yeah, so all that it is is a coordinate change. So you can think of it as giving you perhaps the essential dynamical variables. But as I, as I mentioned, in trying to answer prodrugs question, it. You know, basically there are, there's an infinite set of these maps that will do the job just as well, right? I can take these one dimensional representations and stretch them however I want and they'll still work as long as I do some sort of Y is smooth then, but it still works. And so that's sort of muddies up how to interpret these things, right? Because if there's The map that will work. Any small variation on these maps I'm showing here. Then it's perhaps a little bit hard to give meaning to what these maps really mean. And so for the most part we're just thinking of it as, as a tool that takes us. There were reduced representation but it's yeah, I'd say it's a little bit hard to give an honest interpretation. So I'm going to leave the queue can grow from tours will go line 51, map phi inverse swamis store. Really amazing to go from line three, students. So remember that this works is preview, compressive throats. Yes, of course. It's only an inverse when it's restricted to that part of the manifold. Yeah, attribute more colored stone bridge. But of course, when you think about inverses z obviously have to think about your domain and range. That's something. Okay. Let's keep going to the next example. So again, just a little bit harder here, still staying on a torus. But now we're going to have quasiperiodic periodic dynamics. So I just changed the frequency of motion in the theta direction so that it's in commensurate with one in the phi direction. And that gives us quasiperiodic periodic dynamics. And so now, if you wait long enough, they'll densely cover the surface of this torus. So again, we just sort of follow the same procedure. This time we have to break it up into six pieces. Though there's, let's see, maybe you can see a red, blue, yellow, green, purple, and orange. So top left is showing what each of those pieces look like. And then bottom-left to showing how these points look when, when I make all the pieces overlap with each other. So now it's a little bit messy red because you can get up to four pieces overlapping with each other at the time. And in case that's that's what we were just following our same procedure now that we have are our six patches now. And again, we, we train autoencoders to map this to a reduced representation. So in this case, we can map it down to a two-dimensional representation and to actually use clustering to partitions taller or shorter, or you could do it manually. In this case, we use k-means and incus case, so it was automatic. Okay, as so now we have our two-dimensional representation of each of these patches. And so again, we just do the same thing. Now we learned the dynamics in this two-dimensional state by learning the time delta T flow map. And on the top two graphs, on top two plots on the right there on the x, y, and z coordinates under the Learn dynamics and the true dynamics. And then on the bottom two plots, we're showing the reconstructed reconstructed theta and phi coordinates here, which are a bit more bit easier to interpret. So you can see that we're doing pretty well, except that our face, these are a little bit, right. The, especially in the bottom right panel, you can see that the, The learn dynamics are just a little bit off in phase speed for the true dynamics. But this can get better if you, if you just keep on training. And some maybe one thing that I'll note for this group is it's actually, we've actually found it quite hard to get quasiperiodic dynamics in our model. So, so the plots that I'm showing here truly do have quasiperiodic dynamics. But what we've often found is that we end up getting periodic dynamics that have a very long period, somewhere on the order of 1000 timesteps. And what we think is going on there is our we're getting we're getting Arnold tongues, that form. And so if you take, you take a quasi-periodic system menu, perturb the dynamics a little bit, then, depending on the parameters, you can end up getting a periodic system that's attract. And so, so that's what we've, what we've seen in some instances I just wanted to mention. Okay, so now let's move on to a system that comes from a PD, though, we'll be dealing with the Komodo. So the shin CPD, just as we had the last few weeks, I'm sure this crowd is it's pretty familiar with what it is. And in the first case where we want to set this hyper diffusion parameters so that we get beating dynamics. Okay? So here's what our data looks like. It covers one time unit, one time unit. And you can see that it covers about maybe a little bit more than two periods of promotion, right? And so again, what we do is we break this up into it, the passions. So again, we've used three and that's represented by the blue, red, and yellow bars that are on top of the plot here. And then once we have each of those patches, we again learn a reduce, in this case, one dimensional representation for them. Okay, and this second plot and B, this is showing you the boundary of slavery. Hard to see. What's up girl or supports foods, murmurs, a yellow. There's not going to have three approaches. And I can see red and blue. I can see sort of more of a caricature of our borders are doing. So. I don't think is doing anything particularly special. I wouldn't think of it as any different from blue and red. I mean, I haven't looked close into which may capture you. Just read and you were to succeed. I'm saying that, you know that this example is essentially just the same as the previous examples where we're doing this clustering in state-space. And I wouldn't, I want to think of red and blue and yellow as being any different from each other. I, I would treat them all as is basically the same thing in this example. So if I understand correctly, basically your opinion, how you choose k-means, basically how large should the clusters are? You just get different partitions, right? Yeah, I would say so. It's a particular example. Exactly. Okay. So right, so we have this clustering into three clusters. And we can learn one-dimensional representation since it's just a periodic orbit again. And this plot in B, the second color, color map, that you are showing them pictures of space-time. But you will toss funding is done in state space in Berkeley or six dimensions or something. That's correct. Okay, sir, you're in Fourier modes for your class that presumably, right? Yep, yep. Yep. Okay. And this second, a spacetime plot. This just shows the reconstruction of the data from the one dimensional representations that we have. Okay, so again, we just follow the next step and we learn the dynamics in, in each of the one dimensional spaces. And then we saw everything together to get a global dynamical model. And when we take an initial condition and evolve it under our model, we have somebody again that's very faithful to the true dynamics. So probably not surprisingly, it's just a periodic orbit, so it should be simple enough to do that, but we're just demonstrating here that we can get it in higher dimensional example. And so maybe I'll be explicit. So here, not only the dimension is 64, and we're able to reduce it to a one-dimensional representation, which makes sense, right? It's just a periodic orbit. So topologically it's just a circle. And so we're able to, to get that essentially automatically with Dartmouth in the program is each video will be Corbett separate. Absolutely has its own set of charts. Don't don't know yet what the floor is doing more Guomindang under COVID. Yeah, that's totally true. And that will force come up for different values of this hyper diffusion coefficients up. Though in this case for the value that we've chosen, just hear, I, heard out of marbles is attracting and so presumably it's, It's not embedded in some sort of larger manifold. It is the inertial me. But, but we'll get to some more complicated examples. Okay? So, so now we change the hyper diffusion coefficient and we do it so that we get this beating traveling, right? Yep, so they'll just look at the top set of plots first. So again, I'm Sean, spacetime plots. On the top left. I'm showing it over 100 time units, so they can see this sort of very slow drift. And on the top right I'm showing a zoomed in version just over one time where you can see it's much faster beating period. Okay? Um, and so we have now an example where we have disparate timescales. And so this is the example where we, where we build in the continuous symmetry that we mentioned at the beginning. And that's what's shown in the second, third row. So what we've done is we've just separated the field and to shape, which is shown in the middle row, and a phase which is shown in the bottom row. And because of that symmetry, we know that the Shape Dynamics should only depend on the instantaneous shape. And the face dynamics should also only depend on the. On the instantaneous shape. And so, so we just have to learn a representation of this shape essentially and then we'll get the dynamics from that. And so that's what we do. We take our, our shape data. So in this case, our data consists of just just one time unit. So if I go back, it only consists of one time. Yes. So that's a very small fraction of this sort of long drifting period. And we follow the same procedure that we have before. We cluster it can state-space into three patches. We learn a one-dimensional representation for each of those patches and dynamics, the dynamics from them. So we're learning both the Shape Dynamics and the phase dynamics from this one-dimensional representation. And then once we put everything back together, we can get essentially the correct, Correct dynamics, the correct tree dynamics. So again, I'll mention that the, the beating period is quite close to the true dynamics that are the true beating period of the drift period is it tends to be within, let's say, five to 10 percent of, of the true church drift period that we see in the data. Nevertheless, quite good. And you know, it's, it's almost amazing that we're able to get this drift period quite closely, considering that we're only using one time unit of data, which is 1% of the drift period to actually train the entire dynamical model. And as I mentioned, what are they? Okay, Can you comment on the introduction part, basically the code, you go back the bar that they don't understand this. How much will that assembled medically? ** how much would you do manually? So that was that was done manually. So we manually separated the data into a shape and a phase variable. But then from there we automatically learn the dynamics of the shape and the phase variables they sought. So the separation is something that we do by hand explicitly. But then the dynamics are something that are learned. Automat, good looking guy, the real through your core with a row of the periodic orbit. So you know, both exert because, you know the relative roles. This is no periodic orbit movie Rousseau, constantly growing federal laws after each period. And of course, though, uh, you know these things exactly. But you say you get the what you call drift by 5, 10 percent through this procedure? Yes. Somewhere to the exact answer yet. And so the phase velocity isn't exactly constant, right? So if you look at the bottom row at our phase variable, DC, as well as their source of smoke period. Yeah, and so I went on, we are capturing those oscillations as well. So what happens if you don't separate out the symmetry? The drift of the Sui direction. Yes, owl. So in that case, it becomes topologically the same as our torus example. And so they're presumably we should be able to do just as well. Although I can't I might have tried to very early on, but I can't remember what happened exactly. But I imagine that from a practical standpoint, it becomes harder because it's, the data is in a larger dimensional space. Especially though on paper, we shouldn't be able to do it if we don't do this. Dimensionality reduction. But practically, it might be a harder thing to do. And so I guess the main thing to take out there is if you know that there's a symmetry, then you should explicitly handle it. Well, but the practical cases are the interesting practical cases. So note one or two-dimensional array, so they'll go with them. Self-help. Shouldn't joke if you give it to this higher-dimensional. Though. That's something I should try them to actually see what happens if we do not explicitly give symmetry, but state-space here is infinite. The orbicularis either two-dimensional, we forgot about some material warm dimensional. It gets hard very soon. In never Stokes who will need at least 10, 2200 dimensions. And Martha, she Russians could or a physical dimensions. I don't know if you have got to it for a small, small cell. Okay. Gone. Okay. So, so I'll just end with one last example here. Again, it's going to be curve. Notice that Michelle risky. But now with intermittent dynamics though, here's what the spacetime plot looks like. But it's more informative to look at a few of the leading Fourier coefficients, then you can see that what's happening in state spaces that were basically staying very, very near a saddle point and then bursting along you an opposing saddle point. Okay? So the key challenges with this example are, again, it's a, it's a higher dimensionality, so it's 64. We have disparate timescale, so we have the fast burst in-between these pseudo steady cells. The state-space has a very thin structure, as you can see. And we have sensitive non-periodic dynamics that we're going to follow. Again, just follow the same procedure we had before. So based on, based on what this state space looks like, there are the leading three Fourier coefficients look like. Well, we decided to use six patches. The idea being that you could have one for each of these saddle point sets, can buy them red and blue here. And then one for each of these four that are pulling it orbits. And so we learn an atlas with six charts. And in this case we actually do give some structure to the auto-encoders. So the structure that Alec, Alec planet came up with and it's something that, that Carlos spoke about at last week's webinar. And the basic idea behind it is that instead of just training a neural network, what you do is use. You start off with using PCA or the projection. But then you build a sort of add a neural network on top of that, you get a non the instructions. So the way I think, to think of this would be the PCA sort of gives you like the linear term of the Taylor expansion and then the neural network gives, gives the non-linear terms. But any case, this is just a special structure that we've used for the auto encoder in this example. And, and from that, what we found is that we can get a 3D dimensional dynamical model. That's qualitatively correct. And of course it's, it's minimal dimensions so you can't get the correct qualitative dynamics with lower dimensions and three. So, okay, so a few things that I want to point out is that it looks like we're getting the correct state-space structure, at least approximately, right? If you look at the middle model, I mean, the result looks a little bit mostly compared to with your training on the right. I'm going to die. I'm going to yes. So, so the biggest discrepancies obviously involve length of these quiescent periods. The burst and parents are fairly quiescent periods are honestly, I think they're about no 100% of the true quiescent periods. And so the reason why we think that we're getting this is because then near the saddle points there's a logarithmic divergence. And so if you're just a little bit off, then your period will will be quite a bit off, right? And so we think that somebody that's inherent to the system, not to, to our method. And that's because of the structure and saddle points here. Well, I think it is to some extent the UK to the method. Sense that, you know, your a level of resolution needs to account for how fast or how slow the dynamics IS, or basically how are other, It's not really so much how slow or how fast it is, but basically how unstable it is. Rape have EVA, high degree of instability here. A lot of KL divergence rates near the saddles and hue basically resolve with the same kind of level of accuracy, the entire state space, you are clearly CPU, are they under resulting those regions near it settles? Write them. If you're under resolve them, you know, there's no way you can get this part right. Qualitative of the earliest, right. But quantitatively I'm not going to get it right. Okay. And so what you're suggesting is maybe ratcheting or her or shrinking portions of time. That's one possibility. So basically use different colors for different sized clusters. I mean, you really need most resolution you the settings, right? Everything else is we're simply Yeah, definitely. Okay. Yeah, that's a that's a fair point to point up and use re red and blue are very Vince someone who is doing yeah. Yeah. And I guess the problem is that now, even though they're red and blue are quite dense, they're much, much denser. Write out these at the cusps than they are at the outer points. So maybe to Romans point, you might want to break this up. Ideally, you might want this orange cluster to eat, reach much further into, much closer to the customer, is, as Raman say, you just might want to break up into even more clusters so that you get, are philosophers cluster size of your different, right? So if you allow the cluster size to vary, then you can use larger, sort of larger clusters where you don't need the accuracy and the smaller one, smaller bite-size in the state-space, in the whole state space where your network on, hopefully that. So we, so like I said, we haven't done anything special what the clustering, right? We've just used certainly out of the box k-means. But they're obviously might be much smarter ways to meet the clustering. So I think actually mike, mike Graham had a paper back in the day with Yanis camera case where he was at doing that sort of clustering in a different kind of way where he was essentially massaging time. You get you get the result that you described. Let me Could it be that, you know, you need charts of different dimensions. So the charts for the boring part of the floor. One, the moon. So maybe we've got two or three for the cliff hurt. The coldest dumps be tricked. The gradient. I, I would, I would avoid that because accion is the global property of the system. Um, and so that's not necessarily, I mean, you know, in our initial manifold has a dimension, right? Yeah, we agree on that. That's a global property. But locally or might be just sitting on lines or you might be piecing it together from pieces which has more dimensional. Worse when you look at corner motor skill, your number of unstable directions is changing all along along the flow. Number of unstable directions is the first indication of the, it's accrued notion of dimension because for every unstable direction immediately mentioned. But then we know you need more dimensions to gather inertia than the chloride. Your noodles subcontracted months. But it's possible that the parts of the state-space a low-dimensional little pieces of art that provide your comments. Yeah. I mean, if you were to look at, let's say the the yellow or the orange posture and just run out encoder on depth. You're probably just get the one D structure right here with these very well with just 1D, right? So notice you reconstruct nice plots. These plots have that orange and green or whatever, right? So if that is executing earlier, conventional pieces that you will need effectively a larger dimensional manifold Because otherwise you will not be able to resolve the dynamics near the saddle, right? So you'd probably have to use 2D, but I don't think you need 3D. Or a yellow, orange. I mean, if you don't do 3D, then you won't get non-periodic dynamics. I think you will. Okay. I mean, we've Friday and Emilia. Yeah, it seems very important because when you go to turbulent, Korematsu versus turbulent, especially important to room. These are the two things you see parts of the dynamics. So we'll collect urine, orange, green spots. They're not regular because that thing's going to turbulence. And the, and then the, the defects are very important, conveying your moods or getting a role, average role at some instant that happens very quickly. It happens for you. Here. These are two basic building blocks when you go to More to the inertial manifold or five-dimensional, that mural right now. At most three-dimensional version. It might be interesting to try what you suggested where you have different dimensions for your different parts at, you know, at least from a practical point of view. Also kinda conceptual. You're, you're being very fearless. And because you go from 64 dimensions down to three and back to 64, and also define phi inverse phi. And you're not afraid of doing that going from your manifold to the flat charge and back and forth. And this says that made me also within the challenge themselves. You might be going between charts of different dimension. And it's little bit scary, but because how do you maintain the full dimensionality of chaotic flow of it? And it's only some places to what happens. I think what happens for cooler water shiver when you go to turbulence. In our defines the physical dimension or dimension of inertia manifold is the fact that this covariant vectors can go parallel in the floor. And that happens at non-linearities happens, and the unstable manifolds folded down squish my stable manifolds. And that's the reason why to meet so many unstable, I mean, contract directions to have a correct dimensionality of the flow. But these things local and they kind of have some normal form and they're the things that be what would be simple motions into a higher dimensional inertial manifold space. I think this is an example. The picture they're showing us here. Locally, you cannot do one dimension because to accommodate this, you need to do, I think this is good. Emilia, piecing it together this frame, anybody help post on a conceptual level, much as numerical sizes to have. Basically one more thing that I wanted to say about this is just a comparison to previous methods. And it might say previous methods, I mean, methods that don't use multiple pieces. I'll just try to do everything at once globally. So in that case, the strong when the embedding theorem is, is very informative. What it basically tells us is that if you do not use an atlas of charts, you should still be able to reduce the dimension to two twice the true dimension of the manifold and the worst case. And so what we did is we tried to do exactly that. So let's suppose that the manifold really three-dimensional. In that case. We should be able to come up with just a global model. A model that just uses one patch that is six-dimensional, in principle should work. And when we try that, we consistently get this kind of result where we're getting something that's qualitatively very different. So here we're getting at this attracting fixed point always, which is obviously very far off from the true dynamics. And moreover, we can also look at how the reconstruction error. So this is really where when he's strong embedding theorem comes in. What it says is that in principle, a six-dimensional one chart model should be able to describe the manifold perfectly. But when we compare their reconstruction error of our three-dimensional multi-term model from the previous slide to this six-dimensional one chart model with the same number of trainable parameters. What we see is that the multi-track model does an order of magnitude better in reconstructing the data. And so the main thing that I want to say here is that even though we have theoretical almost guarantee or guarantee on paper from when he is embedded there. What we're finding is that I'm in practice, we're getting a lot more out of using multiple charts. So, so, you know, on paper these two should be able to do just as well. But in practice, using this multi chart approach does much better than just using a global approach. And, and especially, let me just highlight again, this is with the same number of trainable parameters here. Okay? So we're getting, we're getting something more out of using multiple charts. So just to summarize quickly. Inspired by differential geometry, develop this Alice of charts approach to learn simple dynamical model models directly from data. And the way that our approach works is by learning a patchwork of local models and then summing them together to get a global model. And at least on the examples that I've shown today, or method outperforms global methods. And maybe I'll just leave up this slide here with some. Now what we see is three major benefits. One is the ability to obtain dynamical models of the lowest possible dimension. The second is that the sort of divide and conquer approach has a number of Computational benefits. So one is that it's, it's scalable. The second is that we can, we have the ability to adapt models locally and maybe as you guys mentioned, you could eat, can have these different pieces having different dimensions. Maybe that has some, some practical benefit. And, and this algorithm is also embarrassingly parallelizable. So every patches when we're training the neural networks are trained independently of each other. And so you can just do everything in parallel. And the last thing that we think is the benefit is that That's this directly gives you a way to separate state space into regions of distinct behaviors which could be beneficial for modeling. And I started mentioned this at the beginning that neural networks are not required. You just need some other, you just need some way to approximate these functions. So for example, one thing that we, one thing that you could do, for example, when learning, when training the auto-encoders is do a linear projection for the dimension reduction step and then non-linear reconstruction to get the inverse of that, of that coordinate map phi. I mean, that's something that works in some examples too. Okay, so, so that's all I wanted to say. And so I know we've had good meal. You have to remind me to do my view also to wn WE DO Loop member and walk. Hmm. Okay. I'm trying to, I guess let me see if I can remember what it was. But we could be doing overcook. Yeah. Hmm. I don't remember what it was and sorry. I might have to know that's not the only me, but also the speaker doesn't. Yeah. Anyway, thank you. The floor is open for discussion as well because it was almost certainly. But then you'll do have a sense of how the results, the accuracy you, maybe you don't even need your help. We can see better. He said eigen. But how do your results depend on how much training? Do you have a sense of that? We don't. I mean, you need to fill the manifold, but you have to cover all that. Hey, I think for this to work, at least the way that we've done it right there. It's pretty crucial. Yet. Definitely. And so I think that's I've tried thing about this a little, a little bit. You know what? If you have data that partially covers a manifold gives you use the equations of motion to perhaps add some structure to your autoencoders. Let's say it's a sort of correctly cover the pieces that your data are missing. But at least in this iteration of the method, you need to cover the entire manifold. For it, get good results. How density you need to cover it. It's, it's unclear, but I guess that's enough to get all all the wiggles that might exist in space. So not not a great answer. I know, but like yeah, that's that's my answer for you. I don't know. Maybe maybe it had some experience with this. Yeah, I mean, for, for turbulent flows when you have the dimensionality, which is let's say 10 or 20. If you use a relatively small piece of a turbine trajectory, I mean, basically you see that the data only covers a set of much smaller dimensionality then the actual dimensionality of, of your attractor a meal. Can we ever even hope to generate enough data in order to? One interesting thing that I like that was the, you remember I had this one example where I had orbit that sort of densely covered the surface of the torus. So what I've also tried is to make a taurus that's kind of like, you know, instead of plain doughnut, It's like a crueler Dona. So it has all these ridges to it. And what I found there was that when I would train these autoencoders, I can basically get for the course shape of the Dona. But I wasn't able to get the richness of that sort of domain. And this is something that's, that's a known feature of, of neural networks is that they tend to get. It's our sequentially get frequency information so they can very quickly get low frequency information, but it takes a long time to get high frequency information. And so that shell, right, maybe not necessarily new players come in and get networks. I mean, I think it's, I think I've seen that in deep networks as well, at least. By example, you start get frequency information sequentially from small to large. But I think what's interesting about that is that, you know, maybe you can't get perfect models, but maybe you can get course models just because of that feature of, of neural networks. And, and I think getting horse models is probably something that would be more useful, especially as you move to higher dimensional systems. Because in the end, I don't think that these sorts of approaches will be super-useful to get very detailed dynamical information. But I think that there'll be good to sort of give you Forest dynamics. But that's just my own opinion. Alright, thank you. I have another question. You were mentioning that the less light that neural networks are not necessity. How do you plan in capturing the nonlinearity is 0. If your surface is, you're not using something like the neural networks. Yeah. So one way that you could do it, for example, is for the projection to the lower-dimensional space, you could just do linear projection on that. You can do that by ECA, for example. And then to get the approximate inverse mapping, you could use something like Like like ridge regression. So just, it's just some other non-linear function approximator that's not a neural network. And so that's all I meant by saying neural networks are not necessary. He use or other nonlinear function approximators use. Okay? Unless it was how, how you would flow efficiently and give the beginning we go partitioning the data lake both B and a and B haha, how that is sensitive. Like if you have any sort of like taking like sea planes through the list and dopamine. So tonight if you took like five points or the list and seven children, how sensitive your algorithm would be to like extending the petition. We're shaping the coefficient that it's a bit more. Yeah. In my experience, it has not been very sensitive to that. Now you could imagine if you go to extreme cases, things could change. So one thing is if your partitions become enlarged where they actually close on each other, then, then that's a problem, right? If we go back to the very first example of a circle where I had these three partitions. And let's say I expanded each of them so much that they eventually, you know, each of them eventually ended up being the whole circle. Now you have a closed object that you can't produce two to one dimension anymore. And so that's maybe one trivially could run into. But for the most part, we haven't found too much sensitivity. But of course, you do have to. Yeah. There's probably a range of the correct number of partitions to make. And we spoke a little bit about that earlier about how you might try to get the correct number of partitions. What we've done here is just basically that the first thing that you would think to try it, which is just to do k-means clustering. But I'm sure that there are better methods, better methods that you can think of. It. This was really interesting. Yeah, Thanks. Yeah, thanks. No more question. This was fascinating and beautiful. Thank you very much. All right. Thanks so much. Thanks everyone for, for the questions and and for coming for the talk as well. Thank you. Progress through your career or THC is that it was really fun. Good luck. Thanks. Hello. And thanks for having all of us as well for these past four weeks, I think it was a great opportunity for all of us to talk to you, a different crowd that we don't normally get to talk to you that has a lot of interesting ideas. Thanks.