All right. It's 330 or 230, depending on where you are. Good afternoon, everyone. And welcome to the CHB weekly seminar. I'm very happy today to be hosting Professor Victor's of Allah, who is the bulge of an Associate Professor in the Department of Chemical and Biomolecular Engineering, Biological Engineering at the University of Wisconsin, Madison. And he's also a computational mathematician in the Mathematics and Computer Science Division at Argonne National Laboratory. So Professor Isabella holds his BS degree from Universidad below Americano and his PhD degree from Carnegie Mellon University, both in chemical engineering. And I remember first meeting victor when he was a graduate student at Carnegie Mellon. Probably during a visit of mine to Carnegie Mellon Bank. And so I've enjoyed watching his work, following his work and watching his career over the years. When he was a researcher at Argonne National Lab. And then, and then after several years they are moved to University of Wisconsin and maintains an appointment also at Argonne National Lab. So he has received numerous awards, the NSF and DOE early career awards and the P case award. And I really, one thing I really appreciate about Victor's work is, is his work on software development, algorithm development, and also very strong engagement in applications in a, in a very serious way. And today we were talking about his work in distributed fertilizer production. I don't know that he's going to be talking about that with us today. That I am very much looking forward to hearing about the work that he's going to share with us. So thank you very much for limiting us. And out. I'll turn the floor over to you and to everyone in the audience, please do share questions as they as they arise in the Q and a. And then we'll take we'll take questions at the end. Thank you. All right. Thank you, Mark, for that very kind introduction. I know if I remember the first time that we that we met when I was a grad student, it's always been a pleasure to interact with you. So it's been a fantastic day even for a virtual visit. So I apologize for not being able to be there. I think is the time right, the times that we live in. So hopefully will things will move on. But I'm really happy to have the opportunity to share with you some of the work that we've been doing in the, in the area of data science and a little bit of machine learning, but mostly in, more in the space of data. So the word that I'm about to talk to you about is work led by my student, Alec Smith, which is about to graduate. Unfortunately, bruce Yang and also Amy accion. All right, so without further ado, let me get started. So the first thing that I want to emphasize in the talk is that when we're thinking about data, we have quite a collection of possible ways to represent data. And these representations come from mathematics, right? So then mathematicians have provided to us very powerful ways for representing different objects. So these are things like graphs, matrices, vectors, manifolds, functions, right? And I'm going to show you a little bit about how those different representations arise in different types of applications. And the reason why I think that these representations are important to understand and to really nail down is because they provide an abstract framework to think about very different problems on there are common lens. And if you have the ability to think about very different problems on there are common less. I think that that's something really cool because you can start transferring knowledge across disciplines. So how do people in the systems biology community are doing dealing with data and how can we transfer those knowledge, right, that, that type of techniques to something that might not be in the systems biology you feel. So I think that this is while the powers of thinking about these representations carefully and also tried to find ways to unify all these different concepts. So let me get started. I'd like to motivate this representation with a very simple example. So here what I'm encoding in the top pisa, a pattern, right, that comes from a white and black dots, right? So that is encoding some sort of sequence. So I can represent this in the form of an image, right? So this is something like computer vision will do. Body. You're thinking, talking to a computer, right? So you can also encode this in the form of a vector. So it's a sequence of 10 numbers here. But you can also represent the same type of sequence in the form of a function or something that mathematicians call a manifold. And I'm going to explain to you why call it a manifold in a second. But these, you can see that this encoding also the pattern, right? So this is kind of like a 01 type of staircase. So this type of representation comes very naturally for people like Martin myself, because we're in the controls field. So there's something like a bang, bang, bang type of control representation. So the, the key message in this slide is that we're really talking about the same data object, but we're really represented in different forms that are kind of complimentary because different representations will highlight different features of the data when you look up, think about it in different ways. Now the example we constantly a bit more interesting when you go into dimensions. So here what I'm doing is looking at an image which is just a collection of pixels that have a certain intensity of light. So here what I'm showing the, this white region is the one that has the highest intensity of light. And dark regions cause less intensity. The way that you process this image in a computer is by representing it as a matrix, right? So every pixel is an entry. And then the whatever number we have encoded in that pixel, which is capturing the intensity of light. Now similar to what I showed you before, mathematicians will also tell me, Well, wait a second. I could also represent this object as a manifold. And the idea that I do these here is that I put my x-axis, my y-axis here, right? Y and x. Then what I'm going to do is introduce a third dimension, right? And that third dimension is going to be precisely the intensity of light. And you clearly see that this representation start revealing a little bit better the topology geometry of this data object. And this is going to be a secret that I'm gonna be showing you throughout the presentation. How this alternative ways of thinking about the data can sometimes highlight different patterns or different characteristics that are not obvious from other representations. This will cause a little bit more exciting when you actually look at a real image. So this is a picture of my son when he was six months old. So we were somewhere in Norway. He was wearing a Viking hat here. So, so these are big image, right? And similar to what I showed you before. So this is our encoding a matrix, right? And I can also think about this in a three-dimensional, right? By looking into this manifold IAEA. And you see that these manifold, we start highlighting some interesting new geometrical features about the image that are not directly obvious from the two-dimensional projection. So for example, this is telling you places where the light is actually changing very abruptly when the derivative of the light is moving in different directions very abruptly. And that type of characteristics is important when we are thinking about identifying the boundaries of the object, for example. So this is used quite a bit in computer vision for detecting object boundaries. Now another interesting thing is how close the human ambition extract information from, from, uh, from different types of objects, right? And typically the, the, the human eye is not only looking at geometric patterns, but it's also looking at color. So color is a very powerful source of information. So the way that we encode color in computer vision, for example, is why you're thinking about this color image, decomposing this into three color channels, typically RGB, red, green, and blue. And then the secret is that each one of these images, but these channels is actually a matrix, right? And this gave rise to the concept. So for those that aren't familiar, a tensor is just a high dimensional generalization of a matrix, right? So in this case a matrix is into the, if I add a third dimension, then I have a three-dimensional tensor. An important observation here is that color casts a lot of information embedded in it, right? So there's a lot of data, a lot of potential patterns that you can extract from it. And now things get even more interesting because the human eye, it's also powerful, right, is very good at extracting information from dynamics, right? So, so we're very good at identifying when things are moving slower, things are moving fast. All right, so here I'm just showing you an example of a video of a liquid crystal going to elaborate a little bit this, this application later, a syllables when it's exposed to a chemical contaminant. And you see here that there's a very rich source of information. So we have temporal patterns, we have a spatial patterns, we also have color patterns that are evolving over time, right? So the way that you represent a video is typically in the form of a four-dimensional tensor, right? So it's adding one more dimension, which is time. So the ideas that I have, each one of these snapshots in time being a three-dimensional tensor, which is this, these multiple channels. And then I'm just gonna stacking these up in four dimensions, right? And this is going to create a four-dimensional tensor. So the message here is that this type of representation contains a huge amount of data, right? And there are a lot of interesting things that you can do with video. They encode a lot of interesting patterns of interesting things that you might want to explore to make decisions. Alright? Now, an interesting idea that has been gaining quite a bit of traction in the, in the data science machine learning space is this idea that you can encode a lot of very different data objects in the form of graphs or networks are colloquially more known as networks with a graph. So just to give you an example, I can actually represent a matrix in the form of this mesh, which is our graph. Every node is a pixel and then I'm representing the links as the adjacency between the different pixels. And I can encode data inside each one of these nodes. So for example, I could put the intensity of light and implicitly in these nodes, right? And this kills me. Put more information attached to this graph. Now, what a wise wire, graphs becoming really popular in machine learning and data science. Well, one of the reasons for that is that graphs are rotationally invariant data objects. So a graph really does know have notion of space and time. It doesn't matter if this node is at this latitude and longitude and disorder. Notice that these are the latitude and longitude. So strictly speaking, a graph doesn't think about it that way. All that matters about in a graph is the topology and the topologies capture implicitly by the connectivity. So what that means is that I can twist around this object. I can use bended. I can even move around the objects and the notes. And as long as the edge topology, that connectivity is not being affected, fundamentally, we're thinking about the same object and that costs a lot of benefits in a lot of applications. Now, I will explain a little bit later. Another reason why graphs are becoming powerful is because they are naturally provide a framework for representing things of interest to chemical engineers, right? So things like molecules. So for example, nowadays there's a lot of machine learning techniques that are trying to predict emergent properties from molecules like toxicity or their ability to form myself on things like this, right? So I'm going to show you an example of that a little bit later. So this provides graphs provide a really flexible way of representing molecules. And this is bringing benefits that were not possible in the past when we use Taylor representations. For example, this is miles strings that were used in the past and computational chemistry. There are still use quite a bit, right? And they have some nice benefits. Both the graphs provide a more general, the more generalizable framework for representing this type of molecule, sun and an optics. Alright, so now, once I have given you a little bit of an overview of these different types of representations that exist. Now let's think about what do we do with those representations? And the secret is going to be that depending on what we're presentation to use, you can apply different types of techniques to be able to highlight different aspects of that data object. So just in summary, typically we're trying to do transformations are operations on this data. Most of the time where we're trying to do is to reduce it. We're trying to compress the data object just to facilitate visualization analysis or storage. Sometime we're trying to mine it. So what we mean by mining it is just trying to extract patterns, knowledge, information, or features from it. Sometimes we're also interested in just cleaning it, right? So we're trying to get rid of some component. There might be actually like hiding some of the important aspects of the data, right? So these are things like filtering and smoothing. And one of the fascinating things I think right now about data science is that we have a lot of techniques available for doing this transformations that come from many different communities. So you're thinking about communities like a statistics algebra, Fourier transforms that come from signal processing, convolution, and topology. And all these techniques are currently converging into creating this very powerful approaches for trying to perform all these different functions on the data. So let me start giving you some examples of some transformations and how do we typically think about this data objects and how we tried to extract interesting information from them. And one example that I like to give to people in general, this on this concept of statistics, right? So I think we as chemical engineers were trained in statistics. So it's natural for us to think about the world in this way. So typically the type of workflow that we encounter in many applications is someone provides to us some sort of complicated data object. There could be some sort of set of observations in a statistics. What we're trying to do is to characterize these in the form of probability density function, right? So think about a histogram or something like that. But then you say, well, this histogram is complicated. So what we want to do is summarize it. So we want to compress this information into some number represented ic, number of interests, right? So we looks at things like averages and variances and tell some things with that. And once we project this data object into these numbers, into these summarizing statistics, then we tried to correlate those statistics are something of interest, right? So we're trying to establish correlations on linear regression models and things like that. All right, so these are very natural workflow. For us. The problem is that statistics is not necessarily the best approach for characterizing certain features of the data. In this example, what I'm showing you is ten representative cloud points or point clouds that are trying to just convey some particular data object. It is very clear that these ones, when we look at them with our eyes, they have very different topological features. Body gu, we're trying to summarize these data objects. We basic summarizing statistics. You wouldn't be able to tell them apart, right? So that means that sometimes the statistics my, means certain features from the data. And this is a simple example, of course, that things are more complicated the real world. But hopefully the message is clear that sometimes is not that obvious how you proceed with the statistics. Another example I'd like to give is principal component analysis, which is a very powerful approach that is used for everything, right? And, um, one of the interesting kind of examples that I can give you is this one. So think about that you have a field that was generated from a diffusion equation. So I just project a random field over a diffusion equation. And this creates some interesting texture. Right? Now, what we're trying to do here is to say, well, this data object is complicated to observe directly this way. So I'm going to try to projected into a low dimensional space, right? So I do my eigenvalue decomposition of the matrix associated with this. And then I project these to visualize it in two-dimensions, right? And they start revealing when these fields are close or when they are far apart. And that is very useful, right, in, in visualizing data. But the problem is that sometimes we PCA is not that easy to separate certain objects just because it may also mean some important information. So for example, if I were to represent this field using these three-dimensional manifold idea that I was telling you about. You will notice that there are a lot of interesting topological features about this field that cannot be captured directly to correlation, right? So PCA is approach that is trying to structure from variance and correlation. So those types of features not necessarily can be extracted from PCA. In some cases, it might not be a problem, right? Putting some cases, you may actually need this type of information. Alright? Another superpower of Paul approach to try to extract features from data is convolution. So convolution was started in the signal processing community. The idea is super-simple and I'm just going to walk you through some of the basics. I'm going to take an input signal. This signal can be anything like a vector, a matrix, or a tensor. A tensor, you can go to any arbitrary dimension that you want. Of course, things are going to become very computation intensive as we go into high dimensions. Also convolution can be applied to graphs, as I'm going to show you a little bit later. So the idea is I'm going to pass the signal. I'm going to apply an operator that is called a filter. And this is going to generate a new signal that is going to activate certain interesting features from the data. All right, The interesting thing about convolution is that it's a very general approach to extract different types of features from your data. You can extract patterns, you can extract, or you can perform a smoothing functions that tries to extract certain frequencies from your data. You can also extract something like derivative information. I'm going to show you an example in a second. So for example, you can extract gradients of light. And that's something that you can use to characterize the geometrical patterns in your, in your data object. Just to give you a quick idea. So this is my, this is my image, right? Very simple example. This is my operator, which is also a matrix, right? The image is also a matrix. Then what I'm going to do is do an element by element multiplication of the convolution operator. We are given neighborhood of my image. And this is going to result in another little matrix. And then what I'm gonna do is move around this window. And I'm going to parse all the way around this matrix or the image. And this is going to generate a new image. And you see that in this case, when I apply the Gaussian filter, what I'm doing is actually extracting some dominant frequencies from my data. And this is important when you're doing that Fourier or filtering on the inside. Where you can also extract geometrical patterns. And this is a very interesting topic that I think is of interest to chemical engineers. Because what I want you to do is to think about this image actually a subfield, the same way that we think about of a thermal field, right? So think about this is a thermal field or an image, right? And then I'm going to apply this operator. Looks a lot like a finite difference, a stencil. So remember that every time we apply a finite difference formula in two-dimensions, we come up with these little stencil that tells me how am I applying the derivative operator in both dimensions, right? And when you apply the same operator here, for example, this is highlighting places where the light is changing very abruptly and when the light is not changing, right? Because you're computing the first or second derivative, especially the first derivative, right? So that's important. And this is super important when you are highlighting the geometrical partners or the boundaries of your image. Now as you can in mind, when you think about it from a finite difference perspective, then you can say. Well, I can actually apply different types of derivative operators to highlight different types of interesting derivatives, right? So for example, there's an interesting operator that actually is used quite a bit in photography called the Laplacian, which as the name suggests, is something that is very similar to diffusion, right? So, so this is trying to highlight the second derivatives of the, of the field in different dimensions. And as you can in mind that this can also help you highlight places where the light is at its maximum point or with the light decides minimum point, right? Because the secondary, what characterizes these type of features? Or I. But now the question is, well, effectively this convolution operator can use any operator that we ever dreamed of. All that I need to do. I can put any sort of weights associated with this filter. And the sky's the limit here. So there are many things that you can do. But I think that the big boom in machine learning over the last year has been the ability for us to automate the selection and the discovery or what are the best operators to characterize our data. Sometimes it's not obvious. So for example, if I give you this liquid crystal dataset, it is not always obvious what are the dominant features that best explain the behavior of this object. We don't know if it's color, we don't know if it's the geometric patterns. Sometimes these things are just hiding from us, right? The typical example people get in machine learning is that cat or a dog, right? So we, as humans have some intuition about what are the geometrical features of that. But sometimes it's not that obvious to distinguish certain types of cuts from different types of dogs, right? So, so I think that that's the key idea. So the idea with these techniques is that you identify the filters through optimization. So you're trying to solve a very big parameter estimation problem. That is using the filters as the parameters that you're trying to estimate right? Now, the important thing is that these operators can be vectors, matrices, or tensors. And as you can in mind, the number of parameters here explodes. And this is one of the reasons why this technique has been very challenging to apply, unknown until recently has become actually very mainstream. One of the reasons why it has become very mainstream is because you can use graphical processing units, which have become a very powerful cardboard to or applying a lot of these techniques. Now, an important message that I just want to convey out of this slide is that the input to convolutional neural nets do not have to be images. And when I started working in this area, all the papers that I read assume that the CNN was processing images and iOS as well. These techniques are always images, right? In reality, anything that can be expressed as a vector or a matrix or a tensor wherever you have, can be fed into a CNN. And that took me a little bit of time to realize until I understood this data representations a little bit better. And that's why things so important. Think about how do we represent the data and feed it to a tool like this. Now convolution can also be applied to graphs. The idea is very similar. The only difference is that our graph is an irregular object, is an irregular mesh, while an image or a matrix, right? Is actual irregular mesh, right? The way that you define the window is just affected by diary regularity. So typically in a graph, we are defining the neighborhood of a node as your, as your particular window for the convolution applies very similarly. And this has been the concept behind a new set of tools called graph convolutional neural nets, which is basically the same idea you're trying to learn these filters either forgiving graph object to try to best explain the features of that graph object. Alright? Now, convolution is a really powerful approach to characterize this geometry of of different types of data objects. Particularly things are appearing in images like micrographs and things like that, right? So that's really powerful. However, it does not the only approach that we have to characterize geometry. So an interesting field that has been gaining quite a bit of traction is topological data analysis. So the idea, as the name suggests, is that we're going to be using tools from geometry and topology to try to characterize some of those features from a different angle. Okay, instead of applying this convolution operations, we're going to do something different. The best way to, I think for me to explain to you what this is, is to start with this object, which is a graph. And this graph has two very interesting topological features that people in graph theory always think about. Their thinking about the number of connected components and the number of cycles. The number of connected components tell you a little bit about something, the connectivity of this graph. So it is telling me the graph is fully connected or if it's disconnected. So in this case, for example, the graph is fully connected, so a number of connected components is one. If I were to split this graph into components, then you will come to write simple idea. The number of cycles, as the name suggests, is the number of nodes or return to themselves, right? So there's always a path back to itself. That's another important topological feature because if you don't have cycles in the graph, then this becomes a three, which is a different type of a graph object, right? Please. A very powerful approach. So there's this very interesting number called the Euler characteristic that basically captures the different topological features. And this object is super interesting. It's a number, is a number that summarizes the topological characteristics of an object and can be used for many different things as I am about to explain to you. So it's a very simple idea, but with a lot of potential applications. So let me walk you through how this is done super quickly. All right, so in mind for a second that you want to characterize the topology of something like our image. And remember that an image is just a matrix, right? You say, well, this actually has notes and Barrett has our data imbedded in the nodes, right? So the light intensity. So this means that this is a graph, but I also have numbers embedded in the graph. Okay? So how do I deal with those numbers? So the idea is to apply a technique called a level set filtration. So the idea is that I say, well, you know what, I'm going to start with the lowest number. The lower value that I have in those notes in these cases are won. And I'm going to eliminate everything that is above one, right? And this is going to generate a graph. And I can compute the Euler number associated with it, the Euler characteristic associated with. Then I'm going to go to the second largest value, right? In this case it's a three and I'm going to eliminate everything else. So degenerates a different graph and this generates a different number. And I keep going on with the process until I include all the entries and I recover a perfect mesh, right? So these records, a set of values that is summarized in a curve. Curve is called the Euler characteristic curve. And it's a super powerful approach to summarize the geometry of a field or of a matrix. Okay, it's a really powerful approach for doing that. Now, these Euler characteristic actually is used quite a bit in a field that is super fascinating, which is neuroscience. I had no idea about this until my stool and Alex told me about this. So this is the good thing about having grad students they go investigate stuff that you were not aware and then they can't communicate it to you. So turns out he took a course on neuroscience and data applied to neuroscience. And turns out that what they do neuroscience is something really interesting. They're very interesting characterizing the topology networks, right? So something like, for example, how different parts of the brain are interacting. So turns out that this is nothing but a correlation matrix. But instead of analyzing this correlation matrix using standard approaches like a PCA, what they do is actually represent this as a, as a matrix. And as we have observe, a matrix can be represented as a graph. And what they do is they do this level set filtration. Well now applied to data that is attached to the weight and the debt is attached to the edges. Because the edges in this case is what they're representing, how strongly two parts of the brain are connecting. Okay, So these are very similar idea will not apply to edges. And this information is summarized in this Euler characteristic curve. And this gives them an idea of how the, the, what is the topology of that connectivity of the brain and how strongly these different parts of the brain are actually interacting. The Euler curve can also be used to characterize manifolds. So these are data objects that look a little bit more interesting. So typically in chemical engineering we think of functions, right? So think about like a one-dimensional function. Well that function has topological features. They have minima maxima, you have saddle points. So you can characterize also the topology of that function. And you also do it using this concept of labor set filtration. So what you do is a slide of, you are going to slide slice the function right at different levels. And this is going to start revealing the different parts of the connected components and also the different holes that you're creating in this, in this object, right? And this becomes super-useful precisely when you're analyzing manifolds that are known as selling one dB omega in higher dimensions. So for example, this is a micrograph that appears from liquid crystals, right? So same example that I show you the beginning. So as I have told you, I can introduce a three-dimensional represent the intensity of light. And turns out that these field can be represented in this way. And then what I do is apply this level cell filtration. That all that is doing is it slicing off the image at different levels. Every time that you slice off, there are going to introduce these calls into the field, right? And this start revealing what parts of the image are actually connected and which parts of the image chart disconnected. This also start revealing when you have reach a peak or when you are reaching a valley, right? And this conveys a lot of very interesting geometrical information. So at the end of the day, what you do is summarize this image in a magical curve that is called the Euler characteristic curve. Well, it turns out that this Euler characteristic curve is super powerful as I'm about to show you and can be used for many different types implications that arise in chemical engineering. All right, so just to kind of give you a perspective, so nowadays we have been applying anthropological data analysis with many different collaborators. So we have been very lucky to get data from many, many different types of applications. So we're starting to work with people on trying to understand how copolymers assembled and create some interesting topological structures like muscles and things that in some worms and things like that. You can also analyze our space time dynamics, right? So like computational fluid dynamics. So turns out that you can actually monitor how the dynamics are evolving over a phase plane, something like that. And then that is important in understanding when the system is about to enter a chaotic mode or something like that. You can characterize surfaces similar to what I just showed you with the liquid crystal. You can characterize atomistic positions that come from molecular dynamics, trying to understand the topology of that object. And also our favorite for process instance, people that you can characterize also processed networks. And if you think about a network like similar to the brain, right? Which is, I'm going to give you an example, right? All right, so now I have entered the face in my presentation where I promised not to show you anymore concepts. And we're actually going into real applications. All right, so I want to make sure that everyone gets a good idea like the types of things that you can do with these type of tools. And hopefully this will motivate some interesting discussion. So let me start with super interesting application that came from our collaboration with Nick covered. So I had no idea that this thing existed. So interested in detecting endotoxin in liquid solutions, right? And turns out that the state of the art technique FDA approved for detecting endotoxin. Things like surgical equipment, relies on these LAL test, which uses the blood of the horseshoe crab. Horseshoe crab is one of the most fascinating animals that are still around on earth after many, many times, right? So they can develop a very, very powerful immune system that actually allows them to detect the presence of these endotoxins. This is the most medieval thing that I have ever seen in my life. And he's actually the state of the art for detecting in the toxins. While the reasons for that is that is supersensitive, you're detecting these things are picograms, right? So it's very, very sensitive. Now as you my mind, this is not sustainable and it's also very expensive. So now, nowadays, like ghosting the United States, hundreds of thousands of crops are harvested every year. Now they're returned back to the ocean. But you can in mind that this is not a pleasant procedure when they extract the blood for the animals, right? So that's not good. So people in material science haven't even tried to develop right synthetic approaches to try to mimic some of these high sensitivity and selectivity that, that the, that this test has. One of those approaches is actually using microemulsion where you're going to put this very small droplets full of liquid crystal. When the endotoxin attaches to the surface of this droplet, is going to trigger an orientation transition in the liquid crystal that can actually be optically transduce, right? So you pass this through something like a flow cytometer and this will generate these two signals. And this gives me this really interesting data object, which is a point cloud, right? And what is really obvious to you from your rice is that these are the geometry of this point cloud. This actually very interesting, right? It actually changes as you change concentration. For us is very obvious that there are differences here and here. He gets trickier is when you get into places like this, right? So that means that you need to rely on numerical techniques or, or, or other types of techniques for analyzing this data object. So the state-of-the-art technique for dealing with this type of data object is based on binning or counting, right? So you focus on a particular region of the, of the flow cytometer scattered and you count how many of those droplets are there. And people have found over time that this region actually concentrates more strongly as you change the concentration of the contaminant. And that allows him to construct a sensor, right? So they can correlate the concentration of endotoxin to the actual field that they actually observing so effectively you can create a sensor. I've also been using this technique to actually detect the source of the endotoxin. So it's something like a spherical calling pseudo Normans, right? So actually does create signatures also in there. So what we decided to do was to say, well, let's try to do something better, right? So what we're gonna do is try to apply these convolutional neural nets. So the idea is that I treat this flow cytometer scatter as a field. And I'm actually going to put a three layer field in here. So I'm going to put my target and I'm going to put two reference to references. So that creates a three-dimensional tensor. Then we pass this tensor to the convolutional neural net. And turns out that this does a fairly good job at predicting the concentration of the endotoxin. You can also classify what is the source of the endotoxin, which is also super interesting. Now the dirty secret about this is that as I explained before, the convolutional neural net cost, a lot of parameters and a big criticism typical in this type of our publications. Reviewers always complain about this, is, well, you have 1 million parameters. What does that even mean? How well does this metal general license right? And they are correct, is very hard to make. These models generalize, especially when you don't have that much data, right? However, one thing that I always tried to explain to the reviewers is that one of the super powerful things about convolutional neural nets is that you can use that as a first pass to detect if at all, there are some geometrical features hiding in there that explains your data. And that might help you make the prediction, which is one of the first sources of kind of validation that you want. Because if the convolutional neural net cannot detect those features, is going to be really hard to find something else that will detect those features. These, these approaches are truly, really powerful. So 13, that's one of the benefits of using this technique, even if it doesn't end up being what you end up doing, right? So precisely multivariate by this idea that I don't want a model with a million parameters, right? Which doesn't seem like something you might want to do. You say, well, let's look at topological data analysis. What I'm gonna do is try to represent this point cloud as a field. So what you do is just apply a little filtering here and represent this as a field. And you see that they start rebuilding geometrical patterns, something like a contour plot, right? And then I introduce a third dimension. And if I introduce a third dimension where you realize, which is kind of obvious is that this point cloud is not in both a probability density function into, in two dimensions, right? The third dimension is the density. And this start revealing the geometry of this data object. Now, if you know a little bit about statistics, you will realize that characterizing geometrical features of high-dimensional probability density function is not easy. When you have one-dimensional distributions, you can use quantiles, you can use moments to characterize those features, putting high dimensions as possible. So what we're doing is precisely use this idea of the Euler characteristic so we apply the levels and filtration and that allows me to construct an Euler curve. And then what we do is correlate the distance between the Euler curves with the concentration. What you can find something really interesting. First of all, the first important discovery is that there's actually a systematic deformation of the topology of this data object. So as I increase the concentration, this does not arbitrarily jumps between topology is actually a continuous deformation. Alright? And that's important, right? Because you understand that there's structure associated with this phenomenon. The other thing that is interesting is that turns out that there's a linear model that explains how this deformation happens with concentration. So that means that I can use linear regression, right? So all of the sudden, I can go from a highly sophisticated convolutional neural net, something that is much more tractable and much more interesting to, to apply. Alright. Another use of liquid crystals is for detecting air contaminants. So this is the response of a liquid crystal is a very thin film when it's exposed to something like awesome, right? So you see it's very obvious that this response right? In time, there's also spatial patterns and also their, their scholar, right? And those are the calories are very powerful source of information here because precisely this is a crystal, right? So, so crystals induce very interesting color information. And what we've been trying to do is trying to extract information from these response patterns to try to see if we can build a sensor. That tells me if I have also present in this air contaminant, in this area environment or some sort of environment. Or if I have other types of contaminants present like chlorine, right? And what we find is that indeed you can do it, right? And this is one of those applications that I told you about where the convolutional neural net becomes a very useful approach because it can simultaneously extract information from spatial patterns, color and temporal patterns. We can do it all at once. So it seems like a machine gun. But I think that that's something that you might want to do first because you cannot predict the concentration of the contaminant based on all those sources of information is going to be hard to do anything else, right. So I think that's why it gives you like a like a preliminary first screening of this, right? So what we did was precisely build a convolutional neural net. We're actually passing this spacetime response signals and tried to understand if those dynamics are actually being influenced by the concentration of the contaminant. And indeed, you find that yes, they're being strongly influence. And this is reflected in this kind of parody plot. So we're predicting the concentration of chlorine and ozone as a function of the spacetime response pattern, right? So that tells us that there's actually a lot of information embedded in it. And right now what we're working on is to try to use topological data analysis to extract those patterns and try to correlate them. But as I said, I think it's important to kind of first do a first pass with this CNN to see if there's something interesting that you can extract the data. Now, these convolutional neural nets can be applied to many different data objects. I'm just going to tell you super quick about a couple of other applications we've been working on. So people know they are very interested in quickly sorting plastic. Specifically what they do is they grind plastic into these little flakes. So this is not like plastic bottles or anything like that. So these are little flakes. And then what you do is pass this through a very fast our ATR FTIR spectro device that is going to be generating a lot of spectra in real time. And this generates very high frequencies, right? And what we wanna do is degenerates a lot of data, which is great for a machine learning too. And so what we do is encode this into something that is color gradient and your angular field. And then pass this through a convolutional neural net. And we've been able to find some pretty good classification of different types of plastics. One of the interesting things that we have found is that certain types of plastics kept very similar signature, so they're difficult to classify, but some others are more easier to classify. And this gives us some intuition into what aspects of the instrument we need to be refining to be able to tell disorder plastics apart, right? So this is something that we're still working on. Another application is we're trying to use this graph ideas to predict properties of surfactants for example. So in this case, I'm showing you different types of surfactants, right, with different types of charges or no charges. And one of the emergent properties that we want to predict just from the molecular structure is the critical micelle concentration, which cause a lot of applications from a experimental and industrial standpoint. And one thing that we have found is that indeed we can extract a lot of these geometrical features directly from the molecular structure of the, all of these different surfactants to predict this concentration. And we, with this, we're trying to understand what are the exact features that are actually driving this cooperative behavior or collective behavior? And this is some, one other interesting applications, neural nets that are actually quite virus. All right, now I'm going to show you a couple more slides, and I'll finish this slide. Alright, so we have five more minutes. I think I'm still doing good because I don't see Johnny jumping or anything like that. All right. Okay. So this is an interesting one close to my heart because I'm a process engineer. So in process engineering we're really interesting. Process monitoring. So chemical processes are fascinating, right? So they're complex pieces of equipment. We monitor them with a lot of different sensors, right? And one of the tasks that we want to do when we monitor sensor is identify different types of faults. So can you tell me when the catalyst is degrading? Can you tell me when this thing has filed and you cannot remove heat anymore, right. So those are the types of things that you want to determine. And it's not obvious, right? Because you're just seeing something simple like a Tennessee Eastman Chemical process, which is a benchmark. You have 52 process variables that you're monitoring simultaneously. And you have many things that can happen, right, left, 20 different types of faults. So one of the interesting things that we said is, okay, let's think about this chemical process as a brain, right? So fundamental is the same problem from a mathematical standpoint. So I am monitoring different parts of the process, in this case the brain and collecting time series data. I'm going to build a correlation matrix, and I'm going to build this correlation network that I was telling you before about. This is going to start revealing what parts of the process are strongly interacting and which parts of the process are not as strongly interacting. And the topological information of this interaction is hopefully going to reveal when things are going well, on, where things are not going well. All right, so this is similar in spirit to how people in neuroscience tend to detect certain types of abnormal behavior of diseases like Alzheimer's and things like that. Now, I am no expert in neuroscience, so don't ask me any questions about this. I use note that this is one of those techniques that they use, right? Um, precisely what they do is they construct the Euler characteristic of this network. And through these they tried to reveal when they're topological differences. Alright? So precisely what we did was to apply the very same technique to the chemical process. You'll see that when you are monitoring this time series is nearly impossible to make sense What on earth is going on there? There's a lot of things going on. So what we do is summarize these different types of datasets in the Euler characteristic. And you precisely see that there's a clear difference to when the process is behaving under certain faults and when the process is does not have faults. When you do not have false, the structure of this process is very different, okay? Now, at the same time, there are certain types of faults that you cannot tell apart based on this. So that means that those two folds are generating similar signatures in the data, right? So you cannot tell them apart. So maybe this tells you something about this. Your question. Yeah, and I'm monitoring the questions here, so OLAP uncertain. When I'm done, I'm promised that I'm about to be done. All right, the last example I want to give you is we're actually monitoring the topology of solvent environments. So turns out that my colleague read one length here in the department Molecular Dynamics person. And one of the things that they have found out is that the sovereign environment plays a key role when you are processing things like biomass, right? So you form this very interesting hydrogen bond structure that actually can influence their activity of different types of molecules in there. So what they do is they do molecular dynamic simulations. I'm not an expert on this, but one of the important things here is that they see random fluctuations. Are you going in the dynamics, right? So this is natural in this type of applications. And they're trying to understand if you can find early signatures in this three-dimensional field than mice suggest that this molecule would be highly reactive and non-reactive in this environment. And they want to do it early on because every time that they run the simulation, it takes them a lot of time, right, two hours to days of simulations. So if you can detect this early on, that's good news. So one of the things that we observe is that indeed the top of the topology of the solvent environment early on is strongly correlated to the performance of the solvent. And that tells us something about God or reactant is about to race to react in a particular environment or not. We've been able to correlate these two reactivity, right? So we can predict if a given solvent environment will induce the activity or will not induce reactivity. Now we're also trying to understand like what types of solvents are good, what types of solvents are no good, right? And that gives you some high-throughput approach for February in absorbance. Another interesting thing that we have found is that even in this dynamic simulations, you see these random fluctuations over time. Inherently when you compute the topology, the Euler characteristic, you find that those actually are not at topo, topological invariant at some point, right? So what that means is that the topology actually converges very quickly in time. Am I still fluctuate from a three-dimensional perspective? The atomistic positions are fluctuating. Pull the topology has actually converge. So that tells us something about when can you stop the molecular dynamic simulations that can actually save quite a bit of time. All right, so with this, I'm going to conclude. I want to thank all my students semester and Alec Smith looking for a job, also looking for a postdoc. So you want to hire him, Go for it. It's also a bruise Jang and imaging which are going to be looking for jobs in the foreseeable future. Let's Pope, then also my collaborators. So I'm very lucky to have colleagues that provide challenging datasets and motivate a lot, a lot of this development. And also if you're interested in our research. So if I have any on the graphs in the, in the audience or you have on the graphs in your groups. Please encourage them to apply to Wisconsin's aware, very exciting about all the stuff that we have going on. Some new hires in the Departments of new faces. Also, we have an open faculty search. So if you're a postdoc or a senior PhD student, also feel free to reach out to me. I'm more than happy to provide information. Alright, so with this, I'm going to stop my presentation and answer any questions that you might have. So thank you for your attention. Thank you, Martin. We have a number of questions from the audience. Start with one from Pang thing. In Fei Chang, is there a data dimension limitation? If we're using the Euler characteristic curve, your examples are, in a 3D low dimension, how to deal with this limitation? Yeah, nodule come up to arbitrarily high dimensions. It becomes increasingly computationally, as you might expect, complex. I will say that what is tractable, these four dimensions. So we are characterizing a space-time fields with four dimensions in computational fluid dynamics and heat can be done. The idea is very similar to apply filtration. So Julius, I look at regions in a feel, for example, that certain value and you get rid of everything above that value. And you characterize the connected components, the calls, and different other topological features. So yeah, it applies to high dimensions, but it can get expensive. Not as expensive as convolutions, but it can get expensive. Yeah, that's a great question. The question grammatically well, yes, the graph presented as an adjacency matrix. Presumably you could find convolution operators and matric, yes. Yes, yeah, absolutely. So there are different types of convolutions, approaches that people have been looking into the graph. By exploring the adjacency matrix, by exploiting some of the topological features like the, like, the Laplacian, things like this. So, so yeah, there, there are different types of convolutions that you can do. Nowadays. Just to give you an example, my looking into defining this window a little bit brother. They say, well, the problem with this windowing approach is that it only capture certain local features around and note body may actually miss on global features of the graph. So now looking at convolutions that go beyond the local region, they call these message-passing. So I'm so gather a lot of interesting approaches for designing that graph convolution. Yeah, VPs convolutional neural nets for AFM images looking for artifact. Yeah. But then we kind of wondered, well, these systems are designed to find cats and dogs. And maybe the filters that we're using are not really exactly. It filters for AFM artifacts. Maybe that same question, I'm not sure. Yeah, Absolutely. So absolutely. I think that the the convolutional neural net, if you train it, is going to find the best filter that works for your target application, right? And are now making sense of those filters might be challenging because they're just numerical interests. So really when you interrogate what were the optimal filters that these to identify, you really will see a matrix and you have no idea what, what on earth is going on. But when you apply the convolution to the matrix, it starts telling you what are the geometrical features that is looking for. So yeah, the, the, the filters do not have to be tailored to the cats and dogs. I think probably what you're describing is that sometimes when people are trying to avoid training the convolution unit because it's really expensive. They apply existing convolution filters that have been obtained by Google and places like that? Or is it transparent? Is a pre-trained neural net, right? Yeah, but, but yeah, you can tailor it to your own, to Europe. And yeah. There's another question. Complain about possibility for real time implementation for fraud detection. Yes. Yes. So you can do it. The way that we are applying this TDA approach for process monitoring is that we actually look at a certain window. So it's similar to how Posner fall detection works. So do you look at a given time window and then you record all the signals from your process over that time window, let's say one hour, right? And then you look at the Euler characteristic associated with our window. If that departs from the, the other data, the dataset under normal operation, then we can say, yeah, this is on the faulty detection. There are other approaches for doing this like convolutional neural nets as well. But they use these very same idea of the, of the window is a window operation. Okay, There's a question also, maybe from wrong Lu, from 80. I'm wondering about the example of chemical processes. How would one figure out which variables, parameters, duration of measurements, et cetera, to use it. Seen these choices would be important. Yeah, that's a great question. We're coming done much into that, but look much into that. But that's definitely something that I've been wondering about. I think one of the interesting things, and now you can do now you have these tools is precisely identify which variables are actually driving the topology, topological differences. Sometimes you, yeah, do you discard a certain component that is strongly correlated with others so that you can in mind that that really does not contribute that unique signature to your topology. So you can start doing analysis, like Leave One Out type of analysis to identify what Barabbas actually makes them make more sense and are actually driving that topology. But those type of analyses tend to be a little bit brute force, right? So if you just try different combinations, I don't think that there's a good systematic approach for doing. But I think these tools at least start giving you some indication of what might actually be relevant or not. How do you properly select your information? Or potentially what new sensor I'm, I need to write two to actually highlight the aspects of the process. We have another question. Have you tried to use ed serial learning approaches at all your work? Absolutely not math. It. So I guess I tried to stay away from machine learning as much as I can. Sometimes is unavoidable because there's so many techniques and interesting stuff happening. But so I, I agree having using machine learning more as a tool and really focus more on the data representation to be honest, but I'm familiar with it. I just haven't tried it, but definitely you will be interesting to do some of those techniques for some of those computer vision type of applications. Some of the things that we see is that precisely, sometimes there's confusion in the computer vision when you introduce the small amounts of noise. So that trolls and our neural net to crazy regimes. So, so yeah, so I think that these approaches can help robots defy some of those generalizations. We just haven't done it, but definitely it's an interesting area. The question. So, so the trucks with feature selection and we have an image or something that's not an image, matrix or cancer. There's not really any notion of time in the feature selection, yes. But a lot of times we might want to pick out features that are changing a lot over time or longer, important to protect, predict future evolution of a feature? Yes. Yeah. Maybe I'll just be interested to hear your thoughts on this point. Yeah, that's a great thing. So I think that goes back to this idea. So there are two ways of thinking about it. So the way I think about this Euler characteristic is that I'm taking these snapshots over time and I'm trying to project them to a low dimensional space. And maybe you can actually build a dynamical model over that low-dimensional space to see how those topological features are evolving over time. So that's one of the things that we've been trying to also doing computational fluid dynamics. So that's one way of doing it. The other thing that we've been trying to do is to characterize directly the topology of the spacetime feel. So to understand how time and space are getting convolved in there. And tried to see if there's some interesting features of that behavior, joint behavior that might be explaining the date or the emergent behavior of the system. For example, we have done with the liquid crystal type of things. So you're absolutely right. I think that monitoring how these features evolve over time or extracting it inherently dynamic of features, right? Like timescale some things. Yeah, I think that that is a very powerful source of information for sure. In a lot of, a lot of things that we're doing, right, like vibration, sound, and oscillations things. Okay, funny. Can you summarize again, maybe with an example, the advantage of the graph representation versus other representations. Yeah. So, um, so one of the things, first of all, if you maybe let me go back to the slides pretty quick. One graph, one just to explain it. No, I got two all the way to the beginning. So let me go with this one. So first of all, when you are comparing against a structural data, something like a mesh. So think about a vector or a matrix or a pencil. What happens is that there's a predefined notion of order, right? So do you know that these notes are always adjacent? Right? And that introduces rigidity in the representation. Just to give you an idea in, in imaging, that is a problem when you apply computer vision, that's a problem because if I rotate the image, all of the sudden this becomes a different object than the original image. So in computer vision, you need to do a lot of sophisticated approaches. People call it like a rotation, right? You just expand your dataset by introducing all the possible rotations of the same object so that the neural net doesn't get confused by the same object, right? So this is what people call rotational variance, right? So that means that the way that I rotate the object will actually induce a change in the, in the neural net. With graphs, you don't cut that problem, right? So you eliminate the dependence on, on, on a space. Now that might be good or bad, right? So in certain types of applications is notion of a space is important, right? Some other types of application notion of space might be irrelevant, right? And, and one of the things that we have found is again, some applications that you don't need to know where, where, where you are located in space to actually make a prediction, right? The other thing with graphs is that you can introduce an arbitrary number of attributes. People call it in the notes and in the edges. So it's a very general approach to actually capture very sophisticated data objects compared to other types of approaches, like a matrix and vector that might require a high dimensional tensor to be able to represent all those different types of attributes. So, so it gives you a little bit more flexibility in the expression. All right? So, but again, in some applications is better, in some applications it might not be the best. Okay, Well, I think it's probably time to wrap up, so thank you so much. And we really appreciate you're visiting us today after the presentation. Great. Thank you, Martha. Thank you, everyone for attending. Feel free to reach out if there's anything I can go take care. Buh-bye. Thanks a lot. Thank you. Take care. See you then.