[00:00:10]
>> Figure out the right so this is like my little. It's easy to figure out why or because here who are questions and who are not so here's a beer for you then we publish it feeling the need for 2700000000 can't seem to get it people said to do it gracious but it is the rest of the day possibly one who publishes other people he really isn't it dealing for every block in United States he when he will as many people are on that walk.

[00:00:42]
Yet another table which is how many people are very raids all 63 combinations here on that walk to get me to another table how many people who are over 18 is very very they did not hit 100 people over the border to Stansted very serious. So that a lot happens.

[00:01:02]
And sundered bottom line we have every age but not life walk we have that bike walk route so we got. Next day if you construction we actually be constructed that one of them is a stick from the populace the 5th which is a problem because it's nice that.

[00:01:26]
We could do that big the best job protecting people. So we didn't know it is time that it is you construction was possible we knew it was a theoretical possibility but we didn't hear of it it was actually piss off the boat and partly because commercial off the most elusive We use an optimizer you really need a sax all her life yes.

[00:01:53]
You're. Right all that off. Book you know that this is life so. So to do it we actually needed our. They're all. Paying for needs big $5.00 an hour and it would set a good time at $64.00 who are here to $88.00 gigabytes of RAM so we're probably right on a few $100000.00 you're probably doing it and and use this commercial optimizer because if you concluded you have to use optimizers and he's proven culpability to do this if you're here programming solutions and all.

[00:02:38]
The people on how to grow is expensive but you get a pretty good license here so you can probably do it here right. And then get a set it. Is 538 people. So we have created 8000000 micro directors which when you tabulate that you get the published of this that we wanted to let me for it.

[00:03:06]
We did it we can start off you just sit here but the question is how accurate is SR multiple weak instructions that reduce the cities the same statistics. So that did your goal is not reconstruct the hundreds of my fear that the real goal the single that John lives on this block was John's race and how many kids this job happen are made the same rights that's the sensitive data that people want the census there are that.

[00:03:36]
Stupid. We didn't need to or should be available databases that are available for him. And he walked in and he coded it created by a need address age and sex and he took our city didn't really get to create our reconstructed migrated from him not on it but on address needs that right then we could look at a name and then we could learn it in a city and rights.

[00:04:04]
Groups are called that would be cool so that's all a litigious hack and realty that did it would have it with grace that not update to the commercial available it would have made but not records and we we very With only 45 percent of the records are played it for 45 percent of the records needed backlash for address age and sex.

[00:04:31]
And it also so we can cheat we go with the real hazards are so all the ones that we link we've got 38 percent of them are worth 17 percent of the earth's population. So if you do this with your grandkids like that person is you're going to need some sort of extruded for it and this it is if you can purchase or you get like you say stalker.

[00:04:55]
But you want no child you talked. Her into it if your goal is that no John that is you might not get that right but if if your goal is to find 500 people or maybe John and freak them out good each one say the Senate is fairly this information about you and for 70 percent of the Days him I doubt if you got that right those are the ones that will publish the others the small bottle right there so this is a problem for us it's a it's a special group it gets less of a problem for a specific individual but compared to the other research that we have done where we've done an hour it took it to attack with our.

[00:05:37]
In the database of construction the reeducation grid will play less than one percent was I point 01 percent. So so our track was good it wasn't perfect and outside attacker would have a harder time because they don't know the answers but we were able to do confirm we had occasions for 70 percent of the US population which is a problem we did not reconstruct the families that is still research for all of your profit to do that and publish he's doing it to be really close to the all of them and we did not really cover the detail self identified greatest service for example there are Americans and there are Native American publications we have all identified which right there we did not cover the article.

[00:06:27]
THAT SAID IT sector would not nobody should be education for correct David after conferring with an External Data Source Of course if they did confirm with an extra bit as for us they could then word other factors that lead really identifications to be more correct than not right so that is probably correlated variables that if these things are true then the reactor occasionally does really hot.

[00:06:55]
Because their side lost a lot of people under some oxygen if you get one there's some bosses there heterogeneous and somebody out there who genius and their son and so when you take into account those other kinds of verification bigger out why the reconstruction to where it was that there could be so.

[00:07:17]
So the Protect we can do protection mechanism for 200-2000 that that protection and it was his fault possible swap so what he did is he looked for households which were very similar in size. It could be swapped with each other it looks cold it were very distinct the bird it works or it was a one walk with one household on it it had 3 people aren't needed swap them with maybe with another bought with household 3 people are not with one person but need a whole bunch of those needs call that a double her household and eat did not explain this is more detail than I just did we did not explain how the selection criteria or the geographical range of or which swapping took place which thread which if your bits are soft and which ones would not fly off the impact are there all the the great none of that stuff.

[00:08:15]
It was at the slaughter of stated that the field was secret. So we told you what we were doing a little did give a lot of details. And other reasons to get a lot of details as it turns out that the neck in this room is not very strong here the details you can do to spot it.

[00:08:35]
So we can know that it was a lot for a whole bunch of reasons. This walking assume that you will be able to get access to the soft 100 percent detail file who yelled at it we can reconstruct the 100 percent he thought and he also did not spot it 100 percent sources tell you that.

[00:08:58]
Answer will give us a good swap with 100 percent you didn't even get stuff out a lot of. Who did it that's it created for you then. You could easily get really big past the chains of 2010 you couldn't get really good off the wise use of one that people were focusing on this is a problem in 2010 and yet it is that there was the man for architect a census of the net for protecting the census was invented down to sex.

[00:09:32]
And it really is still not quite understood here for several years after that. So we we know that there are basically 2 mathematically valid approaches were protected in general that the data publication would get published less data and leave his publish less accurate data so clearly it would publish less data I think it is if you start shit would have been harder didn't work Riette and vigorous reconstructions that some of you just can't do because for.

[00:10:07]
So little local about publishing last state that's called good the protocol very often but you traded reconstructions as if you published it which stated you didn't do everything for thousands if you go up a 2nd stage so this idea goes back to the computers and for Richard back to life in there he cited it they called it periodic crude tracking.

[00:10:30]
And in this case that up because there were these interactive that this is called databases the satisfactions one of the researchers just as she queried it realization was that they could learn about it a vigil for example you could ask the average age of everybody and my clock and then you could ask the average age of everybody is such a great growth if you've got it made you can know something it's about paying it's attractive who lives in something.

[00:11:00]
So it turns out early on we need is hard on and a good example of why prerogative heart so you really have that that the physical good will come the beginning of the cough and this is a couple sophistic and it turns out that you can use a sites all over and precisely reconstructed right there as a way to do that you shouldn't construct the saddest sight loaded thing that are pretty variables where every aisle 1st of that a graphic parable there's only one solution.

[00:11:30]
And all my computer. Modeling I'll find a lack of this off Actually I can do that reconstruction of course to some. Breakthrough it's the sort that get very good for you 2nd if it block out this route. There are 2 solutions. And they have this in conduct so we know that that sort of need all solution.

[00:11:57]
For both universes there must be a card like you know this gave us this 2 possibilities so if you need a set solver to do this and it's a good part right in series out on it isn't it hard that's really bad if you want to use regarded as your way of protecting your data that you're publishing you need it doesn't really learn from the dinar this is a degree not only his career out of it the part but it happened in karate existence as he's off you can just make rid of the group is against the system and indeed it is reconstruction and the probability one if you ask that question is you can recover the entire database.

[00:12:39]
And because the attacker has no idea when you asked and of course. So we just can't say we're going to publish last week Joe know when we cross the threshold that we publish too much data and call as it is other databases out there right said there's other databases are also good this is the feel of it you can use your database your structure so not only this we are getting at the heart but we don't know all the pool all the data available it happens and it could be that there is that they don't have enough data to date but next week they're.

[00:13:20]
Sitting at publish less accurate. Now job that's it started coming across the Mississippi coast from the well they're going to commission property. Q That is it has it's called the strikers it accurately through confidential data it is however my existing hire competition that pays we've got the right let's talk about it but we just know that the problems should at least if there's a few accurately a soccer match.

[00:13:46]
So the question is. How accurate should the Republicans be. Still and I as I mentioned this is on the cover. So if you have this extra information. Your head and figure out which of these 2 solutions is the right solution. That's right. So. Did Bridget privacy gives us of this illusion of how much noise out in after our publication should be.

[00:14:20]
To protect our lives it is a mathematical definition of privacy law caused by a religious. Definition is no different it's on the attackers external creation Newtons of the hacker to hear that power in its composable you know nothing about it. But the cool thing about this is that the series at the Tigers competition about it.

[00:14:45]
So it's not like we're busy A.T.'s and needs to have the attacker kid try all to the 256 possible that he's. Recently acquired right off to the 56 possible. And they just won't know which is the correct heap because it will be up to recognise that part. This is the kind of graph that we use on.

[00:15:09]
We get to harvest the current between privacy laws and accuracy so we don't say privacy because we know nothing of privacy the individual's out. Privacy laws called by the did publication indeed get that so it was already. So this tells us this is a valid one that's on the right you see last part of it a publication.

[00:15:38]
Accuracy is a harbinger of scale you just made from 0 to wonder there are 100 percent. Their privacy laws. Is the gal with a video did they call us on. It has do you know it's. So it gives us provable barrels on the maximum brightness he wants. And a little no good nor is it needs to be added to guaranteed that other program on privacy.

[00:16:10]
Is called differential privacy because it models the differential that each person experiences from having their data. Inserted passing always gets roughly proportional to the lead to the back of the single person database so that the release is based on a 1000000 people there was a very small bit that there really is a space from one person and the amount of those is just proportionately very large.

[00:16:41]
So the basic idea. Is that you take this that this is it you want to publish the radicals so I like to visualize it this way I said there's a voice barrier we wrote this is this it's for the it's very well get your music. It's really simple to explain it.

[00:16:59]
And this is some of the statistics go they did it so that we updated $0.00 black males on that walk. Which it still says that one of her right but if you're a statistician you're adding up a lot of clocks that make sense because you know on the earth box they get averaged out of the other walks in the north pole either because.

[00:17:21]
It's hard to figure out what you do on these mediums that means and if you deny your project just said you know it's directly to the media that means there's a lot of very efficient use of the art to turns out that there are some other mechanisms you didn't get Tiger back on the media is a means to the same privacy it was.

[00:17:42]
It's a clever thing to have to do this is we have to figure out how to add it was in the most efficient possible. So we can preserve the statistics of the terror. While not preserving this existence it or not. So the cool thing about this is it was free as a noise you can still get it is reconstruction you'll still get a solution but it probably will get across that.

[00:18:12]
It will have no way to know if this approach solution or not. So good you could now correlate with external data you need to say OK it's not the right solution had it at Drexel did it is working the straight through here it's all over. But you still won't know how it works only see you must know whether the reality was one here minus whatever somewhere else.

[00:18:36]
So you can also take the statistics and make them consistently to move the zeros we can make everything. You did yours yes 1st processing as long as the post processing happens to the right of the noise barrier it doesn't use any kind of values you're still good friends with part of the cool things about the products it was.

[00:18:59]
So good we're going to be using different privacy for this week on the census and here's how we're going to do it we paid. It is send your response file and we create a city called United file we count that up and those are the that's a good portion.

[00:19:16]
For House of Representatives good had for the states right now is outside the purple. Then he added it this really confuses me when I joined the census variation and it people's responses did if they had dogs or people on we also would look at the active ministry of life for records stuff earth.

[00:19:39]
We had a disclosure avoidance system to take citizens is it by mistake is the price it was for if they did not exactly see this it was what they should be more accurate British of the less accurate my slides are on the Internet people in. The U.K. We want you to put it.

[00:20:00]
And then to create this is that it might predict that it's looking at a private it's private if you put it that we may publish it that hasn't been cited their own bugger he said it made it that need to be published. So. We checked what we've all really very nourishing stories of policy for birds of their privacy that's been really hard like we needed her to do get rid of floating point numbers like you know you hear her talk to her like you need 148 bits of the story like the media accounts of this friend could meet.

[00:20:39]
You know in bits of her and it's crazy that that's the part the bit that we say that our sound is or that the entire country has because the funds for accuracy it is that there's a bit of the power of it in the parts of. So little heat every day you can't use the product there.

[00:21:01]
Are So there was no off the shelf system for doing this we had to create one and we needed to have this characteristic that we would be making hide our hair all statistics for like states that for blocks all the sooner we got such a creates that consistent variables which is why are making market it actually rather than just making it so back to the system works is a big cables and then we figure out who did it.

[00:21:31]
Here's how so the flashes look at 2 out of the 1st algorithm we take every block we add always to a community we bought it turns out that we call it a pocket is that of a bottom up algorithm that in the system. But we do have to talk down to.

[00:21:50]
Where we take our created a 1000000000 records we get it absolutely we have a national history and all the different kinds of records then we. Hugh the real data we give all our sick little time to wish we had was that if it were the easy state this year we did the same thing to tell you the old you just said here at the track of these same thing for example walk so we end up with a lot of measurements but it all onto the noise barrier so they're all safe to tabulate and then what we do is we started the national level we need a consistent history and then we take that marker data records we our people every state that we consistent his signature every state and then we do that down to the county got the tracking of.

[00:22:39]
So this is his hair so straight out with those on the left. Are top down over and this is accuracy so right here we know 5 that receive roughly and this is at Salon from 0 to 6 and you can see that the blue line which this was done on the next thing for the dataset is the fax it's a new version districts and it's somewhat There we see down the bottom it's only point $15.00 after it but when we act together all the tracks it depends a little it's very accurate and it is national level it's basically distinguishable from too much of the.

[00:23:16]
Variously bought by Bach or appeared at this rate by this or that group of it never gets better yes because we never were holding the top with accuracy so when you add a lot of the easy numbers together you get those numbers depend on the distribution you're using this week.

[00:23:37]
Yes. Or. So we haven't decided epsilon for 2020 but we did decide it was over 2018 and happily disappointed 80 data to that it used was point 25 which is here and the reason we use point 5 is that it was it tests. On there we didn't do all the other things that we would normally do practice it like not response follow up like.

[00:24:07]
Really good used to it was intense was like all over par so the idea was why are we going to get people's our privacy laws if the data isn't accurate or right. But we have decided we're pretty pretty good at it but registered it was asking people what levels of that received it required and the answer is we've got a good number they think there are questions there are about some of them.

[00:24:34]
So. It's your educated you know tough life this is right that the block by block out really isn't as good at even immigration just their body to worship this or this was a surprise pictures of the certain point at the top there. So it's the same privacy laws assisting with this your purse but won't hurt if you better accuracy that's important lesson just because you see different of privacy doesn't mean you're going to get it if you are afraid off of accuracy versus privacy loss it means you're going to be on their current but different mechanisms have different curves so our goal there is a come up with the very best heard that will come off.

[00:25:13]
The public policy choices which is a critical either Sunday and where should the accuracy of the. Cell phone. So this slide is kind of a look at say but. This is what Google was doing our great highs responses I mean the noise that individual might prevail through that that's the least efficient way to view this is what some people think when you say different pricing management is the right to deliver the results are not very.

[00:25:47]
This is our pipe are bought out for them it is it is like the top down and really we want to use the most efficient that are possible but where you go my goodness my cars. This is life and he's a columnist as he talks about the marginal social cost of the marginal social benefit and.

[00:26:08]
Because he says it what we want to do it want to set up slower growth in marginals with ripoff equals a marginal social budget and he has a paper where he computed that for a particular database He looked at the increase of cost by get anything that versus the benefit of having particularly on federal funds part of it.

[00:26:28]
And heat so if you need a difference of various layers it's really funny like all of these people they say I'll call the River Center for anywhere they want to be and you can assume that it's going to be possible right and so I can just figure if it did come up with actual on the table for.

[00:26:45]
So we don't know what the correct value will be and clearly if you're economists here at the Pentagon all these extra you. Know your prices for. Your sake I put. You through for. Chapbook by. Yap yap one record per person that's what we mean by recruiting that's another statistician thing.

[00:27:12]
For the other questions at this point. Though there was a place for a. Center Yes. Yeah so we had 8. Data storage executive policy hit. And it consisted of the assistant director and has chaired by the deputy director so the deputy director is running during his career in public service.

[00:27:43]
And his business from it here is the director who's point be administrator does not sit beside that that is purely a citizen of opposition to this and. That's the good. I used to get to sit in a room with these separated at about this stuff because I used to be a therapist this is a good review board but now I'm so busy with somebody else or the slippery floor so I do it because.

[00:28:13]
That's the only girl it was going to slip through Forster or her or both of choice for sure not those were the sort of girl feels like it's just a good question that the secret was. So that's a really good question we publish that to simple standards and some of our standards say very very clearly that we have to stop meant to be exact that it like this and traditionally we had held this flows or informal methods off the table we published documents of the kinds of methods that we use but the methods were not strong enough to be published and still be held back the actual methods that we were using were the actual practice and it's very similar to what happened in cryptography between that made 6575 I used it needed cryptographic algorithms were kept secret and then there was this realization that secret algorithms work if it was strong and the National Bureau of Standards decided national or you know I.

[00:29:27]
Designated person standard for the United States it would be strong enough that it could be needed public and the only thing it would be kept secret would be the starting gate of Iran they'd get rid of heat so for us our goals and everything to call them except the random number sequence of the is and you could say we used to say the initial setting of the real number generator and that was in that light we had one random number generator and now we're actually using cryptographically strong random numbers for every draw which is good a chance.

[00:30:01]
But. We're using the is the already various structure and if you look at the AT THIS isn't it turns out that only one out of every part of Paul random numbers are actually read the rest are based on. So this sounds suspicious it never had to deal with this question about more than reading the numbers really rare but we did yes.

[00:30:28]
There. Was a slow. Thank you for asking that question so give this a that you could if somebody had it in the back rooms that's a lot of it you should get back to original data right is that there again we have to see the data so. What are the things that we do do is we've calculated it differently private manner the relationship between the original data and the result we actually populate the other.

[00:31:02]
And. Now you can say will you get back to college so that it reports a higher or even deadlier 0 right then course again because of that the output is equal to the end so that someone who you have to trust that the software that will run 80 it will bit see over them so that we say that.

[00:31:21]
We are doing things is it will be easy for people to verify that the results are actually not in words what it is that we are one of the things that we are doing is that we are not holding that hand at the population per watt. So the result walks were you can go to that Washington review live here and you did look really needed a lot more than 50 people there or other blocks really are another 1000 people here did you know did it there's hundreds of people present apology so because the blog cancer not being a poet said that this is an easy critic that the input is not equal.

[00:32:03]
But you have to plug in politically very far because you don't really know that you are going to say it's art but people don't like that. Idea that writers at this point. Yes So. That's correct. Yeah and so I think it's the most happening in the head of people at this year we haven't really discussed it yet but there's clearly from other projects that I've worked on so I will tell you about the other project by this prompted by my work on the terrorism project 55 years ago and we created this is better for our office.

[00:32:49]
And the network have nice people who are terrorists right here so the problem was. If we publish says no to the no work database of the republic is there. Any kind. A bit like that would be bad if it were sent here. But if it is another terrorist and you don't find it then that's bad because I think that it is right here like that but people getting a distance of it all so who publish the I.D.F. and people go do it and they said you know I've got myself AND YOU'RE MY CREDIT FOR LIFE Yeah but that's not there like actual barrels of T.V. He's not.

[00:33:36]
But we're not you're not there to use that to your. Life there are so you can play it so a lot of people really understand it works and that it is and it's just not clear that he can parent that kind of public education with him so one of the things that can only be published symbols and then you can say well you know that's what you're good at somebody else's example and whether you work in sample or not some potential but the Census is not as simple as that it's a universal language and so there's that challenge.

[00:34:13]
So that despite it and steered with this I said was it like well we're City accuracy the right answer is that's like the real data are you could be reported certain and rested recorded or heap instructor differences right algorithms so that these statistics this split between men and women is more important than this but between the ages or you get needed sort of the split in ages it's more here that this should be pushed there's a that's another cup of coffee prices paid where should it accuracy go pretty much the guys on what you want to do with the data if you're interested I will give the services the children into adults then you probably want each distribution get more or if you're interested in.

[00:34:56]
Your city and political organizing you might be more considerate of race. To react to what we get. Right now. It's really hard to get these out on this one he didn't write but this talks about overdoing this in science and then this one I did great with the season this is a summer intern and this one actually we're suited entire get a basic instruction on that hypothetical circle and you download our code from the website and run databases and search yourself.

[00:35:29]
So it's now clear the partners to put it all parts of. Their life right here in this life will be needed these are available all of the stuff works like that they're all from my personal work site you know you can keep that in mind so that this story here is that I'm really fascinated by clicking here.

[00:35:52]
You know and I'm really interested in how to get the. Rights it through very hard to change the number get here on a keyboard. And nobody picked it up for me. To be corrected sorry about what changed my mind there and then like let people start from here again because.

[00:36:20]
Yes. These are. All. Well settled let me give you an example OK. This one did need uses of be this so that the Census is not where the action is for the Census Bureau stayed here for the start date here about the thing called the there. Where the we ask what are 38 people United States 8.

[00:37:04]
Agent questions like the questions and they are legally required answers and begin it for me said something to their houses. This was called a riddle Margaret is part of Billy's started by probability sample so we can draw lines in demographics to know that we are a lot of people each of these categories this is a had arisen a few people who get more response are OK I'm not talking about this yesterday.

[00:37:37]
So the good news for this and you know census data is to calibrate the population down to the C.S. Of course it's to be a force in the House of Representatives all the skids use it for driving their legislative this or so in the old days. In states like this one legislative just this groups were drawn to be really big in urban areas and start out to be roughly what they did there'd be a legislative district by Per urban area there'd be a legislative districts for growth areas in that population down to this huge city like 10 times as it is people who are representative for the birth of the people as for the wrong people it isn't that the wrong people had this proportionate political power it back with the subject of several Supreme Court rulings in there it's what we call one person one vote and the cosmic Asia than the body Rights Act is that legislative districts should have roughly the same number of people.

[00:38:45]
Will have to figure that out will you count tally up the number of blockage on each walk in the legislative district that legislative districts are drawn up block out. So that's the run of the need uses for this and your sense of State is by the Justice Department.

[00:39:04]
Evaluate redistricting exist see if they are evil or not that is GROSS If you're interested in drawing a science if you're interested in using either the racial information or the agent permission to capture in your district the kind of voters if you want to have it because today.

[00:39:25]
You no longer Schuster voters said those are the uses of the data and the question is how accurate does it did I have to be to support this uses so you could take our data and you could you could run the you could run the next innocent name many times on real data and then you could see it how often are decisions made on their real data or on because of the noise etc then you can see how walking Are those decisions wrong different levels of that so.

[00:39:59]
That this is not hypothetical long ago by the British never ever ever go live. In a forum such as this but I'm going to have it right. Fortunately at the 2nd sitting here you don't. Seem to live in. Cuba W W W 2 doesn't census are out. And he's the right person who there is you get it that he did he cheated on census aka your fine our official they call yet these are the birth of Florida these are more right civil rights are.

[00:40:37]
These are are called the state. And Medicare are census we have the night imported and we don't publish the 948 that we need more right it's actually it was made available all by the National Archives and Records of the destruction of Iraq but it's replicated on the whole I don't.

[00:40:58]
Know which is a slight 6 for the most beginning as you can look at the icons data. And he took the 8 different road system for the 28 and 6 and we made it more difficult times with different levels of ice off so here we have waited 4 rounds for the next 1.25 forwards that we have supported by because it's takes a lot of work to run the thing that is really optimizer But if you really want to write yourself there's the source code here.

[00:41:27]
And see if you can download our source carrier and if you go to a few of them here the items website I'm just going to it will probably work on the i Pod a sense this disclosure of court is. On and look at that demonstration did it really US Census so you can actually hear a load Big 8 group or not redistribute their data you can download the data that we use.

[00:41:55]
And you can find out whether or what levels of noise we need the Department of Justice for most of the wrong information for history and if you want to do that please email me because this is like a current research project bars. Guess who. Would work. So he began this project because yeah we're going to work with you and what causes the error why was it higher or lower yet I offered you.

[00:42:35]
A full 5 to. Say that it is our reactive cation the other swap it. For help us move forward from. There so I can answer that question. The other question. I guess. Is. Was. Was is. So that Mrs Wright well over 20 times but that is not our correct model because it's very easy to control and it isn't exactly that that is is an underhanded computationally and it is on their information and found it all to assume.

[00:43:32]
That effectiveness all the information in the universe except for what we are trying to protect. That is what you get with different parts. And of course a good actor has all that information already you know the additional privacy laws caused by our public are really this is that it to this much right that's literally what we are doing this is the 1st sign that the man really matches the requirements of the law very.

[00:44:04]
Wary of previously the metadata national referred to the law because previously there is like this clip function where he published rather this or not gave gave it all away and we don't know where the heck with this. Other crisis. PENN Well it's probably before we like them all for coming it's been great our agent asked the best questions for 00.

[00:44:36]
0 if you were there is one more question but Lou just say that if you have further questions and one is a tail off about my email address right now can he see treatment Reno for us. On. This it is so that you don't passage here that. My personal email is.

[00:45:01]
Citizen Gee that easy I got work and my fish will you know grasses since it got out of our people and since the stock up. To probably. Now versus about. 40. So that's a great question but it does it's not a hypothetical there are a few social scientists who are apoplectic about a nor'easter that data in day one is not to do that often so we're a gauge in the number of forms right now and one of the things that we are counting on is it we were at a noisy folk are we were doing slopping we needed to do it other days we didn't tell you about it we don't tell you what the accuracy was.

[00:46:13]
But in point 48 you're going to get any impact on the actor's it so for that 1st time you get the now actually played by either. You know. And they're not happy with that they're just like me some of the results that we came up with this morning Carol Braun there were like Yes Yesterday the things that we did in fact on was that just the globality of your research for.

[00:46:42]
OK Great Fish Walker fish. Which.