[00:00:05]
>> Morty's most important perhaps I don't know if you would agree with us. In the past twenty years has this interesting work. Memory applies very. Well. And it was just wonderful it was rationed people. Reason. Was. Actually. Now. He's a really nice we're showing that for example. Task intrusive thoughts arise for some people on Sundays.

[00:00:53]
On those they are worried about their performance. Or. Really. And now he's telling us I think about some of his courage which is about how to think that perspective. And evaluation and translate. Well thank you Chris it's really great to be here and see a bunch of all trends are as you re marking that Dan I first met I want tell a story but we first met in one thousand nine hundred seventy.

[00:01:35]
Right hold kleagle invited us to a thing in possum I've known Paul since forever Chris says forever fell for a long time so it's really nice to be back at the home of cognitive aging and by the way the story about Tim I do want to my greatest strengths is my lack of creativity so what I just do as I look to what other people are doing and build off of that and you know I think the work like the general slowing the processing speed theory really provided a lot of like that's how I got into this right reading about that that.

[00:02:07]
So it's nice to be at the home of that and you'll see some work from Tim on here as well I can't just leave it alone but. So I'm going to talk about some of the recent work I'm doing and when I was meeting with Audrey earlier she asked she made the size comment saying you've done a lot of things which I heard my research is not programmatic and kind of thinking.

[00:02:29]
So what why do I do it so I had to make the slide so I feel better so thank you. So my research interests fall into sort of three broad domains I've always been interested in cognitive aging and in dimension but really pre-clinical dimension so prior to when people are diagnosed but very interested in psycho social factors that affect cognitive performance and health like stress rumination worry and then even way back when in the one nine hundred ninety S.

[00:02:56]
and early two thousand when I was at Syracuse I was very interested in what people have called the science of daily life how to take some of the phenomena we study in the laboratory questionnaires and see how. Unfolds in every day experiences and the advent of you know things like this very powerful computers pervasive technology that people carry around really opens up some new opportunities so that's kind of what I'm going to talk about today a little bit.

[00:03:24]
With some dabbling into these two two areas so my plan for today due to simple things going to talk about what I view as the most fundamental challenges in the measurement of cognitive change so what I'm all about is trying to figure out how we can measure whether someone's cognitive performance or function has changed from time one to time to whether that time one is right now in ten minutes later or right now and a year or two later as it turns out that's really difficult to do.

[00:03:55]
For a lot of reasons so to talk about what some of the challenges are to that and then ways in which we can maybe try to improve how we monitor cognitive change the sensitivity specificity by relying on pervasive in mobile technology talking about some new opportunities there's a new project that we're working on Chris as part of that and it's I've already talked with a few others about opportunities for connecting with this it's a lot of talk about a little bit later but there could be opportunities for collaboration C connections for that so let's also keep this a little bit informal I know a lot of you will have questions or quibbles I'm going to give a kind of a broad survey of this which means there'll be some things some details I'm not supplying and there's probably some hiding weaknesses of some of the research which I'm sure you'll put your finger right on but feel free to interrupt and ask is I'm going along.

[00:04:50]
So to begin in the beginning right how do people study cognitive change when I teach graduate statistics classes sometimes I have engineers in the class and I'll ask at the beginning design a study of how you would examine how people change over time. And the psychologist and the engineers come up with completely different approaches completely different so psychologists say you do something like this and there's no right or wrong answer but we are talking about engineers so.

[00:05:22]
So the typical way in which people study cognitive change is in prospect of what I call single shot designs and basically this consists of bringing people into the lab or the clinic giving them a test at Wave one we call that the first wave and then we wait some period of time often a year bring them back again do it again do it again right so over the course of a three year follow up we may get for measurement time points on an individual.

[00:05:51]
This is what is typically used to study long term effects effects and changes that transpire over the course of years months maybe decades. What we call the longitudinal single shock now thinking about how do we pay for this we pay for this we're getting grants from and they are in S.F. Let's talk about N H How long will the Grant last five years so what this typically means is in a five year period you can get about three years of follow up because you've got to start up again to bring people in you can't test people instantaneously so you get about three years of follow up now this design is predicated on the assumption that there's not perfect short term stability but it's basically predicated on the fact that if you bring these people in and measure and test them once you're getting a pretty good indication of where they're at and who they are.

[00:06:41]
And you are for sure right but that doesn't necessarily mean the precision sufficient for measuring change. I just I don't know why I threw this in because you know this so what box is that called that's the catell data box right and it describes a dimensions over which you think about measuring different people people can vary you can measure different variables in those measurements can be distributed across time.

[00:07:06]
So. Just to kind of draw a distinction between two different approaches in Randy's lab today when we're meeting we're talking about the differential psychology in experimental psychology right but you can also think of differential psychology as the study of how and why people differ differ. In contrast to a different approach idea graphic right which is how people vary or or change over time so this is kind of really focusing on this variance this is kind of really focusing on here tests can be good for different purposes right so some tests can be really good for differentiating between people other tests can be good for detecting change and they may be the same test or they may be different tests so what we're focused on what I'm focused on I mean we want to do this right but really trying to think of optimal ways of doing this and in some ways it's not just about the test but it's about more broadly the measurement procedure you can build better and better tests but still run up against a problem of intrinsic variability so if you measure someone's weight.

[00:08:12]
The one year retest interval for measuring weight is about point eight nine point eight right but if you were to for those of you in us who have tried to lose weight you measure your weight every day you can see you know five six pound fluctuations from one day to the next right so if you're trying to detect a one pound change in weight over the course of a year and you just measure twice you know if you have pasta the night before you know it's going to have an effect.

[00:08:39]
Now when people when psychologists think about why we measure change. They typically And this went into the design of that experiment you know with the graduate students psychologist want to be able to do this to distinguish between the rates or characters of change between person A or person B.

[00:08:58]
characterizing differential change so why in this case is a declining more rapidly than B. and there could be be a lot of reasons for that so the studies are. Zines with being able to do this in mind they still don't do it very well they can do it if you have large enough samples but there are some real inherent limitations to this type of approach when it's evaluated using the single shot designs.

[00:09:24]
And it also doesn't really capture the true phenomena of what it's like to be a person and move through life so this you can imagine this is an individual's trajectory on some cognitive measure over the course of much of their life and they're declining right and the reasons for their decline could be due to things like their brains getting thinner or there's some amyloid deposition as part of a preclinical Alzheimer's possible process and those are things that really occur and build up over many years or decades right so there can be these long term changes and long term processes but if we were to measure people actually frequently over this period of time there their performance wouldn't follow this smooth conjecture right it would look something like this right and then if we recognize that we can then begin to ask the question well not only why is there a decline from here to here but why is there a change from here to here that it's not noise it's real but it's maybe due to things that are operating at a different cadence maybe chronic stress or inflammation or other kinds of transience conditions that might persist over the course of weeks or months or.

[00:10:37]
You could drill down to an even finer grained time scale and measure people multiple times per day. And see that performance varies over the course of hours or maybe across days and these could be tribute to things like stress fatigue noise and the like so this is the perspective our model of change that we're trying to think of ways of designing assessment procedures to try and measure change across all of these different time scales.

[00:11:10]
So and why is this important so this is important to recognize So just by the way the engineers say what we would do is you just measure someone you know every day and they which want to try and figure out how the system the person is fluctuate fluctuating or changing so that's different approaches now.

[00:11:32]
Let's say your cognitive aging guy or girl and you're like me and you just kind of are interested in these long term effects you say I don't really care about fatigue or stress just because you don't care about it doesn't mean it doesn't exist right it's still there and you can ignore it but perhaps maybe you shouldn't So let's play out what can happen in the context of a single shot longitudinal study where you test people one time one day wait a year test them another time another day there's some variation performance depending on what day it is not maybe you get people on a bad day.

[00:12:11]
Or a person on a bad day you wait a year. And then you test them again and you get them on a good day that's going to look like maybe they improved whereas if you were to look at their typical or average performance they might be declining but so what this means and I'm going to actually quantify a little bit later how large are these effects versus these effects right.

[00:12:37]
But the idea here is that single shot designs don't have sufficient temporal precision to to sort of filter out the short term variability which can then interfere with the measurement of long term change right. Now. The. More. Yeah. Yeah yeah and that's but so you said a couple things more people average it out we.

[00:13:10]
We don't really as we can get more people but conventional approaches really don't allow us to average this out there's a different design that will allow us to do. With more people yeah that doesn't work as it turns out but it does but there's a huge cost to it but because we're very.

[00:13:35]
Ever so the problem. Is. That. Who. Very well. And maybe and I didn't need to be flip with that but the debt but you're exactly that's that's how that's the strategy behind doing it but it's how you do it with your question with your point so like then just instead of doing you know fifteen twenty minutes to eight hours maybe that's worse right because if you happen to pick them on the wrong day then write or you know and here's an interesting empirical question that I don't think there's data to answer this and this is on my last slide posing this if I have thirty minutes of cognitive testing and I want to get the best measurement of your trait do I put all thirty minutes in one session or do you do thirty sessions of one minute in average them distributed across days.

[00:14:35]
Yeah yeah yeah. Yeah well and it could be it also depends what. What are the what are the factors external to the system that could be perturbing right so you can absolutely like what here's what we can't control in the lab that you have a good night's sleep we can't control whether you're worried we can't control whether you're stressed we can't control whether you're fatigue we can give you caffeine a little bit but so there and it just kind of depends how important those things are but I think the bottom line is you're right if you're just interested in measuring a trait what you want to do is do more in a shorter perhaps in a shorter period of time certainly not distributed over that period there are other optimal ways to do that for sure hundred percent.

[00:15:24]
Hundred percent so this is might appreciate this this is from a med analysis that was recently done and basically what we're doing is it could be viewed as not important if you view small effects as something that's trivial so if you look up and if you used you know Dr Rekha P.D. and you go there and you look into effect sizes smaller facts can be statistically significant but not necessarily meaningful I view small effects as just something that are hard to detect that you have to look very closely not necessarily that they're an important but when we think of cognitive ageing How big are these effects what kind it depends on how you quantify them there is a med analysis that was done looking at a bunch of different ageing studies and these are mostly older adults and what they found is that the average rate of change on various cognitive measures depending what the domain was was between point one two and point two six standard deviations per decade which means that the rate of change per year is about point zero one two point zero three standard deviations which means a rate of change per N H study grant period is about point zero three point zero nine step.

[00:16:35]
Deviations a really small effect right so this is this is the challenge right that we're going to get a bunch of money we're hopefully right and then we're going to have to say we're going to study change right can we detect that and then forget about trying to detect that's average then forget about trying to detect variations in this so.

[00:17:01]
And then if you want to move which the N A N I A is pushing toward earlier and earlier middle age samples is going to be even smaller So this is and then I was thinking of this when you're talking about so just testing more people this was a simulation study published in psych methods by Phillipe Ross and Scott Hofer where they took it wasn't as precise as as the Met analysis he just showed you but they took from bunch of different studies the average amount of cognitive change and different longitudinal studies at different variance components they ran simulations determine your sample size and you see this is fifteen hundred two thousand up there and number of measurement occasions you needed to have adequate power and then over different extended periods of time so time is on your side you know that but for an A in A grant you have three years what they found is that to be able to detect differential change individual differences and change requires nine waves over three year period and six hundred people OK.

[00:18:06]
For good power and then if you want to start looking at how change in different variables are related to each other then you're about sixteen waves so you can do this right but it becomes expensive the other I mean there are different things you can do and what they didn't do you talk about is how you reduce the air component so if you can reduce the air component you can fix this and then the question is how best to do that.

[00:18:29]
From CA so. The other point in this is a graph from Tim Salthouse on looking at retest effects so we have sort of this signal to noise ratio right we need a lot of people to detect change and then also we have this graph depicts the magnitude of retest gains by testing the same procedure between zero and one years apart one to three years apart three to seven years apart and then seven to fifteen years apart and you see for a number of different kinds of tests notable retest related gains even up to three to seven years right.

[00:19:09]
And basically what's happening is something like this you know maybe people are declining at a latent process but then they're also improving and if you average those together this is what you end up observing and in through the traditional kind of single shot design it is statistically analytically mathematically logically impossible in an individual to decompose this you can do it in groups by having control groups but not in an individual.

[00:19:36]
So retest effects can overwhelm if you look at the magnitude of these these are like Point two point three standard deviations so huge relative to the expected rates of change in aging so this is our task right if we're trying to study cognitive change now it's easier if you just trying to characterize you know trait level individual differences but if you're trying to detect here's where I am now where am I going to be later than you even want to predict how change is going to be that's even more challenging So one approach is in what you had raised earlier is more measurements right and averaging filtering out that's exactly what you want to do so what we've done is use what are called measurement bursts and it's there's nothing magic about this is basically just more testing more people are not to testing more frequently so a burst consists of a question or of closely spaced measurements some follow a period another cluster it and the reason why we do this is because people won't do this every day forever right so you're this is basically giving them a break.

[00:20:46]
We're going to talk about how do we want to actually measure people every day forever and there may be ways of doing that but the advantage of this type of design it allows you to look at short term effects as well as long term effects if that's of interest but it's not always of interest but of that's of interest so what you can do is you can aggregate across assessments to cancel out.

[00:21:11]
This type of variability and improve the measurement of change that's one idea behind why you'd want to do more intensive assessments to improve temporal precision This also is important in the context of trying to detect when change occurs so in pre-clinical dementia there's this whole issue of when does decline begin and if you only test people once every year once every three years you have this problem interval censoring you don't know where in that follow up period the change is occurring and it can lead to biases so there could be some very specific applications for which you'd want to do something like that.

[00:21:49]
The other problem or the other benefit of measurement bursts is that it potentially allows you to model retest effects so I had this idea when I was in Syracuse but as learning to cross country ski the first year I was terrible and very very slow completing a a route but then I got faster and faster my time to complete got better and better the offseason occurred I came back I was in this fast is where I left off but it wasn't as slow as never having started and then I went back and over the course of time right but they also got older Right so in addition to me getting better I was all are you know getting slower and more decrepit and less less capable so we developed a cycle metric processing model a double negative exponential which basically models learning it allows for gaps it models rapid recovery or warm up but then also separately allows for drifts an asymptotic performance so this is simulated data that came from a grant proposal that we're doing now where we have one two three four five bursts the means are the same right but this is simulated data following this model and it kind of shows there's learning here steeper learning and recovery steeper learning and recovery steeper learning recovery but then also the speed is creeping up but we did a baby burst study where we only had six measurements perk Asian.

[00:23:13]
And these are the data from it from one of our processing speed tests and you can actually visually see you know if you squint that there's learning and then across the bursts there's more rapid recovery and the bass lines are creeping up a little bit so this is a proof of concept that we can maybe apply these types of models to separate retest improvements because we're modeling them over the short term and then over the long term looking at declines an asymptote group sorry.

[00:23:46]
So this is just that data from that particular stay. Where we had. Two processing speed tests where people had to match three or five. Number strings of three or five digits this is their rate of slowing in milliseconds per year an asymptotic performance although what I'm not showing is that they're mean stayed about the same or even got a little bit better this is back one and back to more slowing for and back to then and back one although both are significant so we're able to detect in one millisecond slowing per year with this type of data this is some updating task that didn't quite work out as well.

[00:24:23]
But the other thing we were able to do is precisely measure individual differences no one hundred people in two years of follow up and show prospect of associations between baseline levels of stress and accelerated decline in asymptotic performance but not in average performance so this may be using burst measurements can help us out a little bit by you know reducing the amount of hay we have to sift through and maybe amplifying the signal that looking for.

[00:24:52]
Now. It's hard to do these types of intensive measurement studies so we actually had a lab in a continuing care retirement center so people came in and they came in and over six days over ten day period cost a lot of money people didn't want to come in you can't really do these intensive measurement studies repeated measures using conventional methods so this is where mobile phones come in so the idea here is that by using pervasive technology we can.

[00:25:23]
Collect data in people's Basically anytime anywhere and try and use that to. Improve our measurement of change so before I get in to the specifics you know it really is kind of important to say exe Now exactly why do you want to do this is it just like is it just like making an app because people have this or are there real important problems you can solve.

[00:25:51]
Using mobile technology to assess cognition and I think there are two we're actually three things but there are two things I'm just going to mention and the first is it makes it easier to test more people you want to sample of ten thousand you can do that. Because you're removing constraints geographic constraints personnel constraints.

[00:26:10]
So large and diverse samples. Which is great if you live in Penn State right at Penn State State College if you've never been there. So there's like this is about like what the study would look like they're right to getting everyone who would volunteer this not is not very dense population not very you know sort of diverse so it's difficult for health related research but also importantly to target hard to reach pop populations I have a colleague Jason Hostin who's implemented some of these procedures too for world wide clinical trial and dominantly inherited all summer cities they're doing this in.

[00:26:47]
Eighteen countries seven different languages too expensive to bring because of the frequency is too expensive to bring people in so their test. To see whether or not they can you know there are is already deployed and there they have some really promising pulmonary data already so they can help and in this case also helps to access world populations or generally busy people maybe can help a little bit with the problem of middle age right so the problem of there are a lot of problems with middle age but if you've ever done I don't know maybe other people like and when we do a study and we recruit people like younger people they look like how they should look the older people look like how they should look at the Middle East people are always weird right and then I kind of think about it like what forty year old has the free time and the inclination to come in and so.

[00:27:39]
Anyhow that's that's my own little little thing but hopefully we can help with that so in some ways relying on this type of pervasive technology can maybe improve citizen power for cities and improve inferential strengths so you don't need to do these intensive measurement studies you could do the conventional ones if you want to assess people once every year and then you could get your Sixteen hundred people right or you could get your nine measurement waves right.

[00:28:05]
But then because this is a little bit more convenient for people you can maybe test them more frequently so what I want to talk about is our approach to doing mobile just very briefly talk about our approach to doing mobile assessments which is to embed them in ecological momentary assessments so when we design cognitive tests they don't need to be done this way on the mobile phone.

[00:28:28]
Each assessment takes about for each test about thirty to forty five seconds but what we get seventy assessments over a two week period so if we want to look at individual differences we can aggregate for that but really we're trying to look at ups and downs and such.

[00:28:45]
But I think more broadly whether it's embedded in E.M.E.A. studies or other types of intensive measurements This allows us to improve temporal precision we don't have to have to wait a year or ten years or seven years to repeatedly test people we can do it at low cost there are other barriers which we'll talk about but yeah.

[00:29:04]
At least twice yeah. Yeah so that's a great question right so that it and that's if with perfect measurement right you have to test more frequently if if it's and they are right so. You know it's Ben. How much we can get people to do how much we think we can get people to do and then also depends what we want to do is basically what we want to do is aggregate across some days to look at variation over the over the course of a year then we'll test them you know have like three or four bursts of seven days within that year because basically we're aggregating the births and we just want to look at that yearly change so we get three or four measurements there's not a lot of information to be honest that we have to really inform this right and you know like if we're interested in in sleep like and sleep related variation cognition well that can pretty much only occur every twenty four hours so something like that we measure multiple times during the day.

[00:30:18]
This is I mean hopefully we'll be able to work some of this out but I don't know. Yeah exactly right so we I mean I think the minimum is two assessments pretty We tend to do between four and five per day right absolutely and is that enough I don't it's it's Ben enough to detect some affects it's probably not enough for everything so we're we're one thing we're struggling with right now but don't tell the ON A It's because they're still reviewing it's to look at the effect of glycaemic excursions in diabetics so they'll be wearing their monitor carrying around the phones beeping them and like you know we just hashed it out like how often do we need to do this and the bottom line is you don't need to do it often if you do it at the right times.

[00:31:16]
So Bill talk a little bit I mean there can be intelligent sampling of assessments and I think that's going to be the answer great question. Anything else at this point. So our approach to doing mobile cognitive assessment is not the only approach to doing this is is to think about it as sort of an ambulatory assessment a procedure where we're measuring people and we want to look at real time processes in naturalistic settings so we've done this by embedding using mobile technology typically smartphones to embed in ecological momentary assessment designs to everyone know those things right so you carry around a device it signal continued it can prompt you to do stuff could be event contingent you do it any time something happens or time contingent you're told to do it in the morning or the evening right so so we do that we give very brief performance tests I'm going to show you two tests we've used.

[00:32:12]
We coupled these with real time reports so ask people what they're feeling what they've been doing and we also in some of our cities have some passive sensing so we're having people wearing at active graphs as well as ambulatory heart rate monitors to look at cardio. Vascular responses throughout the day.

[00:32:29]
And the idea here is that in much in the same way we can make stronger entrances back to a population if we have sound sampling procedures rate we have a random sample of people we know more about the population then if we have some sort of cobbled together convenient sample of the convenience I think that's fine but the same applies when we're trying to make inferences back to a person if instead of saying OK let's schedule one day where you come in and if you're not feeling good that day you can cancel right or if you're feeling too stressed you can cancel if it's a busy day you know you can reschedule we're fine will accommodate right but maybe if we strategically sample occasions we can make stronger inferences back to the individual as well saying same logic.

[00:33:14]
I didn't talk too much about ecological validity that's a whole separate issue some of these some I mean so we can we can talk about whether this actually improves ecological feel it in the of the cognitive measurements or not. So I'm just going to describe one finding from a study called escape the effects of stress and cognitive aging physiology and emotions this was done in the Bronx New York that is the Bronx it's photo shopped.

[00:33:41]
So this is this is called Co-Op City and it's the largest cooperative living development in the world sixty thousand people live there it has its own zip code we sampled from there because we wanted to hold neighborhood and other factors constant. That's a swamp it looks nice when the sun's reflecting off it just just fine but that is a swamp but it's an OK place so we.

[00:34:06]
Recruited a very diverse sample there they participated in intensive repeated measurements study so each burst can sit lasted fourteen days five random times per day they were sampled and then they completed a brief morning and evening survey we picked fourteen days for a very specific reason we were interested.

[00:34:26]
In relating reports of stressful events to cognitive function people report stressful events and about forty to forty five percent of days so we figured we needed to go two weeks to get enough of those events which isn't really enough but you know for that's that's how we pick the fourteen days.

[00:34:46]
A bunch of different measurements different domains we have biomarkers and stuff but I'm just going to talk about some of the self report data the protocol was like this we sent out questionnaires to people they answered a bunch of questionnaires we brought them into the lab we did some cognitive testing and then we sent them on their way to do the piece in the study we're going to be doing next we're going to be randomly or you know randomizing people do whether they get this first or that first and in other kinds of things we should have done we didn't actually plan to do this study we got funded to do a study with netbook computers.

[00:35:18]
And between selling people between the time we got funded and got the money they stopped making that books. So I should have done this but I didn't I just like said OK well that's what do we have now we can do laptops or with smart phones let's use smart phones so I didn't ask if I could reallocate that I just did it and it and it thankfully worked.

[00:35:39]
So or worked sort of another interesting statistic here is people completed about eighty two percent of scheduled assessments which is considered good for these we tell people don't do this if you're in a car walking across the street but it here and is an issue for these types of intensive studies going to show you screenshots from two of the tests.

[00:36:02]
One is a memory test docked grid memory and basically this was modified after a test that Karen said Lucky did it in and did some nice validity work on it you see a five by five grid three seconds remember the dots do a distracter tast touch where the dots are right and the like this because it provides a precision measurement Euclidean distance error we only did two trials.

[00:36:26]
Each time. And then we had a processing speed test where symbol pairs on the top touch the pair on the bottom that you see at the top you can have Lor's where there's one matching We used forced choice because we didn't want to say is that there or not there because our response biases and stuff.

[00:36:45]
As we designed these tests we think about a bunch of different things we have to be brief and part of the reason is because the brief for a test is in the real world the less likelihood an interruption will occur is very important people use their own devices because techs and and phone calls have priority so it'll interrupt the tests so long blocks are you know it's just a trade off we wanted to have task demands so whatever screen they're looking at they should know exactly what they should be doing and we want to discourage cheating you know so make it more difficult to cheat than not they still could cheat if they wanted to on this although it would be that have to make a whole bunch a little grids and carry them around and fill them out.

[00:37:32]
So there are some concerns. You know that we are still concerned people will cheat and then this is naturalistic but we don't want these kinds of settings so there is some concern that the test might be unreliable just very very briefly I'll go through. Results that we had published where we and we apply G.

[00:37:51]
theory a generalizability theory to calculate the liabilities and for a single two trial measurement the intricate last correlation was point three nine which is lower than what you want but still higher than expected just for two trials for the processing speed task was point five four if we average over one day of assessments which is about for the memory test it's about you know three or four minutes four minutes of processing speed test about the same we get into a class correlations this high and then if we average over all before.

[00:38:26]
Days we get really good into class correlations which means the like the the between person variance relative to the daily fluctuations so this is kind of an assessment of how many observations do we need to filter out the daily variability for a burst design so at least if we have a couple days of assessments which are four to ten minutes we can have a reliable indicator of cognitive function and this next slide is actually something that I'm sure would like to talk a lot about it up you know I don't want to spend too much time on it but we also we didn't intend this to be a validation study so in the lab we had.

[00:39:05]
Accounting span operation in a backward span task the mobile tasks we had the dot memory and back task and then for speed we had you know the standard things we had a simple match computerized letter mass number match and then we had the simple search and we did a factor analysis and the ambulatory tests loaded on the factors they should this is a reversed sign because it's an ear score.

[00:39:32]
So this and there is in variance across age so that to us this suggested that these could be at least aggregated valid indicators of individual differences in cognition so at least we're measuring something connected to things we do in the lab now. That would have been a good idea.

[00:39:57]
So we didn't because we didn't you know when I was designing it was to answer these questions about stress and rumination I wasn't thinking let's do a psychometric validation study and we should have we're doing that now for standard protocol and we do it in the lab before and then also in the lab after the burst right.

[00:40:17]
They'll be differences in level of performance for sure yeah. That's yeah that's a great question in fact in the project Chris and I are working on one of the key features as we're developing out the mobile platform will be to validate against in person tests Yeah. Sure. So what we did not days of the month were season.

[00:40:46]
Yeah you know I had not done that but yes days a week so so we distributed except for Friday because the hospital's A Jewish hospital so we closed early on Friday and couldn't get on board people were actually no it wasn't Friday there was that they could start that I guess it was Thursday one of the days we we didn't but we we sort of street had equal uniform assignments for when they began their burst across days of the week days of the Week matter a lot for Africa and mood cognition not so much time of day matters a lot for cognition.

[00:41:22]
But yeah you know I had this idea that I want to get a science paper it all just never it'll never happen but I thought we could do it so we went back and we got all the weather reports for the days in which people did this and because I have allergies and I feel like I lose a few I.Q. points every May So we looked at pollen count and unfortunately in the Bronx they changed where they collect the pollen data midway through the study so but we can still look at you know other kinds of things but yeah day of month is actually should be important because of paydays.

[00:41:58]
You have and once you start to you know peel the layers off the onion there are all these other background variables that would be important to try to line people for. But the last thing a minute mention is the affordance that this type of this type of data collection allows for looking at real time analyses critical Chris alluded to this earlier so you know we can look at whether cognitive performance varies as a function of whether anything stressful is happened earlier in the day what their mood is whether they're fatigued to their time of day so I'm going to show you one I'm going to answer the question is whether it's possible to wake up on the wrong side of the bed but be.

[00:42:39]
Get to that and that's going to show you some background results. To answer this question are these effects large relative to age so remember how big the age of fx are which this hour raise and d'être you're right apologies for the accent but that's why we study this because we think aging is important.

[00:42:59]
But it's really small so here we're going to look at the effect of time of date doing an assessment one hour later in the day whether or not anything stressful as happened in the previous twenty four hours that they reported what their current negative mood is at the time of taking the test whether they're fatigued and whether they're worried about anything so we didn't actually use worry we asked Are you having.

[00:43:24]
Train of thought you can't get out of your head and are you thinking about unpleasant things. So this was the cross-sectional approximation for when your age effect would actually aligns very closely with the Met analysis so the effect of taking a test one hour later in the day and this is a worst score.

[00:43:43]
And these are in standard deviation units so one hour later has an effect that's larger then a one year age increment So what this means that when we do our longitudinal studies we schedule a person to come in at eleven AM burst one right or wave one wave to we can only fit the minute nine we've wiped out any thing potentially that we want to be able to see.

[00:44:10]
Because we're dealing now not over the course of ten years or twenty years but Randy. I'm sorry it was on the this in this sample it was twenty five to sixty five yeah yeah yeah so we're now doing this in a sample average age of eighty and. We pick twenty five to sixty five we didn't want to deal with preclinical dementia I didn't want to have a lot of retirees I didn't want students because the study of daily life so you know there wasn't any so.

[00:44:46]
Well you know so we're going to see I think so remember this was not at seven am so their first cognitive test of the day was between was about two and a half hours after they woke up two and a half to three hours and their last cognitive test was a few hours before they went to bed.

[00:45:05]
In our so we might not have gotten at the far ends we're now doing when they would. You know and when we didn't ask did you have coffee we didn't we didn't get any of this stuff right in this first study. Yeah yeah so that's a great question so how we do that how you do these studies you know before you ask each individual how you know what time do you typically wake up you add an hour to it because people lie they like to say they wake up earlier than they do and you don't want this beeping while they're still awake so we tailored it to their self report.

[00:45:50]
We had report measures of sleep in the morning when they went to bed you can't trust you can trust the are you refreshed question right but you can't trust when they went to bed or are you know so we're using acting goofy now to do that or. Why yeah yeah well.

[00:46:13]
Yeah well not exactly except for except for like the refreshment piece like if it's like subjective sleep then you asked subjectively the objective so it was a sleep study. We you know and I didn't know anything at that time so now we know like so now we're slapping on active graphs.

[00:46:33]
Previous stress or effect is huge so we did a study and this is actually talked about that the last time I was here we're having a stressor on a particular day affects working memory performance by about an amount that's equal to four years that's about what we see here.

[00:46:49]
If you're in a bad mood you do a little bit of were a little bit worse yeah. Self report. Yeah and we allow them to describe it so. Stressor type is totally subjective right totally so so appraisal Right absolutely and then there are can comment problems with how you equate those across individuals fatigue did not matter for this although it did matter of course for the processing speed test stress didn't matter for the processing speed test.

[00:47:22]
In fact there's an argument to say why you might be faster because of a rouse all associated with threat perceptions worry was associated with worse performance on the memory test but also not with the processing speed test. So essentially this type of variation. Could result in. Noise that wipes out your age affects so I'm going to show you just one more one more quick result we just published a paper to address this is how we unwittingly gets up on the wrong side of the bed right and can you actually get up on the wrong side of the bed and if we run out of time the answer is yes so what we did is when people woke up in the morning we asked them how stressful Do you think today's going to be when they went to bed we asked them how stressful Do you think tomorrow is going to be so we wanted to know whether or not stress anticipation predicted worse memory performance this is important in the stress lit literature because usually when people do daily studies they never ask about are you stressed about something that might happen they only ask about events that have happened and what's most stressful like things that are on the horizon and we also wanted to know whether older adults were more vulnerable to this and they aren't.

[00:48:37]
So here's essential either result in the design in the morning on a slide or how stressful Do you think today's going to be they did that soon as they woke up then later in the day they did five of those tests memory tests right and then a dead time we ask about tomorrow how stressful so then.

[00:48:54]
We could take yesterday's And then today's stressful rating and it turned out that on days where they said Today's going to be stressful they did worse on the memory test Remember those are errors scores that's where the coefficient is positive but last night didn't matter. Which it was kind of interesting and we controlled for whether or not anything stressful actually happened so it doesn't matter if something stressful actually happens doesn't matter matter if they were in a bad mood so just waking up saying Today's going to suck.

[00:49:26]
We can predict not with a great degree of accuracy but we can predict that's going to your memory is going to be worse in that day. Yeah you know which is which provides some interesting opportunities for interventions and I'll talk about those in a second but what I'm just going to just to slide through three slides to show kind of where we're going now we're we've been charged with developing a platform for mobile cognitive assessments or so we're going to develop a basically E.-Prime for the smartphone.

[00:49:59]
That will allow people to do experiments on smartphones do ecological momentary assessment studies every all the source code everything will be open that is mean it's free to use because you still have to figure out how to use it and have a secure backend but it'll really be affordable very usable and we're going we have the extension and dissemination piece so as the project is going on we'll be looking for collaborators people who might want to do something like this there can be cost savings I've already gotten some great ideas for input into new measurements from people so this is actually this is going to be one of these open science projects and hopefully we can we'll be reaching out as we've just started a couple months ago yeah.

[00:50:47]
Yeah and fact we will probably beginning in in the summer be formally reaching out to the community so contributing by like if you will use these tests in your study along with these criterion measures you know we can do it for free right because you'll help develop valid validity data normative data suggesting nominating measures that you'd like put into the platform.

[00:51:14]
And the way we're developing the platform things will be very very flexible and ultimately would like your tech savvy grad students to be able to program their own tests as well. But here's kind of what we're thinking to use this in the context of being able to look at time of day effects maybe looking at sundowning looking at the effect of lifestyle factors mental health and thinking about different.

[00:51:35]
Kinds of approaches to interventions rather then cognitive training to do whatever maybe optimization right so if for example we know when you wake up in the morning if you think is going to be a bad day a little mindfulness intervention or you know a happy face intervention could have an impact optimized what you have maybe.

[00:51:57]
Like I mentioned we're doing we're we're involved in into clinical trials were using these types of mobile assessments to improve the sensitivity of detection and. Mediterranean diet intervention and the other is the one I told you about earlier the dominantly inherited. All summers one so and I just had mentioned that we're trying to create a tool box for you guys and as you're thinking about these things I've already had some good interactions.

[00:52:28]
We really want to think about tests that can be done in very brief blocks now bearing in mind you may get an hour of testing over the course of an assessment but every block need to be brief intuitive interface and task demands. And we have a colleague who's on this project who runs a website called test my brain and she has this phrase death by a thousand instructions and she's actually done experiments where she tested people to recall the instructions after a test and they can't remember right so.

[00:53:01]
Anyhow that's so there's some design considerations we're also working on making test have a robust two devices we have a robot that is we're buying every device that's going to come out. At Android I.O.'s and going to be calibrating on speed and the sensitivity to battery life battery saver status process C.P.U. processor load to see what is viable in terms of timing measures as well so those are things to think about and finally you know there are other possible extension so we're very.

[00:53:35]
We interested we're working with a company called sage bio networks who has done a lot in passive measurement so they can measure and reproduce clinical. Clinical grade characterizations of balance and gait in Parkinson's patients by just having them carry their phone around. So we're actually trying to think of planning for what could be a two stage screening assessment approach where we have some sort of passive measurements that could trigger maybe now is a time where we do want to do a cognitive assessment to help solve this problem of how frequently how often.

[00:54:12]
And then finally there are a lot of really interesting questions will be trying to address like how best to distribute ten minutes or twenty minutes of testing and then actually how to design good cognitive tests that are a static an appealing invalid how to design gauging platforms to improve adherence and reduce attrition in studies so that's basically a bunch of people to acknowledge but thanks a lot for your time and attention and if you have any more time I'm happy to answer questions here or then people can go off and study and and if you want to want to escape the high around a little bit.

[00:54:45]
Thank you. Great. Morning donuts at the reception if there's an OK. Yes. Yes. That's that's the goal of all of us who knows what all of us is so it's a large It's a it's a. Ten quadrillion dollars study by the end I it's where they're going to measure one hundred million people they want a hundred million people they'll get thirty or forty and they want to measure everything there is to measure about them.

[00:56:01]
You know genomic So they're going to do genetic sampling and then they're going to be a mobile or digital health piece and that's the idea behind this and by the way if anyone has ideas if you are going to pick you know five to ten minutes of cognitive testing you're going to give on everyone in the United States and they were going to do it once a quarter four times a year what would those tests be.

[00:56:26]
If anyone has any good ideas what Tetris just right I know you know well yeah. But. Yeah. You know yeah yeah yeah. OK So the so these are different here those questions of the first one is how much within subject variability. That's you know it's kind of funny it's like it.

[00:57:23]
So yes we do and for these two tests the within subject variability assessed and that in this protocol in our paper we apply G. theory to calculate within subject variability was point five so about fifty percent of the variability every couple hours on the. Dot memory test was point was point five it a little bit more for the speed test so that's fifty percent of the variance was coherent from occasion to occasion.

[00:57:51]
And fifty percent not just right fifty percent of the within subject variance was. So that's sort of an answer. Yet Well we showed that. Yeah we showed that. That's. Well I mean it depends what you it depends if you're talking like an engineer or like a psychologist right so well I know you're the one person who can say that but I mean the thing is I'm thinking.

[00:58:24]
Frankly yeah yeah. Yeah. Yeah. Correct. Very. Correct. Chorley with other stuff yeah yeah so. Yeah. Right. Right now. So you just said I don't have the ability to do it right now so that would be a change in the trade or so but like so here is like is it an dodginess or X X dodginess right so if there is a change in an ability because you're stressed is that a change in the ability to write suppose if it's because like in most we could get in a whole discussion about motivation but just to answer the question about prediction so this was a lagged analysis into the extent lag counts as prediction you know it is so this is things that happened earlier.

[00:59:40]
This would be prediction as well in the sense that. You know what what the state in the morning by knowing your state in the morning I can predict now when I said like an engineer is not really prediction because I need the whole data set to be able to formulate these so real prediction would be forecasting to where you know I don't have the data set ahead right so and psychologist really don't often do that but that's what we want to do I'm not saying they never do that.

[01:00:12]
Free they're. Very. Yeah OK well but. I don't you know but not everyone cares about that I mean and you know like maximalist maximal the maximal effort or maximal performance which one. Was sometimes when I try too hard I don't do as well so which is it. That's not.

[01:00:45]
So well I mean OK we can we can get. Right to our. Own lives or not too much. You know so and you could be right maybe talking about academic things but you know if someone has an athletic background I know that in certain types of right there is really this effort you can be trying too hard.

[01:01:13]
For you. You know. That you're. Coming. Well it's. So so here's So here's let me just show you a quick thing here if I can get to it then I think we can actually get maximal performance better this way and the reason is how do you know so here's here's the deal we're going to go bowling after this and when we go great great so I'm a ball we're going to go bowling and here's what's going to happen we're going to put you in the on the alley we're going to give you a ball and do your best right and we're all going to watch or at least maybe just one of us will watching all be a professional bowler right or what we can say when you go to the lanes and you can bowl whenever you want right or ball in these times you're going to do this over a period of time you're going to look at your distribution of performance and then we're going to look at your best performance you do and then use that So again even if you want to get at maximal performance I would say having more than one test would be better able to get that yeah.

[01:02:32]
That's true yeah yeah so so then so then so in so then when you're in the lab you bring people in some people you're measuring their traits in other people you're not because they're not not everyone's at their best Yeah yeah and I think we all kind of grapple with these issues and like I'm not saying this is the only way to do it this is like mainly to try and you know talking about measuring change but I think we can do a good job of characterizing individual differences as well where.

[01:03:01]
There. Are. You know one's ecological one's not but that's what and I say we're out of time here. So this is the whole discussion. Well yeah you're right I mean it depends what you mean by there's three different kinds of ecological validity of verisimilitude a calorie and then contextual and yeah so we were trying to design tests on here that are smaller similar like not strange like like so like you do an activation code or you can use things that look more familiar.

[01:03:46]
To people but yeah you're great these those tests don't have face ecological validity but contextually there is an ecological embeddedness to that so yeah. OK. Thank you it's great always to talk to you.