With  the  brown  bag  series. You'll  know  that  I've  got some  administrative  things  to  do  here  first. First  I  would  ask  who  here  is  a  four  credit  student, other  students,  interested  guests of  which  we  have  several. Welcome.  Before  I  get to  talking  about  our  wonderful  speaker  for  today, some  other  things  the  credit  students  need  to  sign  in. Please  turn  off  your  phones. Please  clean  up  your  trash  when  you're done  looking  ahead  a  couple  of  things. We  won't  have  a  brown  bag  on  Thursday, but  we  will  be  returning  on  November  9. Our  speaker  will  be  Stephanos  Nicolaids  from USC  who  will  be  talking  about algorithmic  scenario  generation as  quality  diversity  optimization. All  right,  so  let's  jump  ahead  to today's  distinguished  alumnus  talk. The  topic  is  going  to  be  augment,  diminish, remap,  reality,  freeing  the  mind  and  its  resources. And  our  presenter  is  our  distinguished  alumnus, Dr.  Anne  Mclaughlin,  who  many  of  you know  went  here  for  her  Phd  in  psychology. And  her  advisor  was  Wendy  Rogers. A  little  bit  more  about  her  background. So  she's  a  professor  in the  Department  of  Psychology  at  NC  State. She's  received  awards  for  her  teaching  and  research  from NC  State  and  was  chosen  as  a  faculty  scholar  in  2016. Her  work  has  been  funded  by  NSF, Nasa,  NIH,  and  others. She's  president  of  Division  21  of the  American  Psychological  Association and  associate  editor  for  the  journal  Human  Factors. She  co  authored  a  book  for practitioners  wishing  to  design  technologies for  older  persons  called Designing  Displays  for  Older  Adults. Recently,  she's  published  a  book introducing  human  factors  psychology to  the  general  public. It's  called  All  Too  Human, So  everyone  go  to  Amazon  right  now  and  order  that. And  her  current  research  focuses  on  the  design  and  use  of cognition  aids  to  improve performance  and  learning  of  challenging  tasks. So  when  she's  not  thinking about  the  next  research  project, she  loves  rock  climbing  and teaching  other  women  to  do  the  same. So  please  welcome  my  dear  friend and  longtime  collaborator  Anne  Mclaughlin. Thank  you,  Mary  Beth.  Thank  you  for  that. Very  warm  welcome  and  great  introduction. I'm  really  honored  to  be  your  speaker  today. And  the  reason  is  that  I  have thanks  for  Georgia  Tech  for  so  many  reasons. And  also  I  wanted  to  share  a  few  memories. I  started  my  career  here  in 2001  Where  tell  me  if  you  know  where  this  is. I  lived  in  Hemphill  in  the  graduate  dorms, and  I  had  a  secret  pet  chinchilla hidden  in  my  room  for  the  duration. His  name  was  Mr.  Pops  Alot. The  psychology  department  back  then was  in  the  naval  gymnasium  building, which  has  been  destroyed  by  now. Back  then  also,  I  had  a  few  questionable  fashion  choices. But  as  the  years  passed  and  the students  ID's  changed  how  they  looked, so  did  I.  I  finally  graduated  in  2007  and  that's  when I  went  on  to  go  found  my  lab at  North  Carolina  State  University. Georgia  Tech  has  been  a  huge  influence  in  my  life  because their  faculty  who  helped  me  become a  researcher  and  taught  me how  to  manage  an  academic  career. And  many  of  them  I  still  work  with  today. Indeed,  most  of  what  I'm  going  to  be  presenting  today is  in  collaboration  with  Mary  Beth  Gandhi. Although  I've  been  at  NC  State  since  2007, It's  fair  to  say  I  think  about something  connected  with  Georgia  Tech  every  day. Let's  get  started.  I  could  not  be  more excited  to  tell  you  about  the  work I'm  interested  in  right  now. And  some  of  the  questions  that my  lab  has  started  to  work  on. A  little  audience  participation.  Raise  your  hand. If  any  of  you  use  noise  canceling  headphones, Yes.  Can't  live  without  them. How  many  of  you  use a  zoom  background  to  change  what's  behind  you? Or  maybe  touch  up  my  appearance,  That's  my  favorite. How  many  of  you  turn  down  the  car  radio when  the  driving  gets difficult  or  you're  trying  to  merge  on  the  highway. Same.  Any  sunglass  wares. Then  how  about  using  do  not  disturb? If  you  have  an  iphone,  the  silence notifications  during  certain  times. How  about  a  rather  new  feature that  Google  Maps  has  called  Live, where  it  will  actually  overlay information  in  the  environment.  Anyone  who  uses  this. Okay.  I  figured  with  this  technological  crowd, I'd  have  at  least  a  couple  then  last. How  many  of  you  sometimes  have  closed captions  up  even  if  it's  in  the  language  you  understand? Yeah.  All  right,  great. Well,  the  spans  quite  a  spectrum, but  these  are  all  technological  ways  that  we  use  to alter  reality  by  adding  visual  or  auditory  information, maybe  remapping  that  information. Removing  information.  These  are  all  cognition  aids. You  are  trying  to  increase  your  limited  memory, your  limited  attention,  and  to  make your  sight  or  hearing  better  than  it  would  be  normally. The  purpose  of  my  questions  was  to show  you  just  how  intimately and  comprehensively  that  we  seek  out  cognition  aids. We  want  to  see  farther, we  want  to  remember  more. We  want  to  notice  what's  important. I'm  interested  in  this  at  every  level, but  especially  at  the  cutting  edge  of technology  and  what  we  can  achieve  and  in  making sure  that  the  designs  of  any  future  cognition  aid  is effective  and  usable  sunglasses  are  one  thing. But  let's  get  serious  for  a  moment. This  is  a  recent  anecdote  that  I  experienced. A  close  family  member  of  mine  was in  the  hospital  for  almost  a  month  this  fall. I  lived  in  her  room  for  two  weeks, sleeping  there  in  those  two  weeks, I  saw  firsthand  what  I knew  intellectually  from  the  research, which  is  that  the  number  of  signals  and  alarms  are  way too  high  because  every  machine  has  its  own  alarm  system. The  number  of  false  alarms  was  astronomical, including  a  bed  that  would  alarm  and  speak  to  you, demanding  that  you  lie  back  down. Even  though  every  nurse  who  came  by  was telling  her  to  get  up  as  much  as  you  can  and  move  around, the  chair  had  the  same  alarm, which  would  go  off  every  time  I  got  up  out  of  the  chair, which  is  a  lot  almost  all  of these  flashing  lights  and  signals  were  local, meaning  that  the  people  that  heard  them  the  most  were  us, the  ones  who  actually  had  no  control  over  any  of  it. The  staff  only  heard  them  when  they  came  in to  turn  things  off  and  change  them. Meanwhile,  there  were  other  important  signals that  had  little  or  no  feedback. There  was  a  bile  pump  on  the  wall  chugging  along, happily  doing  absolutely  nothing until  we  made  some  invisible tweak  and  it  started  actually taking  out  stomach  contents on  when  oxygen  wasn't  plugged  into  the  wall. It  wasn't  plugged  incorrectly. After  a  bed  transfer, the  only  feedback  was  once  a  hour, someone  came  in  and  checked  her  oxygen  and  it  was  at 78%  Where's  the  alarm  for  that  last? The  schedule  for  turning a  patient  seemed  to  be  asking  the  patient, have  you  been  turned  recently? Let's  just  say  we  had  a  pretty unreliable  narrator  in  that, if  I  had  to  summarize, this  whole  experience  was  a  cacophony  of  noise, both  visual  and  auditory. That  was  about  98%  of  our  experience. The  other  2%  was  just  seeking  out real  information  and  often  not  getting  it. Let's  see  where  many  of  the  alarms  took  place. Here  in  yellow,  you  can  see all  the  alarms  that  would  go  off  every time  an  IV  bag  got  low. Meaning  it  was  always  a  different  time  through  the  night. The  other  alarms  were  for  the  oxygen, the  bed,  and  the  chair. In  this  photo,  just  this  one  photo, there  are  12  different  systems  that  could  alarm. But  now  I  want  to  turn  this  around  and look  at  it  like  a  research  question. Why  did  I  demonstrate  it  this  way? Why  did  I  choose  to  highlight the  different  alarms  and  yellow? Well,  obviously  I  wanted  to  draw your  attention  to  those  alarms. But  why  didn't  I  give  you  a  list  of them  with  their  attributes to  give  you  different  information? Why  didn't  I  turn  it  into  something verbal  and  that  way  you  could  easily  see  it. Why  didn't  I  just  subtract  out the  other  clutter  so  that  the  alarms became  more  salient  and  visible  like  this. These  are  the  kinds  of  questions that  we  ask  in  my  lab  today. Each  of  those  choices  could  be  more  or  less  appropriate, depending  on  the  needs  of  you, the  users,  and  that's  what  we  want  to  discover. When  should  we  augment  something? When  should  we  diminish  it? When  should  we  remap  it? What  guidance  is  there already  in  the  psychology  literature? Then  how  should  these  augmentations  diminishments  or remappings  look  or  sound  Last, what  are  the  measurable  benefits  that  they could  bring  to  emergency  situations, non  emergency  situations,  to  learning  to  training  today? Since  I  do  not  have  unlimited  time, I'm  going  to  focus  on one  of  these,  which  is  diminishment. Let's  think  about  why  we  don't  diminish. Like  the  slides  I  showed  you  in  the  hospital  room, diminishing  reality  is  often  not  our  first  thought. Why  is  that?  Why  is  our  first  idea usually  to  add  something  rather  than  subtract  from  it? Even  when  we  complain  that the  world  is  cluttered  and  full  of  noise, we  first  think  to  add  and then  we  pull  attention  with  adding. Rather  than  taking  away  unimportant  things, the  authors  of  this  nature  article  explore this  tendency  to  add  rather  than subtract  with  a  number  of  different  studies, Finding  that  whenever  humans  try  to  solve  a  problem, we  have  an  extreme  bias  to  add  rather  than  take  away. I  think  this  is  important for  those  of  us  who  are  interested  in human  computer  interaction  because once  you  realize  that  people  are  not  willing  to  subtract. You  have  to  make  yourself formally  think  of  that  as  an  option. When  they  did  different  studies on  seeing  if  people  would  subtract, they  did  it  and  found  it across  multiple  different  types  of  tasks. From  balancing  a  screen  grid, can  you  balance  it  by  taking  away  or  by  adding  people added  by  landscaping  a  mini  golf  course to  be  better  or  worse. Either  way,  people  added  then  last, to  create  a  certain  type  of  Lego  structure. Taking  away  would  have  been  more  simple, but  people  would  add  legos  to  it. They  found  that  they  did  these  in  these  experiments even  when  it  would  be  cheaper  or  faster  to  subtract. We're  working  with  a  pretty  heavy  bias  here  now. It  also  explains  why  this  field  is  still so  wide  open  with barely  any  literature  and  diminishing  reality. I  think  it's  because  we  do  not turn  to  that  as  our  first  thought. There  are  a  number  of  different  papers  describing the  engineering  and  methods  for  doing  diminished  reality, but  very  little  to  anything  describing the  human  computer  interaction  or  how  we  might  use  it. Although  the  idea  of  diminishment  is  actually  fairly  old, it  dates  back  to  around  the  year  2000. It's  only  recently  become  possible  to  make pretty  convincing  diminishments  and  to  remove objects  computationally  from  our  environment. I'm  going  to  go  over  a  couple  of quick  recent  works  about how  diminishment  has  been  studied. This  is  not  my  research,  this  is  background  in  2021. Researchers  found  that  diminishing  things, like  removing  the  person walking  behind  that  mobile  phone, could  improve  intention  and perception  as  long  as  it  worked  perfectly. However,  whenever  it  did  not  work  perfectly, as  often  happens  with  a  mobile  phone, and  trying  to  use  that  device  for these  more  computationally  heavy  processes, it  actually  harmed  user  experience. We  have  a  caution  there  that sometimes  diminishment  can  be  distracting  if  it's  not perfect  then  This  work  from  Ki  in 2022  found  that  people  generally  like  to have  a  deemphasized  the  emphasized  images  to have  less  clutter  in  their  visual  field  when  they're trying  to  do  a  task  like  working  with  these  Legos. Because  it  does  retain  the  context. That  seems  to  be  an  important  idea  is  that  you  want  to keep  the  context  of  things  rather  than completely  deleting  them  from  your  environment. Also,  people  like  to  customize. One  thing  to  note  is  that  very  few  of  these  studies, in  fact  neither  of  these  used  measures of  learning  or  performance. They  use  measures  of  acceptability  and  usability. Then  last,  this  experiment sought  to  accomplish  a  lot  of the  things  we  want  to  accomplish  with  diminishment, where  they  wanted  to  make  work easier  in  an  open  office  environment. They  did  measure  performance. However,  rather  than  diminishing, they  added  to  the  environment. Again,  we're  seeing  that  bias  to, if  you  want  to  help  people  concentrate, add  things  rather  than  taking  them  away. Let's  get  on  to  a  few  of the  experiments  that  we're  working  on  in  my  lab. One  of  the  major  initiatives  that  we're  doing right  now  is  to  look  at how  to  change  learning  in  the  classroom. How  many  of  you  have  ever  taught or  learned  in  a  large  auditorium? Okay,  well,  this  task  and  environment  were inspired  by  my  personal  teaching in  Poe  Hall  on  NC  State's  campus. For  those  of  you  who've  been  around  a  while, does  anyone  remember  Skiles  Scales  building? Okay.  Pohll  at  NC  State  is  rather  analogous. It's  old,  it's  loud, They  are  poor  acoustics, And  the  auditorium  style  seating  means that  you  can  see  the  screens of  every  person  in  front  of  you. And  let's  face  it,  most  of  the  students are  watching  ESPN.  During  the  lecture. We  have  movement  in  the  periphery that's  grabbing  your  attention, even  if  you  want  to  pay  attention  to  the  lecture. Here  we  have  an  environment  where  it's  almost impossible  not  to  attend  to  these  distractions. And  yet  this  is  where  we  want students  to  pay  the  most  attention. Now  one  I  hope  idiosyncratic  attribute  of Po  Hall  is  how  often  you  hear the  sound  of  loud  drilling  into  concrete. As  unknown  repairs  are  made  during  the  day, they  seem  to  target  my  exam  days for  doing  these  kind  of  repairs. It's  a  place  where  it  becomes  very  easy  to  imagine. If  I  could  just  put  on  some  glasses  and headphones  and  delete  those  different  distractions, I  could  have  a  much  better  classroom  experience. One  of  my  students  focused his  master's  thesis  on  the  problem, and  we  started  small  with  just  a  video. He  manipulated  whether  or  not  visuals  or  audio  were diminished  and  then  also  included a  control  condition  where  there  was  full  distraction. He  was  interested  in  two  different  outcomes, How  much  students  are  going  to  learn, and  also  how  much  these  diminishments  are going  to  take  away  from  your  situation  awareness. After  all,  it  might  be  great  to  just have  the  lecture  available  if  you're  sitting  in  the  room. But  do  you  really  want  to  be  so  isolated  that  you're  not aware  of  fire  alarms or  people  coming  in  and  out  of  the  room? Or  if  someone  asked  you  for  a  pencil, we  don't  want  to  just  have  complete  blinders  on. We  think  we  wanted  to test  what  the  effects  of  that  would  be. He  made  a  couple  of  videos  for these  conditions  and  also  a  test  of  the  lecture  content. He  also  included  a  measure  of  attention  deficit  disorder. To  see  if  there  were  people  for whom  diminishment  would  be  more  important, but  I'll  talk  some  more  about  that  later. Then  last,  he  included  a  measure  of  presence. So  how  much  did  people  feel  that  they  were  in  this  video? Because  this  was  during  covid  and  we  were  very aware  that  people  are  just  watching  a  screen. They  weren't  really  in  the  classroom. All  right,  let  me  see  if  I  can  make  this  play. You  see,  there  we  go. All  right,  this  is the  control  condition  which had  some  added  distractions  added  to  it. I'm  giving  a  lecture  on  signal  detection  theory, which  was  chosen  because  we  assume  students wouldn't  know  much  about  it  and  they also  might  find  it  boring. Well,  it's  a  little  bit  longer  than  that. Well,  okay,  let's  go  back  here. Seems  to  be  set  to  go  really  fast,  normal. Let  me  see  if there's  a  there  seems  to  be  no  sound  on  the  laptop. Yeah,  I'm  pressing  the  unmute  on  the  laptop.  I  see. Does  anyone  know  how  I  might  unmet? I  plug  it,  I'm  gonna  have  a  lot  of  sound, so  I  might  as  well  fix  this  problem quickly.  Thanks  for  your  help. I  bet  that's  how  students  wish  they  could  watch  lectures. I  do  teach,  I  teach a  distance  class  where  I  have  recorded my  lectures  and  they  have  told me  that  they  watch  them  on  fast  forward. So  let's  try  the  other  video, Okay,  This  is  interesting  behavior. Well,  I  can  help  you  to  imagine the  scene,  Picture  it  Sicily. In  the  control  video, we  added  cell  phones, ringing  students  getting  up  and  leaving  the  classroom, coming  back  and  sitting  down, asking  each  other  for  notes  and  pencils. Little  bits  of  video  would  appear  on  the  screens in  front  with  their  own  audio that  would  suddenly  interrupt  the  lecture. Then  on  the  right  we  have  one  of  the  diminished  versions. We  have  diminished  high. This  is  the  high  diminishment  condition. Everything  that's  happening  outside  the  lecture, we  also  had  a  low  diminishment  condition where  it  was  still  pretty  transparent, but  it  was  darker  then  all  of the  sound  has  been  attenuated  so  that  it  is  much  lower. All  right,  What  did  we  find? Well,  interestingly, even  with  such  a  simplified  set  of  videos, we  did  find  some  interesting  results connected  to  their  retention  scores. These  were  all  NC  State  students who  were  getting  credit  for  psychology  class. I  want  to  call  your  attention  to  all  of the  correlations  with  their  retention  score, which  is  highlighted  in  yellow  because  I augmented  rather  than  diminished  of  first, I  want  to  call  your  attention  to  presence. The  more  students  felt  present  in this  strange  little  video  environment, the  worse  they  did. We  thought  that  was  interesting  and  we want  to  follow  up  on  that  to  see  if it  was  the  feeling  of  being there  that  made  those  distractions more  distracting  than  those  who  could really  just  think.  This  is  just  a  video. Also,  those  who  indicated  that  they  had higher  tendency  towards  ADHD, They  also  scored  lower  on  the  retention  test, even  though  the  test  we  used was  a  clinical  measure  that  was  just  supposed  to  say, do  you  have  it  or  not,  that's not  really  a  sensitive  measure. We  were  still  able  to  find  this  prediction. We'd  like  to  follow  up  on  this  with better  measures  of  individual  differences  in  attention. Like  a  test  of  attentional  control, Then  one  small  point, not  really  related  to  diminished  reality. Notice  that  those  who  had a  higher  interest  in  psychology, which  we  measured  before  the  experiment, did  worse  on  a  test of  the  signal  detection  theory  content, which  I  found  a  little  worrying. Now  we  also  looked  at  it  by  diminishment  condition. We're  looking  at  whether people  had  the  high  diminishment, which  would  be  the  high  condition  for  audio, or  the  high  diminishment  for  video. High  means  that  there  was  more diminishment  and  low  means  that  there  was  less. We  did  find  an  interaction  and how  much  diminishment  there  was  and  what  kind. If  we  diminish  the  auditory  distractions, that  was  a  good  thing. It  seemed  to  be  more  important  than  diminishing the  visual  distractions  when we  only  diminished  the  visual  distractions  a  little  bit. So  that's  the  low  visual  that  preserved. We  thought  the  context that  group  did  worse  when they  were  auditory  distractions. I  think  that  tells  us  that preserving  context  at  all  costs  is  not really  always  in  line with  what  you  want  to  have  as  an  outcome. This  might  be  more  similar  to  the  experiment  where if  diminishment  isn't  complete  and  you  notice  it, it  might  actually  distract  you  more than  just  ignoring  the  video  on  its  own. But  wait,  there's  more.  Let's  take a  look  at  how  our  control  condition  did. First  of  off,  you  can  see  on the  retention  scores,  Nobody's  doing  great. We're  talking  here  on  retention, which  our  test  was  a  little  bit  hard. If  you  look  at  the  control  group, they  actually  did  as  well  as  any of  the  good  diminished  conditions. We  wondered  if  this  might  be  because  we're  used  to ignoring  a  lot  of normal  distractions  that  are  in  our  lives. If  the  video  looks  normal, you  can  just  focus  on  the  lecture. But  when  we  added  some  of  the  diminishments, they  may  have  actually  created more  distraction  by  trying  to  diminish. In  a  way,  this  left  us  with some  more  questions  than  answered. Being  present  in  the  environment  did  seem  to  matter, and  that  seemed  to  make  performance  worse. Maybe  that's  why  distractions are  so  challenging  in  an  auditorium, but  maybe  not  as  bad  when  you're  watching  a  video. Overall,  they  didn't  have  great  situational  awareness. Maybe  they  weren't  processing all  the  different  things  that  we  had  going  on. Even  in  the  control  condition, we  think  that  they  were  able  to pretty  well  ignore  what  we  thought  would be  pretty  bad  distraction  for  our  next  step. This  is  a  study  that  we  developed. Now  we're  moving  this  experiment to  virtual  reality  so  that  we  can  test these  different  types  of  diminishment  in virtual  reality  in  a  situation  that  we can  entirely  control  here. They  can  actually  sit  in  a  chair,  look  around. They'll  have  spatial  audio  that's making  different  sounds  left  and  right. People  will  be  close  to  them  or  further  away and  they'll  be  able  to  really see  the  screens  in  front  of  them. We're  keeping  with  a  long  established  tradition of  testing  augmented  reality, or  augmentations  in  virtual  reality  first before  we  try  and  do  it  in  an  actual  environment. Mostly  because  this  is  less  glitchy  than  any, even  the  best  augmentations  that  are  available  now. Then  once  we  establish effects  with  experiments  like  this, we'd  like  to  develop  a  system  that  will test  this  in  an  actual  classroom. Let  me  show  you  this  video of  the  control  section  session. Will  there  be  sound?  Probably  not  try. I  will  narrate  for  you. There  are  many  sounds  happening  in  this  environment. All  over  the  place,  people  are  clicking  pens, moving  around,  asking  each  other  quick  questions. We  made  the  control  condition  here  much more  auditorially  and  visually distracting  than  our  previous  experiment. Because  we  think  that  it's  going  to  mimic more  what  a  student  might  have  in  actual  classroom. Right  now,  I'm  going  to  move  on a  bit  because  the  focus  of those  first  two  experiments  is  on  learning. Which  can  be  a  difficult  outcome  to  measure because  learning  has  a  lot of  other  variables  that  are  part  of  it. In  these  other  follow  up  experiments, we  are  looking  mainly  at  performance. What  about  doing  well  in  the  moment  when it  doesn't  really  matter  whether you  retain  that  for  later. In  fact,  most  cognition  aids  are designed  for  performance,  not  learning. I  hope  none  of  us  find  ourselves  in  this  situation. But  maybe  you'll  be  in a  situation  one  day  when  you  need  to  grab the  defibrillator  off  the  wall  to administer  to  someone  who  is  having  a  heart  attack. You'll  need  the  assistance  of the  cognition  aid  that  comes  with this  machine  and  you  need  to  use it  immediately  and  accurately. Let's  change  our  focus  to  trying to  help  people  perform  in  the  moment. This  next  work  was  inspired by  talking  with  and  reading  about the  experiences  of  astronaut  on  a  very  space  station, which  is  where  they  have  to  do  all  of  their  work. We  went  to  the  plenary  session  of  Dr.  Mashburn, a  famous  astronaut  who's  spent  a  lot  of  time  in  space. He  mentioned  how  distracting  it  was to  try  and  work  in  a  small  environment  with lots  of  people  doing  things  all  around  you and  never  being  able  to  really  focus  or  concentrate. When  he  was  asked,  how  do  you deal  with  these  kinds  of  distractions? These  are  the  answers  he  gave. He  was  able  to  answer  only  as an  individual  because  he didn't  have  system  level  control. He  said  things  like,  make your  world  as  small  as  possible. Remind  yourself  that  what  you're  doing now  is  the  most  important  thing  in  the  world. He  said  that  he  noticed  some  people were  better  at  this  than  others and  thought  maybe  we  could train  people  to  be  better  at  it. Not  so  sure,  but  this  gave  us  some  food  for  thought. What  can  we  do  to  reduce  those  distractions? Well,  we  turned  to  the  hierarchy  of  safety. Anybody  familiar  with  this  hierarchy? What's  the  least  effective  way  to  make  a  system  safer? Someone's  seen  this  talk  before. Maybe  a  warning  or  training  about  what  the  hazard  is, trying  to  train  it  away. What's  the  second  most  effective  way to  keep  people  from  getting  harmed? To  come  up  with  some  guard  that prevents  you  from  being  able  to  interact  with  the  hazard. But  then  the  absolute  best  way  to prevent  being  falling  prey  to a  hazard  is  to  design  out  that  hazard. Now  here  the  hazard  are  distractions. We  could  train  away  the  hazard, probably  could  we  guard  against  the  hazard? Maybe,  but  diminishing  reality  could  conceivably design  out  the  hazard  because those  distractions  will  no  longer  be  present. We  focused  on  the  idea  of  designing  out  these  hazards. We  designed  a  task  in  which  to  test the  effects  of  removing  distraction  via  diminishment. This  was  the  assembly  of a  medical  ventilator  in  an  emergency. Now,  when  we  wrote  the  proposal  for this  experiment  back  in  2019, we  had  no  idea  that  everyone  would  know  what a  medical  ventilator  was  just  a  year  later. This  was  a  project  that  was  originally  funded  by  Nasa, and  we  situated  the  assembly  of the  ventilator  in  long  term  space  flight. There  it  would  be  assumed that  the  crew  members  trying  to  do this  would  not  have  additional  help or  communications  from  the  ground. As  in  the  classroom  study, we  also  measured  situational  awareness and  how  well  they  assembled  the  ventilator. Before  I  go  into  what  it  looked  like, I  want  to  acknowledge  the  massive  contributions of  my  other  PI's  on  this  project. I'm  a  psypologist,  not  a  programmer, and  neither  of  my  students,  however, Mary  Beth  Andy  and  her  team  at  Georgia  Tech came  up  with  a  way  to  run  this  entire  study  remotely, which  was  necessary  as  we were  still  in  the  time  of  covid. This  meant  designing  the  environment  and a  task  according  to  what  we  specified, a  psychologists,  and  then  making  it  possible  for  us to  control  what  the  participants  saw  and  heard  remotely. Vicki  Byrne,  my  other  PI  in  this  project, has  worked  at  Johnson  Space  Center  for  years. And  she  was  able  to  make  sure  that  we  had  good  content, that  we  had  task  analyses  and  other  ways  of  knowing exactly  what  kind  of  tasks  people  would  be  doing on  the  ISS  so  that  we  had  good  face  validity. None  of  this  would  have  happened  without  them. Here's  the  design  of  the  study. We  chose  a  task  of  using  the  procedure  to  assemble the  medical  ventilator  with our  flight  surgeon  as  the  incapacitated  patient. And  that's  why  you  would  have  to assemble  it  because  the  surgeon  is  the  patient. There  are  many  events  that  relate  to why  the  surgeon  is  injured  and what's  going  on  with  all  the  other  crew  members around  the  participant.  We'll  call  that  the  plot. We  divided  the  assembly  procedure  into three  equal  sections  and created  three  forms  of  diminished  reality  for  each  one, so  that  each  participant experienced  all  three  types  of  diminishment. There  was  a  no  diminishment  control. A  universal  diminishment  where  all  sounds  and peripheral  visual  items  were  diminished. And  a  context  aware  diminishment where  we  removed  the  clutter. But  we  kept  all  the  important  pieces in  the  important  audio  that  we would  then  quiz  them  about  later. Our  measures  were  typical  performance  measures, but  also  their  scores  on situational  awareness  and  how hard  they  felt  they  had  to  work  to  succeed. This  next  video  is  pretty  important. I'm  going  to  unplug and  I'm  not  really  sure  how  to  do  this. I  really  need  you  to  hear  the  audio  for  this. Do  you  have  any  ideas  for  me? I've  been  doing  it  may  be  a  problem  with  the  computer. I  could  also  play  the  audio  on  my  computer and  then  the  video  on  this  computer. Let's  see. So  this  is  going  to  try  to  do  it through  the  computer  speakers,  we'll  see  if  it  does  that connect  IV  support  patient, okay,  that's  IV  line  a  patient  give  red  set  of  shot. Connect  IV  support  to  patient, connect  IV  line  to  patient, give  red  sedative  shot. I  support  connect  IV  support  to  patient. All  right,  so  what  you  saw  there  was a  participant  doing  the  task  and  me  of  course, because  it's  a  demonstration, what  we  had  was  a  very  difficult  task that  we  made  intentionally  difficult. Where  you  were  reading the  assembly  directive  screen to  see  what  the  next  step  was, but  all  of  the  words  in the  procedure  for  the  different  objects  didn't  match. Names  of  those  actual  objects  in  the  environment. You  have  to  look  at  the  assembly  directives and  see  that  you  needed  the  pulse  oximeter. And  then  look  around  and  say, out  of  all  these  objects, the  one  that's  most  likely  that  is  the  oxy  pulse. And  then  you  would  give  a  voice  command to  do  something  with  the  oxy  pulse. It  required  a  lot  of  working  memory and  a  lot  of  attention  to  do  this. You  might  think  that  that  is  a  little  bit  strange, that  maybe  we  made  it  too  hard  on  purpose. But  once  we  ran this  with  personnel  at  Johnson  Space  Center, they  said  that  this  is  actually a  very  common  type  of  task, that  they  even  have  a  name  for  it. They  call  an  op  nom  task  or  operations nomenclature  because  the  nomenclature is  almost  always  wrong. Those  are  the  three  different  conditions that  people  were  exposed  to. We  conducted  the  study  entirely  Wizard  Oz. We  had  a  teleconference  with participants  as  they  did  the  task, and  we  listened  to their  voice  commands  for  assembling  the  ventilator. And  then  we  controlled  whether  or  not it  moved  on  to  the  next  step. If  they  were  incorrect,  we  clicked  a  button  that  would play  an  error  sound  to  the  participants. It  seemed  like  we  had  the  best voice  recognition  system  ever, much  better  than  Alexa. Because  the  graduate  students  were incredible  natural  language  recognizers and  they  could  really understand  what  a  person  was  saying. The  only  criteria  was  that  they  had  to have  their  own  Android  phone  so that  they  could  do  this  using a  Google  Cardboard  own  homes. We  gave  the  cardboard  to each  participant  by  mail  if  necessary, and  they  got  to  keep  it  so  we  had  no  hygiene  issues. Then  they  did  all  of  our  pre  study  materials  online, installed  our  app  onto  their  phone, and  then  during  the  zoom  call, they  completed  the  study. Here's  a  video  of  what  the  experience  looked  like  for the  participant  and  for the  experimenters  who  were  controlling  it. We  recorded  this  in  the  lab so  that  you  can  see  it  a  little  more  clearly. But  this  is  what  was  happening  also  via  zoom connect  to  one  to connect  tube  one  to  human  base, connect  tube  one  to  vent. All  right,  So  it  was  a  little  intense, but  I'm  not  going  to  go  into  all of  the  technical  issues  that  we  experienced. But  if  you  would  like  a  lessons  learned  talk, I'm  happy  to  come  back  and  give  a  workshop. So  what  happened?  What  are  the  effects  of  diminishment? Well,  we  had  two  samples, one  were  NC  State  Stem  graduate  students. And  we  consider  them  to  be astronaut  analogs  because  they  had similar  training  to  astronauts in  terms  of  having  a  Master's  in  science. At  least  then  also with  employees  at  Johnson  Space  Center. I'm  going  to  show  you  their  results together  because  I  think  that their  similarities  and their  differences  are  very  interesting. It  also  shows  how  important  it  is  to know  your  sample  generally. These  are  the  demographics  most  had some  augmented  reality  experience  in  both  groups. The  age  group  was  much  more  limited  for the  Stem  participants,  obviously,  than  employees. These  were  the  first  of  the  performance  measures, the  number  of  steps  that  they  were  able  to complete  in  any  one  of  the  scenarios, we  converted  their  steps  to standardized  scores  because  each  section was  counterbalanced  and  was  also slightly  different  in  terms  of  the number  of  steps  you  could  have. But  just  in  general  know  that once  you  start  comparing  conditions, a  zero  would  mean  average  performance. If  we  look  at  this,  you  can see  that  the  first  thing  to  notice  is  that the  control  condition  has a  lower  number  of  steps  completed  for  both  groups. It  did  differ  significantly from  the  context  of  were  group, we  saw  the  same  pattern, Johnson  Space  Center  participants. It's  interesting  to  me  that the  universal  diminishment  condition, which  you  might  think  would  get  rid  of everything  around  you  so that  you  could  complete  the  task, wasn't  actually  the  best  in  terms  of  performance. The  context  aware  was  Next  up, let's  quickly  look  at  the  errors  per  section  here. The  findings  were  a  little  less  clear because  the  NCSU  sample  did  show a  difference  between  the  universal  diminishment and  the  context  but  not  the  control. Also,  the  errors  were  higher than  the  context  aware  condition. This  might  be  because  they  were  able  to  see  what the  next  step  might  be  and  try  something more  frequently,  even  if  they  were  wrong. Skip  that.  Then  here  were  the  results  for  afterwards. We  asked  them,  where were  these  different  objects  in  the  room? And  these  were  objects  that  they  didn't  have  to  use to  see  if  they  had  any  situational  awareness. Somewhat  predictably,  those  in  the  context  of where  condition  were  more  unaware  of  their  surroundings. Because  we  did  diminish  those  objects, they  were  just  shown  as transparencies  as  you  saw  in  the  video. They  were  more  easily  ignored, which  ended  up  in  them  being  ignored  and  then showing  lower  spatial  situational  awareness. We  also  asked  them  about  things  that  were  just  happening. Other  crew  members  were  talking  and  discussing  how to  address  this  emergency  that  was  happening  on  the  ISS. And  NCSU  sample,  those who  were  in  the  universal  diminishment, really  did  not  know  what  was  going on  compared  to  the  control  group. Even  though  the  control  group  had a  lot  of  sound  and  a  lot  of  things  going  on, they  had  more  access  to keeping  on  top  of  what  was  happening  situationally. For  other  people  in  the  plot, it  did  surprise  us  that our  context  aware  condition  did not  do  better  than  this  because  they had  access  to  the  information  that  was  on this  test  and  they  didn't have  all  the  extra  stuff  happening. The  only  idea  I  have  here  is  that  our  task  was  incredibly difficult  and  people  were  really focused  on  the  task  in  all  conditions. And  no  one  did  great  on  the  situational  awareness  test, as  you  can  see  just  from  overall  scores. Last  one  thing  I  think  is  maybe the  most  interesting  measure  was their  own  feelings  of  mental  workload. I  created  the  scale  by  using some  of  the  subscales  from  the  Nasa  Tl  X, which  is  a  traditional  measure  of  workload. But  I  only  used  mental  demand,  temporal  demand,  effort, and  frustration  because  there  really was  no  physical  demand  in  this  task. They  were  just  giving  voice  commands. They  didn't  even  have  something to  operate  in  the  VR  environment. For  the  NCSU  participants, those  in  the  universal  diminishment  condition reported  lower  workload  than  those  in  the  control. This  might  be  an  issue, they  didn't  know  what  they  didn't  know see  or  hear  some  things  and  so they  didn't  know  they  were  missing  them. They  had  lower  workload  even  though  if  you  remember from  the  number  of  steps  completed, they  also  didn't  do  as  well. Now,  the  Johnson  Space  Center  group was  very  sensitive  to  workload. They  had  significant  differences between  the  control  and  each  diminishment  condition. I  think  this  is  what  speaks to  the  differences  in  the  samples. Are  JSC  participants,  you  could  just  tell  when  you ran  them  that  these  are people  who  problem  solve  for  a  living. They  are  determined  to  perform  no  matter  what. So  they  did,  they  didn't  like  making  mistakes, they  didn't  like  failing  to  complete  all  the  steps, and  they  were  going  to  work  harder  to  do  so. Whereas  maybe  our  stem  graduate  students  were  a  little more  relaxed  in  taking  on  more  workload, I  think  our  JSC  participants could  feel  that  difference  in  workload, even  if  it  wasn't  that  visible  in  their  performance, which  they  maintained  as  high  as  they  could. Last,  we  included  a  new  measure, just  with  the  Johnson  Space  Center  study, about  what  participants  might  have  noticed  going on  around  them  that  we  could consider  to  be  something  pre  attentive. This  included  questions  like what  gender  or  perceived  gender spent  the  most  time  talking, who  talked  the  most,  Who  came  through  the  room  you're  in, who  came  through  Most  often. We  base  these  questions  on pretty  standard  early  findings  from  cognitive  psychology, where  people  are  pretty  good at  noticing  when  the  gender  of a  speaker  changes  even  if  it's not  in  the  channel  that  you're  attending  to. And  indeed  there  was  that  effect where  those  in  the  diminished  condition, we're  more  likely  to  answer  these  questions  correctly, which  was  really  interesting  than  the  control. Even  though  the  signal  in the  diminished  condition  was  quite low  or  difficult  to  see. There  was  wider  variability when  people  were  in  the  context  of  were  condition. I  think  this  is  something  to  follow  up  on and  I'm  not  sure  I  totally  have  an  explanation for  why  those  in  the  diminished condition  did  much  better  at this  task  in  our  future  directions. I  have  many  other  studies  to discuss  and  many  other  ideas. I  have  a  senior  Phd  student  who  is currently  using  some  funding  to  look at  how  we  might  use  diminishment  in  training. Perhaps  you're  going  to  be  expected  to function  in  a  highly  chaotic  environment, and  maybe  we  can  scaffold your  introduction  to  that  chaos  using  diminishment. This  could  apply  to  a  lot  of  jobs  that require  work  in  an  unexpected  or  chaotic  environment. But  we'd  also  like  to  replicate the  study  using  actual  augmentation, diminishment  in  the  physical  environment  now  that  we  have a  better  idea  of  how  it  should  or  shouldn't  look  last, we'd  also  like  to  see  for whom  diminishment  matters  the  most  by including  more  sensitive  measures  of  individual  ability. I'm  going  to  close  with  some  of  the  theoretical  models that  bring  all  of  this  work together  and  organize  our  thinking about  how  to  do  research  in  this  area. The  first  is  from  a  2020  paper that  we  published  in  Human  Factors, about  the  fact  that  people  tended  not  to  think  heavily about  what  kind  of  cognition  was being  aided  when  they  create  a  cognition  aid. People  would  casually  throw  around  terms  like  this, checklist  is  for  memory, when  really  the  checklist  was  more  about  encouraging a  social  situation  where  people would  have  to  talk  to  each  other  about  a  patient. The  idea  of  a  cognition  aid, I  think  should  be  embedded  in  the  idea  of what  cognition  is  being  supported. And  then  once  we  establish  that  better, then  we  can  look  for  better  patterns  and theoretical  ways  in  which  to  make  those  aids  even  better. For  example,  you  could  first  look  at  research on  selective  attention  to get  an  idea  of  what's  important. And  then  see  if  there  are  any  studies  that  looked  at exactly  that  problem  to  help  design your  aid  that  would  help  with  selective  attention. However,  right  now,  it  is  very difficult  to  find  literature  that  goes to  the  level  of  what  kind  of attention  do  you  want  to  support? We've  now  taken  this  basic  advice  to put  the  cognition  back  in  cognition  Aids, and  we've  created  this  framework  on  how  to  study  them. Generally,  we  situate  inputs  seen  here  at  the  top  level. Then  we  consider  how  they  might  map  to  the  aid according  to  what  cognitive process  it  is  that  you  need  for  the  task. This  will  help  us  map  to  the  kinds  of  techniques  that a  designer  can  use  to help  achieve  that  goal  most  successfully. We  have  the  inputs  at  the  top, Mapping  to  the  cognitive  processes and  then  mapping  to  the  different  kinds  of tools  you  might  use  as a  computer  scientist  to  create  such  an  aid. Then  last,  all  of  those  have  to  map  to  what  kind  of platform  are  you  going  to  be  using  to  deliver  this  aid? Each  of  these  is  interacting  with  each  other. I  think  that  what  we  need  is  advice  at  each one  of  these  levels  on  how  to  progress  to  the  next  level, what  interactions  there  are,  and  how  to  match  it, so  that  we  can  make  it  easier  to  tell  designers. Here's  what  you  should  do  to create  the  cognition  aid  you  want  for  this  task, this  environment,  in  this  population. In  closing,  we  humans  can  do  amazing  things, but  our  perception,  attention, and  memory  will  always  be extremely  limited,  especially  mine. They  say  that  every  research  project is  an  urge  to  scratch  a  personal  itch. I  would  say  that  every  day  I  think  about  how  I  would  like to  use  a  diminished  reality  aid to  find  focus  and  peace  in  my  life. Diminishing  reality  is  something  I do  think  about  absolutely  every  day. I'd  like  to  leave  now  with  opportunity  for  questions  and acknowledgments  of  all  the  wonderful  students  who worked  in  my  lab  and  my  collaborators.  Thank  you. Thank  you  very  much.  Wow,  that's  crazy. Thank  you,  Anne.  Now  we  have  a  few  minutes to  take  questions  from  the  audience.  Anyone? Chris?  So  one  of  my  pets. That's  one  of  my  pet  peeves  too. Yeah,  I  think  that's  really  interesting.  I'd  say. Why  do  we  have  to  decide to  make  something  safer  by mimicking  what  we've  had  in  the  past? It's  not  as  though  people  didn't get  hit  by  loud  cars  in  the  past. If  we  have  an  electric  car  that's  very  quiet, why  is  our  solution  to  decide that  we  need  to  make  it  loud  like  old  cars  used  to  be? I'm  not  saying  I  know  the  solution, but  maybe  we  could  think  about  removing  something else  from  the  environment  to  make  it  so  that the  car  is  more  perceptible  in  some  way. Maybe  the  car  itself  could  have  different  systems  that could  help  interact  with  pedestrians  in  a  safer  way. But  I  actually  totally agree  that  maybe  just  making  something louder  is  not  necessarily  going  to  help the  whole  environment  because what  else  are  you  inhibiting? I  don't  know  about  Atlanta, but  we  have  some  very  loud  motorcycles  in  Raleigh. When  those  go  by,  you  can't  have  a  conversation. Things  are  definitely  impacted  by  that. I'm  not  sure  that  having  that  loudness  in the  motorcycle  is  worth  the perceived  safety  that  it  might increase  by  having  such  sounds  on  the  road. What  do  you  think a  adding  extra  noise? We  actually,  I  think your  work  probably  shows  a  lot  of  other  examples. Probably  the  mindset  to come  out  the  design  remediation  in  a  different  way. Thank  you  Lori.  You  have  a  question. It  plays  off  that  this  is  the  way  to  get positive  results  by  masking,  say  one  thing. It  existence  is  quotes in  your  neighborhood  but  you  can't  go  turn  them  off. But  I'm  wondering  if  it's  a  white  noise, if  there's  something  in  any  of  these  situations, whether  it's  audio  visual  or  anything else  that  you  could  introduce, stimulation  can  actually,  well, I  think  that  would  be  the  entire  idea  of  having some  diminished  reality  system that  you  customize  and  control. I  think  the  problem  you're requesting  help  with  is  maybe  one  of  the  easier  ones. That's  probably  something  that  we  could  do  now that  you  could  have  a  leaf  blower  eliminator whether  or  not  it's  going  to  be  context  aware to  know  that  maybe  it  should not  eliminate  the  leaf  blower  that's  about  to  hit  you. I  don't  know.  But  in  terms of  selective  sound  attenuation, that  seems  like  something  that's lower  hanging  fruit  than  most.  I  agree  with  you. I  also  love  a  leaf  blower attenuator,  a  student  questions. I'm  sure  there's  a  student  has  questions. Don't  you  want  to  diminish  your  reality? You  don't  have  any  experiences  in  classes,  anything? Yeah,  I've  Brandy's  playing  Gamy. Brandy's  right  now  is  taking  me  to classes  and  whenever  I  put  around  here, she  has  the  stability  just  to  shut  things  out. It  doesn't  matter  what  noises  around. So  have  you  tested  that  in  the  sense  of  looking  at various  age  groups  and  saying  what  age  groups  diminish? I  myself  know  that. I  do  not  have  the  capabilities  then. I'm  76,  she's  18. I  know  what  I  grew  up  with. She's  grown  up  my have  these  amazing  capabilities to  diminish  the  noise  around  ten. Yeah,  that  actually  connects to  something  I've  always  been  very  interested  in, which  is  my  research  really  started  in looking  at  age  related  differences  in  cognition. But  since  I  started  getting  this  sort  of  funding, let's  just  say  Nasa  doesn't  care about  the  people  who  are  older. But  I  could  make  some  predictions  that  we tend  to  get  less  good  at  inhibiting  as  we  get  older, inhibiting  sounds  and  distractions. And  I've  often  thought  that, wouldn't  it  be  great  to  have  diminished reality  glasses  that  got  rid  of all  the  billboards  on  the  highway? Pretty  sure  the  Chamber of  Commerce  is  not  going  to  be  happy  with  that, but  why  not  get rid  of  especially  the  flashing  ones  and  maybe that  would  be  especially  good for  older  drivers  That  said, I  think  that  our  18  year  olds  are  better  at  it, but  they're  still  going  to  be  limited. There's  still  going  to  be  lots  of  situations where  they  get  distracted  when  they  don't  want  to  be. Even  when  we're  at  our  best, we're  not  that  great.  We  only  get  worse. That's  a  really  good  point  and  I  often  do  that  too. You  know,  coffee  shop  noise or  having  noise  in  the  background. I  think  that's  partially  because  of what  technologies  we've  had  right  now. Our  technology  for  adding  white  noise to  cover  distraction.  Pretty  good. Pretty  easy.  It  would  be  a  lot harder  for  you  to  get  a  good  diminishment  system. But  I  think  what's  interesting  would be  testing  which  of  those  is better  if  we  do  eventually have  the  capability  of  making  a  system  that  diminishes, is  it  worth  going, investing  in  that  system versus  having  a  noise  covering  system, which  is  what  we  have  access  to  today. Can  you  elaborate  on  that  a  little  bit? Yeah.  Yeah.  Well,  first,  can  we  see  pictures? Second,  I  think  that  that  is  in  a  way, what  our  universal  diminishment  condition  did. Because  it  really  blurred the  periphery  and  kept  you  where  you  had  to, like  wherever  you  were  looking was  where  you  were  focused. But  it  really  eliminated  a  lot  of peripheral  information  rather than  selectively  removing  things. And  I  think  that,  that  didn't  work all  that  well  because  it  did  change where  people  had  to  physically  look  to  see  something. I  think  it  would  be interesting  to  test  a  universal diminishment  condition  that  did something  like  desaturate  objects  all around  that  you  don't  need  rather than  changing  something  about  what  you  physically  see. But  I  also  think  that  we  can use  this  in  a  lot  of  different  ways. Think  about  the  mechanisms  of  some  computer  games. Have  you  ever  been  in  one  where  you need  a  flash  light  to  see  well? The  purpose  of  that  is  to  make  you  behave  differently  and to  probably  build  suspense and  build  all  these  different  feelings. We  can  harness  that  diminishment for  depending  on  what  the  user's  task  is. Sometimes  it  might  be  a  good  thing, a  lot  of  times  it  might  be  a  bad  thing. And  I  think  that  creating a  taxonomy  of  when  to  use  for  what  purpose, that's  my  ultimate  goal. I  have  a  question  for  you  as  the  person  responsible  for that  layer  of  designing  and implementing  these  AR  techniques. You  said  something  that  really  struck  me about  that  when  the  diminishments  aren't  perfect, they  can  in  fact  make  the  situation  worse. I  know  that  as  a  computer  scientist, I'm  never  going  to  be  able  to  make  something  that's perfect  or  even  our  noise  canceling, headphones  can't  cancel  out certain  types  of  auditory  signals. Do  you  have  any  thoughts  on, at  the  HCI  level, what  could  I  do  when  I  create? In  what  ways  could  they  fail  in  a less  like,  detrimental  way? Yeah,  and  I  think  about  that  a  lot, and  I  probably  shouldn't  use  the  word  perfect because  I  don't  think  that  we  have  to  have  perfection. I  think  what  we  have  to  understand  is  what  is  good enough  to  change  learning  and  performance. If  we  can  find  out  what's  good  enough,  then  maybe. Goal  we  shoot  for.  But  also, I  think  that  it  has  to  do  with  understanding  more about  every  task  than  just  saying, we're  going  to  diminish  things. You  have  to  understand  what  is  the  goal  of  this  person? What  do  I  need to  diminish  what  would  have  the  biggest  effect? And  then  how  can  I  do  that in  a  way  that  won't  be  distracting? I  don't  think  there's  any  clear  answer  of  if  this, then  that  it's  always  going  to  have that  piece  of  the  art  to  it  where  you  say, well,  I  cannot  remove the  people  from  this  table  without looking  like  a  little  bit  of  the  matrix. Sometimes  they're  there,  sometimes  they're  not, but  can  I  put  some  sort  of  filter over  them  that  reduces  their  salience? That  might  have  the  same  effect, but  without  trying  to  go  for a  Holy  Grail  of  diminishment. I  think  that  that's  part  of  the  HCI  piece  here, which  we're  looking  for these  big  effects  on  learning  and  on  performance. But  there's  also  the  importance  of  acceptability and  perceiving  the  different  types  of  HCI, diminishment  eye  play  a  role  there  folks  are  interested because  they  haven't  seen  this  type  of  it.  Could  be. It  could  be.  Are  you  saying  you  want  to  fund a  longitudinal  study  We  do  throughout  the  whole  semester? That  yeah,  I  agree. That  could  be  part  of  it  because  there  is always  a  little  bit  of  novelty. The  people  in  the  last  experiment  I  discussed, they  experienced  all  three  conditions. They  were  also  probably  mentally  comparing those  conditions  because  they  saw  each  one. Sometimes  I  feel  like  the  task  was  so  hard they  weren't  necessarily  aware  of  the  differences. But  it  was  pretty  clear  that in  the  context  of  were  diminishment, we  took  all  of  the  clutter  off  the  walls. In  terms  of  visual  search, it  should  be  easier  to  visually  search  for objects  because  there  are  fewer  objects  that  are  visible. And  they're  also  not  against  the  cluttered  background. I  think  that  novelty  or  not, when  you're  comparing  the  different  types of  diminishment  against  each  other, I  would  have  expected  to  see  effects  there, even  if  all  three  of  them  had  some  novelty  to  them. If I'd  definitely  like to  stop  the  diminishment  before  it  becomes  unethical, I  think  that  this  is  where our  other  processes  and  human  factors  come  in. You  have  to  understand  who  you're  designing for  and  you  have  to  design  with  them, have  participatory  design,  and  get  their  buy  in. I  think  that  anytime  if  we  designed a  health  care  diminishment  system and  then  it's  rolled  it  out, it  would  absolutely  fail. It  would  only  be  if  you  worked with  health  care  workers  in their  context  and  iterated and  changed  things  and  tried  to  come  up  with, well,  what  is  it  that  you  need? At  the  same  hospital  I  was  at, there  was  a  intercom  every  day  where  it  was a  Catholic  hospital  and  there was  a  lengthy  prayer  that  happened. If  that  happened,  you  couldn't  really hear  what  was  going  on  in  the  hallway. I'm  not  saying  that  that might  not  be  critical  for  health  care, but  it  might  be  critical  to  someone's  well  being. I  don't  want  to  make  the  decision  about  whether  or  not that  should  be  something  that  is  deemphasized. But  I  do  think  that  the  healthcare  workers themselves  could  make  that  decision. One  last  question. You  briefly  addressed  how  students and  equal  take  on  the  stress. First  of,  are  you able  to  find  an  objective  way  to  measure that  analysis  or  take  that  out  symes? Second,  in  terms  of  long  term  solution, the  end  be  probably better  learning  environments,  learning  outcomes. Do  you  think  the  solutions  lie  in  flying,  diminishment, automate  correct  things, or  training  ourselves  made to  handle  the  stimulating  environments  to  some. Yeah.  Let  me  did  that  last  part  first. I  think  that  there  are situations  in  which  we  will not  be  able  to  diminish  things. Mary  Beth  has  a  great  example. I  think  you  mentioned  that  social  workers  might be  thrown  into  a  very  chaotic  home  environment. That  they  have  to  assess  that  things  could  be  very  loud, children  running  around,  all  kinds  of  things  going  on, and  they're  going  to  perform  in  that  environment. So  the  question  is  how  do  you  get  them  ready  for  that? So  maybe  you  could  scaffold  them into  it  in  a  way  that  is  training. But  I  would  say  that  just like  with  the  hierarchy  of  safety, training  to  operate  in  chaos is  not  as  good  as  operating  in  non  chaos. If  we  can  come  up  with  ways  for other  jobs  where  you  can  have  some  kind  of  diminishment, you  could  imagine  having  headphones  that  could  reduce the  loudness  or  intensity of  a  child's  crying,  but  you  still  know  it's  there. You're  not  going  to  like,  oh,  there's  a  baby. You're  still  going  to  know, but  you  don't  necessarily  have the  emotional  impact  and stress  of  the  full  volume,  something  like  that. I  think  that  it's  going  to  depend  on  the  situation. Then  we  should  always  be  trying  to go  for  that  hierarchy  where  we  can either  guard  against  people having  distractions  affect  their  performance, or  eliminate  them  so that  they  don't  have  to  worry  about  them. Then  I  think  you  had  a  first  part  to your  question.  I  forgot  already. What  was that  Yes question? Yeah,  yeah. You  hit  on  something  that  we would  definitely  like  to  do  more  of  in  the  future, which  is  look  at  individual  differences. And  I  think  that  we  probably  could  have captured  this  with  motivation  level. So  if  people  were  honest  about  how  motivated  they  were, the  experiment  using  that  as  a  control  might  have  showed that  difference  between  the  NC  State  sample and  the  JSC  sample. But  I  also  think  that  other  individual  differences  about ability  to  control  your  attention or  how  much  working  memory  you  have, even  though  everyone  is  pretty  limited, there's  still  a  spectrum. And  knowing  where  people  fall on  that  might  help  us  predict how  much  they're  going  to  need  or not  need  our  interventions. Thank  you  very  much,  Ann.  Let's  give  her one  last  round  of  applause.  Thank  you  everyone.