[00:00:10]
>> Well. You know. It's true yes which. She said. So it's. Sort of old. So let's just. Everyone thanks for having. I got the microphone on I'm just Roberts as you just heard I'm faculty new assistant professor in interactive computing here at Georgia Tech and I'm going to talk about my work in learning Sciences which is the study of how people learn and how to design the learning environments most my research takes place in out of school learning environments like museums museums are free choice environments where people are going and exploring according to their own interests so Curiosity plays a major factor in the museum experience as we start putting more and more digital interactive into museums.

[00:01:32]
In recent years technology is also becoming a major mediating factor in museums so today I'm going to talk about some of the work bringing those 2 together and I'm going to start with a pretty big question this is a question that in some form or another every curator has faced in their career and that question is.

[00:01:51]
Who cares about this pot right particularly when this pot isn't a case with some other pots and some pieces of pots and some pieces of some other things a space surrounded by any number of other clay pots. And to the untrained eye this pot doesn't really look like anything particularly notable But what if I told you that it turns out that this little clay pot helped usher in a new area era and human history.

[00:02:22]
And this is the challenges faced by curators all the time every item in this case represents some innovation in human history it was carefully chosen from among a lot of other things to tell a particular piece of this story and the theme of this case is in fact Bronze Age innovations and I'm guessing I feel pretty confident in saying that everyone looking at this somebody of objects for the 1st time did not immediately think innovative right because it just looks like some clay artifacts but that's the magic of museums that we have this opportunity to be in a learning environment that that provides an opportunity to think about commonplace objects in a new light.

[00:03:06]
So today I'm talking about 2 very different projects one is the Natural History Museum where we used touchscreen design to inspire curiosity about objects that were not directly explorable they were behind glass and off limits for for exploration so how can we get people to become curious about these things that they didn't necessarily think about before and then also in the Citizen Science context where we created a digital resource that help volunteers expand to a deeper understanding of invertebrates now because this is a brown bag and I decided that it might be prudent to save the high resolution insect imagery till later in the talk so I'm going to go ahead and start with the Natural History Museum just in case.

[00:03:49]
One of the most important considerations in designing for places like museums is that you were never designing in a vacuum this is not a lab study where you are just there and you can control the environment and what people are coming in with this is a place that is in a particular context and the exhibit that I'm going to talk about today has actually been under construction for 10 years in the planning from beginning to opening it was already open in the space to the public when we came in and the technologies the touchscreens that we used were already in the space as well.

[00:04:22]
This is a very different experience than designing from scratch I'm going to use this project to talk about how we approach the design not from a blank slate but how do we figure out what was going on and how can we use what is already happening in the space to augment these objects I'll talk about the form of testing to figure out what to study and then the design driven by theory and then the clothes with how we evaluated it in the museum with actual visitors.

[00:04:50]
So this exhibit was relatively new at the time permanent exhibition in a large natural history museum covered 5000 years of Chinese history through 350 artifacts arranged in 5 thematic galleries and like most object based exhibits the China hall featured objects they're arranged in glass cases with some interpretive text printed large around the space and then also reading rails that give information about each of the individual artifacts like what they are where they were found what they're made up there proximate age and in a traditional space on these reading rails you might also have a paragraph or 2 either talking about the theme of the case or calling out one or more objects that the curators wanted to say something special about it but what made this space unique is that here the content was offloaded to what we call digital rails touch screens that were mounted below each case.

[00:05:48]
So these touch screens you saw on the left they all had thematic text for the case they had all of the objects that were in that case menu either individually or in groups and then each one had typically 3 stories about a paragraph long the visitors could select and then some had additional media like photos or or illustrations to go along with them so mostly for the most part this is a direct translation of what might have been printed on that rail but it's just kind of more of it.

[00:06:17]
And the idea is that visitors can decide what they want to explore what they want to learn more about so rather than the one thing that the curator decided was most important somebody can go to a case him look at and say that's really interesting I wonder what that weird thing is.

[00:06:32]
But we wondered whether the technology could actually be doing a little bit more than that whether we could redesign these screens to maximize them as tools for learning. So does zoom back out to the exhibit again this is a $6000.00 square foot exhibition space and it was a ticketed exhibit at the time so everyone and every all the rule following visitors at least would start at the front which is on the left side of the panel and then go through and flow all the way to the back through these 5 thematic galleries the design of the galleries was meant to be very subtle very call and.

[00:07:07]
All the technology was designed in a way that it would augment the artifacts without distracting from them. So the research in the space started with 1st of formative testing what what's going on in the space already Howard visitors interacting so that we can know where to focus our design efforts and then the Phase 2 was redesigning one of the touch screens and then testing it in the space can we designed it to specifically to foster curiosity about artifacts.

[00:07:37]
And I'm going to talk about each of these starting with the Phase one studies the 1st phase I was spending a lot of time just looking around seeing what people were doing doing some eavesdropping doing a little bit of just kind of watching how people interacted in seeing what they were interested in and then I did a more formal timing in tracking study which is a common method in museums to try to figure out how people are moving through space so typically this is done on paper I did it with a tablet application so I had a map of the space that I could annotate with where people stopped as they went through and then I could code those annotations with the kinds of behaviors they did their things like what did they look at which one.

[00:08:14]
Of those touch screens do they touch when did they talk to each other when did they take a photo things like that and I could put together for each of these visitors a map of their interaction through this exhibit. In total $49.00 groups this way I did not recruit them in advance I just asked I just kind of tried not creepily at all follow them around without seeing them until the end and I was trying to see it without any intervention.

[00:08:42]
On my part what were they doing I picked as a group to me and I only did groups I did not look at single visitors because I wanted to have the opportunity for people to talk to each other so I could keep at least that constant And I looked at whatever the 1st adult or teenager in the in a group that came in the 1st one who did one of the behaviors I was looking at was the one I did up tracking through their space and I spent on average 18 minutes in the gallery which is about what is typical of what the museum expected some of them sailed right through with 3 minutes is practically a sprint through the 6000 square feet some of them took a long time and spent almost an hour the median and mode about both about 14 minutes them if you look at it by gallery.

[00:09:25]
A pretty typical trend in museums the 1st gallery people spent about 4 minutes and then it went a little bit down from there as people got ready to move on the exception of course is gallery for that tall blue bar that had in that space a puppet show a video that was really engaging and people spent a lot of time watching and then of course they would realize they just spent a lot of time watching this movie in the space and zoom past through gallery 5.

[00:09:52]
So now looking more closely at each case how they interacted they on average visitor stopped to look at $22.00 of the $45.00 cases so about half. In those 18 minutes they were there they stayed between 15 and 50 seconds at any one spot. And they touched only 4 or 5 of the rails as they went through 30 percent of the people I follow didn't touch any of the touch screens at all some of them used them a lots of them to use just a few.

[00:10:22]
So now that I knew in the real world what people were doing in the space I wanted to see what they were doing according to the analytics data I looked at the all of the events that were captured in the 3 week study period so not just the times that I was there interacting but the whole time to visit the museum was opened what was happening on those touch screens there were over 200000 unique events and I was interested in 3 particular ones when they chose an object from that many and when they chose a story about that object and then when they chose a slide on the slide show for the stories that had slideshow so now I have analytics data I have live tracking data and I want to compare them so I compared them here is what the 2 look like together and line in the middle the 0 line is the average for each for each metric and then.

[00:11:09]
A plus and minus standard deviations above the me above and below the mean. So we see again that typical overall trend is a little bit more interaction at the beginning and then it kind of drops off as you get to gallery $4.00 and $5.00 with a. Couple notable exceptions and gallery 2 there you see that one long blue bar descending that screen happened to be broken during a chunk of the study period so there were no analytics data being collected and then you seeing gallery 3 this tall drink of water here which is the imperial robe was large striking beautiful robe and people stopped to look at it and also wanted to learn more about it so they went through the screen and read some more of the information about it we have some of that work sort of mismatch so these tiger shoes people stopped and looked at and were drawn to look at but not necessarily to read more about.

[00:12:01]
And then there were some that were below average on both which is expected it for all that is how averages work something has to be below average but this one caught our attention because of what it was it's happened to be a case that the the curators going in had pointed out as being magically very important to understanding that this 2nd gallery and it doesn't actually look on this graph too bad in terms of being below but if you go back to my timing in tracking map of all the 45 screens this is the one that literally 0 of the people that I had followed stopped and used so we thought that this one case that was under appreciated but to magically very important would be a good place to focus our efforts.

[00:12:48]
So of course we're back here. These pieces we know are all on the theme of Bronze Age innovations and on the surface I think we agree they don't look very innovative but we're seeing on the left Oracle Bones that are some of the earliest surviving writing and records of civilization at that time in the front we have some of the 1st currency and carry shells and those pots those pots were fired at such high temperature as it turns out that they not only changed the way cities were built specialized craft districts started being constructed to keep the ash away from residential areas but that techniques for making them pave the way for bronze casting and the Bronze Age So these are interesting objects but people would have no idea unless you know to be interested in them you don't know what questions to ask and what to be curious about.

[00:13:38]
Now the concept of curiosity has been heavily studied in psychology and there are a lot of different definitions we were drawn to this one about Curiosity is a desire to know to see or experience that motivates exploratory behavior that's what we want people to do we want them to be exploring in this space and in particular we looked at 2 models that aligned with kind of behaviors we wanted to come up one is this.

[00:14:01]
The idea of curiosity as an information gap something that increases when we find out that there's something that we could know that we don't know and it's got to be a negative thing like tension or frustration and similar to hunger it keeps increasing until we get the information we need and then it starts to diminish once we get the information and we're satisfied.

[00:14:19]
On the other hand you could look at interest as a positive thing something that's a combination of wanting and liking in different amounts and you anticipate pleasure from gaining new information this can be typically driven by and aesthetically pleasing or viscerally engaging experience so these are 2 possible options the way we could promote curiosity so how do we translate that into an interface design for these touch screens.

[00:14:43]
Our 1st design for the information gap model we call big question because there was a literally large font question we had 6 or 7 of them that scrolled across an attract screen every few seconds would change to a new question and these are all piece of information that were in the text that people wouldn't necessarily know to look for and when you clicked on the question you would take it into that text that answers the question and then you'd have access to the same content as before so this big questions designed to bring out the idea that there are interesting questions to be asked about the subject if you just know what they are.

[00:15:16]
The 2nd design came from an observation that we noticed people were very engaged in a particular slide show in the original design this showed the evolution of Chinese characters over time and people had a lot of interesting conversations and were really excited to learn more about it so we brought that up to the surface and made that an interactive experience so you could scroll along and see which were in use together when different characters were in use and then the buttons on the right still let you drill down and get that access to that same content so this aligns more with other interactive museum exhibits emphasizing engagement on the sensory motor aesthetic and emotional level to draw people in.

[00:15:54]
So now we had these 3 designs all trying to get at this idea how can we foster engagement and curiosity about these objects that are locked away behind glass. We studied them in the museum space and recorded video and audio of what people were doing with the screens and what they were taught saying about them and we looked at how people interacted the none of the designs were intended to draw people in from across the room so we only kind of people who actually came near the case and looked at the case those people had the opportunity to engage so we looked at every zone visit to see what they did once they got near and their eyes looked at it.

[00:16:38]
All of those 800 visits were classified according to whether people saw the case and stopped or looked at and kept going. Whether they interacted meaning they use the touch screen did any of those events on the touch screen that I was interested in or whether they didn't and whether they were alone or in a group.

[00:16:59]
If they were in a group we looked at what they said to each other Did they have any substantive talk meaning did they say anything about any of the objects or any of the information on the rail or do they have no talk or only talk about things that were off topic and we looked at all sessions that were in English Spanish and Mandarin English and Spanish were the most common languages it was us institution but also because of the nature of the exhibit there were a lot of Mandarin conversations and the Spanish in Mandarin were all translated by native speakers and then those interactions were transcribed segmented and coded for conversation.

[00:17:38]
We looked at a couple different measures of success for this one was can we get more people interested remember not that many people were seeing this space seeing this particular exhibit initially and the good news is yes both of our redesigns has significantly more people who got there actually stop and and do something with the screen or with the case so that was fantastic the next thing we want to look at was could we change the way visitors talk to each other we get people we know that talk is a very important metric in a very very important part of learning in a museum environment and so we want to know if we could change what people said and yes we could but not the way we were hoping turns out the original design and the timelines has had about the same levels of conversation but the big questions designed actually had a little bit less people had fewer interpretive statements in that category which was surprising at 1st but then if you think about it that actually might aligned to that information got model that the design was based on people had got the question got the air.

[00:18:38]
So that question and then their curiosity was satisfied so they weren't necessarily driven to keep exploring. And finally as we're watching people use these 3 different designs we noticed there was actually a significant difference in the content of the access how they how they interacted with the with the.

[00:18:57]
Text of the displays where the original design people generally look through a few different questions on their on their visit they would read a couple of the different stories or at least skim them. In the big question design probably also not surprisingly they would click typically one maybe 2 stories but after that we move on and then the timelines Mostly they just played with the timeline and did not go actually deep into the content.

[00:19:24]
So each of these designs was successful and unsuccessful in different ways the original information on demand approach supported learning talk and content exploration but only for the very few people who actually saw it the asking provocative questions in the big questions designed did help draw people in but at the expense of some of the visitor visitor interaction and using the viscerally engaging timeline supported talk and got people to interact with it but at the expense of engagement with the content provided by the curators.

[00:19:57]
So I showed that these different designs had and presumably the underlying theories of curiosity supporting them they did impact visitor behavior which is not actually surprising what's important is that we were able to use all 3 of these different technologies were able to do something that the standard printed rails were wouldn't be able to do help invoke here yesterday get people to become interested in something that they might not have otherwise thought about and even though it brings up some tensions in terms of trying to figure out like well what's a good interaction what do we you know what's important for people to get out of this I can't help but be a little bit pleased that I think a few more people came away wondering about different.

[00:20:36]
And with that I'm going to switch to the next project so I was just telling about some technology some touch screens that were helping to get people to be curious about something that they weren't curious about yet they would just walked on by the 2nd project though is looking at people who are ready all already curious about what they're doing these are citizen science volunteers who are doing aquatic bio monitoring.

[00:21:01]
For water quality and these people are. Coming in to this this project came to the realization that the people who are coming in have a certain amount of curiosity about the animals living in the Stream but the resources they have access to are very limited in helping them deep in that curiosity to learn more.

[00:21:24]
So rather than not knowing what to ask these. Volunteers have questions but they just don't have ways to answer those questions because the tool that they have a very low ceiling to get to the end of what you can get and then you have to stop. So our goal was to design a tool that works for these volunteers still but also the ability to extend their curiosity and answer those questions that they have and bridge the gap between existing resources and I'll show you the differences in some of these resources that people have with the ultimate goal of bringing the volunteers closer to the science.

[00:22:02]
Ok so we're back here who cares about the stream. This is fortunately a lot of people do care about the stream it's not a major waterway it is not big enough to transport manufactured goods however a lot of people do care it's part of an ecosystem and quite a few parties want to know whether the water's clean ranging from the.

[00:22:27]
Regulatory agencies who want to make sure that nearby industries are doing what they're supposed to be doing following the rules to the couple who whose dog jumps into this water every day as they go on their evening walk the fisherman who brings this kids here to learn how to fish the fish who live here until they're fish by the fisherman or the dog and all of these parties want to make sure that this water is still clean so there are a few ways that these groups at least the human groups can measure the water quality the.

[00:22:56]
A common and sort of. Common measure is chemical monitoring by scooping up a sample and then measuring it for things like ph levels dissolved oxygen nitrates temperature things that give you a pretty standard straightforward measure of what's going on in this water and that's very useful but it's also something that is dramatically impacted by common events like a rain storm so it's a snapshot that doesn't give you a long term stable indication of what is actually going on in this water.

[00:23:31]
So we need something that is more reliable and stable and thank goodness we have a whole lot of life in the street hopefully that these are benthic Makar invertebrates these are the nymph in larval forms of many of the insects we know like dragon flies with flies may fly stone flies and they live in most of their lives in freshwater streams and they will stay in these streams through pretty much any weather event and be that stable indicator that we need measured by the quantity and the diversity of life in any given sample.

[00:24:07]
So benthic macro invertebrates benthic meaning sediment dwelling they lived typically at the bottom of these of these streams and macro invertebrates So they are big enough to see with the naked eye without magnification and they don't have a back room and there are a couple different constituencies that look for these in water systems.

[00:24:27]
One is the professional bio monitoring community these are formally trained scientists who work for places like the Environmental Protection Agency the Department of Natural Resources and their job is to go and sample these streams there have specific extensive training and usually some taxonomic certifications to do this work because the data have to be extremely accurate and reliable for regulation for enforcement and for policy decisions so to do the job they have to take their samples to their lab full of high magnification microscopes specialized resources and a lot of tools to make sure that they are being accurate in their identification.

[00:25:07]
Unfortunately there aren't enough of them we've had in recent decades what's called the taxonomic impediment which is that people have left this profession they are retiring commonly and they're not being replaced and then there's the training gaps the people are not being trained so there what it boils down to is there are a lot of streams of water that are not able to be sampled and samples that are not be able to be analyzed.

[00:25:32]
So to address that citizen science groups have been picking up some of this work and volunteer monitoring organisations are most. Recruit and train volunteers to come out and sample streams so these are retirees who suddenly have more time than they used to fishermen who have a vested interest for their hobby to have the stream speak clean environmentalists activists students who want to know what's going on in there in their environment and possibly take some action for it a lot of these organizations focus on environmental stewardship although some of the data are being used for a watch starting red flagging to bring the attention of some possible.

[00:26:13]
Things through to government agencies they're trained usually in a single day actually more usually a half a day training and they do all their identification stream side using simplified keys and usually just a hand lens or no indication. The process is pretty straightforward samples are collected there's a particular protocol depending on what kind of strain you're in about how to scoop up the sample and get the right sample and then you take it out dumping everything into a net and start sorting out the different things that are the same I don't if I what they are and then each organization has a different way of tallying the water quality ratings based on their own data use goals but basically it boils down to the greater diversity of life in your stream the healthier stream particularly of the sensitive to pollution taxes so this is great the problem is that the data that citizen scientists are are recording are at the order order level and for the datasets to be useful to the scientific population they need to be at the family or genus level.

[00:27:17]
It's been a while. The taxonomic hierarchy starts with Kingdom at the top and citizen science groups are right here at the order level. Where the professionals are a family in genus right below so if you are familiar with other amateur biologists or citizen science effort like bird watching this might seem a little unusual because we do bird watching at the species level an American Robin is a species of bird.

[00:27:43]
So why aren't we looking even even thinking about getting down to species on this well we. Got some data from our partners at the Maryland Department of Natural Resources about what it means for bed think about macro invertebrates at this level and they're all in the same kingdom the animal kingdom the 7 phyla 11 classes 35 orders 142 families 488 genera and way more of them 1200 species just in Maryland they don't even know how many species because the tools that it takes to identify all of these different species are beyond what they had in their strip survey so we're talking about a lot of for a riot a species.

[00:28:28]
So Ok citizen scientists are at this order level we only have to get him down one step to Family Studies have shown that family level data can achieve the same ratings in terms of accuracy of the overall quality of a stream as genus level data so we need to get citizen scientists down one level so see how difficult that is.

[00:28:50]
All right these 2 animals who look very different from each other and the different size different shape they don't seem to have much in common but these are actually in the same taxonomic order these are both may fly names. These 2 however that look a lot more similar to each other are in a different order these are this one is a stone fire and even though they are macro invertebrates they are sometimes only barely visible to the to the naked eye so they're actually looking at these very small.

[00:29:23]
Animals and you can see now how trying to train in just a few hours half a day or a day to people with very limited background in biology not just how to collect but also how to identify and how to calculate the water quality ratings that everything that they need to know to get started you can see how training even to those 35 orders would be very challenging and 142 families is basically out of the question.

[00:29:48]
So pretty much by necessity identification resources forces can find just are highly simplified this key was created by the University of Wisconsin extension program is a commonly used macro invertebrate key and it gives anough information that in theory most citizens find just could get to most of their specimens to the level required by their tally sheets The problem is not that tools for identification do not exist the problem is that is this far as this tool can take them they have a very low ceiling and once you get to the order or the sub order that is on the sheets you don't have much way to get beyond to develop your curiosity about what's next the next leap is to these professional level tools that have a very very high floor with complex terminology very arcane scientific language and the need to see things through high powered microscopes the scientists just don't have access to so of course there are some intermediary tools from various organizations but this project was really born from the realisation that these tools are not accessible to most organizations and most volunteers.

[00:31:00]
So to bring this back to curiosity the citizen scientists volunteers as a group are already curious they want to learn but our challenge is how to help them deep in that curiosity give them something that still maintains that low barrier to entry but allows for growth and deeper engagement than they have with the current tools and to do this we needed a team much bigger than ourselves we had.

[00:31:22]
Participatory design process we had just citizen science training organizations. Partner organizations we had designers h.c.i. researchers learning researchers and we got everybody together to try to figure out how to make a tool that really would work across these different disciplines. I'm not going to go into the full 3 year participatory design process I'm just going to pull out a few of the features that we found as we went through this design were volunteers wanted more than the current tools to give them and I'll talk about how we address it so starting with that question what are these features even look like if you're looking at something as small as a dime and you're trying to see what it's claw looks like maybe you can kind of see what the call looks like but can you really see what's going on there what are you can see maybe that it has the thing that your guide said it needs to have but really what it looks like and what it does is remains a mystery.

[00:32:21]
For the for the different order questions these creatures that look very similar are different for some reason what is it what's actually meaningful about those differences and why are they in these different groups and the ones that are in the same order that looks very different from each other what do they have in common that puts them together taxonomically.

[00:32:42]
So starting with the the actual size question given the real size of these specimens even if the volunteers may be able to correctly identify them based on what they can see that might be curious to really look closer and in this regard the difference between volunteer tools and professional tools in magnification could not possibly be bigger professionals have these high powered scopes they have a lab full of equipment and they have a combination of experience seeing a lot of different organisms and access to resources that point out very specifically what they should be looking for and where they should be looking for it the volunteers just can't access or use.

[00:33:20]
Volunteers on the other hand have a little hand. And sometimes they have a flashlight and they have these line drawings that may or may not be annotated I like this one in particular because it reminds me of the give preschool kids like how are these 2 pictures different so you can see so we can figure out what's club and what slender.

[00:33:40]
Now we can't solve this we can't buy citizen science groups high powered microscopes to take down to the stream where they do their identification where they couldn't even plug them in so we had to use a technology solution there's a little bit different we used your pixel imagery so this rig is a good rig that holds a standard camera and then takes systematically takes photos at increasingly deeper zoom levels and then stacks all those images and takes them in a whole grade and stitches them together to create a full picture.

[00:34:13]
So then we have a picture of this specimen and we can take the same specimen from different angles so we can see all the diagnostic characters but more importantly is that once you have these high resolution images you can start to zoom way way in and really get a sense of what you're looking at and what it means to have these gills so we took these images and had our entomology team annotate them so if you're looking for gills on a stone fly and you don't remember what gills look like on a stone fly you can see what they look like what they should be the definition and show you even an illustration in some cases this is what you should be looking for to help kind of build that visual fluency and get people to know what they're looking at and a deeper and more meaningful way.

[00:34:59]
To the 2nd question about what differentiates these animals we know there are some common lookalikes there are. Just different orders that do look like each other and one of the things that novices who are learning this fall back on is how it's going back and things like size and color that are not necessarily meaningful if they're sorry not necessarily not at all meaningful in identification because they vary so much by genus by region if it was a warm spring or a cool spring if that was and which time of year it was of spring or fall you have a lot of variation in size doesn't really matter that much.

[00:35:37]
And then moreover the rules that they have the line drawings will only have a few different specific cases. So we decided to call out and define those key visible diagnostic characters the ones that are in the panel on the right have the key things at multiple taxonomic level so order level on top and then family at the bottom that show you what to look for if you're trying to find this order what are all the things that are common to this order and then you can see them on all of the different animals which is good for some people if you're trying to confirm this thing in my hand is this it may fly and you see it has something that looks like abdominal girls it looks something like.

[00:36:13]
This and then you can look and see Ok well it looks like this one of these might be there at the right gills but if you have something in your hand and you have no idea what it is you need a little bit more help so we created a version of the key like I showed you earlier but integrating our imagery to give a little bit more information so on this one this is a series of choices so you start at the top does it have jointed legs and either no it doesn't or yes it has Ted It has 8 or it has 6 and then you just follow the correct path according to what you're seeing on the specimen in front of you but sometimes and this is what happens when novices who are not really well versed in the vocabulary try to use these keys they get to something they're like I don't know what this means and that's the case with like piercing.

[00:36:58]
Like mouth parts which are the 2nd choice if you say that has 6 legs and I'm going to warn you even I will say that this one's a little bit gross So if you don't know a piercing or needle like mouth parts are you can see the high resolution zoomed in this is what these look like and then a definition to show you so you can guide for some of them we have also pointing out where on the creature you should be looking.

[00:37:23]
It's going if you looked away. And then finally for the ones that have the same order but look different from each other as most of us in science trainings are set up they teach categories are not related to each other anyway but just like these are mayflies these are stone flies and they're given a set of inclusion criteria I call this the buckets of bugs model like what are the things that go in the May 5 bucket of things that have 3 tails things that have a single claw and the thing is that things that have abdominal gills so that works fine to a point but then in the field say in the stream it could get a little bit more complicated because there are exceptions to these rules there are some mayflies like the one in the corner here that have 2 tails there are some a flies that have no tails because they've had a rough go of it and got a little bit too close to a predator so there are problems it's hard if you don't have a perfect specimen Also if you have variations across regions you are not able to tell for sure and be confident that what you have in your hand is following the rules to put it in the right bucket.

[00:38:27]
So we solve this in a couple different ways with a couple different features one is by. Incorporating a digital version of the kind of the gold standard for teaching and Tamala the students which is a teaching collection that is preserved and accurately identified that everyone all in in a particular taxonomy is lined up and organized and you can see the variation across an order and these specimens in our collection were selected by the entomology team because they are the types of families that you are most likely to see in eastern North America which is where we focused geographically but also because they do show the breath of the kinds of shapes and sizes and colors that you might see so that you can start to build up this idea of what what could be the same and what could be different than if you get to that point where you are trying to look for tails and you know that you don't have any tails we have the expanded character list which then goes into not just the big easy to see features but what are all of the specific traits that every.

[00:39:33]
Animal in this group has in common at the order family and sometimes genus level. And then finally you know you're looking for abdominal gills you think that this has them but you're not really sure what it looks like on this particular creature because there's actually a lot of variation even in individual characters so we added this gallery view that shows you what the specific traits look like why all animals that have that trait.

[00:40:00]
So all of these features together respond to what we know about where volunteers need support to get started and where they get stuck once they get past the level of information that they have and available in their current tools we know that no web site is going to let volunteers bypass the years of training that it takes to to obtain fluency and become a professional scientist however we know also that the they are not and a limited by the tools that they have so they can at least start to explore what goes beyond the initial the initial levels so if they want to learn more if they want to be curious about what makes these different what makes size and color not important they can have the resources to address that and in doing this we found that this tool is also available to professional communities as well which is how we work in the ways we know it was a marker of success so we had.

[00:40:59]
We know that most professionals don't need the order level key that I showed a minute ago however we found that professionals do find useful visible images at the expense of characterless the glossary so all of these features that we put in to help novices also are helping professionals for example the site is being used to help.

[00:41:18]
People who are training professionals who are training to take the notoriously difficult techs anomic family level taxonomic identification quiz society a freshwater science certification test using this to study with this tool. So with that are going to wrap up I talked about 2 projects. Both with a different slant than a lot of tech projects a lot of times when we design technologies we think about designing them to help people do something they already do but do it better make people's lives easier so if you want to lose weight you can get various technologies to help track your movements track your eating count up your calories for you make recommendations that make your life easier rather than trying to write everything down a notebook.

[00:42:01]
The technology that I work on designing an informal learning is to try to make people's lives richer how can we. How can we help them expand what they're already thinking about and that is actually the point of lifelong and informal learning so I'd ask any of you who are designing technologies on your own when you thinking about what you're making are you making something that is helping people to move beyond where they already were using the technology to do something that you couldn't do without it and then how are you building something that can inspire and expand curiosity with that I want to say thank you to the amazing project teams that worked on this and if you are still curious either you have an information gap or something that you didn't get the answer to yet or if you which are driven to learn more and interact with any of the content both of these projects are still available online.

[00:42:55]
Thank you thank you thank. You. I think I have time for questions. So. Yeah that's the fun part so I'm doing a lot of this work on the macro invertebrates project was done in partnership with members of a Georgia adopt a stream there is a really strong bio monitoring community here so we're trying to see what the connections are with fact and then also one of the pieces of this project around the partners on the macro invertebrates project was the Carnegie Museum of Natural his.

[00:43:38]
With the idea that eventually this could expand into a museum exhibit we had students last summer who started working on kind of a pop up display if you're trying to get people curious about these bugs out you know in the science experiment. Like a Table about like a street festival or something what sort of a quick exhibit you can do and there are a lot of different possibilities so I have a couple students here we're starting to think about what can we do.

[00:44:03]
Taking the assets of the site the physical specimens that we do have and all the digital assets and try to figure out what can we do to put them together that could be actually used in Atlanta with the local streets. It's. Just. Actually. That's one of the things that we're interested in so there's a couple things there one is that it took a while to get the sites mobile.

[00:44:39]
Optimized for mobile which is something that is very important if you are stream side you don't want to take your laptop you maybe even don't want to take your i Pad So one of the things that we're also trying to develop sort of an extended project is to develop a mobile version that's actually hosted locally on a on a device so I have a demo version of it basically where you're pointing at the key characters at order level basically is as deep as will go with that one but is just trying to kind of show the imagery The 2nd thing that we're working on that we're designing a study right now is we finally got a quiz implemented because one of the pieces of feedback we kept getting from trainers is this is great people leave the the training is really excited about the project and about you know citizen science and about macro invertebrates and then they forget because you get trained ones you can go to a couple repeat trainings but not very often and you only sample a couple times a year if you start digging up sediment in your stream.

[00:45:38]
Too frequently then you're going to disrupt the life that that is living there they're going to go away so they needed some opportunities to practice so one of the things that we've just implemented is a practice quiz looking at the volunteer level of data so either sub order or order and we're hoping to see people start actually going and practicing that and we are seeing some just from the analytics data exactly how much they're using it it's still an open question.

[00:46:05]
It's. The. Right. So what kind of artifacts to you you have something specific in mind or. That's an interesting idea because there is that would be looking at the process for making something which is sort of a different. A different kind of learning and at its core it's you know learning how to do something versus learning about something so I have to think about it a little but I think there are definitely possibilities for.

[00:47:22]
Using it really using any sort of tangible objects that is augmented in some meaningful way to get people to think about in a way they didn't. There's a lot of work that is being done on maker spaces and how to get people to experiment it's not really so that I've engaged too much with but I think it's definitely possibilities.

[00:47:52]
But it seems like you're dealing with very different sort of. Different. People coming to the museum or. So. Really some understand. That you're not pleased with your use. Of. Novices in your expert. Obviously says science has a much higher knowledge of your average but not much as the expert expert so what does.

[00:48:28]
It feel like you were designing information systems. Are there lessons to take away with your design different. Interests but also. Previous content you know they were created for me to take the next time we're designing something that's about information animators. Care whatever it is for this. Question I think one of the biggest assets that all of these environments have in common is the social aspect of it and that there are people there and different different ages different experience levels different ideas and so that's the way to bring that out is design technologies that are not isolating people into a single space but making something that people that it can talk about and that's why I didn't talk very much about the conversation analysis in that particular space because there wasn't actually that much talk about.

[00:49:22]
These artifacts it was just because of the nature of the space with the music playing in the dark lights people didn't talk that much through the whole space and then particularly when they saw that there was a camera there they didn't talk very much at all so there wasn't a lot of conversation in that example but if you can build things that make people help people talk to each other and that's how you can start kind of start to deal with the different kinds of levels and you know rather than just keeping people in their own heads and just it's not a one on one with the machine it's the social interaction mediated by the rest of the space and the environment.

[00:50:03]
I think the basic takeaway is. It's curiosity like I showed you the 2 models that are that we use that made based on the context of the content that we had but it's the sort of the blow to any sort of project like this is assuming that you know what people are curious about the more open ended in the more opportunities you can give people to engage with whatever makes sense to them on their level with what they're bringing in and that's again part of the social aspect of the space in a socio cultural context people are coming in not just you know with a blank slate in a vacuum just like here to do whatever experiment you told them to do that you're paying them an Amazon card for you're trying to get them to have experience with whatever they're interested in so people come into the museum specifically with a lot of different ideas an interest in reasons for being there so if you try to tell them it is like this is the one thing you need to learn then it's going to kind of them might learn it they might be able to remember some facts about it if you if you quizzed him about it later but really if you can make something that lets people engage with it the way that is of interest to them it's going to have a more positive effect or longer lasting effect.

[00:51:27]
Thank you.