[00:00:06.00] Thank you all for coming and viewing the documentary I have to say To be honest [00:00:10.18] [00:00:10.18] this morning I got up I got out of bed I thought about the layout of the land I put [00:00:15.19] [00:00:15.19] on the appropriate shoes I figured out the best ducking over places [00:00:21.03] [00:00:21.03] you know all the things you do when you're moderating a panel of this kind you know [00:00:25.22] [00:00:25.22] I'm just getting severe storms to go it's hidden in Atlanta nowadays so aren't. [00:00:30.19] [00:00:32.11] Appropriate level you know thank you so my name is Heidi Turkana and [00:00:37.10] [00:00:37.10] I am a program director was seismic and I the pleasure of [00:00:42.08] [00:00:42.08] introducing our 4 panelists here today hopefully they all keep their opinions [00:00:48.05] [00:00:48.05] appropriate today so let me start I'm going to read from my cliff notes here. [00:00:53.06] [00:00:54.09] And I'll try and go in the right order so a keelboat well. [00:00:57.18] [00:00:59.07] He is the senior director of advocacy and advancement at fair test and [00:01:04.22] [00:01:04.22] and educational access consultant he is an educator speaker entrepreneur and [00:01:10.18] [00:01:10.18] testing expert kill has worked every level of the supplemental education industry and [00:01:18.06] [00:01:18.06] vising universities watching multiple companies developing dozens of [00:01:23.13] [00:01:23.13] admissions and test preparation programs training hundreds of instructors and [00:01:28.23] [00:01:28.23] helping thousands of students achieve success. [00:01:31.14] [00:01:32.16] Welcome Akil. [00:01:33.08] [00:01:35.21] Delay who works with me over at seismic is a senior research scientist and [00:01:41.08] [00:01:41.08] serves as the director of go stem and capacity programs [00:01:45.13] [00:01:45.13] her research interests include the impact of Siri stereotypes and social [00:01:50.04] [00:01:50.04] identity facts on the academic achievement and motivation of minority students. [00:01:54.17] [00:01:57.11] Place. [00:01:57.23] [00:01:59.07] She is also the cross cultural variation of motivational processes and Latino. [00:02:04.17] [00:02:04.17] Students academic achievement in stem education welcome delay. [00:02:09.02] [00:02:10.14] Jeff he made it braving the weather all the way from d.c. [00:02:15.20] [00:02:15.20] has written about higher education for 2 decades [00:02:18.23] [00:02:18.23] he is the author of 2 New York Times best sellers a contributor to The Atlantic and [00:02:23.18] [00:02:23.18] The Washington Post and a special advisor at Arizona State University he is [00:02:28.20] [00:02:28.20] also a visiting scholar at Georgia Tech Center for 21st century universities [00:02:34.06] [00:02:34.06] his next book called Who Gets in and why a year in psychology [00:02:39.04] [00:02:39.04] admissions will be published by Scott Simon and Schuster in September. [00:02:43.12] [00:02:45.01] And finally our own Rick Clark. [00:02:47.12] [00:02:48.14] Nice to be taken I think I've said many emails and this is my 1st time while. [00:02:52.22] [00:02:55.08] Rick is the director of undergraduate admissions here in Georgia Tech [00:02:58.20] [00:02:58.20] he lives and breathes the college admissions process every day and [00:03:03.05] [00:03:03.05] knows how stressful it can be for high schools high schoolers and parents [00:03:08.06] [00:03:08.06] alike Rick recently coauthored the truth about college admissions a family guide [00:03:13.13] [00:03:13.13] to getting and staying together and easy to follow comprehensive go to guide for [00:03:18.22] [00:03:18.22] families who are preparing for the college admission process so welcome Rick. [00:03:23.14] [00:03:25.07] Ok so today we have about 40 minutes to run some questions with you [00:03:31.23] [00:03:31.23] each of the pretty the panelists will have a different side to our opinion on [00:03:36.23] [00:03:36.23] the questions that we're going to ask and ask and then at the end we'll open up for [00:03:42.12] [00:03:42.12] some questions from the audience Ok so let's begin so I think the 1st [00:03:49.04] [00:03:49.04] thing that we should talk about a little bit is how what's the advantage [00:03:55.02] [00:03:55.02] of the test nowadays you know we hear about baseball in the news and [00:03:58.21] [00:03:58.21] how the asters got called out for using technology to figure out what the pitcher [00:04:03.18] [00:04:03.18] was saying to the catcher and it's in the news that they're trying to figure out [00:04:08.23] [00:04:08.23] is that it's allowed is it not alone are they just being assertive so [00:04:14.04] [00:04:14.04] I think similarly it's happening in the test so [00:04:19.01] [00:04:19.01] movements like to talk a little bit about that if it's happening in the test. [00:04:23.04] [00:04:27.16] Because of of a Ok I guess that's because of a story. [00:04:30.12] [00:04:33.19] I mean I'm going to test for 30 years I've. [00:04:36.07] [00:04:37.11] Fed my family on prepared kids for [00:04:39.23] [00:04:39.23] just heard one of 30 years now it's hard to believe in these tests when I know [00:04:44.22] [00:04:44.22] 6 weeks of a few $1000.00 and you can risk or significant. [00:04:47.23] [00:04:49.04] I mean that's that's really what it comes down to and [00:04:51.15] [00:04:51.15] that's that's legitimate that's the that's not cheating that's not any of [00:04:56.15] [00:04:56.15] the gamesmanship that can happen right that's just straight up I know [00:05:01.12] [00:05:01.12] the test better than the other person I'm going to tell you every nuance [00:05:06.18] [00:05:06.18] that will allow you to get the extra points right and I think that. [00:05:11.02] [00:05:12.11] There's a few different things that are there worth [00:05:15.09] [00:05:15.09] thinking about we're thinking about test one is most standardized [00:05:20.06] [00:05:20.06] testing in this country holds a position which is far too elevated. [00:05:24.11] [00:05:25.17] The belief that there is a significant difference between a 7 birdie and [00:05:29.16] [00:05:29.16] a $750.00 is false and a violation of the actual science of the exam so [00:05:35.10] [00:05:35.10] the way in which we use the test and [00:05:37.09] [00:05:37.09] the exultant nature that these tests have problematic so [00:05:42.05] [00:05:42.05] without even getting into is the content of the test good fair whatever [00:05:46.07] [00:05:46.07] the position it holds is problematic grade when you have 2 candidates and [00:05:52.01] [00:05:52.01] one potentially walked in woke up that morning walked into the test and [00:05:55.13] [00:05:55.13] seeing it for the 1st time and the other one is delivering a hand crafted artists [00:06:00.10] [00:06:00.10] and all coached you know far into an product [00:06:04.08] [00:06:04.08] that's the result of thousands of dollars and years of preparation and [00:06:09.06] [00:06:09.06] they're 10 points apart which College Board actually tells us is the same score. [00:06:13.01] [00:06:14.06] And then the kid with the 10 points higher from all his coaching [00:06:17.02] [00:06:17.02] his investment returns $10000.00 in merit and [00:06:20.15] [00:06:20.15] admissions and so the whole process around the testing is hugely problematic [00:06:26.09] [00:06:26.09] right before we even get into is the stuff on the tests what we care about. [00:06:31.12] [00:06:32.19] So I got issues with that. [00:06:34.08] [00:06:39.19] You know. [00:06:41.05] [00:06:41.05] I just want to clarify your question are you saying [00:06:43.18] [00:06:43.18] there is the question around who is it going for. [00:06:46.01] [00:06:49.05] Or is that I mean is that outward I think in fact you know he well I think [00:06:53.18] [00:06:53.18] I was looking at it as is an advantage isn't it damage to be able to pay for [00:06:58.02] [00:06:58.02] tutoring Ok to get a higher score or is it it's you know it's you don't pay for [00:07:05.17] [00:07:05.17] it either way so I mean the clear answer that is yes. [00:07:10.04] [00:07:11.09] Is it is absolutely an advantage to get a higher score probably period [00:07:17.10] [00:07:17.10] whether the score of the school is tests or optional or [00:07:21.09] [00:07:21.09] they are giving out merit aid as it pertains to testing or [00:07:26.04] [00:07:26.04] there are a holistic process that uses scores as one of many factors. [00:07:29.20] [00:07:30.21] You know here in the south you don't have to look very far. [00:07:33.19] [00:07:34.20] Actually you could look within our own state. [00:07:36.13] [00:07:37.16] For instance the Zell Miller scholarship requires not just at 37 in high school but [00:07:43.05] [00:07:43.05] in 1200 on a single administration of the test so [00:07:47.16] [00:07:47.16] while some colleges may super score the Zell Miller scholarship is linked to [00:07:52.14] [00:07:52.14] one setting so then you get into your own stuff and talk about in the documentary [00:07:57.06] [00:07:57.06] you talk about college who are trying to just write fill their coffers. [00:08:00.15] [00:08:01.20] Or choice you know where you can hold back scores and [00:08:05.01] [00:08:05.01] then only in Curie the ones you want people to see at all over and [00:08:09.03] [00:08:09.03] a number of our surrounding states that have declining populations they are giving [00:08:12.19] [00:08:12.19] out merit aid when there's literally a $20000.00 difference over 4 [00:08:17.16] [00:08:17.16] years in one point on the a.c.c. so there's no doubt that you know. [00:08:22.21] [00:08:23.22] $5000.00 investment. [00:08:25.09] [00:08:26.16] Out of $20000.00 return is is wise if you can afford to do [00:08:32.15] [00:08:32.15] not say say it's right it's just that is the current state of affairs. [00:08:37.10] [00:08:39.06] When I used to think you'd talk depends on where we're talking [00:08:42.17] [00:08:42.17] right because you know we're here to protect I think among. [00:08:48.01] [00:08:48.01] The highly selective institutions this is the conversation but [00:08:52.12] [00:08:52.12] I think that for an increasing number of institutions and [00:08:57.04] [00:08:57.04] an increasing number of students it doesn't matter as much anymore. [00:09:00.11] [00:09:01.12] You know every it seems almost it seems almost every day you know this better but [00:09:06.14] [00:09:06.14] that someone's going to stop I mean the movement is just growing growing [00:09:11.15] [00:09:11.15] you know with kind of had a a backwards move this week [00:09:14.18] [00:09:14.18] with the University of California system deciding not to. [00:09:18.02] [00:09:19.12] Or recommending not to go test optional as we some people thought they might [00:09:23.14] [00:09:23.14] which I think would have really put out a huge [00:09:27.05] [00:09:27.05] nail in the coffin of the city even as he took over the College Board. [00:09:31.17] [00:09:32.18] But I think that among highly selective colleges this [00:09:35.20] [00:09:35.20] continues to be a conversation although even when you ask people at highly [00:09:40.19] [00:09:40.19] selective colleges a couple of years ago I was at a a luncheon and [00:09:44.03] [00:09:44.03] the president Princeton was there and I asked why they still use the s a t. [00:09:49.13] [00:09:50.22] And he basically talked about it as a holistic admissions right as a piece of [00:09:55.13] [00:09:55.13] many things they work out and I said well if it's only one piece of many things you [00:10:00.06] [00:10:00.06] can easily drop it and he said no we don't know we and I think [00:10:05.13] [00:10:05.13] it's a largely I think for most of these selective colleges that out of fear and [00:10:10.11] [00:10:10.11] that's why I think that if you know when the University of Chicago in test optional [00:10:14.10] [00:10:14.10] your speech about this country always been the odd ball out there right but [00:10:17.20] [00:10:17.20] if University of California test optional Harvard or Princeton or [00:10:21.01] [00:10:21.01] want somebody else it's everybody's waiting for somebody else to do it [00:10:26.02] [00:10:26.02] because they're fearful of what will be wast among their own loans [00:10:31.05] [00:10:31.05] among your students among counselors whatever it might be if they were the 1st. [00:10:35.23] [00:10:37.20] And just to be back a little bit more. [00:10:40.06] [00:10:41.14] I kill and bring we're saying [00:10:44.02] [00:10:44.02] about the last 10 I mean we are using the Task thing is we're thinking. [00:10:48.03] [00:10:48.03] Giving us an indication of certain up to your skill at least we hope so [00:10:51.07] [00:10:51.07] and that's what they sell you right. [00:10:52.23] [00:10:54.02] But we know it gives an unfair advantages to those who may be able to pay for [00:10:58.16] [00:10:58.16] preparation that all the students don't have the ability to afford at the same [00:11:03.20] [00:11:03.20] time there's questions about who I stand the test itself even without that [00:11:08.15] [00:11:08.15] preparation is not already advantage Gene certain students by the way [00:11:13.17] [00:11:13.17] that it is constructed but a way that it is a structure and by the situations [00:11:17.20] [00:11:17.20] that are put in those questions that are supposed to be venturing so [00:11:22.02] [00:11:22.02] numerical ability Sumburgh all ability so the task culture around [00:11:27.02] [00:11:27.02] the test creates a problem already it's a problem that already is said [00:11:32.06] [00:11:32.06] that it's inherent to the task so that in itself it is something that. [00:11:37.16] [00:11:38.17] It is not just as part of the problem that it is already in contests and [00:11:44.19] [00:11:44.19] what the S.A.T.'s trying to do or what it says is trying to do. [00:11:47.15] [00:11:49.02] Everything For one thing if we're paying attention we all fall into great [00:11:53.02] [00:11:53.02] the use of words like attitude and [00:11:54.15] [00:11:54.15] things like that I think is part of the huge problem with testing. [00:11:57.22] [00:11:59.06] Testing has always been free throw contest it's essentially what it is [00:12:03.16] [00:12:03.16] it is one skill in the bigger picture and if they just came out and [00:12:06.22] [00:12:06.22] say hey we're testing these things in this way and [00:12:10.19] [00:12:10.19] that's all of this I would have absolutely no problem with. [00:12:12.21] [00:12:14.01] It because it's a free throw contest great if you choose to choose [00:12:17.11] [00:12:17.11] your basketball team entirely based on a free throw contests and [00:12:21.08] [00:12:21.08] that's how you want to do it probably be your picks then Ok right but [00:12:25.20] [00:12:25.20] whether the least be obvious that you're making a very limited [00:12:30.09] [00:12:30.09] determination around a very particular set of skills I'm a really good [00:12:35.07] [00:12:35.07] test because I was an asshole as a kid really what it was I want to teach the do [00:12:40.02] [00:12:40.02] you have your Ok can I see your homework my response you're not blind. [00:12:43.09] [00:12:44.16] It's an absolutely correct answer absolutely socially acceptable but [00:12:50.07] [00:12:50.07] it makes me really good at test because of the the so [00:12:53.18] [00:12:53.18] weird nit picky miss that that requires it and that's what we're [00:12:59.05] [00:12:59.05] testing for then say so and I think that's the whole problem is [00:13:04.16] [00:13:04.16] the language around aptitude ability college readiness [00:13:09.06] [00:13:09.06] career ready this like I challenge all y'all to go take the S.A.T.'s and [00:13:14.14] [00:13:14.14] say that should like if you don't have a number maybe I'll give up my career. [00:13:18.03] [00:13:19.19] No one would do that. [00:13:21.14] [00:13:21.14] But we're putting kids through this every day with all of those sort of [00:13:25.12] [00:13:25.12] judgements behind this one particular thing which is a huge problem. [00:13:30.03] [00:13:31.14] I think is a huge problem too because the s.a.t.. [00:13:35.19] [00:13:37.05] Does not only task on native. [00:13:39.02] [00:13:41.01] When they have a certain level of clarity on what kind of athletes those are and so [00:13:45.00] [00:13:45.00] research has said was a lot about your working memory which is your ability to [00:13:49.01] [00:13:49.01] process keep things in mind Canet all the information to new information make sense [00:13:54.00] [00:13:54.00] of the test the information the pieces that are in that question yes the s.a.t. [00:13:59.16] [00:13:59.16] it's somehow an indicator of how good you are in doing that but [00:14:03.06] [00:14:03.06] it also missioners your ability to stay cool under high situations of straw hats [00:14:08.10] [00:14:08.10] because we know it has on site it reduces your ass in the scores we know it measures [00:14:13.04] [00:14:13.04] your confidence and hope good you're going to do it that test which may not be [00:14:17.11] [00:14:17.11] an indication of how good you actually are you may not beat as good as math but [00:14:21.17] [00:14:21.17] if you feel very confident in your saw there you're good at math that actually [00:14:25.20] [00:14:25.20] helps you help onerous eighty's course we also know that if you're concerned about [00:14:30.02] [00:14:30.02] just doing well that's as you don't want to be should he also is going to find [00:14:33.23] [00:14:33.23] negatively you're as if he scores so this is a lot of not kind it [00:14:37.21] [00:14:37.21] is not an intellectual process and aspects that are hard or [00:14:43.16] [00:14:43.16] what the S.A.T.'s match errors that it is our kids suddenly [00:14:48.05] [00:14:48.05] compounds on the narrative of who do we really think we should [00:14:53.10] [00:14:53.10] be getting at universities who is that we are actually targeting and he should be [00:14:57.09] [00:14:57.09] getting at different institutions I want to go back to the feeder argument effect [00:15:01.18] [00:15:01.18] really are just one of the fears I think is also of all around the rents. [00:15:04.14] [00:15:05.19] And you know if you can if you go test optional and you don't get with that [00:15:10.19] [00:15:10.19] 75 percent of the students right 75 percent of your students submit [00:15:16.03] [00:15:16.03] easier as the Peace Corps you get Dean in the rankings because they still take. [00:15:21.17] [00:15:21.17] He scores into consideration and so for example I was talking to a. [00:15:25.19] [00:15:26.23] Mission the other day that their institution dropped in the rankings. [00:15:30.17] [00:15:32.06] And one of the reasons they think is because they went below that threshold now [00:15:36.15] [00:15:36.15] their S.A.T.'s course actually we're not among the students who submitted [00:15:41.12] [00:15:41.12] them because the idea to believe is that you assume to get better as he [00:15:46.03] [00:15:46.03] scores are going to submit them and so what I was asked Enos Dean is late [00:15:51.12] [00:15:51.12] because they were trying to figure out what could we do things to encourage more [00:15:55.02] [00:15:55.02] students to submit them but if we did that and got over the threshold were all or [00:16:00.07] [00:16:00.07] if he scores actually go down so they're Which way to balance those 2 things out in [00:16:05.00] [00:16:05.00] this goes back to the institutional priorities and the fear right so [00:16:09.22] [00:16:09.22] this is person who doesn't really believe in the test answers to a president [00:16:15.04] [00:16:15.04] in a war who cares about in this case the rains in this case they're dropping [00:16:20.07] [00:16:20.07] the rankings this goes back to we were fearful of what might happen [00:16:25.10] [00:16:25.10] we're always trying to balance those priorities that other people mostly orbs [00:16:32.09] [00:16:32.09] presidents a long long acres are trying to set for us. [00:16:37.06] [00:16:39.22] When I was watching the documentary In preparation I kind of through this circle [00:16:43.14] [00:16:43.14] which was. [00:16:44.07] [00:16:45.09] Rankings you know in a way we use to test scores and [00:16:51.05] [00:16:51.05] comes back around to it mission requirements which comes back to k. [00:16:55.02] [00:16:55.02] 12 focus in education and in the middle just watching it all and [00:16:59.12] [00:16:59.12] taking up all the money is college board as you see fit. [00:17:02.06] [00:17:03.08] You know and it is highly highly disturbing that that's the case and [00:17:07.16] [00:17:07.16] I think particularly when one of the things that was said among many in that [00:17:11.00] [00:17:11.00] documentary on that's just so [00:17:12.23] [00:17:12.23] disconcerting is that and then we seeing it happening right now because of [00:17:17.03] [00:17:17.03] the power of lobbying that both of those organizations out. [00:17:21.13] [00:17:21.13] That we are taking up a lot of kids time [00:17:24.15] [00:17:24.15] you know in preparation during the school day instructional time and [00:17:28.23] [00:17:28.23] as someone who's got 2 kids in public schools I mean not just that makes me [00:17:33.02] [00:17:33.02] extremely sad because again it has 0 real value on old Smith. [00:17:38.02] [00:17:39.04] In a lack the ability or skills that are going to matter at all in life. [00:17:42.23] [00:17:44.11] I mean when you think about it if the s.a.t. as let's pretend 10 percent [00:17:49.06] [00:17:49.06] as easy to the predictive validity of who succeeds in college. [00:17:54.06] [00:17:54.06] Think about the time effort and energy and money that's being contributed [00:17:59.15] [00:17:59.15] to that 10 percent and most of situations if not tempers a lot of [00:18:04.16] [00:18:04.16] places I see it's 2 percent just think about the time and societal impact [00:18:10.10] [00:18:10.10] we're back to 2 percent Amazon was looking at the z.t. discourse of his eighty's [00:18:14.18] [00:18:14.18] course to place their 2nd headquarters if you cut of the film bond ratings for [00:18:19.12] [00:18:19.12] colleges are based on as a team scores because that makes sense. [00:18:23.09] [00:18:24.12] Or so so like so what I support just optional is because this thing is has [00:18:29.17] [00:18:29.17] has you know is that it's off the rails it is way off the rails and the only way [00:18:36.17] [00:18:36.17] to kind of pull it back is to just stop using it to give it has been literally for [00:18:41.03] [00:18:41.03] college admissions it's so far off the rails you just gotta play back. [00:18:44.12] [00:18:47.13] At the bonus I want because. [00:18:48.21] [00:18:50.10] I'm so I guess what I think what I'm understanding is that [00:18:55.11] [00:18:55.11] all of you have very strong opinions against why not to use the text so [00:19:01.06] [00:19:01.06] what needs to happen in order for [00:19:03.18] [00:19:03.18] that to be told like is it going to be an equity thing is it going to be [00:19:09.06] [00:19:09.06] a Finally everybody says Mark we're done we're not spending this money anymore or [00:19:12.18] [00:19:12.18] what where do you think what do we need to do to basically get it to change. [00:19:19.07] [00:19:22.15] I went off script sorry that's funny I mean I think what I said I mean I think it [00:19:26.15] [00:19:26.15] really will take on highly selective college that people actually care [00:19:32.00] [00:19:32.00] about not the 1st one cause are not come to that other other if you're going to [00:19:36.18] [00:19:36.18] other people in the business you're right I think it will take an id be a stamp or [00:19:41.04] [00:19:41.04] I think it would have to have would have been the University of California largely [00:19:44.01] [00:19:44.01] because of their numbers and [00:19:45.23] [00:19:45.23] the power of California students that's what I think people. [00:19:49.11] [00:19:50.17] Might believe is if you have one of the big. [00:19:54.08] [00:19:54.08] I ve already present you can you read I think within a couple of years [00:19:58.13] [00:19:58.13] everybody all right here agree I also think that [00:20:02.21] [00:20:02.21] everyone in this room can contribute to the groundswell of this is terrible. [00:20:06.09] [00:20:07.16] The p.s.a.t. this just last list in October but they [00:20:12.11] [00:20:12.11] actually literally messed up the scoring according to their own formulas and [00:20:16.10] [00:20:16.10] disadvantage tons of kids and it went over like a blip. [00:20:20.21] [00:20:21.22] I think that this has to stop it's just insane and [00:20:25.12] [00:20:25.12] even if you think once you get in the room and take the test [00:20:29.15] [00:20:29.15] it does show something everything outside of that room is just insane. [00:20:33.21] [00:20:34.23] And so it's going to require politicians having the nerve to stop signing [00:20:41.16] [00:20:41.16] contracts with a college board it's going to require some schools having the nerve [00:20:46.04] [00:20:46.04] to stop requiring it of everybody you know I think once that starts to happen [00:20:50.08] [00:20:50.08] maybe you will get enough of a groundswell to start looking for you know to start of [00:20:55.03] [00:20:55.03] knowledge that it's giving a very little but but causing a lot of damage. [00:20:59.11] [00:21:00.20] And I think best work like a public campaign would help I mean [00:21:04.17] [00:21:04.17] you know with the I think it was pew or Gallup a couple of years ago asked. [00:21:08.10] [00:21:09.23] Right around the time of the Harvard emissions case coming out yesterday you [00:21:13.16] [00:21:13.16] know they do all the American public what do you think should be in college [00:21:17.15] [00:21:17.15] admissions an integrated test scores were well above everything else because but but [00:21:23.09] [00:21:23.09] they were among a number of other things on that survey and I think part of this is [00:21:27.09] [00:21:27.09] we haven't been able to grapple with what it means to do an engine was left at [00:21:31.16] [00:21:31.16] the university and what should count I mean if you asked everybody in this room [00:21:36.08] [00:21:36.08] what you take to get into Georgia Tech there is what 30 people in this room 40 [00:21:40.07] [00:21:40.07] people you will have 40 different answers about what should count more than other [00:21:45.00] [00:21:45.00] things I'm assuming grease and quote and of course it will probably be [00:21:49.21] [00:21:49.21] up there among others but you know you'll have people who say you know legacies that [00:21:54.08] [00:21:54.08] very few of you say in-state residency you want to use a test or so [00:21:59.12] [00:21:59.12] I think part of the issue is that as as a public we've been unable to grapple with [00:22:04.20] [00:22:04.20] what should it take to get into a selector college as a result I think that [00:22:10.04] [00:22:10.04] selective colleges ask for everything because they want to kind of cover all [00:22:13.16] [00:22:13.16] the bases and then that leads to lawmakers and [00:22:17.20] [00:22:17.20] the media and others then saying that test scores are important as a result. [00:22:22.12] [00:22:24.03] And even if they stand that. [00:22:26.09] [00:22:27.21] And we say tell for sorry or after all because other folks are abuse and [00:22:32.01] [00:22:32.01] we are in the us a measure of how successful universities are hostages are I [00:22:37.00] [00:22:37.00] still feel there in these things I can be done a lot of lies in order to at least [00:22:41.17] [00:22:41.17] understand what those discourse are doing at your institution or within your [00:22:46.15] [00:22:46.15] institution you can do studies of the predictability of this discourse for [00:22:50.19] [00:22:50.19] your own institution and I like that with the bottle you so [00:22:53.15] [00:22:53.15] who do you want to be as an institution so it's the s.a.t. [00:22:58.01] [00:22:58.01] predicting equality that females and males are going to be successful at our [00:23:02.17] [00:23:02.17] institution then what can we do to compensate or whether we do to correct [00:23:07.21] [00:23:07.21] that issue is the S.A.T.'s corporate in taking their success or [00:23:12.06] [00:23:12.06] more on the represented made or it isn't easy to share and if we value that [00:23:16.11] [00:23:16.11] we value they were severely situation how do we sell Carette for that so [00:23:20.11] [00:23:20.11] I think to have an understanding of even if we are use in the tax what the test [00:23:26.01] [00:23:26.01] is doing for all situations or not for our institutions against our own values [00:23:31.01] [00:23:31.01] is an important thing that all universities can do in order to [00:23:34.15] [00:23:34.15] at least make my full the see shit but do you think universities are you know [00:23:39.14] [00:23:39.14] who believe you into your room or what does the critter. [00:23:42.12] [00:23:43.20] So I mean if you look at students if you just take 100 point bands you know 1500 or [00:23:49.11] [00:23:49.11] higher those students do have the highest g.p.a. and [00:23:53.14] [00:23:53.14] the fastest graduation rates of other students at Georgia Tech now the in is [00:23:58.21] [00:23:58.21] a little bit lower than it is from 1400 to the 14 but [00:24:03.11] [00:24:03.11] if you just want to looks only test for bands and [00:24:06.12] [00:24:06.12] 3536 o'clock on a city that is true however they also have the highest [00:24:11.20] [00:24:11.20] grades that there's going to have all that other stuff you have reports of course so [00:24:15.15] [00:24:15.15] how can you just aggregate from the year again and again you can't completely and [00:24:19.16] [00:24:19.16] I mean there was a time when we were able to and were using more of a formula to [00:24:24.20] [00:24:24.20] make admissions decisions and over time the predictive value of the task [00:24:30.04] [00:24:30.04] continued to develop in terms of correlations to g.p.a. And so [00:24:35.19] [00:24:35.19] what has evolved here in Georgia Tech is we've become far less reliant on the tests [00:24:40.23] [00:24:40.23] over the years you know not abusing tasks and not using code scores and in fact [00:24:46.05] [00:24:46.05] in some ways going reading blind to test scores through a lot of our process and [00:24:50.08] [00:24:50.08] yet our test scores averages keep going up because for a variety of reasons one [00:24:56.07] [00:24:56.07] as you have a more affluent group of students I mean there's no doubt there's [00:25:01.07] [00:25:01.07] a correlation with socioeconomic status I mean we've had you know. [00:25:05.02] [00:25:06.09] A disinvestment in public education across the country and [00:25:09.23] [00:25:09.23] therefore less state appropriations schools look to you out of state students [00:25:14.06] [00:25:14.06] to you know other revenue sources that leads to a higher S.E.'s band of students [00:25:19.23] [00:25:19.23] that leads to higher test scores which does have an impact on g.p.a. [00:25:25.10] [00:25:25.10] retention rates for your graduation rates that all looks good. [00:25:28.01] [00:25:29.02] But in doing so you then begin to question sort of what what [00:25:34.09] [00:25:34.09] is our mission as a public institution we are less representative of our state and [00:25:39.06] [00:25:39.06] we were a decade ago when our test scores were about much lower. [00:25:43.08] [00:25:43.08] True and yet. [00:25:45.07] [00:25:46.10] We were you know we were far more diverse than all measures so [00:25:51.04] [00:25:51.04] I think that's the other real concern that we have also think there's a thing that [00:25:54.23] [00:25:54.23] overlooked in a lot of conversation is conversations around a test score and [00:25:59.12] [00:25:59.12] blood tests and the like that don't actually design or [00:26:02.05] [00:26:02.05] gate that isn't used revised 15 times since it is that there is no test right [00:26:07.05] [00:26:07.05] there's today's test grades looking at test scores and bans of 100 points [00:26:12.10] [00:26:12.10] bans of 100 points could mean you got a different test for if you get 53 [00:26:17.00] [00:26:17.00] of 58 math questions right and your test score depending on which test for [00:26:21.18] [00:26:21.18] you have my fluctuate 90 points and that's of the test scores that I've been [00:26:26.18] [00:26:26.18] able to actually look at they don't release them so you're talking about these [00:26:31.05] [00:26:31.05] you know it so the belief that it's this magical objective thing what if you took [00:26:36.01] [00:26:36.01] the bad test form day and your practice had all been how to do about it your [00:26:40.13] [00:26:40.13] question that's my normal careful pace and you get the bad test where your scores [00:26:44.15] [00:26:44.15] are 90 points lower than you expected and you can't afford to do it again. [00:26:47.22] [00:26:49.10] Right so than now it's luck of the draw so [00:26:53.04] [00:26:53.04] I think the objective ness of these tests are somewhat in question not to say [00:26:58.08] [00:26:58.08] because of how they're going to partially because of how they're created but [00:27:01.10] [00:27:01.10] also in part just because of the just of taking the test so [00:27:06.06] [00:27:06.06] the minute distinctions being made around the test always fascinate me and [00:27:11.18] [00:27:11.18] a place like Georgia Tech I actually don't believe I can do the math in [00:27:14.11] [00:27:14.11] Georgia Tech I don't get every map question right on the essay t.v. [00:27:18.01] [00:27:18.01] g match g r e all of these tasks but I think [00:27:21.12] [00:27:21.12] you could put $20000.00 in from me right now cash money I can't do a derivative. [00:27:26.12] [00:27:26.12] I remember that these things exist but I don't and I would like to believe [00:27:31.05] [00:27:31.05] Georgia Tech is a place where you probably should be able to do in your riveted right [00:27:35.01] [00:27:35.01] so what's the value of being able to do 10th grade math and [00:27:40.17] [00:27:40.17] George tack and why is that the bar that's a really big test with 300 [00:27:46.13] [00:27:46.13] math with real tradition problems and if you can do it in 12 minutes you're good. [00:27:51.01] [00:27:52.07] Like elves are really the people I want to Georgia Tech so maybe this test [00:27:57.12] [00:27:57.12] the fitting and so weird bar part like clearly if you know if you make anything [00:28:03.04] [00:28:03.04] numeric you will parse some people and not others but are we parsing for [00:28:07.09] [00:28:07.09] the right things are resorting for the right things and is it a useful metric [00:28:11.20] [00:28:11.20] they are all questions that need to be asked that haven't been asked enough. [00:28:15.02] [00:28:16.20] To pay so let me ask the next question. [00:28:18.11] [00:28:19.12] Why is there a disconnect between the questions that are asking and [00:28:23.17] [00:28:23.17] the high school curriculum. [00:28:24.18] [00:28:28.09] Because well I think in part because we've been sold that [00:28:33.21] [00:28:33.21] right let me put it this way for the audience because I love examples [00:28:38.06] [00:28:38.06] do you know the prime number you know all smart people. [00:28:40.08] [00:28:41.14] You know all those and every other prime number is a so [00:28:46.01] [00:28:46.01] what's the least prime number to shout out when you know it. [00:28:48.06] [00:28:51.02] But we got Ok so let's just park that right here and [00:28:54.06] [00:28:54.06] we've got one person say we want all the whole lot of quiet. [00:28:57.11] [00:28:58.20] And several people saying to there it lies the problem with the s.a.t. [00:29:03.11] [00:29:04.12] if I ask a roomful of grown folks some 6th grade math. [00:29:08.12] [00:29:09.18] We've got problems. [00:29:10.20] [00:29:12.14] With the right I think that's that's really important to think about like and [00:29:15.22] [00:29:15.22] all of you who didn't answer were thinking 3 and wondering about other things right [00:29:20.05] [00:29:20.05] like and the next I'm going to put a clock on you. [00:29:23.05] [00:29:24.11] And see if you can do it under time pressure I'm going to revise the question [00:29:27.15] [00:29:27.15] ask you for the least positive integer that has exactly 2 factors and [00:29:32.10] [00:29:32.10] see if you can parse that. [00:29:33.12] [00:29:34.22] Right so I can say that that's a line with a school curriculum but. [00:29:38.09] [00:29:41.08] Right there in lies the problem with a lot of the x. [00:29:43.10] [00:29:43.10] is that isn't a worthwhile question asked in a worthwhile way about things we [00:29:47.22] [00:29:47.22] actually care about I can't tell you the last time I've looked at a prime number or [00:29:51.12] [00:29:51.12] thought about what prime numbers work and I wasn't on a test setting. [00:29:54.20] [00:29:57.10] But I'd love to hear from Georgia Tech people like tell me use primes to [00:30:00.14] [00:30:00.14] having real computer programming all the time I will feel much better about having [00:30:04.15] [00:30:04.15] to learn prime numbers but I have yet to find a use for it. [00:30:08.04] [00:30:10.19] Yeah I mean I think that it's in a lot of ways it's sort of a Kaiser so [00:30:14.20] [00:30:14.20] say a fact I mean you know the usual suspects I mean white. [00:30:19.04] [00:30:20.22] People don't even know anymore why it's being used and I think that So [00:30:25.04] [00:30:25.04] the question of how or what would it take to move away from this I [00:30:29.16] [00:30:29.16] totally agree with what Jeff said in terms of and I think he truly that [00:30:35.00] [00:30:35.00] somehow is still looked at the the guidepost of higher education so [00:30:38.04] [00:30:38.04] there's so much of it and not only it's insane for [00:30:41.19] [00:30:41.19] what it really looks like for most of America but I think that. [00:30:46.19] [00:30:47.20] The other piece is politicians and fours of regions and [00:30:52.16] [00:30:52.16] systems because especially you know I would think this is true nationally but [00:30:56.21] [00:30:56.21] I can speak certainly to the south there is no doubt that the belief is that these [00:31:02.04] [00:31:02.04] or measures of intelligence and that it's an easy you know quantitative [00:31:07.13] [00:31:07.13] board support of that is going to you know indicating a wow for access to college and [00:31:13.22] [00:31:13.22] things like that and if we're really going to make any type of traction because [00:31:18.06] [00:31:18.06] the truth is really I mean we've got to go a bit of a conversation I think about this [00:31:21.12] [00:31:21.12] before like California could have gone test for optional in the South would have [00:31:25.23] [00:31:25.23] just turned the other way if you know that's what we've always done. [00:31:30.03] [00:31:31.16] And so I think what you've got to do is find a way to. [00:31:34.11] [00:31:34.11] Counter which is so difficult because of the money involved the incredible [00:31:39.00] [00:31:39.00] machines that all our a.c.c. and college board and gets you regents and [00:31:44.09] [00:31:44.09] others to hold to convince them that a single setting on the s.h.c. [00:31:49.18] [00:31:49.18] do for 1200 is going to be your measure for all of this morning. [00:31:53.19] [00:31:54.22] Or in the state of Alabama or other places around or you know around here where [00:32:00.05] [00:32:00.05] a lot of this is plains to dollars and that goes back to rankings no one else. [00:32:04.16] [00:32:07.14] In terms of in reporting this book over the last year where I got to the inside of [00:32:12.14] [00:32:12.14] 3 different admissions offices a big public a big private in the city [00:32:18.06] [00:32:18.06] and in a small private. [00:32:20.01] [00:32:21.04] I one of the things I have come to realize in talking to parents not that they [00:32:25.09] [00:32:25.09] necessarily think that or in lawmakers not the necessarily always think that [00:32:28.23] [00:32:28.23] the s.a.t. in these He's a test of the knowledge and they see it as a a it's [00:32:34.09] [00:32:34.09] a wave of helping institutions differentiate between high schools [00:32:40.11] [00:32:40.11] because I think the big concern of parents I see particularly [00:32:45.18] [00:32:45.18] on pretty privileged backgrounds is that if I want a house [00:32:49.15] [00:32:49.15] in this great city in this neighborhood said my kid to this group School or [00:32:54.04] [00:32:54.04] I mean a lot of monies that went into a private school and [00:32:56.18] [00:32:56.18] I want to make sure that they get a fair shot a fair shot at college and they think [00:33:01.15] [00:33:01.15] that the S.A.T.'s score is the way it is that balance wheel which is often years. [00:33:07.06] [00:33:08.21] At least in the 1st Shaking of plots. [00:33:11.07] [00:33:12.19] Between schools so I think that's one another and [00:33:16.02] [00:33:16.02] again I think it goes back to this really discussion of what do we mean by merit [00:33:21.04] [00:33:21.04] like what do we mean what do we mean by this you know especially [00:33:25.06] [00:33:25.06] in places like this where there's tons of obligations in the seats are scarce which [00:33:29.11] [00:33:29.11] is a whole other argument we should be having Why are the seats so scarce. [00:33:32.22] [00:33:34.20] What are the measure who gets him to do. [00:33:37.01] [00:33:38.21] That beat beyond the fact that his parents may have the concern they should know that [00:33:43.03] [00:33:43.03] S.A.T.'s already biased possibly coworkers Hi folks you [00:33:48.11] [00:33:48.11] know students will have an amoeba grounds males and white students I mean there's [00:33:53.16] [00:33:53.16] a lot of evidence that if the sh t. has some predictive ability it is mostly for [00:33:58.06] [00:33:58.06] the students because it has question is that you know and actually. [00:34:03.03] [00:34:05.03] Mills students and white male students had to do better on the easy questions [00:34:10.21] [00:34:10.21] of the on the up their safety compared to African American students who tend to do [00:34:15.02] [00:34:15.02] better on the worst questions of the S.A.T.'s and that is what the result for [00:34:18.20] [00:34:18.20] that is because even what is considered in the s.a.t. normal Larry that people [00:34:25.05] [00:34:25.05] in families was speak around the kitchen table it is a stream we buy as the word [00:34:30.11] [00:34:30.11] professional families families from hives as economic backgrounds they can use their [00:34:35.14] [00:34:35.14] right ascension to feel studies as your occupation that can make up and [00:34:40.23] [00:34:40.23] ologies about particular forms of sports and understand all of which himself [00:34:45.16] [00:34:45.16] apparently a lot of there is a t.v. So there is already certain biases seem bold [00:34:50.15] [00:34:50.15] included in the tabs that Don which is the predictability for all other students so. [00:34:56.11] [00:34:57.18] We should not worry. [00:34:58.16] [00:35:00.09] That if it I think that let me just tell you that with a couple of [00:35:04.12] [00:35:04.12] things one gets it he has straighter be as they remove the book now they don't [00:35:07.21] [00:35:07.21] test it in the same way right but the problem is. [00:35:11.01] [00:35:12.21] How do you remove some of the societal biases about what's good and [00:35:18.12] [00:35:18.12] how we measure what's good what people should know what they should [00:35:22.09] [00:35:22.09] 1st as if he had Latin on it and they claimed it was an intelligence test [00:35:26.01] [00:35:26.01] because some of us are born or in Latin and others aren't really that [00:35:29.23] [00:35:29.23] there's the genetic collapse in this that you know you can either have or [00:35:34.05] [00:35:34.05] don't great like they only think about it how much math were you born knowing. [00:35:39.18] [00:35:40.18] To say that this is an aptitude test Well [00:35:43.22] [00:35:43.22] it's a test that you can because anything can be tested it can be learned and [00:35:47.16] [00:35:47.16] there's almost nothing that you can't test that couldn't be learned so the weight [00:35:52.02] [00:35:52.02] put on some of these things problematic the design of some of these things and [00:35:56.01] [00:35:56.01] privileged families tend to have exposure which even beyond things [00:36:00.23] [00:36:00.23] like test prep is often overlooked if you get into a private school lot of private [00:36:05.01] [00:36:05.01] schools use the i c e which is the s a t 20 years ago it still has vocab on it so [00:36:10.14] [00:36:10.14] you took the i.o.c. to get a 5th grade maybe you took the S.A.T.'s to get [00:36:14.16] [00:36:14.16] into your 8th grade so now you have Pravda like maybe you didn't prep for [00:36:18.12] [00:36:18.12] kindergarten in New York City they have consultants charging [00:36:21.09] [00:36:21.09] hundreds of dollars thousands of dollars an hour to do kindergarten prep for [00:36:25.22] [00:36:25.22] the test to get into kindergarten so you can begin to as you do the 5th grade as [00:36:30.02] [00:36:30.02] the 8th grade as you do that's $87.00 but it's going to roll around those 80 [00:36:33.06] [00:36:33.06] this is old news these strategies are things you've done before the test [00:36:37.18] [00:36:37.18] is something very comfortable and known and even if you just took those tests [00:36:43.11] [00:36:43.11] you had a 6 test advantage when the kid who's never seen it before and then you [00:36:47.12] [00:36:47.12] start talking about things that are like built into the Test one of my favorite [00:36:51.03] [00:36:51.03] examples is the Wechsler test which is supposedly an intelligence test which is [00:36:54.20] [00:36:54.20] often used for gifted kids it wasn't until the eighty's that they revised it [00:36:59.17] [00:36:59.17] one of the best example I remember is that they were asked about the word Rumi [00:37:03.13] [00:37:03.13] which in the African-American community especially at the time that was [00:37:06.02] [00:37:06.02] a name how many poor black kids know what a ruby is as a gemstone versus a name and [00:37:11.23] [00:37:11.23] there was a huge fight with the author the test author [00:37:15.02] [00:37:15.02] to get that question about Rubys off of the test because of the biases and [00:37:19.07] [00:37:19.07] this is in the eighty's right and think about that like so they did fix a lot [00:37:24.06] [00:37:24.06] of these issues until the eighty's so what's the impact of the continued [00:37:29.07] [00:37:29.07] use of these things that sure even you're cleaning it up a little bit now. [00:37:32.23] [00:37:34.04] But there's a whole lot of stuff that still held over [00:37:37.22] [00:37:37.22] from when it was horrible now it's merely that. [00:37:40.12] [00:37:41.16] I mean there's a lot and so there's a lot of that that's ignored around what these [00:37:45.14] [00:37:45.14] tests are what they show what it means who really who are holding out as the standard [00:37:50.12] [00:37:50.12] of what's good and I think that's sort of the same thing with the hard right [00:37:53.17] [00:37:53.17] because if you start to search for Harvard so we're going to go that particular [00:37:57.15] [00:37:57.15] theory as the standard for what's good about influences the creation of. [00:38:01.17] [00:38:03.01] Even a well we whole society has a new mission of what it is and [00:38:07.05] [00:38:07.05] well considering college as if it creates processes that disadvantage also on [00:38:11.21] [00:38:11.21] the represent a minority status or a time threat which is [00:38:16.23] [00:38:16.23] when you have a member of a group that maybe even vote for an identity that they [00:38:21.19] [00:38:21.19] have let's say I'm a Latina woman and I'm going to take a math test I don't [00:38:26.16] [00:38:26.16] think there have many graceful things to be said about either woman in math or [00:38:31.19] [00:38:31.19] Latino thing Matt So if I was afraid that [00:38:36.17] [00:38:36.17] my test scores make them firm a negative stereotype about my group our likely [00:38:41.23] [00:38:41.23] to be under stress in that test and then scored lower than any other student and [00:38:46.14] [00:38:46.14] we know that when we do stereotype threat studies around the s.a.t. [00:38:50.14] [00:38:50.14] results are that the s.a.t. math actually. [00:38:54.15] [00:38:56.00] For females in print this their achievement for [00:39:00.10] [00:39:00.10] lay by 20 points and as a comparable for African-American and Latino students and [00:39:06.02] [00:39:06.02] underestimates their intellectual ability for about I think it was 40 points so [00:39:11.12] [00:39:11.12] just by having the students sitting there wondering what all others may [00:39:16.18] [00:39:16.18] think of my intellectual ability at this right moment when I'm up to take stats so [00:39:21.22] [00:39:21.22] even our own conceptions a while we concede are valuable as intellectual [00:39:26.21] [00:39:26.21] skills as merit you know that's the ways we see ourselves and [00:39:31.11] [00:39:31.11] the stresses that the systems like that pass on ourselves especially when [00:39:36.22] [00:39:36.22] it's really high test when we know that our future is to say that all that said. [00:39:41.18] [00:39:41.18] We have to sort we have to sort these up occasions in some way so I guess that's [00:39:46.15] [00:39:46.15] the question how many are studio version 41041000 rounds off 323200 or [00:39:53.16] [00:39:53.16] so how do you get from 412-3200 I mean that is really the question one by one. [00:39:59.09] [00:40:00.20] But how do you write how do you Ok so [00:40:02.22] [00:40:02.22] even if you're gathering that question if you took. [00:40:06.09] [00:40:07.12] Not the level you selected not the almost selected you know let's take the 3rd [00:40:12.12] [00:40:12.12] level down in the 3rd level of rejections do they succeed [00:40:17.01] [00:40:17.01] Absolutely I mean that's the problem we need a 97 percent graduation rate because [00:40:21.16] [00:40:21.16] if we took the kids that would lead us to a $94.00 life ends and [00:40:26.17] [00:40:26.17] that's what colleges are off the competing for right. [00:40:28.23] [00:40:30.00] Back to some of the information we've been looking at recently I mean our outlook and [00:40:34.06] [00:40:34.06] average even with 41000 is higher than our in the old [00:40:39.01] [00:40:39.01] group 10 years ago when it comes to just you know looking at test scores and back [00:40:44.01] [00:40:44.01] to that point of yes you can see I mean if you look at just again under point b. [00:40:48.04] [00:40:48.04] and with all the flaws we know that within it but just for a clean look at. [00:40:52.01] [00:40:53.02] You know scale. [00:40:54.06] [00:40:55.12] You go to the 1300s of $1390.00 which is was all below our average test for [00:41:02.14] [00:41:02.14] right now it's just something like a $1450.00 overall average but [00:41:06.23] [00:41:06.23] $1339.00 t. those kids still up for [00:41:09.12] [00:41:09.12] their 1st year here in Georgia are to 3.36 Well that's that's a higher g.p.a. [00:41:15.04] [00:41:15.04] than we've ever had in our in since these history until just 2 years ago but [00:41:20.03] [00:41:20.03] you're taking fewer of those kids that are 813 to 39 than we used to we're going to [00:41:24.11] [00:41:24.11] have fewer men around him I mean our outlook and pool continues because [00:41:29.19] [00:41:29.19] again because they look at those numbers and they say I'm not even going to apply [00:41:34.06] [00:41:34.06] I not there for actually and that's one of the things that concerns me the most is. [00:41:38.05] [00:41:39.13] Who doesn't apply I feel I feel very confident not perfect [00:41:44.02] [00:41:44.02] about the way we view those who do what concerns me or [00:41:47.17] [00:41:47.17] this seems to look at the middle 50 percent range smirk it's you know what [00:41:51.11] [00:41:51.11] average the definition of average is right and they were given bands and [00:41:56.23] [00:41:56.23] they say I don't have this I'm not going to apply so you know we've tried to pull [00:42:03.03] [00:42:03.03] all this stuff all former patients we've tried to you know deemphasize this work. [00:42:07.07] [00:42:07.07] Because that's the big concern that we have is they're not coming into our pool [00:42:10.16] [00:42:10.16] for us to then give them an advantage which is the you know the u.c.. [00:42:14.00] [00:42:15.07] Executive summary that just came out about the committee that's reviewed this you [00:42:18.21] [00:42:18.21] know history repeating itself has said that's what they're doing and u.c. [00:42:23.20] [00:42:23.20] system is you know if you haven't given them disadvantaged background [00:42:27.02] [00:42:27.02] you're scoring well or they're getting a minute at a higher rate that is true [00:42:30.10] [00:42:30.10] Georgia Tech as well however they have to apply they have to apply and [00:42:35.01] [00:42:35.01] that's what that's what that study did not as far as I've read overall So [00:42:39.22] [00:42:39.22] how do we get Georgia Tech and 5 of its peers [00:42:45.07] [00:42:45.07] to just stop reporting to us news and stop reporting test scores just asked [00:42:50.20] [00:42:50.20] to start there just stop telling people test scores and keep it internal right. [00:42:54.18] [00:42:56.23] Sure that sounds like that but I support and [00:43:00.13] [00:43:00.13] I think you know I think that's the kind of thing it will take a grid that. [00:43:04.21] [00:43:06.09] Now they can look at that and say I can't get it because you know [00:43:11.11] [00:43:11.11] you have Was it your question I did a keynote work in Illinois seat and [00:43:16.07] [00:43:16.07] right before I have got done it more questions president came out and [00:43:19.23] [00:43:19.23] said we look at scores in context I looked up their scores they like we know 12 [00:43:23.23] [00:43:23.23] school means something different 12 they had admitted 12 if you like 6 years right [00:43:27.21] [00:43:27.21] here like so we know what this means to kind of when you look at the numbers like [00:43:31.10] [00:43:31.10] their mitts are like 1400 not so it's one thing to say you know [00:43:36.22] [00:43:36.22] what the numbers mean frightful when if I'm an applicant and I see that 2 percent [00:43:42.10] [00:43:42.10] of students with my score got it in it's really hard to be that $70.00 back to what [00:43:47.05] [00:43:47.05] you were mentioning there about you know inequities you know whether gender [00:43:52.17] [00:43:52.17] you know sick anomic racial etc I mean let's be honest the kid for [00:43:57.18] [00:43:57.18] home you know Marietta is got a double legacy and [00:44:02.02] [00:44:02.02] that 1320 their peers like you're applying you know that falls below [00:44:06.21] [00:44:06.21] profile whereas the 1st generation doesn't have that kind of background [00:44:11.04] [00:44:11.04] they're looking at the numbers and saying I'm never going so [00:44:13.09] [00:44:13.09] what if you're just what if you just search by of lower numbers. [00:44:17.08] [00:44:17.08] Which we didn't Ok yeah we do mean and [00:44:19.10] [00:44:19.10] that doesn't have an impact on applications or by the Photoshop. [00:44:22.10] [00:44:23.23] App that we do that in our state because we have the you know we have the space and [00:44:30.05] [00:44:30.05] the money to back it up maybe I don't know if everybody knows about certified maybe [00:44:33.18] [00:44:33.18] you want to yes I'm going to read here's a way that we use test scores than [00:44:37.11] [00:44:37.11] actually we would even if we were to be a test optional school [00:44:40.20] [00:44:40.20] is that you can go in if you look at your instances over ours which was brought up [00:44:45.06] [00:44:45.06] before Georgia Tech one of those is enroll more well so we go and [00:44:49.01] [00:44:49.01] we say we're going to buy all women in Georgia who score x. [00:44:55.02] [00:44:55.02] to y. And you know this is true without institutional [00:44:59.18] [00:44:59.18] Brownian it's true with others we buy lower for women than we do from it partly [00:45:04.08] [00:45:04.08] because of the partly because it's due for our college board Evey's Yeah so [00:45:09.05] [00:45:09.05] we pay $41.00 so just another way the College Board makes money without a doubt. [00:45:12.13] [00:45:14.00] So that every single name sold every single time she was multiple schools also [00:45:18.18] [00:45:18.18] search Yes exactly so that is that is sort of you know one use and [00:45:24.08] [00:45:24.08] one tactic I guess for trying to build your applicant pool. [00:45:28.07] [00:45:29.09] But. [00:45:30.19] [00:45:30.19] And just another thing about the Cold War They will every time I write that they [00:45:35.01] [00:45:35.01] sell names they always call up and say the license things they don't like me to use [00:45:40.00] [00:45:40.00] the words they don't like any reporter use the word so obviously Holborn has a very [00:45:45.12] [00:45:45.12] large public affairs definitely boxer's my book you know return the medal. [00:45:51.15] [00:45:53.14] I don't know about that life in the Senate Ok so [00:45:56.09] [00:45:56.09] it sounds to me like money is a big part and [00:46:00.16] [00:46:00.16] the idea of exposure is a 2nd really think hard and things that we can do and [00:46:06.17] [00:46:06.17] then it sounds like it's just now and spilling people's knowledge of [00:46:10.23] [00:46:10.23] the test and how the application and admission price process works. [00:46:15.21] [00:46:17.05] Only last part is problematic in this idea of transparency. [00:46:21.01] [00:46:22.02] So I was there you were there to it [00:46:25.13] [00:46:25.13] this year when I moderated a panel on transparency. [00:46:29.03] [00:46:30.12] The piece for role in admissions across the country News in the way diversity [00:46:34.12] [00:46:34.12] blues and all these admissions and deans up there is that yes we [00:46:39.14] [00:46:39.14] want to become more transparent what was funny about that is that when I was doing [00:46:43.13] [00:46:43.13] this book 2 years ago and I approached a bunch of perch 24 universities [00:46:48.07] [00:46:48.07] mostly selected all sort of to sit inside their emissions process for [00:46:52.23] [00:46:52.23] the poor 21 said No including by the way one of the deans that was [00:46:57.21] [00:46:57.21] on that stage talking about transparency right and so part of this is they don't [00:47:02.02] [00:47:02.02] want transparency part of it is the contradictory nature of emissions right so [00:47:08.03] [00:47:08.03] it's hard to tell people what counts when you [00:47:13.01] [00:47:13.01] as an emissions Dean don't even know what counts or [00:47:16.17] [00:47:16.17] cross the border I mean that's the problem everything's in context and so it's really [00:47:21.03] [00:47:21.03] hard to say this counts for more than that except when and there's always [00:47:26.04] [00:47:26.04] exceptions there's always exceptions and it's amazing that you received. [00:47:30.08] [00:47:30.08] Fornia so I have at home the University of California. [00:47:34.01] [00:47:35.04] Emission standards from 150 were true or in their undergraduate catalog and [00:47:41.06] [00:47:41.06] I think we're less than a 1000 words about this long if you look at the University of [00:47:45.07] [00:47:45.07] California today their admissions standards as they put them on the web [00:47:49.23] [00:47:49.23] they go on for about 7 pages thousands of words and basically at the end it [00:47:55.06] [00:47:55.06] is we are in a holistic you know use holistic conditions and [00:48:00.18] [00:48:00.18] basically we decide what we decide at the end of the day it's and it is [00:48:06.04] [00:48:06.04] impossible to transfer your yeah I mean again it's about supply and [00:48:12.09] [00:48:12.09] demand I always say admissions is just it's very simple it is just 2 [00:48:16.20] [00:48:16.20] things supply and demand and institutional power that is college admission that's all [00:48:21.05] [00:48:21.05] the things that drive it and you know when I got here we were at mit we had like 9500 [00:48:26.09] [00:48:26.09] applications we were big 64 percent of the students that came you know those kids or [00:48:31.22] [00:48:31.22] kids anymore in 2003 you know they're running companies they're phenomenally [00:48:36.12] [00:48:36.12] successful in every single way it was a lot higher ses he was a lot lower [00:48:41.16] [00:48:41.16] I don't think we're fundamentally different in both ways now but [00:48:45.20] [00:48:45.20] we used to be able to go down to make him and go down to Albany and [00:48:50.01] [00:48:50.01] say you need 83781250 that was true and now you [00:48:55.14] [00:48:55.14] know 41000 applications is exactly what you said it's just like a jet I'm on trick [00:49:00.08] [00:49:00.08] Good luck you know I don't know it's going to happen because it's not that we deny [00:49:05.08] [00:49:05.08] 15 forty's every day people are literally in my office doing that right now and [00:49:10.13] [00:49:10.13] you know because back it happens to be from Silicon Valley and [00:49:15.00] [00:49:15.00] they want to see us and we had 110 applications from one public [00:49:20.04] [00:49:20.04] school in Silicon Valley more than we got from Brady high school which is [00:49:24.00] [00:49:24.00] one of the close of schools to us. [00:49:25.12] [00:49:26.13] $89.00 of the $1101.00 in c.s. we're not taking all those kids. [00:49:30.07] [00:49:30.07] So then they get denied by us and they get into Dartmouth and [00:49:32.21] [00:49:32.21] that makes 0 sense when you look at rankings or a minute rate or [00:49:36.07] [00:49:36.07] anything else that makes total sense when you look at institutional priorities [00:49:40.02] [00:49:40.02] we are full up on Silicon Valley gives perfect right and [00:49:44.17] [00:49:44.17] I think that I think it also reflects society great to be a college admissions [00:49:49.09] [00:49:49.09] test scores all of that when you look at the charts for income inequality [00:49:54.06] [00:49:54.06] right the wealth your portion of the you know there's a there's a growth [00:49:58.08] [00:49:58.08] among wealthy people in America there's a growth among poor people in America and [00:50:01.22] [00:50:01.22] the middle is getting squeaks right so no longer do you so you have the wealthier [00:50:06.17] [00:50:06.17] families who can afford all this stuff all the consultants all the Photoshop [00:50:12.02] [00:50:12.02] all the donations to help their application to all the schools and [00:50:17.05] [00:50:17.05] apply to a whole lot more schools than they ever did before won't work is going [00:50:21.05] [00:50:21.05] to do it so you have more students looking to go to college you have less ones or [00:50:26.06] [00:50:26.06] colleges more expensive to go to so [00:50:28.18] [00:50:28.18] more rich students are getting in there doing more things to compete [00:50:33.04] [00:50:33.04] to get him so you have all of these divergent things that he and [00:50:36.17] [00:50:36.17] when they went college were nice talk about tests are great inflation [00:50:40.04] [00:50:40.04] they don't talk about test inflation and they don't talk about that they've created [00:50:43.12] [00:50:43.12] the great inflation because who runs a peak which kept accounts for [00:50:47.21] [00:50:47.21] the majority of the waiting g.s.a. so they're creating great inflation and [00:50:53.10] [00:50:53.10] arguing over a guy grade inflation is terrible look at test scores [00:50:56.09] [00:50:56.09] well actually test score inflation is huge to the top level of test [00:51:01.02] [00:51:01.02] scores the volume of people scoring above what is it 1400 or [00:51:05.20] [00:51:05.20] so I think is a poor 100 percent what it was 10 years ago things that trouble him [00:51:10.14] [00:51:10.14] something crazy like that sadly like over 5 when you look at what [00:51:15.13] [00:51:15.13] does that test print that we only really look at freshman g.p.s. you're not looking [00:51:20.16] [00:51:20.16] at you're going to be you know running at top 50 company in 20 years you're not [00:51:25.09] [00:51:25.09] looking at Career Success you know being a freshman g.p.s. and there's a lot. [00:51:30.07] [00:51:30.07] Happens also in your freshman year that actually contributes to that freshman [00:51:34.17] [00:51:34.17] g.p.a. that you're a Taney so and that is also differential among groups and [00:51:40.10] [00:51:40.10] in the past at the university that you go to so [00:51:42.21] [00:51:42.21] that's a lot of it that is affecting that relationship but [00:51:47.13] [00:51:47.13] you're right I mean it makes perfect sense that a few years back even though the test [00:51:51.12] [00:51:51.12] scores are not so high that doesn't mean those individuals who are not very very [00:51:56.05] [00:51:56.05] very successful in their careers the connection between income and score test [00:52:01.20] [00:52:01.20] scores is critically important for some of the school because they they want what. [00:52:05.14] [00:52:06.23] They claim they need. [00:52:07.23] [00:52:09.01] And so this goes I mean they can find out well [00:52:11.16] [00:52:11.16] they can measure welcome other ways beyond test scores obviously but. [00:52:15.05] [00:52:16.09] You know the fact is that many of these highly selective schools are wind and [00:52:21.19] [00:52:21.19] they claim to be all need And the only way they think [00:52:26.05] [00:52:26.05] although they're sitting on billions and billions and billions of dollars [00:52:30.04] [00:52:30.04] the only way they think they can do that is that I have enough for new students. [00:52:34.22] [00:52:36.18] To pay for the lower income students they promised me $44.00 again I don't buy that [00:52:42.09] [00:52:42.09] argument because their city I'm going the dollars that they could afford this but [00:52:46.04] [00:52:46.04] it's one of the reasons why another reason I don't think they want to retire [00:52:50.05] [00:52:50.05] because there's no other way to sort of buying [00:52:53.06] [00:52:53.06] a home when they need a certain percentage of their clients to have that income. [00:52:57.15] [00:52:59.10] Ok so I want to be conscious of your time and. [00:53:02.07] [00:53:04.20] What we can now hear from her one question from the audience that's [00:53:08.23] [00:53:08.23] really gotten certified I know. [00:53:10.16] [00:53:15.17] It's. [00:53:16.05] [00:53:26.16] Pretty strongly. [00:53:27.11] [00:53:29.21] I. I think that. [00:53:30.23] [00:53:34.10] With the people that. [00:53:35.23] [00:53:37.04] Run the p.c. on the road as they are today there [00:53:40.12] [00:53:40.12] is a few 100 of us who are even if there were nothing. [00:53:44.19] [00:53:45.19] To do with that. [00:53:46.15] [00:53:47.23] And. [00:53:48.11] [00:53:49.19] If you think that's. [00:53:50.14] [00:53:53.21] Great for now I'll do that and that. [00:53:57.12] [00:53:58.13] Right and I rewrite the ground with the reviewers. [00:54:03.11] [00:54:05.23] And yes I read that. [00:54:08.00] [00:54:09.03] After not. [00:54:09.18] [00:54:11.14] That. [00:54:12.02] [00:54:14.05] There's a certain line with. [00:54:15.11] [00:54:16.20] That but I think that there are that are. [00:54:19.04] [00:54:20.13] Very well now that we're through the public this. [00:54:24.00] [00:54:25.09] And I hope the rest. [00:54:27.00] [00:54:28.13] Of the company to stand up for that. [00:54:32.03] [00:54:34.07] Downtrodden. [00:54:35.00] [00:54:36.13] And know that could let me let me let me point out what you're missing [00:54:41.01] [00:54:41.01] in the California thing didn't quantify any of that. [00:54:43.20] [00:54:45.02] The thing that often happens with better predictor they're often talking about [00:54:50.04] [00:54:50.04] g.p.a. alone being point 61 correlation to 1st your g.p.s. [00:54:55.03] [00:54:55.03] when you go up to adding test scores that you get 2.63 So [00:54:59.23] [00:54:59.23] again my question isn't does it do something is it worth it is it work [00:55:04.18] [00:55:04.18] the drama it's creating regularly what you're saying is that yes it does predict [00:55:09.02] [00:55:09.02] right but the question is so but so great it's right Also would you like me [00:55:14.06] [00:55:14.06] to choose college admission based on one or [00:55:19.02] [00:55:19.02] 2 Saturday mornings or based on 4 years of grade like it seems insane to say that [00:55:24.15] [00:55:24.15] that is more representative of what a college experience is the 4 years. [00:55:30.06] [00:55:30.06] Of classwork right so I think there's a lot of there's a lot of dancing [00:55:35.00] [00:55:35.00] that's done around testing and also the California one I'm still working my way [00:55:38.06] [00:55:38.06] through it but so far I've got 50 references to college work research and [00:55:42.19] [00:55:42.19] largely a lot of their stuff is based on college work research which I find [00:55:47.13] [00:55:47.13] really weird because no called research has ever said the tests aren't great [00:55:53.04] [00:55:53.04] despite independent research showing that you know this is where I do disagree I [00:55:57.05] [00:55:57.05] know a lot of college work researchers and I think that they're good people and [00:56:00.11] [00:56:00.11] I think that they are right and so I don't drink I don't think it's the framing [00:56:04.15] [00:56:04.15] I mean I do agree the board is a huge lot of your organization I do think that [00:56:08.16] [00:56:08.16] the research is is salt I think the numbers are right I think [00:56:13.16] [00:56:13.16] the framing is questionable just just like the book the measuring success that [00:56:18.17] [00:56:18.17] just came out there's a book that's out about how great inflation is terrible and [00:56:22.07] [00:56:22.07] when you look at the acknowledgements like the op or a College Board [00:56:26.12] [00:56:26.12] big knowledge with acknowledged college boards communication team. [00:56:30.02] [00:56:31.03] In the creation of an academic book this is the 1st time I have ever seen that [00:56:37.02] [00:56:37.02] and just like even if it's true why would you ever put that in print. [00:56:41.10] [00:56:42.19] Like your book includes test scores are better than g.p.a. [00:56:45.22] [00:56:45.22] and I could call it work thanks for the help. [00:56:48.06] [00:56:49.06] Really bizarre and suspect so I say to you to look at the numbers and [00:56:54.16] [00:56:54.16] see if the numbers actually mean something to you right now and [00:56:58.06] [00:56:58.06] I think that's one of the big issues. [00:56:59.16] [00:57:00.23] That I wondered if this time a. [00:57:03.12] [00:57:04.16] Lot. [00:57:05.23] [00:57:05.23] I think the things that were. [00:57:07.14] [00:57:12.00] Not kept you. [00:57:12.13] [00:57:13.18] Sit there and that's the politicians get their own way and it isn't it's. [00:57:20.12] [00:57:21.22] Really. [00:57:22.10] [00:57:25.11] There's nobody 3 who. [00:57:26.17] [00:57:27.18] Member there is that you more $1000000000.00. [00:57:30.22] [00:57:32.11] Then I believe we're. [00:57:33.08] [00:57:34.14] Glad to welcome you with that [00:57:37.19] [00:57:37.19] from the profit that makes their job so much more valuable. [00:57:42.10] [00:57:43.16] Then why there is no objective measure in it and nothing higher than. [00:57:48.10] [00:57:49.10] The object that. [00:57:50.07] [00:57:53.14] Was regularly. [00:57:54.09] [00:57:55.23] At this in. [00:57:56.12] [00:57:58.19] All of them are worth it and I think you talk about yeah. [00:58:04.14] [00:58:05.17] That's worth it every so great that your circuit is it is so great and [00:58:11.20] [00:58:11.20] there's it there are so many stakeholders in a mission everybody has that [00:58:17.03] [00:58:17.03] has a view on everybody and everybody's biased on this because everybody has [00:58:21.09] [00:58:21.09] something of stick right us News and World Report rankings the institutions [00:58:26.16] [00:58:26.16] the independent college counselors you know so right you have test prep and [00:58:30.15] [00:58:30.15] you also have independent college counselors those sensually charge also [00:58:33.17] [00:58:33.17] a lot of money mostly to write essays that are we know what those really provided [00:58:39.12] [00:58:39.12] by your own admission your age so there's a you can go through every single piece of [00:58:44.01] [00:58:44.01] the visions process I think this gets back to my earlier point is what really really. [00:58:48.17] [00:58:50.22] And that's the piece we haven't been able to grapple with and [00:58:54.19] [00:58:54.19] if I'm going to answer your question and by the way thank you for being that was of [00:58:58.14] [00:58:58.14] Akkad I love it makes me excited so somebody has to stand up for the test. [00:59:02.17] [00:59:04.06] This is how I would answer that question is not that it doesn't predict anything [00:59:08.08] [00:59:08.08] we know we predict things we know it would be a certain functionality you're working [00:59:11.22] [00:59:11.22] memory if it helps you interface student who can connect previous information [00:59:16.09] [00:59:16.09] you print this things that are more efficient all that are emotional it that [00:59:21.07] [00:59:21.07] the problem is that it seems to produce a better person than others [00:59:24.13] [00:59:24.13] the predictability is different like you want your tests 1st of all to have [00:59:29.05] [00:59:29.05] all the items in sight trying to measure the same thing we know that's not the had [00:59:33.18] [00:59:33.18] has not been the case in previous S.A.T.'s they have issues with items that they just [00:59:37.23] [00:59:37.23] don't relate to the other ones that have biases that have to do with compass and [00:59:42.10] [00:59:42.10] language and culture and then you want them to predict what they say [00:59:46.16] [00:59:46.16] they want to prevent which is that you know that freshman g.p.a. [00:59:50.22] [00:59:50.22] Well it does that but he does it better for some students that others he tends to [00:59:55.10] [00:59:55.10] underestimate particularly female grades now if you use more like the essay [01:00:00.08] [01:00:00.08] questions to that that I don't even melt into the better of those questions so [01:00:04.09] [01:00:04.09] you may depend on actually that you know that the area you know your task all your [01:00:09.12] [01:00:09.12] city it because it actually predicts differently depending on the university [01:00:14.18] [01:00:14.18] that you are attending you know that if you go to a school that has like more [01:00:19.17] [01:00:19.17] of your city or has laid requirements on a more highly selective that you're [01:00:23.18] [01:00:23.18] predictability actually the changes if you're a female versus if you're a medal [01:00:29.15] [01:00:29.15] and then it also prevents that differently depending on the groups of students that [01:00:34.10] [01:00:34.10] are applying so it's not that it doesn't prevent things it's just that they [01:00:38.10] [01:00:38.10] have believe it to do with fair job of predicting that success [01:00:42.20] [01:00:42.20] it is very different depending on the university does looking at it [01:00:46.13] [01:00:46.13] depending on the outcome that you're looking at and the. [01:00:48.20] [01:00:48.20] Then you are the person that is applying so it is so [01:00:51.13] [01:00:51.13] complex that is nobody asked objective s. we think we are so [01:00:56.03] [01:00:56.03] the problem is to consider at the level of t.v. there is this all the things that we [01:01:00.11] [01:01:00.11] claim are more subjective Well the cause may not that that be that objective for [01:01:05.14] [01:01:05.14] everybody so then there is the question how we should be we [01:01:10.10] [01:01:10.10] just get company in comparison to those other aspects don't. [01:01:15.06] [01:01:17.13] You think you can start I walk into a room and I'll say v.s.e. I would guess a cheap [01:01:21.22] [01:01:21.22] and emissions people don't know what it is people don't understand just [01:01:25.20] [01:01:25.20] big think it's objective they think it's pure it is that is that is just not true. [01:01:29.08] [01:01:30.13] And you know it might. [01:01:31.13] [01:01:32.14] Work I thought still a bit more but. [01:01:35.07] [01:01:36.18] Just a good question and your patients are great groups. [01:01:40.10] [01:01:41.19] In the documentary you are talking about as if they were your cooling [01:01:47.17] [01:01:47.17] the requirement to go so it was bad of course to do something quickly and [01:01:53.12] [01:01:53.12] this someone some of that you were forced to to find these kind of patterns [01:01:58.12] [01:01:58.12] to recognize the patterns now you know I did the additions were old and [01:02:03.21] [01:02:03.21] for the record I don't. [01:02:06.06] [01:02:07.16] But if you did I was worried my employer and you say you were things really working [01:02:12.22] [01:02:12.22] for your employees and are hiring someone to do that actually. [01:02:16.02] [01:02:17.07] Mutilated you something that is often pattern recognition like that's actually [01:02:21.07] [01:02:21.07] something really important that you know there's some kind of will so [01:02:24.23] [01:02:24.23] the question should be is there any research this shows that you really [01:02:30.04] [01:02:30.04] are down the line I do agree with you know you if you have these hires or worse. [01:02:35.06] [01:02:36.10] You know the kind of recognition being tested [01:02:39.03] [01:02:39.03] you know people are learning you know. [01:02:40.05] [01:02:41.09] Measuring how fast you can do something are they are the more successful [01:02:44.19] [01:02:44.19] when you're going to review on just the 1st year really it's the. [01:02:48.21] [01:02:48.21] I'm. [01:02:50.02] [01:02:50.02] Curious you know there's research out there [01:02:52.22] [01:02:52.22] that I would answer that sort of 2 ways one is. [01:02:56.19] [01:02:58.07] Once you go beyond 1st year especially 1st year but [01:03:01.02] [01:03:01.02] one special what your college is as a black man I'm like [01:03:04.19] [01:03:04.19] all of this stuff starts to mean nothing right because we know. [01:03:07.04] [01:03:08.11] Resumes that are identified as African-American get hired Let's hope like [01:03:12.13] [01:03:12.13] all of that just becomes money here the further and further you go to 20 years ago [01:03:17.07] [01:03:17.07] so Elizabeth there how that would be really difficult to do for [01:03:22.12] [01:03:22.12] what the other part about that I think is important to recognize is the the nature [01:03:27.23] [01:03:27.23] of the test the thing that those folks were really proposing is a lot of sort of [01:03:33.21] [01:03:33.21] what I'm saying your terms of our belief in its objective and right and all of the. [01:03:39.23] [01:03:41.11] Resulting determinations that are made based on the belief of the tests being [01:03:46.12] [01:03:46.12] objective are false bill that is that that test prep in preparing you for [01:03:51.19] [01:03:51.19] those patterns I can give this person a 200 point advantage that this person [01:03:57.05] [01:03:57.05] doesn't tap So then what's the value of that test what is it actually showing [01:04:02.18] [01:04:02.18] right and I think it's yes and you risk or on a one to 5 scale I have very little [01:04:07.12] [01:04:07.12] issues with it because there are very few false positives you can't accidentally get [01:04:12.07] [01:04:12.07] a 700 right you can accidentally get a 400 right like [01:04:17.09] [01:04:17.09] I think false negatives happen all the time False positives are fairly rare but [01:04:22.05] [01:04:22.05] the problem is the way we use this you put a 700 from somebody and [01:04:26.10] [01:04:26.10] a 720 and they're going significant difference and it's actually [01:04:30.04] [01:04:30.04] immaterial difference it's actually been in what colors were called the standard [01:04:34.02] [01:04:34.02] error measure they expected they give you a test today and a test tomorrow [01:04:38.03] [01:04:38.03] your score will vary by up to 40 points that's what the college board admits to. [01:04:42.08] [01:04:43.10] So this is so with the track track like sure of what you know I'll come [01:04:48.18] [01:04:48.18] through very we'll be here yes the test has the test has so [01:04:52.20] [01:04:52.20] little to do with that with a little bit and you. [01:04:55.15] [01:04:56.22] Look at it we could there be some study done where [01:05:00.19] [01:05:00.19] you know some natural experiment where you can do the test or [01:05:04.08] [01:05:04.08] the suspects if there's any correlation I think I'm just [01:05:08.13] [01:05:08.13] interested in proving that question I would doubt I don't know I doubt it. [01:05:12.03] [01:05:13.16] I think the studies and you know I usually have to make a disclaimer and [01:05:17.06] [01:05:17.06] speak going to visual psychologist like I speak of pretty much kind of within my [01:05:21.17] [01:05:21.17] discipline and for the things that I know from my discipline. [01:05:24.08] [01:05:25.13] I think a lot of the studies you know when I was working about work in [01:05:28.11] [01:05:28.11] a couple of our working memory which is your ability to keep things in mind is [01:05:31.12] [01:05:31.12] a resource of your memories how fast you process and [01:05:33.21] [01:05:33.21] keep things in and how fast you can connect that with previous experience and [01:05:37.09] [01:05:37.09] things that you're seeing that is something that is highly correlated not [01:05:40.06] [01:05:40.06] only for their safety but for other canid ability tests so [01:05:44.07] [01:05:44.07] when you look at around Conny ability that's a lot of them and [01:05:48.20] [01:05:48.20] you see that what they have in common is a certain sense that if you have a very [01:05:55.11] [01:05:55.11] Cup acidy heart large capacity you work in memory that you do welcome the ability [01:06:00.15] [01:06:00.15] this that said there was an interesting study by Barry and colleagues and [01:06:06.07] [01:06:06.07] I think it may be 201120122 me that they actually looked [01:06:12.00] [01:06:12.00] at the validity across different kind of ability tests and they looked at [01:06:16.13] [01:06:16.13] a locational test like there is a team but they looked at test using the military and [01:06:20.13] [01:06:20.13] they use also looked at committee tests used in employment and [01:06:25.04] [01:06:25.04] they tried to look you know what was common and [01:06:28.19] [01:06:28.19] if they're worse any differences between groups they still found the same [01:06:32.06] [01:06:32.06] differences in groups so. [01:06:34.00] [01:06:36.01] There really is the 1st They're also getting the view that I think that they [01:06:41.09] [01:06:41.09] offered to 1st off there is a very and it was a very interesting study because they [01:06:45.06] [01:06:45.06] I think they looked in that when they looked at it the city had raised and [01:06:48.09] [01:06:48.09] these 2 found you don't see certain level of that most of their colleagues [01:06:53.08] [01:06:53.08] test is just that they believe to be it is more closely related to one group [01:06:57.18] [01:06:57.18] to European American 2nd that kind of the United States means that they just do tend [01:07:02.03] [01:07:02.03] to do the task pretty it's better their performance it tends to predict wars [01:07:07.00] [01:07:07.00] the performance of all the groups they try to google try to use their city for [01:07:11.20] [01:07:11.20] test scores for careers and they found it was crap or they got rid of it. [01:07:14.08] [01:07:15.10] So there is us I think there is actually a study but Google deftly and [01:07:19.23] [01:07:19.23] then also credits. [01:07:21.13] [01:07:22.23] Years ago in terms of using data analytics to [01:07:25.21] [01:07:25.21] Irene they want to find out after 5 years of Credit Suisse what. [01:07:29.22] [01:07:31.03] Will predicted success rate would predict if people who stayed [01:07:33.20] [01:07:34.21] in a found of their very measures. [01:07:36.23] [01:07:38.09] Most of which they would not share it was in my last book this is how one knows. [01:07:41.18] [01:07:43.04] But one of the things that they noticed were. [01:07:45.10] [01:07:47.16] Actually when they were really surprised major didn't matter [01:07:50.20] [01:07:50.20] because they were already mostly from. [01:07:53.02] [01:07:54.06] Commentator. [01:07:54.22] [01:07:55.22] Major. [01:07:56.10] [01:07:57.17] Math majors and science majors. [01:07:59.07] [01:08:00.10] They found that a lot of decisions in memory so [01:08:04.01] [01:08:04.01] if you were elected governor president didn't matter as much as being elected [01:08:09.00] [01:08:09.00] he kept it right because the idea of team captain would probably get because of your [01:08:14.22] [01:08:14.22] Because of what you did on the team anyway so [01:08:17.23] [01:08:17.23] what they did was they started what Christmas did after this analysis [01:08:23.00] [01:08:23.00] when they started to recruit at a much larger set of institutions but [01:08:28.03] [01:08:28.03] then what happened was is hiring at some point becomes personal and [01:08:33.03] [01:08:33.03] you would have you know somebody interviewing a kid from x. [01:08:37.14] [01:08:37.14] State University for game and they wouldn't believe that this is good for [01:08:42.18] [01:08:42.18] next a university could really do the job and so the people would be hired So [01:08:46.21] [01:08:46.21] what they found was that they actually expanded their recruitment wall but [01:08:51.04] [01:08:51.04] they're hiring boarded up in the same institutions so [01:08:55.10] [01:08:55.10] it's like the top up also in the Paul Cup It's out there is a section on it was [01:09:01.01] [01:09:01.01] hiring practices and look cross a lot of the banks like to Ireland cross kids [01:09:07.06] [01:09:07.06] because of whatever which also struck me as the same sort of thing just the biases [01:09:12.02] [01:09:12.02] of who we like them all yes Reese. [01:09:15.12] [01:09:15.12] He's quoting research from. [01:09:17.03] [01:09:18.03] Rivera which her book called. [01:09:20.22] [01:09:22.05] Well to remember you know the book no award Rivera though she's a researcher at [01:09:27.18] [01:09:27.18] Northwestern that you wrote a book about she got into the hiring process at [01:09:32.20] [01:09:32.20] Big Bangs at Wall firms and ad consulting firms and what she found was that [01:09:39.04] [01:09:39.04] those those types of things on the resume were somebody new like you see so [01:09:43.18] [01:09:43.18] do I ski right you played lacrosse or did I write you went to their games I did not [01:09:48.22] [01:09:48.22] write those things mattered a lot more in the hiring process and inhales. [01:09:53.08] [01:09:54.18] Yes I think if I think your question is it would. [01:09:58.17] [01:10:00.09] Be exactly where. [01:10:01.06] [01:10:04.22] I. [01:10:05.10] [01:10:07.01] Wish it were as you really think. [01:10:11.11] [01:10:12.13] It will. [01:10:13.01] [01:10:16.12] Not them. [01:10:17.23] [01:10:17.23] But I think that you're going to see a couple more publics going test for [01:10:22.22] [01:10:22.22] optional same way that we have the Pacific Northwest. [01:10:26.10] [01:10:27.18] I think it's fair for young that. [01:10:29.23] [01:10:31.10] Are going to face what I would say the next 6 months but [01:10:34.13] [01:10:34.13] at the Oregon public's both major organ publics we're going to start. [01:10:38.14] [01:10:40.05] I think that Georgia Tech has a president right now who's very open to listening and [01:10:45.08] [01:10:45.08] looking at data. [01:10:46.09] [01:10:47.10] And that he's open to the conversation I mean back to your comments or. [01:10:51.23] [01:10:52.23] About testing the validity of it you know we don't do this anymore but when we used [01:10:57.01] [01:10:57.01] to make formulaic decisions the students who performed the worst here were high [01:11:01.20] [01:11:01.20] testers low grades now I don't have any data on that anymore because we put so [01:11:06.14] [01:11:06.14] much emphasis on rigor of curriculum and performance because we don't take it like [01:11:10.01] [01:11:10.01] that anymore but they were about 12 percent lower attention rate after a year [01:11:14.23] [01:11:14.23] when you have the real discussions here between test scores and grades so [01:11:18.07] [01:11:18.07] there is something to the data there I've a friend who's now running admission that [01:11:22.17] [01:11:22.17] you 1st of all families said the exact same thing is true there as well so [01:11:26.08] [01:11:26.08] there is no good to go there behind that concept. [01:11:29.18] [01:11:31.03] I think that you have a growing test for a whole movement but whether or [01:11:36.07] [01:11:36.07] not a Princeton Harvard one of those that would take off so [01:11:39.18] [01:11:39.18] many dominoes will fall in the next 5 years. [01:11:42.13] [01:11:43.21] I don't doubt an i.v. [01:11:45.10] [01:11:45.10] will go to us were optional actually it's just well one of the ones that has [01:11:49.03] [01:11:49.03] the power to have a massive shaft do that I would predict that. [01:11:53.21] [01:11:55.23] Nothing nothing can. [01:11:57.14] [01:12:00.19] I'm not sure on Terra want to. [01:12:02.07] [01:12:03.23] Get better at the moment horses standardized testing is just growing it [01:12:09.00] [01:12:09.00] continues to grow it continues to become even more pervasive in kit of education so [01:12:13.11] [01:12:13.11] I don't see it up in bills changing any time soon I think [01:12:19.17] [01:12:19.17] we can concentrate on higher education universities addressing issues that [01:12:24.01] [01:12:24.01] are important for us concentrating on are making sure that when students come [01:12:29.02] [01:12:29.02] here we give them the resources that they need to be successful and I think that we [01:12:33.21] [01:12:33.21] should definitely pay close attention to those kids who are not applying who we [01:12:37.23] [01:12:37.23] are not accepting that making sure that if we open the door to hold on to [01:12:42.20] [01:12:42.20] students all the distance that different students that we have practices and [01:12:47.03] [01:12:47.03] higher education that make sure that this is to us are successful all of them and [01:12:52.00] [01:12:52.00] I feel that is her responsibility so [01:12:54.23] [01:12:54.23] I don't know that that's a pretty Chen and I hope is what we do. [01:12:58.21] [01:13:01.23] Or what I think will happen I do know what I hope will out and [01:13:05.15] [01:13:05.15] I hope that as more institutions become cognizant of big issues around testing [01:13:11.07] [01:13:11.07] that it will become more clear what it actually is and what it actually isn't and [01:13:17.01] [01:13:17.01] I think that until now we've accepted a lot of things around testing that aren't [01:13:21.21] [01:13:21.21] actually supported by the information and they've been largely misuse So [01:13:24.23] [01:13:24.23] what I hope will happen is we'll get better information as more schools [01:13:29.20] [01:13:29.20] used to test I don't forget all your you don't have massive swaths of schools like [01:13:35.03] [01:13:35.03] dropping I don't think that's going to happen but I do think that the use of test [01:13:38.17] [01:13:38.17] will become more informed more nuanced more thoughtful and. [01:13:43.11] [01:13:43.11] Hopefully that will be a good one of the thing I'm going to say there could be some [01:13:47.12] [01:13:47.12] disruptors out there like drooling know is doing some great work and [01:13:52.16] [01:13:52.16] you know right now with the corona virus and [01:13:54.08] [01:13:54.08] China I mean a lot of people are really trying to figure out you know how are they [01:13:58.18] [01:13:58.18] it's these submissions and this visa issuance another of these type of issues [01:14:03.23] [01:14:03.23] you know continue to be a problem you know what's that going to look like for [01:14:08.11] [01:14:08.11] the administration that's has outside of the fact that you know [01:14:11.11] [01:14:11.11] these types get passed around and they're constantly hacked and [01:14:14.23] [01:14:14.23] they have to cancel all these scores you know a place like Michigan State right now [01:14:18.20] [01:14:18.20] is probably really is what I mean 1100 of their 1st your kids come from China the. [01:14:23.07] [01:14:24.12] Delays and things like that I think doing those one that people are going to start [01:14:28.09] [01:14:28.09] to look to would not surprise me about the disrupter in the marketplace and [01:14:33.10] [01:14:33.10] people like them obviously. [01:14:34.19] [01:14:38.00] Thank you. [01:14:38.12] [01:14:39.13] Thank. [01:14:40.23]