[00:00:13]
>> Good afternoon everyone I want to thank everyone here for showing up also want to think we can lead for it for you know initially extended invite I think I was here last year some time. I saw a a video by Dr Ty say Dr Titus OK. I was with Microsoft and I was Mike Walker and.

[00:00:40]
They were talking about codifying your intuition and I thought wow that's a great that's a great way and I did a paper or did an article and I wanted to me Dr Kim I was saying I was in a land for something and I wanted to come by I mean anyway because I thought that was a profound way to kind of think about how we approach adversary behavior so I did an article for dark reading called entire codify your intuition into secured of ops where pushing for left and there's thing that we learned about building breaking fixing software that we can cooperate into building better software and understanding how can we leverage our intuition early on in the software development life cycle to build better software so that's kind of use here in that we can but we know from from my my years I detest I spent about 6 years a D.H.S.S. running a R. and D. program so I came in I came to detest in August of 2012 I think around a time we research was you know winding down or something but I met Winkie through through that initiative so I D.H.S.S. N T I ran a one of the largest R. and D. programs in science and technology directorate.

[00:02:02]
In the cyber security division around software security how can we push forward the state of the art to create better ways to analyze and detect potential weaknesses a software that can expose want to belittle so I funded everything from static analysis dynamic analysis so one of my big things that I'm very focused on is tech transition to commercialization So typically when I get a research paper either through a long range B.A. or a broad agency announcement or whether it's through Cibber.

[00:02:35]
I'm kind of different I work my way to the end because I want to understand commercialization plan the tech to transition plan use cases how you play an operational as a technology because it helps me make better decisions on how or whether it's a fun the research because you know obviously D.H. as we have operators we have in users in a process is that need innovation and I wanted to figure out a way to maximize the R. India investments by really understanding the commercialization plan the strategy take transition that helps me realize I'm better understand the novelty in the research so that's operating so so today I'm talking about for leaning and trying to reshape everything cyber R. and D. Many of our current capabilities in approaches are very reactive they're very active they're not able to do with some of the complex challenges that we see from our adversaries.

[00:03:49]
So we're falling behind the power curve in terms of our capabilities to defend our critical infrastructure our systems our software. And what we really need is more proactive strategies approaches and capabilities that can solve challenging problems and complex problems new cyber challenges that arise I think if we can think in that mindset of of pushing for at the state are we can be in a better position to be a little bit more forward leaning and more innovative I found this caption on internet somewhere as I was present for my presentation.

[00:04:35]
And it talks about the gaps in research and some of the challenges and I've always believed that there is there is there is gaps in not only the state of the art but the state of practice and if the state of practice is lagging it becomes very hard to innovate and create for leaning capabilities I give an example in terms of state of practice when I was a D H S I funded a research project called static analysis to static and static to analysis modernization project and essentially it was a idea that I had.

[00:05:17]
Working with in a say in their center for sure a software trying to understand why static analysis tools are or are not keeping pace of modern software development this is around a time when hardly hardly want to believe came out and none of the static analysis tools were able to detect that we can instead expose they hardly want to believe so I was watching T.V. with my wife 11 night.

[00:05:43]
How many people from new T.V.. Was your Imagine a danger to learn from H.D.T.V. so I mean people are familiar with the Property Brothers so there was probably did too probably roseate one brother who finally collect their homes you have the other brother who takes money infuse in and in the neglected homes and the renovate the homes.

[00:06:08]
I'm going to do the same thing for static analysis I want to infuse money to modernize that economics is one of the problems we have a static analysis in terms of we talking about now some of the gaps instead of practice one of the things we're studying as we don't know is static and I was too can and cannot do we don't have a precise way to measure the tools if you look at the N.S.A. has been doing over the last several years with their their tool study.

[00:06:36]
The problem with this to study their synthetic test cases so what I want to do with stamp was to not only modernize static and I was capabilities but come up with the. A very scientific way that has some foundational bearings in it to be able to measure the effect is of 2 So ultimately what I wanted to accomplish with this is so when someone wants to buy or purchase a stack and I was to be under they would have a good understanding of the strengths and weaknesses of it but today we have no idea of measurement so we can't modernize the tools we don't know where the gaps are in the tools so those are the things are not talked about this the connection between the state of practice and the state.

[00:07:20]
Also looked at some of the top 5 challenges that we see for not only 2100 but 2018 so for someone and as you look at this list some things remain a saying who phishing I mean we've been talking about phishing for the last 10 years and fishing is probably the most prevalent attack vector for system compromise right we see with the software supply chain in PM crypto coin stealing Mauer that made his way into the in PM.

[00:07:57]
Ecosystem so now we are seeing different attack veganism ways the adversary is trying to attack our systems ransomware is on the rise I mean is this the new Then this and this out in advance percentage threats are always always the bane to a lot of operators. Problems to add all to this talk about cyber challenges J G A O came out with this study and it identified the nation's top cyber security challenges.

[00:08:41]
And what I gather is these these these these challenges here are a result of active cyber capabilities approaches and strategies establishing copper is a cyber strategy and performing oversight so there's not a comprehensive strategy how we're going to build the foundations to 4 leading research it is no comprehensive cyber strategy security federal systems more often than not the federal government is slow always last to adopt innovation so the question becomes that we start thinking about research working with government agencies and government institutions.

[00:09:26]
How can we get capabilities to the in user faster how can we get innovation to the the in users faster and obviously critical infrastructure has been very important as well as privacy and since the ration but one of the biggest things that we all struggle with. Is cyber hygiene and we look at from the stats are any 4 percent of these fail to meet goals encrypt in data rest we know that pretty much all breaches pretty much all breaches have occurred over the last 1015 years could potentially have been prevented with encrypting your data is that in a day adversaries after the data there is there's a there's a crown jewels is there sensitive data that needs are protected we saw that with the massive open or opium breach.

[00:10:18]
They did was encrypted we saw they were Equifax I mean Equifax I mean that was we've all been impacted by at least I mean I was I was a federal employees I was impacted by both O.P.M. as well Equifax so we all become vulnerable to to not. Conducting and doing the basic cyber hygiene which is so essential but part of that problem is is the reactive capabilities that a lot of these State of the art tools have.

[00:10:48]
So how do we continue to push for the state of the are and. Look a little bit more forward leaning too to deal with some the complex challenges that we see from the spectrum I thought this is a very interesting take here I did a before and now and I pulled this from sofas 200-1000 threat report so before the defenders will would have execute execute was early in the kill chain but now we're seeing.

[00:11:24]
Tactics and techniques changing and they went into the very tail end of the kill chain and as a result we have to shift our thinking how many heard the term think like a hacker no one just me right we have to put on adversaries mindset and really try to understand ways in which they can attack our systems and if if they are moving towards the latter part of the kill chain you know when the executable occurs then we need to be able to adjust our tactics and capabilities to be read to known detective stoppie here's something worth noting right.

[00:12:15]
This was something also pulled from the 2001000. Sophia's report. And if in fact ever serious tact is a method are changing involving the question becomes how do we predict and disappear the future take needs to capabilities the adversaries we use so instead of being one step behind we need to with our research so as you all are sitting Well let me take a let me pause for a 2nd and just do a quick check of folks in a room how many I actually are doing computer cyber research here.

[00:12:56]
How many folks here actually applied to grants from the photogrammetry decide to research how many folks you want to apply for grants into several research how many people here understand the importance of tech transition to commercialization So as we are you guys are doing research and trying to solve challenging problems with novel capabilities in our approaches.

[00:13:26]
You really need to model your research approaches to be a good to be able to predict in anticipation So whether you're doing something on I'll. Be Doing something on a privacy or you do some more of my where I think having that right having that cyber approach is probably something that is worthwhile investing so I've been impacted by this so pride to leave in I left before I left the.

[00:14:03]
In of October of 2000 the 17 right at a time where programs are being cut my Software Assurance program was 10000000 so I meant out of one of the largest programs it did as I can see it was 10000000 I think mine is got cut to about 2000000 and as a writer around the time I decided a staff for me to leave a kind of you know you kind of feel things and I didn't want to put I don't want to initiate any new startup programs because I knew I would not be around to see them actually get awarded so I kind of left but part of the problem is with this approach is if if ever series a costly moving a lot faster than us and one of the things with this this this line here.

[00:14:51]
Increases cyber spending. By the cuts in research so meaning we're going to spend on the quote unquote stately our technology is occurring out there that we know are not for need leaning enough and not someone behind a power curve so now we're we're increasing spending to buy technologies and capabilities that we know don't really do a good job at defending against some of the more complex and challenging and sophisticated threats so the chef was May so that operators.

[00:15:28]
Can have more fun as a research well when I was a S N T I led a A I B T process integrated product team process where we would work with all the composer within D.H.S.S. gather requirements to help shape our indeed to make so make transition of our day more appropriate to homeless creating a prize and you view surprise when you work of operators is no this is no slight Sobber it is the type of requirements you get they are not requirements they are more features functionality histamines and they are all centered around your current infrastructure your current environment.

[00:16:09]
So the question becomes researchers will have to start working more closely with stakeholders and operators so essentially always look at research and development either by the call the BIG are small are big D. small D. right meaning big armies more research to develop you know more research big big these big you know more development but because of the change in landscape they escape in federal federal budgets we're moving from big art to small are so it is very very low research that is being done when operators are influenced in research requirements is more so hey I have this development I have this gap in my current infrastructure or my current capabilities can you develop a bridge or a gap to the help close this is not really anything that research related where it solves you know or is or is aggressive in solving challenges or problems so how do you account for this when you start engaging with government agencies especially did this in research and novel ideas and how does this impact the way in which you do transition and commercialization because some folks here or academic institutions they're more in the research side less under them aside now research is very important and so you have to counter this some way or another so I just wanted to bring that up I think this is this is something that I've noticed over over the last year or so now working for Myer which is A F R D C Where where we research a development load differently we're working with the operators and it's really not big R. is really OK we're going to develop and we want to transition this is an example.

[00:18:24]
Other F. 1900 federal cyber strategic R. and D. roadmap and one of the things I want to emphasize here is is is while we're still trying to close the gap that exists in the state and state a practice the government does recognize the federal strategy does recognize the need for more aggressive research technologies and capabilities we need to really focus on proactive research and development activities that can disrupt adversary activity so detect.

[00:19:05]
Right well I would miss that would deter deter you know we need to have technologies to deter adversaries. So having a. A Well formalized strategy that incorporates a a mixture of these different is the fit of these different defense mechanisms or defense of strategies is important to leverage I know that deception moving target offense is huge and something that folks are starting to incorporate into the research because it's very important we need to be able to adapt you know have adaptive capabilities.

[00:19:44]
So there's a lot of things that we have with new machine learning artificial intelligence that can help to help us with more defensive strategies as we. Conduct our research so rich researchers have to adjust a little bit now it becomes very important for you to understand the mission so you have to connect your research with the mission Otherwise your research has no value and it will never get over the valley of death so understanding the mission is very important so working with stakeholders and operators is very important to understand the mission balance and leverage scientific discoveries.

[00:20:33]
And take technological opportunities be risk takers early it's OK to fail I rather as a program manager when I have my program reviews I make sure that I have a really really good defining decision points for no go go decision points because it's important to be aggressive for research was also important to take risk and the sooner you can fail at least for me as a program manager the better off we are because then we can adjust their approach talk about it adjust their approach address adjust the strategies figure out what needs to be done maybe we develop new use cases that more properly to transition targets maybe even use cases that were initially defined up front in the proposal are not properly aligned with the mission or with a potential customer so failing early and take a risk is very important and obviously as a mentioned I'm a huge proponent of tech transitions so crossing the valley of death is very important and really really focus in aligning your your research with mission because that's important.

[00:22:02]
So as we move to operators influencing R. and D. right one of the things we have to realize is that we need to take the take the cyber activities out of the hands of and skill operators who are overwhelmed who are costly find fires and don't have a real a real pulse on the actual gaps that exist within their army I've always say that humans you know this is no slight on folks humans are the weakest link in cyber activities.

[00:22:46]
And often becomes a killing field the kill the seal so it the more we can train machines I mean folks are are familiar with the dark cyber Grand Challenge where they're using machines to to look for now we want to believe become a with patches to mitigate a particular issues in software the goal is to really create more resilient software so the more we can take cybersecurity out of the hands of unskillful or move to machine leverage machine computers to do other stuff I think we are moving in the right direction because obviously humans can be the weakest link so in terms of cyber domain cyber research remains here are some research remains that I think are important as as as computer sciences and research just to kind of you know understand and one important thing I want to stay in only across the main will cross-cutting all these you know a lot of these things are very crosscutting mobile is cross-cutting with cloud computing environments.

[00:23:56]
Bar medical devices even V. who aviation so the A lot of these things are known only within their own domain but they are cross-cutting and one point thing to realize is software security becomes the fabric that ties all these things together I've always say software security is our 1st layer defense if we're not doing good design good software development then we're creating want to Billy's on the fly and no fire all no mower detection no deception technology can can can can mitigate poorly developed software just can't and because software is so ubiquitous it becomes the underpinning to everything we do so.

[00:24:45]
These are some things to take in consideration these domains as you look for research topics engage with different research organizations and agencies and one of these I would advise researchers is to get out and talk to operators get out and talk to folks talk to startups understand a problem understand the problem the current market understand your competition understand your domain you're doing your research in so innovate with speed in the jelly We need better capabilities and we need to get known these capabilities we need to get them out to operators sooner and faster we have to align as I mentioned align our research development prototype early to validate the primary technology requirements because we can't do research in a vacuum there's a whole enterprise environment where your research material need to go into what's the best set of use cases to help operationalize this research I think is important to align alignment is very important to make sure that your research can be successful so any time you can do any type of demos early demos up early on proof of concepts is very important to do that allocate is also important allocate enough resources to do in an op really integration testing right when you do research you need transition Parness you need to validate your research somehow in these ago in some operational environment so as you do in budgeting for your research your proposal right understand if any resource resources need to be allocated to do integration in Iraq in Arbil you testing especially if you have a transition customer in mind you want to engage them early and often.

[00:26:56]
Get requirements get a feel for the operation of Vironment some of the pain some a technology is the deploy in the operation environment ways in which your technology can integrate in Iraq with other technologies already there to create a new capability that you probably mean realize when you study your research how do you maximize your research that's important there's no silver bullet there's no we're technology we're seeing things being you know we're seeing blended cyber attacks which will which would which which we need a integrated approach to do with Excel or understand adoption match understand adoption and match that pace with technology in the environment for better innovation meaning if if you're a transition point and you're looking at is leveraging the cloud a private cloud then well when you do your research those need to be that that dynamic need to be accounted for in your research early on are they use a 0 trust in their environment will do a trust Yeah well how do we account for that our research aligning research allocating resources and that will help accelerate into the market faster and one of the other things is if you can adopt some of the technologies already there in a giving operating or enterprise environment that's to your advantage that's what you have and don't recreate the Will it doesn't make sense to leverage was there build on top or was there so you could accelerate and get to market faster or these get get your research to Operation environment fast to know only validate but help help you help potential sponsors solve some really really challenging problems so opportunities for competitive advantage there's so much data out there the tons of data out there we have the leverage data also sort of data.

[00:29:04]
Especially with the the rise of big data analytics. What new insights can we gather from all this data about cyber challenges and problems that we didn't have visibility into before how can we slice and dice data how can we pivot into all this data to really give us Buz into things that worked before or may have not worked but provide a way forward for us to be in a position to do more for lean research partnerships to rapidly absorb new technologies there's a tons of integrate is out there who would be willing to work with you especially we have a really really novel research or technology that solves major problems there's there's any great is that will work with you to get your technology into operation environments because you need to validate within it whether or not your research or your technology actually works and the feedback the most important thing us just say is feedback from a sponsor or potential transition customer is very important is very important leverage investment by industry and take forging to build capabilities as I mentioned why why reinvest the why reinvent the we'll take foraging is a also awesome approach that you can use to help accelerate your research ideas but also help accelerate getting your technology in the hands of end users who need the who can use it so this is my last.

[00:31:00]
Talk about my parting shots I think the work that has been done here at G T R I I think to leverage your past excesses dumbbell is is a great example of research I came out here that not only did it not only transition got over the valley of death but was commercialize and was acquired.

[00:31:23]
That's a huge success Weekley was a huge part of that learning from that is very important land code. In as well as others is things you can learn from prior research that came out of this. To be successful in Creve for leaning capability because I believe at the time done bhalo was considered for leading technology when it was and when it hit the market and understanding in the lessons learned from that is very important connect the dots to accelerate.

[00:31:57]
Or connect the dots to accelerate innovation is to say and what I mean by connect the dots is understand a mission but before that when you start to research understand how this research technology will be operationalize in operational Ramat what I use cases are these use cases align with a mission can I leverage investments in prior new discovery I mean prior technologies and research are there new discoveries I can leverage to create a new capability all these things are important connecting the dots lean forward we don't fall over and what I mean about it is sometimes we can be too aggressive right sometimes we have tunnel vision in terms of the research we want to do there has no operation of operational value to anyone people need they is going to solve their problems today and sometimes being 2 for a living is not a good idea so how do you balance being aggressive in attacking a problem or addressing a problem and being too aggressive so looking for but don't fall over.

[00:33:28]
Context is everything and I mean it is mean to use cases are very important as as as every see the federal landscape shifting and in Him By law it's this knowledge the photo side is industry as well operators are getting more involved in formulating flues and research So context is everything and I can't stress enough the importance of narrowing your research to a group of use cases that are important to the operator or stakeholders once you validate that you're on your way so how do you take a large problem shrink it to accept a set of use cases that addresses we know root pains in environments and align your research with that becomes a path to suspects who to success of tech transition is a mission fall fast and get back up realigned to reshape and don't get stuck in the valley of death meaning don't create research that is meaningless when I say meaningless I mean it doesn't solve a problem today so if we could do these things I think we have a better opportunity creating more forward leaning capabilities and approaches to address the cyber research so this is the end of my talk appreciate your attention and our open up the floor to any questions or comments then when we have at this time and please them before don't be shy X. questions I like engaging I think is very important I want to be able to help and assist folks who are very interested in research to make sure they are on the right path and right approach to doing this I have a question here 1st and I'll take your question yes or no you need to rethink it.

[00:35:57]
Well you know you know you thought of. This one. I read that. While you. Made it very. Well to your 1st question I think one of the things is everybody who do research may not have the visions of a formulaic startup but for those who are interested in transition I would say having a really really good idea of not only competition in the market but also what do people want like it's good to do research but if no one's going to use it was a purpose of doing the research understand everyone has their own research interests understand and I'm not knocking that by think it's important to really understand the value in transition in research and in the validating 2nd question I think the more we expose ourselves the data to understand the kinetics across those I mean I think the better off would be because there is there is there is a because things are across the Main in cross-cutting the attacks surface changes to a certain extent because everything has kinetic points and I think the more we expose myself to data meaning understanding the threats within those the mains understanding how those threats can be explored it within those the mains.

[00:38:05]
And finding the connection points across those different domains is important and you'd be surprised probably what you can learn in terms of the kinetics across those different cyber domains in how we go into aviation right how other domains play into that and we may find that there are similar tech services and if we can solve a problem in one domain right if there's commonalities to another domain I think we can there's lessons learned it's things we can learn from understanding the connection but across all those different domains Hopefully the answer your question next question.

[00:39:03]
So my answer there's all different right so matter we work with a lot of sponsors so we have insight into potential pain points working on different sponsor organizations we have something called a miter innovation project where. Engineers can submit research novel research ideas to be funded right so obviously this small investments it's not a $1000000.00 to $34000000.00 a small investment but the investment is focused on solving a particular customer need I say look at the work we've done with attack that was that came from a pain point from a customer and there was some of best moves that were put into that to develop that framework that attack framework to help solve some of the issues we see on the threat landscape so.

[00:40:05]
So it's a little different for F A R D C We we do do some our stuff but I think a lot of times we knew when your so engaged with the customer and in the sponsor you're in you're instinctively getting requirements by doing work for them or ideas I say they can formulate a requirement or understand a use case that can enjoy drawdown requirement so I think that's that's a little different from submitting a low res be a force to meeting some type of white paper to a broad see enough of a more questions yes or.

[00:40:58]
No. Well. Those are new partnerships that are talked about they need to be formed right there's no one there's no one entity. That holds the problem space to themselves right and one of these ideas. And I has some as a sex which is so I looked at the software screen promise and listen in you alluded to some there was a point in this recent software security differently from academia academia see the problem different from government and vendors see these differ from everybody so I wanted to get everybody I want to at least in the onus on the same side of the street so I started to work in groups.

[00:41:59]
Because I want folks to understand what we all understand is an issue a software security but everyone has a different way in which they or the importance of how it should be saw but let's figure out a way working on the same page and work to address it was 2 or 3 issues I think that's that's that's that's achieving some momentum in trying to attack one problem but it would there were required a unique relationship of partnership and the researcher to me in my opinion would have to seek that out and have a relationship that's what I'm saying is very important to have relationships with not only the stakeholders in operators but also understand your competition because on if I have a research idea and I know this vendor here is struggling I have not added it to is is backlog of products.

[00:42:56]
That's the opportunity to leverage our commercialize and potentially have some type of partnership with the vendor to not only get your your research developed fastening cellaring to get it to market but this opportunity for exit strategy so that's why it's important to really you know. Strictly research and development or do I have ideas for transition commercialization because part of that is really having understanding of which are exit strategy is because this is the more you talk with vendors in focus industry you create your chances of someone wanting to acquire your tag or your research because what I found out is and I hate to say this sometimes vendors can be so large they felt innovate and they always looking to gobble in perches and acquire new tech and new research.

[00:44:00]
Because everything's revenue driven if I'm making money selling the same set of capabilities there's really no incentive to innovate. We're. In the other thing is I think is important also is what this is a few I'm not to say a lot of the the resources I work with in the past there are few who are taking the research into thought leadership around the research and what the thought leadership piece is it is socialize your ideas and concepts with the larger with a larger audience it makes them aware of the things you're working on and what you're doing by doing a thought leadership is your sort of clear and a pathway or the barriers for people to embrace what you're trying new concepts and ideas you know look at the industry we're always slow to adapt something that's new and different so sometimes providing that thought leadership so yeah academic papers are finding academic journals of fine but I really don't have Operation context of attached to it so if you provide thought leadership or all your novel ideas and research and connect it with the operational contacts in India the thought leadership I think that will help bed be very successful and you will see a lot of interest and you can potentially you know start working groups around it because that's how you shift people's mindset don't lease it was a unique way to help shift people mindsets in more questions one for 0.

[00:46:07]
Well I appreciate everyone sticking here and those Friday to pry have a better they do like Christmas shopping and everything but I appreciate everyone being here and. You know you all have have. Really set the bar high I've been where we have a lot of research institutions and academic institutions and universities and I've always thought very highly of Georgia Tech and the work they have can that have come out here and some of the challenging problems that you able to address and keep up the good work and everyone needs to reach me I can be reached at only then at Kevin eager NG with even Or you can follow me on Twitter at think it's I am kept Tories or kept Tories on Twitter and.

[00:47:06]
We'll see maybe acting it will be in my pocket also have a park Ascot so I was cool incisive prospectus is on the Sound Cloud and maybe Winky you could be on what my next guest more Pica us talk about R. and D. in the challenges in transitioning into a for living R. and D. So thanks everyone for your time appreciate it thanks.