Title:
You can lead a horse to water...: Representing vs. Using Features in Neural NLP
You can lead a horse to water...: Representing vs. Using Features in Neural NLP
| dc.contributor.author | Pavlick, Ellie | |
| dc.contributor.corporatename | Georgia Institute of Technology. Machine Learning | en_US |
| dc.contributor.corporatename | Brown University. Dept. of Computer Science | en_US |
| dc.contributor.corporatename | Google AI | en_US |
| dc.date.accessioned | 2021-04-06T15:39:09Z | |
| dc.date.available | 2021-04-06T15:39:09Z | |
| dc.date.issued | 2021-03-24 | |
| dc.description | Presented online on March 24, 2021 at 12:15 p.m. | en_US |
| dc.description | Ellie Pavlick is an Assistant Professor of Computer Science at Brown University where she leads the Language Understanding and Representation (LUNAR) Lab. She received her PhD from the one-and-only University of Pennsylvania. Her current work focuses on building more cognitively-plausible models of natural language semantics, focusing on grounded language learning and on sample efficiency and generalization of neural language models. | |
| dc.description | Runtime: 57:05 minutes | |
| dc.description.abstract | A wave of recent work has sought to understand how pretrained language models work. Such analyses have resulted in two seemingly contradictory sets of results. On one hand, work based on "probing classifiers" generally suggests that SOTA language models contain rich information about linguistic structure (e.g., parts of speech, syntax, semantic roles). On the other hand, work which measures performance on linguistic "challenge sets" shows that models consistently fail to use this information when making predictions. In this talk, I will present a series of results that attempt to bridge this gap. Our recent experiments suggest that the disconnect is not due to catastrophic forgetting nor is it (entirely) explained by insufficient training data. Rather, it is best explained in terms of how "accessible" features are to the model following pretraining, where "accessibility" can be quantified using an information-theoretic interpretation of probing classifiers. | en_US |
| dc.format.extent | 57:05 minutes | |
| dc.identifier.uri | http://hdl.handle.net/1853/64417 | |
| dc.language.iso | en_US | en_US |
| dc.relation.ispartofseries | Machine Learning @ Georgia Tech (ML@GT) Seminar Series | |
| dc.subject | Language models | en_US |
| dc.subject | Natural language processing (NLP) | en_US |
| dc.title | You can lead a horse to water...: Representing vs. Using Features in Neural NLP | en_US |
| dc.type | Moving Image | |
| dc.type.genre | Lecture | |
| dspace.entity.type | Publication | |
| local.contributor.corporatename | Machine Learning Center | |
| local.contributor.corporatename | College of Computing | |
| local.relation.ispartofseries | ML@GT Seminar Series | |
| relation.isOrgUnitOfPublication | 46450b94-7ae8-4849-a910-5ae38611c691 | |
| relation.isOrgUnitOfPublication | c8892b3c-8db6-4b7b-a33a-1b67f7db2021 | |
| relation.isSeriesOfPublication | 9fb2e77c-08ff-46d7-b903-747cf7406244 |
Files
Original bundle
1 - 4 of 4
No Thumbnail Available
- Name:
- pavlick.mp4
- Size:
- 215.4 MB
- Format:
- MP4 Video file
- Description:
- Download Video
No Thumbnail Available
- Name:
- pavlick_videostream.html
- Size:
- 1.32 KB
- Format:
- Hypertext Markup Language
- Description:
- Streaming Video
No Thumbnail Available
- Name:
- transcript.txt
- Size:
- 53.09 KB
- Format:
- Plain Text
- Description:
- Transcription
- Name:
- thumbnail.jpg
- Size:
- 65.2 KB
- Format:
- Joint Photographic Experts Group/JPEG File Interchange Format (JFIF)
- Description:
- Thumbnail
License bundle
1 - 1 of 1
No Thumbnail Available
- Name:
- license.txt
- Size:
- 3.13 KB
- Format:
- Item-specific license agreed upon to submission
- Description: