Title:
EvalAI: Evaluating AI systems at scale

dc.contributor.advisor Batra, Dhruv
dc.contributor.author Deshraj
dc.contributor.committeeMember Parikh, Devi
dc.contributor.committeeMember Lee, Stefan
dc.contributor.department Computer Science
dc.date.accessioned 2019-01-16T17:21:52Z
dc.date.available 2019-01-16T17:21:52Z
dc.date.created 2018-12
dc.date.issued 2018-12-06
dc.date.submitted December 2018
dc.date.updated 2019-01-16T17:21:52Z
dc.description.abstract Artificial Intelligence research has progressed tremendously in the last few years. There has been the introduction of several new multi-modal datasets and tasks due to which it is becoming much harder to compare new algorithms with existing ones. To solve this problem, this thesis introduces EvalAI, an open source platform for evaluating and comparing machine learning and artificial intelligence algorithms at scale. This platform is built to provide an open source, standardized, scalable solution for evaluating learned models using automatic metrics as well as with human-in-the-loop evaluation. By simplifying and standardizing the process of benchmarking, EvalAI seeks to lower the barrier to entry for participating in the global scientific effort to push the frontiers of machine learning and artificial intelligence, increasing the rate of measurable progress in these communities.
dc.description.degree M.S.
dc.format.mimetype application/pdf
dc.identifier.uri http://hdl.handle.net/1853/60738
dc.language.iso en_US
dc.publisher Georgia Institute of Technology
dc.subject Machine learning
dc.subject Artificial intelligence
dc.subject Evalai
dc.subject Deep learning
dc.subject Computer vision
dc.subject Reinforcement learning
dc.subject Systems
dc.subject Scale
dc.subject Data science
dc.subject Kaggle
dc.title EvalAI: Evaluating AI systems at scale
dc.type Text
dc.type.genre Thesis
dspace.entity.type Publication
local.contributor.advisor Parikh, Devi
local.contributor.advisor Batra, Dhruv
local.contributor.corporatename College of Computing
local.contributor.corporatename School of Computer Science
relation.isAdvisorOfPublication 2b8bc15b-448f-472b-8992-ca9862368cad
relation.isAdvisorOfPublication bbee09e1-a4fa-4d99-9dfd-b0605fea0f11
relation.isOrgUnitOfPublication c8892b3c-8db6-4b7b-a33a-1b67f7db2021
relation.isOrgUnitOfPublication 6b42174a-e0e1-40e3-a581-47bed0470a1e
thesis.degree.level Masters
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
DESHRAJ-THESIS-2018.pdf
Size:
13.97 MB
Format:
Adobe Portable Document Format
Description:
License bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
LICENSE.txt
Size:
3.86 KB
Format:
Plain Text
Description: