Series
Master of Science in Computer Science

Permanent Link
https://hdl.handle.net/1853/73328
Series Type
Degree Series
Description
Associated Organization(s)
Associated Organization(s)
Organizational Unit

Filters

16 9
16
25
25
Reset filters

Settings
Start date: 2000 ×

Publication Search Results

Now showing 1 - 10 of 25
Thumbnail Image
Item

A First Look at Autonomous Systems Recurrently Causing BGP Origination Conflicts

2024-05-06 , Bemba, Olivier

The Border Gateway Protocol (BGP) plays a key role in the Internet as it provides the path for packets to travel between independent networks (Autonomous Systems) on the Internet. However, it also allows multiple networks to announce reachability for the same prefix, which makes it vulnerable to attacks and misconfigurations that modify Internet traffic. This is known as an origin conflict. According to the Global Routing Intelligence Platform (GRIP), a software that detects this kind of event, less than 1\% of all Internet networks is responsible for almost 40\% of the most suspicious origin conflicts detected between January 1, 2020, and January 1, 2023. Therefore, it is important to try and understand whether these networks are not causing these conflicts for malicious purposes, or whether it is a matter of new routing habits, or simply misclassification from GRIP. As a first step, we leverage GRIP to isolate autonomous systems (ASes) that have been involved in at least one origin conflict on 50 different days. Then, for all these ASes, we collect data about their organization, their location, their type, and their GRIP events. In parallel, we also retrieve routing data using the RIPE Stat API to provide more context. Finally, we combine these data and analyze them to find indicators of malicious activity, configuration error, or legitimate behavior. Thanks to this first look at GRIP data from an AS perspective, we were able to observe some already seen legitimate behavior, such as the use of private ASNs or Internet exchange point prefixes. These two use cases need to be added to the GRIP classification system. Next, we also observed several business relationships, such as hosting providers with their customers, IP space lessors with their lessees, and DDoS mitigation providers with their customers. These business relationships need to be studied in greater depth to better characterize and detect them. We also present two use cases: a global mobile operator that is known to make frequent configuration errors, and a hosting provider that is known to provide services to malicious customers. This study is a first building block towards finding new insights into the routing habits of Internet operators and thus contributes to improving the global routing monitoring system.

View
Thumbnail Image
Item

Improving Real-world Aerial Scene Understanding With a Synthetic Dataset

2024-04-29 , Khose, Sahil Santosh

Real-world aerial scene understanding is limited by a lack of datasets that contain densely annotated images curated under a diverse set of conditions. Due to inherent challenges in obtaining such images in controlled real-world settings, this thesis introduces SkyScenes, a synthetic dataset of densely annotated aerial images captured from Unmanned Aerial Vehicle (UAV) perspectives. We carefully curate SkyScenes images from CARLA to comprehensively capture diversity across layouts (urban and rural maps), weather conditions, times of day, pitch angles and altitudes with corresponding semantic, instance and depth annotations. Through experiments using SkyScenes, this thesis demonstrates that (1) models trained on SkyScenes generalize well to different real-world scenarios, (2) augmenting training on real images with SkyScenes data can improve real-world performance, (3) controlled variations in SkyScenes can offer insights into how models respond to changes in viewpoint conditions, and (4) incorporating additional sensor modalities (depth) can improve aerial scene understanding.

View
Thumbnail Image
Item

Domain generalization in vision models for aerial imagery datasets

2023-05-04 , Agarwal, Aayushi

In this work, we present SKYSCAPES, a large-scale densely-annotated high-resolution synthetic dataset of aerial images captured from an oblique UAV perspective, for two aerial scene understanding tasks – semantic segmentation and object detection. SKYSCAPES has been designed by keeping in mind several important desiderata specific to aerial images in mind – scale, diversity, class-representation, etc. Experiments conducted on SKYSCAPES demonstrate that (1) SKYSCAPES coupled with a real dataset can serve as a syn-to-real generalization benchmark, (2) SKYSCAPES synthetic data can augment real-data for better generalization and (3) intra-source weather and daytime variations in SKYSCAPES can be used to systematically study robustness of aerial vision models. We hope our dataset and subsequent experiments enable the development of improved visual scene understanding models for aerial viewpoint images.

View
Thumbnail Image
Item

Mitigating Racial Biases in Toxic Language Detection

2022-05-05 , Halevy, Matan

Recent research has demonstrated how racial biases against users who write African American English exists in popular toxic language datasets. While previous work has focused on a single fairness criteria, we propose to use additional descriptive fairness metrics to better understand the source of these biases. We demonstrate that different benchmark classifiers, as well as two in-process bias-remediation techniques, propagate racial biases even in a larger corpus. We then propose a novel ensemble-framework that uses a specialized classifier that is fine-tuned to the African American English dialect. We show that our proposed framework substantially reduces the racial biases that the model learns from these datasets. We demonstrate how the ensemble framework improves fairness metrics across all sample datasets with minimal impact on the classification performance, and provide empirical evidence to its ability to unlearn the annotation biases towards authors who use African American English. ** Please note that this work may contain examples of offensive words and phrases.

View
Thumbnail Image
Item

Multiagent debate among vision-language models improves multimodal reasoning

2024-04-29 , Murugappan, Ganesh Meyyappan

We propose a framework for improving the multimodal reasoning capabilities of vision-language models through multiagent debate, where multiple models engage in a structured debate process, taking opposing perspectives and exchanging arguments about a given multimodal input containing text and images. Through this iterative debate, the models can complement each other's strengths, surface relevant evidence across modalities, and arrive at more robust and well-reasoned conclusions compared to using a single model. Evaluated on the ScienceQA dataset, models involved in a debate significantly outperformed their individual baselines, with prompting strategies resulting in further improvement. The debate process allows models to identify flaws, provide additional evidence, and negotiate stronger final answers by combining diverse skills, highlighting the potential of constructive disagreement and debate for overcoming limitations in current multimodal AI systems.

View
Thumbnail Image
Item

Impressions: Understanding Visual Semiotics and Aesthetic Impact

2024-04-29 , Kruk, Julia

Is aesthetic impact different from beauty? Is visual salience a reflection of its capacity for effective communication? We present Impressions, a novel dataset through which to investigate the semiotics of images, and how specific visual features and design choices can elicit specific emotions, thoughts and beliefs. We posit that the impactfulness of an image extends beyond formal definitions of aesthetics, to its success as a communicative act, where style contributes as much to meaning formation as the subject matter. However, prior image captioning datasets are not designed to empower state-of-the-art architectures to model potential human impressions or interpretations of images. To fill this gap, we design an annotation task heavily inspired by image analysis techniques in the Visual Arts to collect 1,440 image-caption pairs and 4,320 unique annotations exploring impact, pragmatic image description, impressions, and aesthetic design choices. We show that existing multimodal image captioning and conditional generation models struggle to simulate plausible human responses to images. However, this dataset significantly improves their ability to model impressions and aesthetic evaluations of images through fine-tuning and few-shot adaptation.

View
Thumbnail Image
Item

Language Models: Generator and Labeler

2023-05-02 , Rungta, Mukund

Over the last few years, there has been remarkable progress in the capabilities of language models. These models have been trained on massive amounts of data using advanced learning algorithms, enabling them to perform a wide range of Natural Language Processing (NLP) tasks with great accuracy. This has made them highly reliable and robust, with state-of-the-art or comparable performance on various NLP benchmarks. In this thesis, I explore two uncharted territories of using large language models: hierarchical text classification and generating training data without human supervision. The proposed approach and design choices for both tasks demonstrate superior performance over different baselines. To support these claims, I examine different components of the model and analyze their contributions to the overall improvements. Although there are several limitations to using language models directly for generating training data, such as ensuring label accuracy and preserving dataset diversity, this work can inspire further research on exploiting dataset-generation-based zero-shot learning using large pre-trained language models.

View
Thumbnail Image
Item

Can Language Models Protect Personal Information

2024-04-29 , Mendes, Ethan

Large multimodal language models have been shown to memorize and leak pre-training data, raising serious user privacy and information security concerns. While data leaks should be prevented, it is also crucial to examine the trade-off between the privacy protection and model utility of proposed approaches. In this thesis, we introduce PrivQA— a multimodal benchmark to assess the capabilities of language models to follow privacy protection instructions such that a selected group of personal information is protected while remaining useful to respond to other queries. PrivQA consists of both annotated human-AI dialogues as well as repurposed queries from existing question-answering (QA) datasets. We propose an access control instruction technique to iteratively self-moderate language models’ responses, which significantly improves privacy. However, through a series of red-teaming experiments, we find that adversaries can easily circumvent these protections with simple jailbreaking methods through textual and/or image inputs. We believe PRIVQA has the potential to support the development of new models with improved privacy protections, as well as the adversarial robustness of these protections.

View
Thumbnail Image
Item

Integrating LASCO Corona Images into Spatio-Temporal Attention Model for Improved Solar Wind Speed Prediction During ICMEs

2024-04-29 , Perlman, Zachary Adam

This thesis explores enhancing machine learning-based solar wind speed prediction models by incorporating C2 images from the Large Angle and Spectrometric Coronagraph (LASCO). Solar wind predictions are critical as they significantly impact Earth, disrupting everything from power grids to communication systems. Traditional prediction models have used images from the Atmospheric Imaging Assembly (AIA) and historical wind data. This research introduces integrating an additional data source—LASCO C2 images—which provide a unique perspective of the solar corona and highlight solar anomalies like interplanetary coronal mass ejections (ICMEs). To accomplish this, we augment a baseline multimodal prediction model (which includes convolutional, recurrent, and attentional components) with a new processing branch for LASCO C2 images, which parallels the existing branch for AIA images. This integration aims to leverage the distinct characteristics of LASCO images to boost the predictive capabilities of the model. The performance of this modified model was evaluated against the baseline through the root mean square error of the predicted solar wind speeds during identified ICME events from 2011 to 2016. Results indicate that the LASCO-enhanced model achieves a noticeably lower error compared to the baseline model during ICME-dominated periods. This improvement highlights the model's elevated capability during these critical times and represents a promising step forward in improving the accuracy of predictions during the most impactful solar events, potentially improving how we prepare for and mitigate the hazardous effects of space weather.

View
Thumbnail Image
Item

Effective qubit mapping, routing and scheduling for trapped-ion quantum computers

2023-05-02 , Gupta, Vima

Trapped-Ion Linear Tape (TILT) architectures offer a scalable way to realize ion-trapped quantum computers through tape-based shuttling and routing operations. Modulo a cost model for tape movement and gate application, the quality of qubit mapping and routing (QMR) targeting TILT architectures has a tangible impact on circuit fidelity. State-of-the-art QMR techniques either don’t account for the cost of tape movement or rely on heuristic-based approaches. To address the shortcomings of existing qubit mapping and routing (QMR) techniques, this thesis introduces and evaluates MALT, a comprehensive extension of a MaxSAT based QMR technique by Molavi et. al. MALT generates efficient swap insertion and tape movement schedule for shuttling based architectures, beating the SOTA heuristic for all configurations and benchmarks. The thesis also addresses the issues in scaling a pure constraint-based approach and showcases the optimizations made to scale. This technique incorporates noise-awareness in terms of adding a swap gate versus moving the tape, allowing the user to customize for their noise model.

View