Research Experience Undergraduate

Aakash Lilaramani

PhD Student

Zhiming Zou

Dian Jia

MS Candidate

Raghuraman Venkatesh

Sarthak Agarwal

Xiaoqian Ruan

Visiting Scholar

Kun Xia

Hyeonjeong Park

The National Science Foundation has awarded EVL a Major Research Instrumentation (MRI) grant to build the Data Observation and Computation Collaboratory (DOCC) in the new Computer Design Research and Learning Center (CDRBC) building. The Data Observation and Computation Collaboratory (DOCC, &rsquo;the dock&rsquo;) is a much-needed visual data science instrument for UIC science and engineering research and research training. DOCC provides high-performance computing, advanced networking, high-speed data storage, interactive high-resolution displays and software and services for Artificial Intelligence / Machine Learning (AI/ML), data analytics, data simulations, visualization, and virtual-reality requirements. Given how varied modern data-science application requirements can be, DOCC is designed as a configurable, heterogeneous computing environment that prioritizes real-time, interactive, collaborative applications and remote data streaming, but can be scheduled off-hours for large, batch-mode processing. 
DOCC will empower users to work with their data and their colleagues in real-time, test codes, view a &rsquo;window in the oven&rsquo;to watch large simulations run, and simultaneously view results along with related data and ensemble runs on large display walls. DOCC has a visual front end that enables users to easily access familiar collaboration tools, scientific notebooks, AI/ML workflows, and popular visualization software. 
Alternatively, users can add new AI/ML models or create them from scratch. And because DOCC will be connected to the campus research network, data and/or computer codes can be ported to remote computer and cloud resources located on campus, regionally, nationally, or internationally. 
A room-sized instrument, DOCC will be located in the high-trafficked atrium of the new CDRLC, home to the UIC Computer Science Department, part of the College of Engineering, situated across from an auditorium and multi-purpose event space. This location will promote UIC data-science research initiatives and the instrument&rsquo;s collaboration potential. DOCC will also be used to conduct campus research meetings, programs and training initiatives. 
Award Number (FAIN): 2320261 
Period of Performance: 10/01/2023 - 09/30/2026 
Project Title: Research Infrastructure: MRI: Track 2 Acquisition of Data Observation and Computation Collaboratory (DOCC)

EVL Awarded NSF MRI Support to Build the Data Observation and Computation Collaboratory (DOCC)

Roberto Cesar Jr., professor/department head of Computer Science at the University of Sao Paolo and director of eScience Research Center presents his research in computer vision and image processing for urban visualization. Dr. Cesar&rsquo; invited talk will be held: 
EVL&rsquo;s Continuum 
Engineering Research Facility, Room 2068 
Friday, September 29th, noon 
&ldquo;Computer vision and physical models for urban informatics&rdquo; 
Computer vision techniques have played an important role in urban data analysis. Such techniques are useful for capturing and analyzing visual information about cities&rsquo; infrastructure, social manifestations and mobility, to name just a few. However, the adoption of computer vision techniques in urban environments faces important challenges, such as the restricted heterogeneous hardware typically involved, outdoor image variations, and other uncontrolled aspects. In this talk, we present some recent results to overcome these limitations. Specifically, we discuss computer vision techniques for rain scenes, pedestrian mobility analysis, and physical models for urban green regions and graffiti distributions. 
Bio: 
Roberto is a professor at the University of Sao Paulo (USP) since 1998 (BSc in Computer Science - UNESP - 1991; MSc in Electrical Engineering - UNICAMP - 1993; Ph.D. in Physics - USP/Brazil, IPT-UCL/Belgium - 1997). He is currently a full professor in the Department of Computer Science - IME - USP working in the Data Science Research Group. He is currently a special advisor for Physical Sciences and Engineering at the Sao Paulo Research Foundation - FAPESP. He served as the Director of the eScience Research Center at USP and as the head of the Computer Science Department. He was a member of the Image and Vision Computing and the Signal, Image and Video Processing editorial boards, chair and invited speaker of conferences and workshops (Sibgrapi 2003, CIARP 2010, Sibgrapi 2011; SHAPES 2.0 - 2012, eSon - IEEE eScience 2013, IEEE eScience 2014). He has experience in computer science, with an emphasis on computer vision, machine learning, and artificial intelligence. 
<a href="https://www.ime.usp.br/~cesar/">https://www.ime.usp.br/~cesar/</a>

Roberto Cesar Jr. Invited to Present &ldquo;Computer vision and physical models for urban informatics&rdquo;

Unsupervised semantic segmentation (USS) aims at partitioning an image into semantically meaningful segments by learning from a collection of unlabeled images. The effectiveness of current approaches is plagued by difficulties in coordinating representation learning and pixel clustering, modeling the varying feature distributions of different classes, handling outliers and noise, and addressing the pixel class imbalance problem. This paper introduces a novel approach, termed Imbalance-Aware Dense Discriminative Clustering (IDDC), for USS, which addresses all these difficulties in a unified framework. Different from existing approaches, which learn USS in two stages (i.e., generating and updating pseudo masks, or refining and clustering embeddings), IDDC learns pixel-wise feature representation and dense discriminative clustering in an end-to-end and self-supervised manner, through a novel objective function that transfers the manifold structure of pixels in the embedding space of a vision Transformer (ViT) to the label space while tolerating the noise in pixel affinities. During inference, the trained model directly outputs the classification probability of each pixel conditioned on the image. In addition, this paper proposes a new regularizer, based on the Weibull function, to handle pixel class imbalance and cluster degeneration in a single shot. Experimental results demonstrate that IDDC significantly outperforms all previous USS methods on three real-world datasets, COCO-Stuff-27, COCO-Stuff-171, and Cityscapes. Extensive ablation studies validate the effectiveness of each design. Our code is available at https://github.com/MY-LIU100101/IDDC. Our code is available at https://github.com/MY-LIU100101/IDDC. 
Keywords: Unsupervised semantic segmentation · Imbalance-Aware Dense Discriminative Clustering · End-to-end training ·
Deep clustering 
Funding: The COMPaaS DLV project (NSF award CNS-1828265)

Imbalance-Aware Discriminative Clustering for Unsupervised Semantic Segmentation

Though the object detection performance on standard benchmarks has been improved drastically in the last decade, current object detectors are often vulnerable to domain shift between the training data and testing images. Domain adaptation techniques have been developed to adapt an object detector trained in a source domain to a target domain. However, they assume that the target domain is known and fixed and that a target dataset is available for training, which cannot be satisfied in many real-world applications. To close this gap, this paper investigates fully test-time adaptation for object detection. It means to update a trained object detector on a single testing image before making a prediction, without access to the training data. Through a diagnostic study of a baseline self-training framework, we show that a great challenge of this task is the unreliability of pseudo labels caused by domain shift. We then propose a simple yet effective method, termed the IoU Filter, to address this challenge. It consists of two new IoU-based indicators, both of which are complementary to the detection confidence. Experimental results on five datasets demonstrate that our approach could effectively adapt a trained detector to various kinds of domain shifts at test time and bring substantial performance gains. Code is available at https://github.com/XiaoqianRuan1/IoU-filter. Code is available at https://github.com/ XiaoqianRuan1/IoU-filter. 
Funding:: The COMPaaS DLV project (NSF award CNS-1828265)

Fully Test-time Adaptation for Object Detection

Semi-Supervised Temporal Action Localization (SS-TAL) aims to improve the generalization ability of action detectors with large-scale unlabeled videos. Albeit the recent advancement, one of the major challenges still remains: noisy pseudo labels hinder efficient learning on abundant unlabeled videos, embodied as location biases and category errors. In this paper, we dive deep into such an important but understudied dilemma. To this end, we propose a unified framework, termed Noisy Pseudo-Label Learning, to handle both location biases and category errors. Specifically, our method is featured with (1) Noisy Label Ranking to rank pseudo labels based on the semantic confidence and boundary reliability, (2) Noisy Label Filtering to address the class-imbalance problem of pseudo labels caused by category errors, (3) Noisy Label Learning to penalize inconsistent boundary predictions to achieve noise-tolerant learning for heavy location biases. As a result, our method could effectively handle the label noise problem and improve the utilization of a large amount of unlabeled videos. Extensive experiments on THUMOS14 and ActivityNet v1.3 demonstrate the effectiveness of our method. The code is available at github.com/kunnxia/NPL.

Learning from noisy pseudo labels for semi-supervised temporal action localization

Estimating the human posture from an image or a video is a fundamental task in computer vision. It not only enhances other vision tasks like action recognition, person re-identification, and virtual try-on but also facilitates many real-world applications including robotics, healthcare, sports, and retail. An effective and efficient human pose estimation system can enable robots to acquire skills from demonstrations, support physical therapists in diagnosing and rehabilitating patients, assist sports analysts and coaches in tracking and training athletes, and empower retailers to establish employee-free stores. Thanks to the advancement of deep learning and the availability of large-scale datasets, the performance of state-of-the-art human pose estimation methods has drastically improved in recent years. These approaches can accurately estimate postures in daily activities and sports. However, several challenges persist. For example, (1) it remains challenging to estimate postures that occur rarely or are entirely absent in the training data; (2) handling complex scenarios, such as crowded environments, motion blur, low-light conditions, and occlusions, is still a formidable task; (3) there is a growing need to develop efficient models that can estimate human poses in real-time or on low-power devices; (4) exploring novel applications for human pose estimation that can bring societal benefits or transform industries is an exciting avenue of research. This special issue presents a collection of high-quality papers on human pose estimation and its applications. They cover a wide range of topics, from fundamental human pose models with improved accuracy and efficiency to applications in human behavior understanding and assessment, sign language recognition, and multi-camera calibration. Through this special issue, we would like to open the path toward more discussion between practitioners and researchers in this very important yet challenging field of human pose estimation.

Guest editorial: special issue on human pose estimation and its applications

Weakly-supervised Video Anomaly Detection (WVAD) aims to detect abnormal events in videos given only video-level labels for training. Recent methods relying on multiple instance learning (MIL) and self-training achieve good performance, but they tend to focus on learning easy abnormal patterns while ignoring hard ones, e.g., unusual driving trajectory or over-speeding driving. How to detect hard anomalies is a critical but largely ignored problem in W-VAD. To tackle this challenge, we propose a novel framework, termed Abnormal Ratios guided Multi-phase Self-training (ARMS), for W-VAD. It includes a new abnormal ratio-based MIL (AR-MIL) loss and a new multi-phase self-training paradigm. The AR-MIL loss guides the learning of hard anomalies by enforcing a minimum ratio of abnormal snippets in an abnormal video and no abnormal snippets in a normal video. Our multi-phase self-training paradigm sequentially performs bootstrapping, hard anomalies mining, and adaptive self-training so as to address pseudo labeling on easy anomalies, detect hard anomalies, and setting adaptive abnormal ratios for different videos in a unified framework. Experimental results on three benchmark datasets, i.e., ShanghaiTech, UCF-Crime, and XD-Violence, show that ARMS outperforms all previous state-of-the-art methods and has a great advantage in detecting hard anomalies. 
Index Terms: Anomaly detection, weakly-supervised video anomaly detection, multiple instance learning.

Abnormal Ratios Guided Multi-Phase Self-Training for Weakly-Supervised Video Anomaly Detection

Developing sensing circuits and systems capable of perceiving the physical environment is an essential step toward next-generation cyber-physical systems that leverage computation, communication, and sensing integrated with the physical domain. The recent introduction of a class of artificial intelligence (AI) methods known as deep learning has drastically advanced vision-based environment perception. However, the applicability of these methods is significantly limited by their reliance on vast amounts of 3D annotations for training, which are expensive, unnatural, and often unobtainable. Acquiring 3D annotations necessitates specialized data captured by 3D sensors rather than utilizing abundant and readily available images or videos. Moreover, it requires a controlled environment or compromises with inaccurate subjective labeling. This project aims to address real-time environment perception in a very challenging, largely unexplored, yet highly practical setting, where only 2D annotations are available during training, by modeling and learning rich space-time structures of the environment. The outcomes of this project will significantly impact cyber-physical systems and facilitate a wide range of applications, from robots in manufacturing and personal services to autonomous vehicles that enhance people’s mobility and safety. Furthermore, this project will tightly integrate research and education at the University of Illinois Chicago (UIC), which is a Minority Serving Institution (MSI), through curriculum development, research training for high school, undergraduate, and graduate students, broadening the participation of female and minority students, and community outreach. 
This project will address real-time environment perception when only 2D annotations are available during training by systematically pursuing a novel approach that decomposes the complex physical environment into geometry and motion substructures in 3D space and models their rich space-time interactions. From streaming video, the trained system will not only recognize objects and scene layouts in the environment but also estimate their 3D geometry and 3D motion in real-time. This project is focused on (1) establishing a self-supervised framework for lifting 2D objects to 3D, by bridging geometry and motion, brightness constancy, and differentiable depth rendering, (2) scaling 2D-to-3D lifting of individual objects to perceive the entire environment, by modeling and learning short-term and long-term dynamics of the environment, and (3) achieving flexible and generalizable environment perception by handling articulated motion and out-of-distribution environments. The new approach is expected to be efficient in terms of the amount of annotations required for training, scalable to a wide range of objects, and robust to the complexity and diversity of real-world environments. This project will advance and enrich the fundamental research of visual sensing, perception, and learning. Moreover, this project will demonstrate the usability and robustness of the proposed approach in real cyber-physical systems.

CCSS: Modeling and Learning Space-time Structures for Real-time Environment Perception

EVL affiliated faculty and assistant professor of Computer Science Wei Tang has been appointed by Science Direct as associate editor of the journal &ldquo;Pattern Recognition&rdquo;.  The journal publishes developments in the fields of computer vision, image processing, text and document analysis, and neural networks. Dr. Tang&rsquo;s expertise in these fields makes him the perfect candidate for this position.  We congratulate Dr. Tang on his appointment.

Computer Vision

Bibliography

pubs (5)

Related Topics

people (8)

events (3)

research (1)

Contact