[23.12.19] Lee Eun-seok speech at the 2nd Digital Sprout Conference

On December 19, 2023, Professor Lee Eun-seok of Sungkyunkwan University (Software Convergence University President/SW Central University Project Director) gave a lecture on "The role of universities in the paradigm shift of education" at the second conference in Gangnam, hosted by the Ministry of Education, 17 municipal and provincial offices of education, and the Korea Science and Creativity Foundation.

The AI Convergence Major Idiots and RMDP Team won the Gold Award and Special Award at the 1st Seoul Metropolitan Univers

AI convergence major idiots (Kim Myung-seop, Lee Soo-hyung, Cho Jung-hwan), RMDP (Kim Soo-min, Kim Hyun-seo, Jang Woo-hyuk), Won the Gold Award and Special Award at the 1st Seoul Metropolitan Universityies Artificial Intelligence (SCAICO) Combined Competition Kim Myung-seop, Lee Soo-hyung, Cho Jung-hwan, Kim Soo-min, Kim Hyun-seo, and Jang Woo-hyuk (RMDP team), who majored in artificial intelligence convergence at our university's Department of Global Convergence, won the gold prize (2nd place) and the special prize (SKT special prize) at the first Seoul National University Artificial Intelligence Coalition Competition held at Sejong University's Ocean AI Center on November 19 (Sun). The Seoul Central Artificial Intelligence Competition (S.C.A.I.C.O.) was organized by the SW-Centered University Project Group of the school, the Sejong University SW-AI-Centered University Promotion Group, the Catholic University LINC3.0 Project Group, the Catholic University of Digital Transformation ICC, the Konkuk University SW-Centered University Project Group, the Kyung Hee University SW-Centered University Project Group, and the Inha University SW-Centered University Project Group, hosted by the student council of each university's related departments and sponsored by 3M, SK Telecom and DEVOCEAN. A total of 13 teams from eight universities in the Seoul metropolitan area, including the school, participated through internal preliminary rounds for each university. SK Devocean Kim Sang-ki, Team Idiots (Artificial Intelligence Convergence Major, 22nd grade Lee Soo-hyung, 20th grade Kim Myung-seop, 22nd grade Cho Jung-hwan) The competition was held under the theme of producing achievements using AI-related keywarts (Trend Korea 2023) in 2023. The daft team that won the gold prize (2nd place) was awarded KRW 1,000,000 and the award was awarded KRW 500,000 and the award was awarded to the RMDP team that won the SKT Special Award. The grand prize (1st place) was awarded to the KHAI team of Kyung Hee University, and the silver prize (3rd place) was awarded to the brown sugar team of Sejong University. In particular, this competition is significant in that the student council of each department organized and hosted the event, leading the academic exchange.

One paper by Intelligent Embedded Systems Laboratory (supervisor: Shin Dong-gun) has been approved for publication at AA

One paper by Intelligent Embedded Systems Laboratory (supervisor: Shin Dong-gun) has been approved for publication at AAAI Conference on Artificial Intelligence 2024 (AAAI-24), an excellent academic conference in the field of artificial intelligence 논문 #1: Proxyformer: Nystrom-Based Linear Transformer with Trainable Proxy Tokens (Lee Sang-ho, master's degree in artificial intelligence, and Lee Ha-yoon, Ph.D. in electrical, electronic, and computer engineering) The paper "Proxyformer: Nystrom-Based Linear Transformer with Trainable Proxy Tokens" focuses on the complexity of self-attention operations. In this paper, to solve the quadratic complexity of the input sequence length n of the existing self-attention operation, we propose an extended Nystrom attraction method by integrating the Nystrom method with neural memory. First, by introducing a learnable proxy token, which serves as a landmark of the Nystrom method, we reduce the complexity of the attraction operation from square to linear, and effectively create landmarks that take into account the input sequence. Second, we learn to effectively restore the attraction map using minimal landmarks by applying continuous learning. Third, we develop a dropout method suitable for the decomposed attraction matrix, enabling the normalization of the proxy tokens to be effectively learned. The proposed proxyformer effectively approximates the attention map with minimal proxy token, which outperforms existing techniques in LRA benchmarks and achieves 3.8 times higher throughput and 0.08 times lower memory usage in 4096-length input sequences compared to traditional self-attention methods. [Thesis #1 Information] Proxyformer: Nystrom-Based Linear Transformer with Trainable Proxy Tokens Sangho Lee, Hayun Lee, Dongkun Shin Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI), 2024 Transformer-based models have demonstrated remarkable performance in various domains, including natural language processing, image processing and generative modeling. The most significant contributor to the successful performance of Transformer models is the self-attention mechanism, which allows for a comprehensive understanding of the interactions between tokens in the input sequence. However, there is a well-known scalability issue, the quadratic dependency of self-attention operations on the input sequence length n, making the handling of lengthy sequences challenging. To address this limitation, there has been a surge of research on efficient transformers, aiming to alleviate the quadratic dependency on the input sequence length. Among these, the Nyströmformer, which utilizes the Nyström method to decompose the attention matrix, achieves superior performance in both accuracy and throughput. However, its landmark selection exhibits redundancy, and the model incurs computational overhead when calculating the pseudo-inverse matrix. We propose a novel Nyström method-based transformer, called Proxyformer. Unlike the traditional approach of selecting landmarks from input tokens, the Proxyformer utilizes trainable neural memory, called proxy tokens, for landmarks. By integrating contrastive learning, input injection, and a specialized dropout for the decomposed matrix, Proxyformer achieves top-tier performance for long sequence tasks in the Long Range Arena benchmark.

Professor Jae-pil Heo's lab was approveed by publication of four AAAI 2024 papers

Professor Jae-pil Heo's lab was approveed by publication of four AAAI 2024 papers Four papers from the Visual Computing Laboratory (Professor: Jae-pil Heo) have been approved for publication at the AAAI Conference on Artificial Intelligence 2024 (AAAI-24), an excellent academic conference in the field of artificial intelligence. 논문 #1: “Towards Squeezing-Averse Virtual Try-On via Sequential Deformation” (Ph.D. program in the Department of Artificial Intelligence Shim Sang-heon, master's program in the Department of Artificial Intelligence Jung 논문 #2: "Noise-free Optimization in Early Training Steps for Image Super-Resolution" (Lee Min-gyu, Ph.D. in Artificial Intelligence) 논문 #3: “VLCounter: Text-aware Visual Representation for Zero-Shot Object Counting” (Kang Seung-gu, Master of Artificial Intelligence, Doctor of Artificial Intelligence, Doctor of Artificial Intelligence, and Master of Artificial Intelligence Kim Eui-yeon) 논문 #4: “Task-disruptive Background Suppression for Few-Shot Segmentation” (Park Soo-ho, Ph.D. in software/mechanical engineering, Lee Soo-bin, Ph.D. in artificial intelligence, Hyun Jin-ik, Ph.D. in artificial intelligence, and Sung Hyun-seok) The paper "Towards Squeezing-Averse Virtual Try-On via Sequential Deformation" addresses the issue of visual quality degradation in the field of high-resolution virtual mounted image generation. Specifically, there was a problem with the texture of clothes being squeezed in the sleeve, as shown in the upper row of Fig. 1(a). The main reason for this problem is the gradient collision between the total variance loss (TV) loss and the adversarial loss, which are two loss functions that are essentially used in the field. TV loss aims to separate the boundary between the sleeve and torso from the warped clothing mask, while adversarial loss aims to engage between the two. These opposite goals feed back the false slope to cascaded appearance flow estimation, resulting in sleeve compression artifacts. To address this, in this paper, we approached from the perspective of interlayer connections in the network. Specifically, it was diagnosed that sleeve compression occurs because the conventional cascading appearance flow estimation is connected with a residual connection structure and is heavily influenced by the adversarial loss function, and to reduce this, a sequential connection structure between cascading appearance flows was introduced into the last layer of the network. Meanwhile, the lower row in Figure 1(a) shows different types of compression artifacts around the waist. To address this, in this study, we propose to warp into the style first worn when warping clothes (tucked-out shirts style), and then partially delete the texture from the initial warping result, and implement the computation for this. The proposed technique confirms that both types of artifacts are successfully resolved. In the paper "Noise-free Optimization in Early Training Steps for Image Super-Resolution," we address the limitations of existing learning methodologies and Knowledge Distillation in image super-resolution problems. Specifically, one high-resolution image was separated and analyzed into two key elements, the optimal centroid and latent noise. Through this, we have confirmed that latent noise in the learning data induces instability in early learning. To address this issue, we proposed a more stable learning technique by eliminating latent noise in the learning process using Mixup technology and pre-trained networks. We have confirmed that the proposed technique brings consistent performance improvement across multiple models in the Fidelity-oriented single image super-resolution field. In the paper "VLC Counters: Text-aware Visual Representation for Zero-Shot Object Counting", we address the problem of counting the number of objects designated as text in images. This paper raised the issue of the two-stage method in previous studies with massive computational volumes and the possibility of error propagation. To solve the preceding problem, we propose one-stage baseline, VLBase, and extended VLC Counters with three main technologies. First, instead of relearning CLIP, a pre-trained giant model, we introduced Visual Prompt Tuning (VPT). Additionally, textual information is added to the learnable token of VPT so that the corresponding object gets highlighted image features. Second, fine-tuning has been made to obtain a similarity map that emphasizes only the important part of the object area, not the whole. This allows the model to increase object-centric activation. Third, in order to improve the generalization ability of the model and to accurately locate the object, we integrate the image encoder feature into the decoding and multiply the previous similarity map by the feature to focus on the object area. Not only does the proposed technique significantly exceed the performance of existing methods, but it also doubles the learning and inference speed with a lightweight model. The "Task-disruptive Background Suppression for Few-shot Segmentation" paper addresses how to efficiently address the background of Support in a new-shot segmentation problem, which refers to a small number of images (Support) and masks to find objects in new images (Query).