题目：Customizing personal large-scale language model using co-occurrence statistic information

摘要：In text generation, a large language model (LLM) makes a choice of each new word based only on the former selection of its context using the softmax function. Nevertheless, the link statistics information of concurrent words based on a scene-specific corpus is valuable in choosing the next word, which can help to match the topic of generated text with the current task. To fully explore such important information, we propose a graphsoftmax function for task-specific text generation. It is expected that the final word choice would be determined by both the global knowledge from the LLM and the local knowledge from the scene-specific corpus. To achieve this goal, we regularize the traditional softmax function with a graph total variation, which incorporates the local knowledge into the LLM. The proposed graphsoftmax can be plugged into a large pre-trained LLM for text generation and machine translation.Through experiments, we demonstrate that the new GTV-based regularization yields better performances in comparison with existing methods. Human testers can also easily distinguish the text generated by the graphsoftmax or softmax.

简介：刘斌，本科硕士博士分别就读于辽宁工业大学信息与计算科学，电子科技大学软件工程和电子科技大学计算机软件与理论，并在英属哥伦比亚大学进行博士联合培养年，香港大学博士后，于2018年加入西南财经大学。研究兴趣为机器 学习和数据挖掘。

题目：Robust Structure Learning And L_p-Regularization For Graph Neural Networks

摘要：Graph neural networks (GNNs) have become one of the most important branches in various deep learning, due to their remarkable power in learning with graph-structured data. Our current report consists of two folds.  First, we provide a lower bound of Rademacher complexity for two-layer GCNs, which motivates us to formulate the proposed robust algorithm for recovering graph structure and learning tasks in GCNs. Second, we also aims at quantifying the trade off of GCN between smoothness and sparsity, with the help of a new L_p-regularized (1 < p ≤ 2) stochastic learning proposed in the work. For a single-layer GCN, we develop an explicit theoretical understanding of GCN with the L_p-regularized stochastic learning by analyzing the stability of our regularized stochastic algorithm. Finally, several empirical experiments are implemented to validate our theoretical findings.

简介：吕绍高，南京审计大学统计与数据科学学院教授，主要研究方向为统计机器学习与数据挖掘，尤其关注分布式学习、深度学习以及强化学习的理论基础与算法设计。

题目：面向社交平台内容挖掘的多模态深度学习技术

摘要：随着知乎、微博、Twitter等社交平台的普遍流行，人们习惯于在社交平台进行分享。针对社交平台中的海量数据进行实时分析，有助于在舆情监控、犯罪检测、灾害防护、情感分析等方面有效提升相关职能部门的管理效率。由于多媒体技术和移动互联网技术的快速发展，社交平台近年来呈现出多模态的特性。除了传统的文字信息，视频、图片等内容在社交平台上普遍存在。为了提升对社交平台的分析质量，多模态深度学习近年来被广泛研究。通过整合语音、图像、语言等不同模态的信息，多模态深度学习能够显著提升对多模态互联网内容的识别能力。本报告将围绕多模态深度学习，介绍其相关技术原理和发展现状，并探讨大模型时代的多模态深度学习研究。

简介：Fengmao Lv is currently an Associate Professor at the School of Computing and Artificial Intelligence, Southwest Jiaotong University, China. My research interests are in multimodal deep learning, transfer learning and their applications in computer vision, natural language processing and social network analysis.He has openings for self-motivated undergraduate students, master students and PhD students (co-supervised). Feel free to contact me if you are interested in my research area

题目：Supervised Random Feature Regression via Projection Pursuit

摘要：Random feature methods and neural network models are two popular nonparametric modeling methods, which are regarded as representatives of shallow learning and Neural Network, respectively. In practice random feature methods are short of the capacity of feature learning, while neural network methods lead to computationally heavy problems. This paper aims at proposing a flexible but computational efficient method for general nonparametric problems. Precisely, our proposed method is a feed-forward two-layer nonparametric estimation, and the first layer is used to learn a series of univariate basis functions for each projection variable, and then search for their optimal linear combination for each group of these learnt functions. Based on all the features derived in the first layer, the second layer attempts at learning a single index function with an unknown activation function. Our nonparametric estimation takes advantage of both random features and neural networks, and can be seen as an intermediate bridge between them.

简介：周井然,西南财经大学统计学院博士研究生。本科毕业于加州大学圣塔巴巴分校数学系，研究生毕业于卡迪夫大学数据科学及分析专业。目前研究方向为机器学习，有限数据量下数据的预测和解释性，及变量间的因果和泛化能力的研究。