The semantic segmentation of drone LiDAR data is important in intelligent industrial operation and maintenance. However, current methods are not effective in directly processing airborne true-color point clouds that contain geometric and color noise. To overcome this challenge, we propose a novel hybrid learning framework, named SSGAM-Net, which combines supervised and semi-supervised modules for segmenting objects from airborne noisy point clouds. To the best of our knowledge, we are the first to build a true-color industrial point cloud dataset, which is obtained by drones and covers 90,000 m2. Secondly, we propose a plug-and-play module, named the Global Adjacency Matrix (GAM), which utilizes only few labeled data to generate the pseudo-labels and guide the network to learn spatial relationships between objects in semi-supervised settings. Finally, we build our point cloud semantic segmentation network, SSGAM-Net, which combines a semi-supervised GAM module and a supervised Encoder–Decoder module. To evaluate the performance of our proposed method, we conduct experiments to compare our SSGAM-Net with existing advanced methods on our expert-labeled dataset. The experimental results show that our SSGAM-Net outperforms the current advanced methods, reaching 85.3% in mIoU, which ranges from 4.2 to 58.0% higher than other methods, achieving a competitive level.

• PAPER
• Cite
• Code
• Project

Regular inspections and timely maintenance of the over-head electric power facilities are crucial to national security. As pioneers of intelligent maintenance with UAV based inspection, we have gained insight into the industrial revolution and find the trend of the technology development from IMLocal( Intelligent Maintenance by the local application) to IMCloud(Intelligent Maintenance by Cloud). IMCloud architecture is reported with several shortcomings. In this paper, a next-generation intelligent maintenance system - IMEdge(Intelligent Maintenance using edge-cloud collaboration) is proposed to overcome the shortcomings by 4 primary architectures: edge cloud security, autonomous inspection, collaborative inferencing, and edge collaborations between robots. Further, 3 general latent collaboration cores hidden in the architectures are found in the IMEdge. A vast range of edge- edge collaboration is constructed taking the advantages of the collaboration cores for the electric power industry. Finally, a cross-industry architecture is proposed based on the IMEdge.

• PAPER
• Cite
• Code
• Project

无人机在各种工业基础设施的自主巡检中，需要为大型复杂待巡检对象设计大量的数据采集航点，以获取高质量的全覆盖影像数据。逐点飞行作业能耗高采集效率低，限制了无人机大规模的工业应用。为了优化飞行作业模式，有效提升巡检效率，针对已知待巡检复杂设备结构提出一种全新的多向视点规划算法——连续可微采样的无人机多向视点规划算法(MD-VPP)。该方法在满足全面覆盖巡视目标以及数据采集质量要求的前提下，大幅减少航点数量。首先，将云台俯仰角、偏航角和相机成像面积作为约束条件，构建视点采集连续可微的视图质量目标函数，优化求解候选视点集；然后利用贪心算法求解全覆盖待巡检对象的航点位姿。通过实验对比分析了针对不同巡检对象的航点数量和航点数减少率，相比领域内其他优秀方法，所提方法在确保全覆盖的前提下航点数量至少降低77%,极大地提升了无人机单次作业的数据采集效率。

• PAPER
• Cite
• Code
• Project

输电杆塔关键弱纹理部件的通用检测方法依赖大量样本的标注和学习。在无相关部件样本训练情况下,本文提出一种可迁移的输电杆塔弱纹理部件检测方法。本文方法结合了孪生神经网络和互相关卷积用于融合样本块与待搜索区域的特征,其中通过样本部件掩膜裁剪以有效滤除背景噪声,最后在尺度、位置、交并比3个方面提出了对应的分数修正策略以提高检测精度。实验结果表明,本文提出的掩膜裁剪和分数修正策略有效提高了弱纹理部件的检出精度,按照AP50的标准,本文检测方法的平均准确率达到了98.0%,能有效避免由于观测视角、尺度以及光照带来的干扰。

• PAPER
• Cite
• Code
• Project

Using unmanned aerial vehicles (UAVs) for performing automatic inspection of overhead power lines instead of foot patrols is an attractive option, since doing so is safer and have considerable cost savings, among other advantages. The purpose of this paper is to design a 3D laboratory test-platform to simulate UAVs' inspection of transmission lines and secondly, proposing an automated inspection strategy for UAVs in order to follow transmission lines. The construction and system architecture of our 3D test-platform is described in this paper. The inspection strategy contributes to knowledge pertaining to an automated inspection procedure and includes two steps: flight path planning for UAVs and visual tracking of the transmission lines. The 3D laboratory test-platform is applied to test the performance of the proposed strategy and the tracking results of our inspection strategy are subsequently presented. The availability of the 3D laboratory test-platform and the efficiency of our tracking algorithm are verified by experiments.

• PAPER
• Cite
• Code
• Project

This paper presents a novel approach to the initialization of an ego-motion estimation technique for autonomous power line inspection. Dual channel vision, consisting of an infrared and optical camera, is typically adopted during inspection. The infrared camera is far more proficient at reliably detecting heated regions of the power tower which can be regarded as a prior relationship between the tower and cameras. Using the infrared camera, which is equipped parallel to the optical camera, an incomplete correspondence between the optical image and a 3D CAD model is established. Depending on the degree of correspondence, the initial pose of the CAD model in the optical image is estimated through two stages of coarse-to-fine estimation. The primary contributions of this paper include: 1) using dual vision for partial initialization; 2) incorporating two-stage algorithms to estimate an accurate pose quickly; 3) implementing an algorithm which functions correctly regardless of the motion blur or background texture. Experimental results consistently show that the initial pose can be estimated efficiently and robustly

• PAPER
• Cite
• Code
• Project

Localisation and mapping with an omnidirectional camera becomes more difficult as the landmark appearances change dramatically in the omnidirectional image. With conventional techniques, it is difficult to match the features of the landmark with the template. We present a novel robot simultaneous localisation and mapping (SLAM) algorithm with an omnidirectional camera, which uses incremental landmark appearance learning to provide posterior probability distribution for estimating the robot pose under a particle filtering framework. The major contribution of our work is to represent the posterior estimation of the robot pose by incremental probabilistic principal component analysis, which can be naturally incorporated into the particle filtering algorithm for robot SLAM. Moreover, the innovative method of this article allows the adoption of the severe distorted landmark appearances viewed with omnidirectional camera for robot SLAM. The experimental results demonstrate that the localisation error is less than 1 cm in an indoor environment using five landmarks, and the location of the landmark appearances can be estimated within 5 pixels deviation from the ground truth in the omnidirectional image at a fairly fast speed.

• PAPER
• Cite
• Code
• Project

TThis paper presents a method to derive optimal viewpoints for flying robot based transmission tower inspection, which applies point cloud model. A safe envelope is established according to safe transmission tower inspection rules. Essential inspection factors are proposed to evaluate the quality of candidate viewpoints, which includes visibility, dimensionality feature of point cloud as well as the distance between viewpoint and transmission power. A score function is constructed to quantify candidate viewpoints, while multiple attribute decision theory is applied to calculate the weight of each factor. Particle swarm optimization (PSO) is used to find the optimal viewpoints set. Both simulation experiment and practical observations are carried out. The results prove that optimal viewpoints have a great contribution for accurate transmission tower inspection. Final results are compared to patch-based method and proved to be feasible.

• PAPER
• Cite
• Code
• Project

Background/Introduction Robot localization can be considered as a cognition process that takes place during a robot estimating metric coordinates with vision. It provides a natural method for revealing the true autonomy of robots. In this paper, a kernel principal component analysis (PCA)-regularized least-square algorithm for robot localization with uncalibrated monocular visual information is presented. Our system is the first to use a manifold regularization strategy in robot localization, which achieves real-time localization using a harmonic function. Methods The core idea is to incorporate labelled and unlabelled observation data in offline training to generate a regression model smoothed by the intrinsic manifold embedded in area feature vectors. The harmonic function is employed to solve the online localization of new observations. Our key contributions include semi-supervised learning techniques for robot localization, the discovery and use of the visual manifold learned by kernel PCA and some solutions for simultaneous parameter selection. This simultaneous localization and mapping (SLAM) system combines dimension reduction methods, manifold regularization techniques and parameter selection to provide a paradigm of SLAM having self-contained theoretical foundations. Results and Conclusions In extensive experiments, we evaluate the localization errors from the perspective of reducing implementation and application difficulties in feature selection and magnitude ratio determination of labelled and unlabelled data. Then, a nonlinear optimization algorithm is adopted for simultaneous parameter selection. Our online localization algorithm outperformed the state-of-the-art appearance-based SLAM algorithms at a processing rate of 30 Hz for new data on a standard PC with a camera.

• PAPER
• Cite
• Code
• Project

The mass and stiffness of cantilever are neglected; thus, the robot and load can be considered as point masses and are assumed to move in a two-dimensional, x-y plane. The coordinates of the trolley and load are (x M , y M ) and (x m , y m ). Triangular membership functions are chosen for the derivative of the robot position error, swing angle, the derivative of the swing angle and force input. A normalized universe of discourse is used for the three inputs and one output of the fuzzy logic controller. Scaling factors k 1, k 2 and k 3 are chosen to convert the three inputs of the system and activate the rule base effectively, while k 4 is selected to activate the system to generate the desired output. To simplify a rule base, the derivative of robot position error, swing angle, the derivative of swing angle and force are partitioned into five primary fuzzy sets as follows:

• PAPER
• Cite
• Code
• Project

To localize inspection robot in an in-service substation costs much. How to discriminate its location among highly similar scenes is the main problem of localizing robot in the substation using vision. A novel approach for visual localization using image retrieval and multi-view geometry is proposed. It is applicable for autonomous inspection of in-service substation without additional modifications of the environment. The experimental results demonstrate the efficiency and reliability of our approach. They have been further discussed by means of parameters for image description, number of neighbouring images for coordinates estimation, training dataset selection and performance evaluation. They verified that our approach is a cost-effective solution to robot localization in in-service substation.

• PAPER
• Cite
• Code
• Project

In recent years, the UAV (Unmanned Aerial Vehicle) inspection for the electrical line has received increasing attentions due to the advantages of low costs, easiness to control and flexibility. The UAV can inspect the electrical tower independently and automatically by planning the flight path. But during the inspection along the path, the UAV is easily impacted by gust wind due to its light weight and small size, which always leads to the crash into the electrical tower. Thus, in this paper, a safe flight approach (SFA) is proposed to make the flight be safer during the inspection. The main contributions include: firstly, the piecewise linear interpolation method is proposed to fit the distribution curve of the electrical towers based on the GPS coordinates of the electrical towers; secondly, the no-fly zone on the both sides of the distribution curve are created, and a security distance formula (SDF) is raised to decide the width of the no-fly zone; thirdly, a gust wind formula (GWF) is proposed to improve the artificial potential field approach, which can contribute to the path planning of the UAV; finally, a flight path of the UAV can be planned using the SFA to make the UAV avoid colliding with the electric tower. The proposed approach is tested on the experiment to demonstrate its effectiveness.

• PAPER
• Cite
• Code
• Project

Combustion systems need to be operated under a range of different conditions to meet fluctuating energy demands. Reliable monitoring of the combustion process is crucial for combustion control and optimisation under such variable conditions. In this paper, a monitoring method for variable combustion conditions is proposed by combining digital imaging...

• PAPER
• Cite
• Code
• Project

Experimental investigations into the combustion behaviors of single pulverized coal particles are carried out based on high-speed imaging and image processing techniques. A high-speed video camera is employed to acquire the images of coal particles during their residence time in a visual drop tube furnace. Computer algorithms are developed to determine the characteristic parameters of the particles from the images extracted from the videos obtained. The parameters are used to quantify the combustion behaviors of the burning particle in terms of its size, shape, surface roughness, rotation frequency and luminosity. Two sets of samples of the same coal with different particle sizes are studied using the techniques developed. Experimental results show that the coal with different particle sizes exhibits distinctly different combustion behaviors. In particular, for the large coal particle (150–212 μm), the combustion of volatiles and char takes place sequentially with clear fragmentation at the early stage of the char combustion. For the small coal particle (106–150 μm), however, the combustion of volatiles and char occurs simultaneously with no clear fragmentation. The size of the two burning particles shows a decreasing trend with periodic variation attributed to the rapid rotations of the particles. The small particle rotates at a frequency of around 30 Hz, in comparison to 20 Hz for the large particle due to a greater combustion rate. The luminous intensity of the large particle shows two peaks, which is attributed to the sequential combustion of volatiles and char. The luminous intensity of the small particle illustrates a monotonously decreasing trend, suggesting again a simultaneous devolatilization/volatile and char combustion.

• PAPER
• Cite
• Code
• Project

Given a large set of unlabeled images taken by drones for power transmission line inspection, how can we efficiently and cost-friendly develop and deploy defect detection models with high accuracy? To date, power transmission line inspection is increasingly based on drones and there is an expectation of over 100K images taken daily in 2020. These images, in current practice, are generally inspected by inspection engineers. In this paper, we present LeapDetect, an agile platform for rapidly developing, deploying and iteratively upgrading detection models for drone-based power transmission line inspection. The platform consists of a labeling system, an online detection service system, and a training system to cover a complete life cycle of a new coming detection task. It supports building models from a cold start or by an upgrade. LeapDetect has helped us to develop a couple of object detection models for power transmission line inspection. We have demonstrated how LeapDetect helps develop and iteratively upgrade models for power transmission line inspection with two case studies (i.e., bird nest and split pin).

• PAPER
• Cite
• Code
• Project

Accurate State of Charge (SOC) and State of Health (SOH) estimation is crucial to ensure safe and reliable operation of battery systems. Considering the intrinsic couplings between SOC and SOH, a joint estimation framework is preferred in real-life applications where batteries degrade over time. Yet, it faces a few challenges such as limited measurements of key parameters such as strain and temperature distributions, difficult extraction of suitable features for modeling, and uncertainties arising from both the measurements and models. To address these challenges, this paper first uses Fiber Bragg Grating (FBG) sensors to obtain more process related signals by attaching them to the cell surface to capture multi-point strain and temperature variation signals due to battery charging/discharging operations. Then a hybrid machine learning framework for joint estimation of SOC and capacity (a key indicator of SOH) is developed, which uses a convolutional neural network combined with the Gaussian Process Regression method to produce both mean and variance information of the state estimates, and the joint estimation accuracy is improved by automatic extraction of useful features from the enriched measurements assisted with FBG sensors. The test results verify that the accuracy and reliability of the SOC estimation can be significantly improved by updating the capacity estimation and utilizing the FBG measurements, achieving up to 85.58% error reduction and 42.7% reduction of the estimation standard deviation.

• PAPER
• Cite
• Code
• Project

Online battery capacity estimation is a critical task for battery management system to maintain the battery performance and cycling life in electric vehicles and grid energy storage applications. Convolutional Neural Networks, which have shown great potentials in battery capacity estimation, have thousands of parameters to be optimized and demand a substantial number of battery aging data for training. However, these parameters require massive memory storage while collecting a large volume of aging data is time-consuming and costly in real-world applications. To tackle these challenges, this paper proposes a novel framework incorporating the concepts of transfer learning and network pruning to build compact Convolutional Neural Network models on a relatively small dataset with improved estimation performance. First, through the transfer learning technique, the Convolutional Neural Network model pre-trained on a large battery dataset is transferred to a small dataset of the targeted battery to improve the estimation accuracy. Then a contribution-based neuron selection method is proposed to prune the transferred model using a fast recursive algorithm, which reduces the size and computational complexity of the model while maintaining its performance. The proposed model is capable of achieving fast online capacity estimation at any time, and its effectiveness is verified on a target dataset collected from four Lithium iron phosphate battery cells, and the performance is compared with other Convolutional Neural Network models. The test results confirm that the proposed model outperforms other models in terms of accuracy and computational efficiency, achieving up to 68.34% model size reduction and 80.97% computation savings.

• PAPER
• Cite
• Code
• Project

Lithium-ion batteries have been widely used in electric vehicles, smart grids and many other applications as energy storage devices, for which the aging assessment is crucial to guarantee their safe and reliable operation. The battery capacity is a popular indicator for assessing the battery aging, however, its accurate estimation is challenging due to a range of time-varying situation-dependent internal and external factors. Traditional simplified models and machine learning tools are difficult to capture these characteristics. As a class of deep neural networks, the convolutional neural network (CNN) is powerful to capture hidden information from a huge amount of input data, making it an ideal tool for battery capacity estimation. This paper proposes a CNN-based battery capacity estimation method, which can accurately estimate the battery capacity using limited available measurements, without resorting to other offline information. Further, the proposed method only requires partial charging segment of voltage, current and temperature curves, making it possible to achieve fast online health monitoring. The partial charging curves have a fixed length of 225 consecutive points and a flexible starting point, thereby short-term charging data of the battery charged from any initial state-of-charge can be used to produce accurate capacity estimation. To employ CNN for capacity estimation using partial charging curves is however not trivial, this paper presents a comprehensive approach covering time series-to-image transformation, data segmentation, and CNN configuration. The CNN-based method is applied to two battery degradation datasets and achieves root mean square errors (RMSEs) of less than 0.0279 Ah (2.54%) and 0.0217 Ah (2.93% ), respectively, outperforming existing machine learning methods.

• PAPER
• Cite
• Code
• Project

Convolutional neural networks have made significant progresses in edge detection by progressively exploring the context and semantic features. However, local details are gradually suppressed with the enlarging of receptive fields. Recently, vision transformer has shown excellent capability in capturing long-range dependencies. Inspired by this, we propose a novel transformer-based edge detector, \emph{Edge Detection TransformER (EDTER)}, to extract clear and crisp object boundaries and meaningful edges by exploiting the full image context information and detailed local cues simultaneously. EDTER works in two stages. In Stage I, a global transformer encoder is used to capture long-range global context on coarse-grained image patches. Then in Stage II, a local transformer encoder works on fine-grained patches to excavate the short-range local cues. Each transformer encoder is followed by an elaborately designed Bi-directional Multi-Level Aggregation decoder to achieve high-resolution features. Finally, the global context and local cues are combined by a Feature Fusion Module and fed into a decision head for edge prediction. Extensive experiments on BSDS500, NYUDv2, and Multicue demonstrate the superiority of EDTER in comparison with state-of-the-arts.

• PAPER
• Cite
• Code
• Project

Weakly-supervised semantic image segmentation suffers from lacking accurate pixel-level annotations. In this paper, we propose a novel graph convolutional network-based method, called GraphNet, to learn pixel-wise labels from weak annotations. Firstly, we construct a graph on the superpixels of a training image by combining the low-level spatial relation and high-level semantic content. Meanwhile, scribble or bounding box annotations are embedded into the graph, respectively. Then, GraphNet takes the graph as input and learns to predict high-confidence pseudo image masks by a convolutional network operating directly on graphs. At last, a segmentation network is trained supervised by these pseudo image masks. We comprehensively conduct experiments on the PASCAL VOC 2012 and PASCAL-CONTEXT segmentation benchmarks. Experimental results demonstrate that GraphNet is effective to predict the pixel labels with scribble or bounding box annotations. The proposed framework yields state-of-the-art results in the community.

• PAPER
• Cite
• Code
• Project

This paper presents a method to derive optimal viewpoints for flying robot based transmission tower inspection, which applies point cloud model. A safe envelope is established according to safe transmission tower inspection rules. Essential inspection factors are proposed to evaluate the quality of candidate viewpoints, which includes visibility, dimensionality feature of point cloud as well as the distance between viewpoint and transmission power. A score function is constructed to quantify candidate viewpoints, while multiple attribute decision theory is applied to calculate the weight of each factor. Particle swarm optimization(PSO) is used to find the optimal viewpoints set. Both simulation experiment and practical observations are carried out. The results prove that optimal viewpoints have a great contribution for accurate transmission tower inspection. Final results are compared to patch-based method and proved to be feasible.

• PAPER
• Cite
• Code
• Project

This article puts forward a novel idea of power tower inspection depending on model-based behavior planning. It mainly focuses on the problems caused by uncertain security, flight time limitation and stochastic noise affection when inspecting with flying robot. Firstly, we construct a safety space and some target viewing regions based on the model of the power tower. Then a reinforcement learning procedure is adopted to find an optimal policy of guiding the inspection behavior. Experimental results show that the model-based behavior planning improves the efficiency of the inspection significantly even with the wind gusts or stochastic interferences.

• PAPER
• Cite
• Code
• Project

This paper introduces Answer-Augmented Prompting (AAP), an innovative approach that leverages the Response Consistency of History of Dialogue (HoD) phenomenon in Large Language Models (LLMs). AAP not only achieves significantly superior performance enhancements compared to traditional augmentation methods but also exhibits a stronger potential for "jailbreaking", allowing models to produce unsafe or misleading responses. By strategically modifying the HoD, AAP influences LLM performance in a dual manner: it promotes accuracy while amplifying risks associated with bypassing built-in safeguards. Our experiments demonstrate that AAP outperforms standard methods in both effectiveness and the ability to elicit harmful content. To address these risks, we propose comprehensive mitigation strategies for both LLM service providers and end-users. This research offers valuable insights into the implications of Response Consistency in LLMs, underscoring the promise and peril of this powerful capability.

• PAPER
• Cite
• Code
• Project

The sequential interaction network usually find itself in a variety of applications, e.g., recommender system. Herein, inferring future interaction is of fundamental importance, and previous efforts are mainly focused on the dynamics in the classic zero-curvature Euclidean space. Despite the promising results achieved by previous methods, a range of significant issues still largely remains open: On the bipartite nature, is it appropriate to place user and item nodes in one identical space regardless of their inherent difference? On the network dynamics, instead of a fixed curvature space, will the representation spaces evolve when new interactions arrive continuously? On the learning paradigm, can we get rid of the label information costly to acquire? To address the aforementioned issues, we propose a novel Contrastive model for Sequential Interaction Network learning on Co-Evolving RiEmannian spaces, CSincere. To the best of our knowledge, we are the first to introduce a couple of co-evolving representation spaces, rather than a single or static space, and propose a co-contrastive learning for the sequential interaction network. In CSincere, we formulate a Cross-Space Aggregation for message-passing across representation spaces of different Riemannian geometries, and design a Neural Curvature Estimator based on Ricci curvatures for modeling the space evolvement over time. Thereafter, we present a Reweighed Co-Contrast between the temporal views of the sequential network, so that the couple of Riemannian spaces interact with each other for the interaction prediction without labels. Empirical results on 5 public datasets show the superiority of CSincere over the state-of-the-art methods.

• PAPER
• Cite
• Code
• Project

Graph clustering is a fundamental problem in machine learning. Deep learning methods achieve the state-of-the-art results in recent years, but they still cannot work without predefined cluster numbers. Such limitation motivates us to pose a more challenging problem of graph clustering with unknown cluster number. We propose to address this problem from a fresh perspective of graph information theory (i.e., structural information). In the literature, structural information has not yet been introduced to deep clustering, and its classic definition falls short of discrete formulation and modeling node features. In this work, we first formulate a differentiable structural information (DSI) in the continuous realm, accompanied by several theoretical results. By minimizing DSI, we construct the optimal partitioning tree where densely connected nodes in the graph tend to have the same assignment, revealing the cluster structure. DSI is also theoretically presented as a new graph clustering objective, not requiring the predefined cluster number. Furthermore, we design a neural LSEnet in the Lorentz model of hyperbolic space, where we integrate node features to structural information via manifold-valued graph convolution. Extensive empirical results on real graphs show the superiority of our approach.

• PAPER
• Cite
• Code
• Project

In recent years, deep clustering has achieved encouraging results. However, existing deep clustering methods work with the traditional Euclidean space and thus present deficiency on clustering complex structures. On the contrary, Riemannian geometry provides an elegant framework to model complex structures as well as a powerful tool for clustering, i.e., the Ricci flow. In this paper, we rethink the problem of deep clustering, and introduce the Riemannian geometry to deep clustering for the first time. Deep clustering in Riemannian manifold still faces significant challenges: (1) Ricci flow itself is unaware of cluster membership, (2) Ricci curvature prevents the gradient backpropagation, and (3) learning the flow largely remains open in the manifold. To bridge these gaps, we propose a novel Riemannian generative model (RicciNet), a neural Ricci flow with several theoretical guarantees. The novelty is that we model the dynamic self-clustering process of Ricci flow: data points move to the respective clusters in the manifold, influenced by Ricci curvatures. The point's trajectory is characterized by a parametric velocity, taking the form of Ordinary Differential Equation (ODE). Specifically, we encode data points as samples of Gaussian mixture in the manifold where we propose two types of reparameterization approaches: Gumbel reparameterization, and geometric trick. We formulate a differentiable Ricci curvature parameterized by a Riemannian graph convolution. Thereafter, we propose a geometric learning approach in which we study the geometric regularity of the point's trajectory, and learn the flow via distance matching and velocity matching. Consequently, data points go along the shortest Ricci flow to complete clustering. Extensive empirical results show RicciNet outperforms Euclidean deep methods.

• PAPER
• Cite
• Code
• Project

Graphs are typical non-Euclidean data of complex structures. In recent years, Riemannian graph representation learning has emerged as an exciting alternative to Euclidean ones. However, Riemannian methods are still in an early stage: most of them present a single curvature (radius) regardless of structural complexity, suffer from numerical instability due to the exponential/logarithmic map, and lack the ability to capture motif regularity. In light of the issues above, we propose the problem of \emph{Motif-aware Riemannian Graph Representation Learning}, seeking a numerically stable encoder to capture motif regularity in a diverse-curvature manifold without labels. To this end, we present a novel Motif-aware Riemannian model with Generative-Contrastive learning (MotifRGC), which conducts a minmax game in Riemannian manifold in a self-supervised manner. First, we propose a new type of Riemannian GCN (D-GCN), in which we construct a diverse-curvature manifold by a product layer with the diversified factor, and replace the exponential/logarithmic map by a stable kernel layer. Second, we introduce a motif-aware Riemannian generative-contrastive learning to capture motif regularity in the constructed manifold and learn motif-aware node representation without external labels. Empirical results show the superiority of MofitRGC.

• PAPER
• Cite
• Code
• Project

Information diffusion prediction is fundamental to understand the structure and organization of the online social networks, and plays a crucial role to blocking rumor spread, influence maximization, political propaganda, etc. So far, most existing solutions primarily predict the next user who will be informed with historical cascades, but ignore an important factor in the diffusion process - the time. Such limitation motivates us to pose the problem of the time-aware personalized information diffusion prediction for the first time, telling the time when the target user will be informed. In this paper, we address this problem from a fresh geometric perspective of Ricci curvature, and propose a novel Ricci-curvature regulated Ordinary Differential Equation (R-ODE). In the diffusion process, R-ODE considers that the inter-correlated users are organized in a dynamic system in the representation space, and the cascades give the observations sampled from the continuous realm. At each infection time, the message diffuses along the largest Ricci curvature, signifying less transportation effort. In the continuous realm, the message triggers users' movement, whose trajectory in the space is parameterized by an ODE with graph neural network. Consequently, R-ODE predicts the infection time of a target user by the movement trajectory learnt from the observations. Extensive experiments evaluate the personalized time prediction ability of R-ODE, and show R-ODE outperforms the state-of-the-art baselines.

• PAPER
• Cite
• Code
• Project

Link prediction typically studies the probability of future interconnection among nodes with the observation in a single social network. More often than not, real scenario is presented as a multiplex network with common (anchor) users active in multiple social networks. In the literature, most existing works study either the intra-link prediction in a single network or inter-link prediction among networks (a.k.a. network alignment), and consider two learning tasks are independent from each other, which is still away from the fact. On the representation space, the vast majority of existing methods are built upon the traditional Euclidean space, unaware of the inherent geometry of social networks. The third issue is on the scarce anchor users. Annotating anchor users is laborious and expensive, and thus it is impractical to work with quantities of anchor users. Herein, in light of the issues above, we propose to study a challenging yet practical problem of Geometry-aware Collective Link Prediction across Multiplex Network. To address this problem, we present a novel contrastive model, RCoCo, which collaborates intra- and inter-network behaviors in Riemannian spaces. In RCoCo, we design a curvature-aware graph attention network ([Math Processing Error]GAT), conducting attention mechanism in Riemannian manifold whose curvature is estimated by the Ricci curvatures over the network. Thereafter, we formulate intra- and inter-contrastive loss in the manifolds, in which we augment graphs by exploring the high-order structure of community and information transfer on anchor users. Finally, we conduct extensive experiments with 14 strong baselines on 8 real-world datasets, and show the effectiveness of RCoCo.

• PAPER
• Cite
• Code
• Project

Graph clustering is a longstanding research topic, and has achieved remarkable success with the deep learning methods in recent years. Nevertheless, we observe that several important issues largely remain open. On the one hand, graph clustering from the geometric perspective is appealing but has rarely been touched before, as it lacks a promising space for geometric clustering. On the other hand, contrastive learning boosts the deep graph clustering but usually struggles in either graph augmentation or hard sample mining. To bridge this gap, we rethink the problem of graph clustering from geometric perspective and, to the best of our knowledge, make the first attempt to introduce a heterogeneous curvature space to graph clustering problem. Correspondingly, we present a novel end-to-end contrastive graph clustering model named CONGREGATE, addressing geometric graph clustering with Ricci curvatures. To support geometric clustering, we construct a theoretically grounded Heterogeneous Curvature Space where deep representations are generated via the product of the proposed fully Riemannian graph convolutional nets. Thereafter, we train the graph clusters by an augmentation-free reweighted contrastive approach where we pay more attention to both hard negatives and hard positives in our curvature space. Empirical results on real-world graphs show that our model outperforms the state-of-the-art competitors.

• PAPER
• Cite
• Code
• Project

Continual graph learning routinely finds its role in a variety of real-world applications where the graph data with different tasks come sequentially. Despite the success of prior works, it still faces great challenges. On the one hand, existing methods work with the zero-curvature Euclidean space, and largely ignore the fact that curvature varies over the coming graph sequence. On the other hand, continual learners in the literature rely on abundant labels, but labeling graph in practice is particularly hard especially for the continuously emerging graphs on-the-fly. To address the aforementioned challenges, we propose to explore a challenging yet practical problem, the self-supervised continual graph learning in adaptive Riemannian spaces. In this paper, we propose a novel self-supervised Riemannian Graph Continual Learner (RieGrace). In RieGrace, we first design an Adaptive Riemannian GCN (AdaRGCN), a unified GCN coupled with a neural curvature adapter, so that Riemannian space is shaped by the learnt curvature adaptive to each graph. Then, we present a Label-free Lorentz Distillation approach, in which we create teacher-student AdaRGCN for the graph sequence. The student successively performs intra-distillation from itself and inter-distillation from the teacher so as to consolidate knowledge without catastrophic forgetting. In particular, we propose a theoretically grounded Generalized Lorentz Projection for the contrastive distillation in Riemannian space. Extensive experiments on the benchmark datasets show the superiority of RieGrace, and additionally, we investigate on how curvature changes over the graph sequence.

• PAPER
• Cite
• Code
• Project

Social network alignment, aligning different social networks on their common users, is receiving increasing attention from both academic and industry. Most of the existing studies consider the social network to be static and neglect its inherent dynamics. In fact, the dynamics of social networks contain the discriminative pattern of an individual, which can be leveraged to facilitate social network alignment. Hence, we for the first time propose to study the problem of aligning dynamic social networks. Towards this end, we propose a novel Dynamic Graph autoencoder based dynamic social network Alignment approach, referred to as DGA, unfolding the fruitful dynamics of social networks for user alignment. However, it faces challenges in both modeling and optimization: (1) To model the intra-network dynamics, we design a novel dynamic graph autoencoder to learn user embeddings with complex network dynamics. (2) To model the inter-network alignment, we design a unified optimization framework over proposed dynamic graph autoencoders, constructing a common subspace for user alignment across different networks. (3) To address this optimization problem, we design an effective alternating algorithm with solid theoretical guarantees. We conduct extensive experiments on real-world datasets and show that the proposed approach substantially outperforms the state-of-the-art methods.

• PAPER
• Cite
• Code
• Project

Social network alignment, identifying social accounts of the same individual across different social networks, shows fundamental importance in a wide spectrum of applications, such as link prediction and information diffusion. Individuals more often than not join in multiple social networks, and it is in fact much too expensive or even impossible to acquiring supervision for guiding the alignment. To the best of our knowledge, few method in the literature can align multiple social networks without supervision. In this article, we propose to study the problem of unsupervised multiple social network alignment. To address this problem, we propose a novel unsupervised model of joint Matrix factorization with a diagonal Cone under orthogonal Constraint, referred to as MC2. Its core idea is to embed and align multiple social networks in the common subspace via an unsupervised approach. Specifically, in MC2 model, we first design a matrix optimization to infer the common subspace from different social networks. To address the nonconvex optimization, we then design an efficient alternating algorithm by leveraging its inherent functional property. Through extensive experiments on real-world datasets, we demonstrate that the proposed MC2 model significantly outperforms the state-of-the-art methods.

• PAPER
• Cite
• Code
• Project

Social network alignment, which aims to uncover the correspondence across different social networks, shows fundamental importance in a wide spectrum of applications such as cross-domain recommendation and information propagation. In the literature, the vast majority of the existing studies focus on the social network alignment at user level. In practice, the user-level alignment usually relies on abundant personal information and high-quality supervision, which is expensive and even impossible in the real-world scenario. Alternatively, we propose to study the problem of social group alignment across different social networks, focusing on the interests of social groups rather than personal information. However, social group alignment is non-trivial and faces significant challenges in both (i) feature inconsistency across different social networks and (ii) group discovery within a social network. To bridge this gap, we present a novel GroupAligner, a deep reinforcement learning with domain adaptation for social group alignment. In GroupAligner, to address the first issue, we propose the cycle domain adaptation approach with the Wasserstein distance to transfer the knowledge from the source social network, aligning the feature space of social networks in the distribution level. To address the second issue, we model the group discovery as a sequential decision process with reinforcement learning in which the policy is parameterized by a proposed proximity-enhanced Graph Neural Network (pGNN) and a GNN-based discriminator to score the reward. Finally, we utilize pre-training and teacher forcing to stabilize the learning process of GroupAligner. Extensive experiments on several real-world datasets are conducted to evaluate GroupAligner, and experimental results show that GroupAligner outperforms the alternative methods for social group alignment.

• PAPER
• Cite
• Code
• Project

The sequential interaction network usually find itself in a variety of applications, e.g., recommender system. Herein, inferring future interaction is of fundamental importance, and previous efforts are mainly focused on the dynamics in the classic zero-curvature Euclidean space. Despite the promising results achieved by previous methods, a range of significant issues still largely remains open: On the bipartite nature, is it appropriate to place user and item nodes in one identical space regardless of their inherent difference? On the network dynamics, instead of a fixed curvature space, will the representation spaces evolve when new interactions arrive continuously? On the learning paradigm, can we get rid of the label information costly to acquire? To address the aforementioned issues, we propose a novel Contrastive model for Sequential Interaction Network learning on Co-Evolving RiEmannian spaces, CSINCERE. To the best of our knowledge, we are the first to introduce a couple of co-evolving representation spaces, rather than a single or static space, and propose a co-contrastive learning for the sequential interaction network. In CSINCERE, we formulate a Cross-Space Aggregation for message-passing across representation spaces of different Riemannian geometries, and design a Neural Curvature Estimator based on Ricci curvatures for modeling the space evolvement over time. Thereafter, we present a Reweighed Co-Contrast between the temporal views of the sequential network, so that the couple of Riemannian spaces interact with each other for the interaction prediction without labels. Empirical results on 5 public datasets show the superiority of CSINCERE over the state-of-the-art methods.

• PAPER
• Cite
• Code
• Project