Schedule for: 24w5263 - Analysis of Complex Data: Tensors, Networks and Dynamic Systems

A buffet dinner is served daily between 5:30pm and 7:30pm in Vistas Dining Room, top floor of the Sally Borden Building.

Meet and Greet at the BIRS Lounge (PDC 2nd Floor)

Breakfast is served daily between 7 and 9am in the Vistas Dining Room, the top floor of the Sally Borden Building.

A brief introduction to BIRS with important logistical information, technology instruction, and opportunity for participants to ask questions.

Network data is prevalent in numerous big data applications, including economics and health networks, where it is of prime importance to understand the latent structure of the network. In this paper, we model the network using the Degree-Corrected Mixed Membership (DCMM) model. In the DCMM model, for each node i, there exists a membership vector consisting of the weight that node i puts in community k. We derive novel finite-sample expansion for the weights, which allows us to obtain asymptotic distributions and confidence intervals of the membership mixing probabilities and other related population quantities. This fills an important gap on uncertainty quantification on the membership profile. We further develop a ranking scheme of the vertices based on the membership mixing probabilities on certain communities and perform relevant statistical inferences. A multiplier bootstrap method is proposed for ranking inference of individual member's profile with respect to a given community. The validity of our theoretical results is further demonstrated via numerical experiments in both real and synthetic data examples. (Joint work with Sohom Bhattacharya and Jikai Hou)

Multi-modal populations of networks arise in many scenarios including in large-scale multi-modal neuroimaging studies that capture both functional and structural neuroimaging data for thousands of subjects. A major research question in such studies is how functional and structural brain connectivity are related and how they vary across the population. We develop a novel PCA-type framework for integrating multi-modal undirected networks measured on many subjects. Specifically, we arrange these networks as semi-symmetric tensors, where each tensor slice is a symmetric matrix representing a network from an individual subject. We then propose a novel Joint, Integrative Semi-Symmetric Tensor PCA (JisstPCA) model, associated with an efficient iterative algorithm, for jointly finding low-rank representations of two or more networks across the same population of subjects. We establish one-step statistical convergence of our separate low-rank network factors as well as the shared population factors to the true factors, with finite sample statistical error bounds. Through simulation studies and a real data example for integrating multi-subject functional and structural brain connectivity, we illustrate the advantages of our method for finding joint low-rank structures in multi-modal populations of networks.

Lunch is served daily between 11:30am and 1:30pm in the Vistas Dining Room, the top floor of the Sally Borden Building.

Meet in foyer of TCPL to participate in the BIRS group photo. The photograph will be taken outdoors, so dress appropriately for the weather. Please don't be late, or you might not be in the official group photo!

The block-model family has four popular network models: SBM, MMSBM, DCBM, and DCMM. A fundamental problem is, how well each of these models fits with real networks. We propose GoF-MSCORE as a new Goodness-of-Fit (GoF) metric for DCMM (the broadest one among the four), with two main ideas. The first is to use cycle count statistics as a general recipe for GoF. The second is a novel network fitting scheme. GoF-MSCORE is a flexible GoF approach. We adapt it to all four models in the block-model family. We show that for each of the four models, if the assumed model is correct, then the corresponding GoF metric converges to N(0,1) as the network sizes diverge. We also analyze the powers and show that these metrics are optimal in many settings. For 11 frequently-used real networks, we use the proposed GoF metrics to show that DCMM fits well with almost all of them. We also show that SBM, DCBM, and MMSBM do not fit well with many of these networks, especially when the networks are relatively large.

The talk introduces a Signed Generalized Random Dot Product Graph (SGRDPG) model, which is a variant of the Generalized Random Dot Product Graph (GRDPG), where, in addition, edges can be positive or negative. The setting is extended to a multiplex version, where all layers have the same collection of nodes and follow the SGRDPG. The only common feature of the layers of the network is that they can be partitioned into groups with common subspace structures, while otherwise matrices of connection probabilities can be all different. The setting above is extremely flexible and includes a variety of existing multiplex network models as its particular cases. The paper fulfills two objectives. First, it shows that keeping signs of the edges in the process of network construction leads to a better precision of estimation and clustering and, hence, is beneficial for tackling real world problems such as, for example, analysis of brain networks. Second, by employing novel algorithms, our paper ensures strongly consistent clustering of layers and high accuracy of subspace estimation. In addition to theoretical guarantees, both of those features are demonstrated using numerical simulations and a real data example.

Multi-view data arises frequently in modern network analysis e.g. relations of multiple types among individuals in social network analysis, longitudinal measurements of interactions among observational units, annotated networks with noisy partial labeling of vertices etc. We study community detection in these disparate settings via a unified theoretical framework, and investigate the fundamental thresholds for community recovery. In particular, we derive a sharp threshold for community detection in an inhomogeneous multilayer block model and characterize a sharp threshold for weak recovery in a dynamic stochastic block model. Finally, we introduce iterative algorithms based on Approximate Message Passing for community detection in these problems. Based on joint work with Xiaodong Yang and Buyu Lin (Harvard).

A buffet dinner is served daily between 5:30pm and 7:30pm in Vistas Dining Room, top floor of the Sally Borden Building.

Breakfast is served daily between 7 and 9am in the Vistas Dining Room, the top floor of the Sally Borden Building.

Tensor regression models are of interest in diverse fields of social and behavioral sciences, including neuroimaging analysis, image processing and so on. Recent theoretical advancements of tensor decomposition have facilitated significant development of various tensor regression models. This talk discusses a tensor regression model, wherein the coefficient tensor is decomposed into two components: a low tubal rank tensor and a structured sparse one. We first address the issue of identifiability of the two components comprising the coefficient tensor and subsequently develop a fast and scalable Alternating Minimization algorithm to solve the convex regularized program. Further, finite sample error bounds under high dimensional scaling for the model parameters are provided. The performance of the model is assessed on synthetic data and is also used in an application involving data from an intelligent tutoring platform. Extensions to multivariate time series data are also briefly discussed.

Lunch is served daily between 11:30am and 1:30pm in the Vistas Dining Room, the top floor of the Sally Borden Building.

Optimal experimental designs on network data are challenging due to the interference between network units. One unit’s treatment status usually affects its neighbors’ outcomes, a phenomenon known as spillover effects or network effects. In this talk, we focus on the experimental designs for three types of networks. Specifically, for a well-clustered network, we consider the optimal design equipped with the Horvitz-Thompson estimator with the sampling procedure considered. We establish a minimum sample curve, which is a combination of a number of clusters and cluster size to be sampled for any given required power. For imperfectly clustered networks, we investigate the optimal randomized saturation design for difference-in-means estimators and re-evaluate two widely used designs, which are believed to be optimal but turn out to be sub-optimal. Lastly, we introduce a novel partitioning method for an arbitrary sparse network without cluster structures, where direct treatments and interference can be separately controlled for a sub-network. By trading data quantity for quality, the proposed method turns out to outperform existing designs on sparse networks.

The centrality in a network is often used to measure nodes' importance and model network effects on a certain outcome. Empirical studies widely adopt a two-stage procedure, which first estimates the centrality from the observed noisy network and then infers the network effect from the estimated centrality, even though it lacks theoretical understanding. We propose a unified modeling framework, under which we first prove the shortcomings of the two-stage procedure, including the inconsistency of the centrality estimation and the invalidity of the network effect inference. Furthermore, we propose a supervised centrality estimation methodology, which aims to simultaneously estimate both centrality and network effect. The advantages in both regards are proved theoretically and demonstrated numerically via extensive simulations and a case study in predicting currency risk premiums from the global trade network.

We study the node classification problem on feature-decorated graphs in the sparse setting, i.e., when the expected degree of a node is O(1) in the number of nodes, in the fixed-dimensional asymptotic regime, i.e., the dimension of the feature data is fixed while the number of nodes is large. Such graphs are typically known to be locally tree-like. We introduce a notion of Bayes optimality for node classification tasks, called asymptotic local Bayes optimality, and compute the optimal classifier according to this criterion for a fairly general statistical data model with arbitrary distributions of the node features and edge connectivity. The optimal classifier is implementable using a message-passing graph neural network architecture. We then compute the generalization error of this classifier and compare its performance against existing learning methods theoretically on a well-studied statistical model with naturally identifiable signal-to-noise ratios (SNRs) in the data. We find that the optimal message-passing architecture interpolates between a standard MLP in the regime of low graph signal and a typical convolution in the regime of high graph signal. Furthermore, we prove a corresponding non-asymptotic result.

The analysis of tensor data, i.e., arrays with multiple directions, is motivated by a wide range of scientific applications and has become an important interdisciplinary topic in data science. In this talk, we discuss the fundamental task of performing Singular Value Decomposition (SVD) on tensors, exploring both general cases and scenarios with specific structures like smoothness and longitudinality. Through the developed frameworks, we can achieve accurate denoising for 4D scanning transmission electron microscopy images; in longitudinal microbiome studies, we can extract key components in the trajectories of bacterial abundance, identify representative bacterial taxa for these key trajectories, and group subjects based on the change of bacteria abundance over time. We also showcase the development of statistically optimal methods and computationally efficient algorithms that harness valuable insights from high-dimensional tensor data, grounded in theories of computation and non-convex optimization.

We study inference in the context of a (large dimensional) factor model for matrix-valued time series, with (possibly) common stochastic trends and a stationary factor structure in the error term. As a preliminary, negative result, we show that both a "flattened” and a projection/sketching-based estimation technique offer super consistent estimation of the row and column loadings spaces, with no improvement in the rate of convergence when using a projection-based estimation. However, the common stochastic trends cannot be estimated consistently in the presence of a factor structure in the error term: in the presence of strong cross sectional dependence, even sketching does not help. In turn, this precludes estimation of the stationary idiosyncratic component associated to the common factors. Hence, we propose an alternative way of consistently estimating the common (stationary and nonstationary) factors, and the row and column loadings spaces associated with both the stationary and nonstationary common factors. Our technique is based on: firstly, consistently estimating the row and column loadings spaces associated with both the stationary and nonstationary common factors; secondly, “unsketching”, i.e. getting rid of the common nonstationary component by projecting the data onto the orthogonal complement of the estimated loadings space associated with the common stochastic trends; thirdly, using the “unsketched" data to recover the whole factor structure (loadings and common factors) associated with the stationary factors; fourthly, removing the estimated stationary common component from the data, and estimating the nonstationary common component once again. In this case, we show that full-blown consistent estimation is possible, and that projection-based estimation improves on the rates of convergence of the estimated loadings space for both the stationary and the nonstationary factor structures. Ancillary results such as limiting distribution of the estimated common factors and loadings, and sequential procedures to estimate the number of common factors are also proposed.

High-dimensional tensor-valued data have recently gained attention from researchers in economics and statistics. We consider the estimation and inference of high-dimensional tensor factor models, where each dimension of the tensor diverges. Specifically, we focus on the factor model that admits CP-type tensor decomposition, allowing for loading vectors that are not necessarily orthogonal. Based on the contemporary covariance matrix, we propose an iterative higher-order projection estimation method. Our estimator is robust to weak dependence among factors and weak correlation across different dimensions in the idiosyncratic shocks. We develop an inferential theory, establishing consistency and the asymptotic normality under relaxed assumptions. Through a simulation study and an empirical application with sorted portfolios, we illustrate the advantages of our proposed estimator over existing methodologies in the literature.

A buffet dinner is served daily between 5:30pm and 7:30pm in Vistas Dining Room, top floor of the Sally Borden Building.

Breakfast is served daily between 7 and 9am in the Vistas Dining Room, the top floor of the Sally Borden Building.

A separable covariance model for a random matrix provides a parsimonious description of the covariances among the rows and among the columns of the matrix, and permits likelihood-based inference with a very small sample size. However, in many applications the assumption of exact separability is unlikely to be met, and data analysis with a separable model may overlook or misrepresent important dependence patterns in the data. In this article, we propose a compromise between separable and unstructured covariance estimation. We show how the set of covariance matrices may be uniquely parametrized in terms of the set of separable covariance matrices and a complementary set of “core” covariance matrices, where the core of a separable covariance matrix is the identity matrix. This parametrization defines a Kronecker-core decomposition of a covariance matrix. By shrinking the core of the sample covariance matrix with an empirical Bayes procedure, we obtain an estimator that can adapt to the degree of separability of the population covariance matrix.

We propose an autoregressive framework for modelling dynamic networks with dependent edges. It encompasses the models which accommodate, for example, transitivity, density-dependent and other stylized features often observed in real network data. By assuming the edges of network at each time are independent conditionally on their lagged values, the models, which exhibit a close connection with temporal ERGMs, facilitate both simulation and the maximum likelihood estimation in the straightforward manner. Due to the possible large number of parameters in the models, the initial MLEs may suffer from slow convergence rates. An improved estimator for each component parameter is proposed based on an iteration based on the projection which mitigates the impact of the other parameters (Chang et al., 2021). Based on a martingale difference structure, the asymptotic distribution of the improved estimator is derived without the stationarity assumption. The limiting distribution is not normal in general, and it reduces to normal when the underlying process satisfies some mixing conditions. Illustration with a transitivity model was carried out in both simulation and two real network data sets.

Lunch is served daily between 11:30am and 1:30pm in the Vistas Dining Room, the top floor of the Sally Borden Building.

A buffet dinner is served daily between 5:30pm and 7:30pm in Vistas Dining Room, top floor of the Sally Borden Building.

Breakfast is served daily between 7 and 9am in the Vistas Dining Room, the top floor of the Sally Borden Building.

One of the main tasks of causal inference is to learn direct causal relationships among observed random variables. These relationships are usually depicted via a directed graph whose vertices are the variables of interest and whose edges represent direct causal effects. In this talk we will discuss the problem of learning such a directed graph for a linear causal model. We will specifically address the case where the graph may have directed cycles. In general, the causal graph cannot be learned uniquely from observational data. However, in the special case of linear non-Gaussian acyclic causal models, the directed graph can be found uniquely. When cycles are allowed the graph can be learned up to an equivalence class. We characterize the equivalence classes of such cyclic graphs and we propose algorithms for causal discovery. Our methods are based on using specific polynomial relationships which hold among the second and higher order moments of the random vector and which can help identify the graph. Joint work with Mathias Drton, Marina Garrote-Lopez, and Niko Nikov.

Consider a stationary, weakly dependent sequence of random variables. Given only mild conditions, allowing for polynomial decay of the autocovariance function, we show a Berry-Esseen bound of optimal order $n^{-1/2}$ for studentized (self-normalized) partial sums, both for the Kolmogorov and Wasserstein (and $L^p$) distance. The results show that, in general, (minimax) optimal estimators of the long-run variance lead to suboptimal bounds in the central limit theorem, that is, the rate $n^{-1/2}$ cannot be reached, refuting a popular belief in the literature. This can be salvaged by simple methods: We reveal that in order to maintain the optimal speed of convergence $n^{-1/2}$, simple over-smoothing within a certain range is necessary and sufficient. The setup contains many prominent dynamical systems and time series models, including random walks on the general linear group, products of positive random matrices, functionals of Garch models of any order, functionals of dynamical systems arising from SDEs, iterated random functions and many more.

Lunch is served daily between 11:30am and 1:30pm in the Vistas Dining Room, the top floor of the Sally Borden Building.

This paper introduces a general framework of Semi-parametric TEnsor Factor Analysis (STEFA) that focuses on the methodology and theory of low-rank tensor decomposition with auxiliary covariates. Semi-parametric TEnsor Factor Analysis models extend tensor factor models by incorporating auxiliary covariates in the loading matrices. We propose an algorithm of iteratively projected singular value decomposition (IP-SVD) for the semi-parametric estimation. It iteratively projects tensor data onto the linear space spanned by the basis functions of covariates and applies singular value decomposition on matricized tensors over each mode. We establish the convergence rates of the loading matrices and the core tensor factor. The theoretical results only require a sub-exponential noise distribution, which is weaker than the assumption of sub-Gaussian tail of noise in the literature. Compared with the Tucker decomposition, IP-SVD yields more accurate estimators with a faster convergence rate. Besides estimation, we propose several prediction methods with new covariates based on the STEFA model. On both synthetic and real tensor data, we demonstrate the efficacy of the STEFA model and the IP-SVD algorithm on both the estimation and prediction tasks.

Population level single cell gene expression data collects gene expressions of thousands of cells for a given individual in a large set of individuals. Such data allows us to construct cell-type- and individual-specific gene co-expression networks. It is important to understand how such a network is associated with individual-level covariates. This talk considers a regression framework with multivariate Gaussian distribution as an outcome with vector covariates, where the Wasserstein distance between distributions is used as a replacement for the Euclidean distance. A test statistic is defined and used to test for no effects and partial effects of covariates. The asymptotic distribution of the test statistic is derived under the assumption of the array of covariance matrices satisfying a CP decomposition criteria. Simulations show that the proposed test has correct type 1 error and adequate power. Results from an analysis of large-scale single-cell data reveal an association between the gene co-expression network of genes in the nutrient sensing pathway and age, indicating the perturbed gene co-expression network as people age. Joint work with Tony Cai and Hongzhe Li.

We develop a novel framework for the analysis of medical imaging data, including magnetic resonance imaging, functional magnetic resonance imaging, computed tomography and more. Medical imaging data differ from general images in two main aspects: (i) the sample size is often considerably smaller and (ii) the interpretation of the model is usually more crucial than predicting the outcome. As a result, standard methods such as convolutional neural networks cannot be directly applied to medical imaging analysis. Therefore, we propose the deep Kronecker network, which can adapt to the low sample size constraint and offer the desired model interpretation. Our approach is versatile, as it works for both matrix- and tensor-represented image data and can be applied to discrete and continuous outcomes. The deep Kronecker network is built upon a Kronecker product structure, which implicitly enforces a piecewise smooth property on coefficients. Moreover, our approach resembles a fully convolutional network as the Kronecker structure can be expressed in a convolutional form. Interestingly, our approach also has strong connections to the tensor regression framework proposed by Zhou et al. (2013), which imposes a canonical low-rank structure on tensor coefficients. We conduct both classification and regression analyses using real magnetic resonance imaging data from the Alzheimer’s Disease Neuroimaging Initiative to demonstrate the effectiveness of our approach.

"It is well-known how to do inference for the change-point in heavy tailed time series is greatly open. This article proposes a winsorized CUSUM approach to solve this problem. We begin by investigating the winsorized CUSUM process and deriving the limiting distributions of the Kolmogorov-Smirnov test and the Self-normalized test under the null hypothesis. Our tests are shown to exhibit a power approaching to one for both fixed and local alternatives. We also extend the winsorizing technique to tests for multiple change-points in cases where the priori information on the number of actual change points is unknown. When the existence of change points is affirmative, we further introduce the winsorized-mean-based least squares estimator for both the jump and location. Our framework is quite general and its assumption is very weak. This enables the application of our tests and estimation theory not only to linear time series but also to nonlinear time series, such as TAR and G-GARCH processes. To evaluate the performance of our procedures, we conduct simulation studies and provide two real examples from financial markets. The empirical evidence supports the effectiveness of our proposed procedures for testing and dating change-points in heavy-tailed time series."

We study low-rank matrix estimation for a generic inhomogeneous output channel through which the matrix is observed. This generalizes the commonly considered spiked matrix model with homogeneous noise to include for instance the dense degree-corrected stochastic block model. We prove the limiting mutual information for such models, which provides a framework to study the signal detection thresholds. We will also discuss an optimal spectral method to extend the BBP phase transition criterion to the inhomogeneous setting. We will show that the transition for the appearance of 1) an outlier outside the bulk of the limiting spectral distribution and 2) a positive overlap between the associated eigenvector and the signal, occurs precisely at the optimal threshold, making the proposed spectral method optimal within the class of iterative methods for the inhomogeneous Wigner problem. This is based on a series of works with Alice Guionnet, Florent Krzakala, Pierre Mergny, Aleksandr Pak, and Lenka Zdeborova.

Making online decisions can be challenging when features are sparse and orthogonal to historical ones, especially when the optimal policy is learned through collaborative filtering. We formulate the problem as a matrix completion bandit (MCB), where the expected reward under each arm is characterized by an unknown low-rank matrix. The ε-greedy bandit and the online gradient descent algorithm are explored. Policy learning and regret performance are studied under a specific schedule for exploration probabilities and step sizes. A faster decaying exploration probability yields smaller regret but learns the optimal policy less accurately. We investigate an online debiasing method based on inverse propensity weighting (IPW) and a general framework for online policy inference. The IPW-based estimators are asymptotically normal under mild arm-optimality conditions. Numerical simulations corroborate our theoretical findings. Our methods are applied to the San Francisco parking pricing project data, revealing intriguing discoveries and outperforming the benchmark policy.

A buffet dinner is served daily between 5:30pm and 7:30pm in Vistas Dining Room, top floor of the Sally Borden Building.

Breakfast is served daily between 7 and 9am in the Vistas Dining Room, the top floor of the Sally Borden Building.

Two main families of Factor Models coexist in the literature: the Dynamic (initiated in Forni et al. (2000) and Forni and Lippi (2001)) and the Static (going back to Chamberlain (1983) and Chamberlain and Rothschild (1983)). The Static is facing problems with the Weak---the so-called weak factors (Onatski (2014)). Recent results (Gersing, Rust, and Deistler (2024)) are reconciling the two approaches while solving the weak factor issue. Based on joint work with Matteo Barigozzi (University of Bologna).

5-day workshop participants are welcome to use BIRS facilities (TCPL ) until 3 pm on Friday, although participants are still required to checkout of the guest rooms by 11AM.