1 Real-Time Dense Field Phase-to-Space Simulation of Imaging through Atmospheric Turbulence

2025-04-24 0 0 6.67MB 11 页 10玖币

侵权投诉

Real-Time Dense Field Phase-to-Space Simulation

of Imaging through Atmospheric Turbulence

Nicholas Chimitt, Student Member, IEEE, Xingguang Zhang, Student Member, IEEE,

Zhiyuan Mao, Student Member, IEEE, Stanley H. Chan, Senior Member, IEEE

Abstract—Numerical simulation of atmospheric turbulence is

one of the biggest bottlenecks in developing computational tech-

niques for solving the inverse problem in long-range imaging. The

classical split-step method is based upon numerical wave prop-

agation which splits the propagation path into many segments

and propagates every pixel in each segment individually via the

Fresnel integral. This repeated evaluation becomes increasingly

time-consuming for larger images. As a result, the split-step

simulation is often done only on a sparse grid of points followed

by an interpolation to the other pixels. Even so, the computation

is expensive for real-time applications. In this paper, we present

a new simulation method that enables real-time processing over a

dense grid of points. Building upon the recently developed multi-

aperture model and the phase-to-space transform, we overcome

the memory bottleneck in drawing random samples from the

Zernike correlation tensor. We show that the cross-correlation

of the Zernike modes has an insigniﬁcant contribution to the

statistics of the random samples. By approximating these cross-

correlation blocks in the Zernike tensor, we restore the homo-

geneity of the tensor which then enables Fourier-based random

sampling. On a 512 ×512 image, the new simulator achieves

0.025 seconds per frame over a dense ﬁeld. On a 3840 ×2160

image which would have taken 13 hours to simulate using the

split-step method, the new simulator can run at approximately

60 seconds per frame.

Index Terms—Atmospheric turbulence, wave propagation,

Zernike basis, Phase-to-Space Transform, Fourier optics

I. INTRODUCTION

Light propagating through the atmosphere suffers from dis-

tortions due to the random spatio-temporal ﬂuctuations in the

index of refraction. Over a long distance, these distortions will

accumulate and degrade the image quality. The development

of atmospheric turbulence mitigation algorithms has received

a considerable amount of interest over the past few decades

[1]–[9]. Deep-learning-based techniques have recently been

reported with some promising preliminary results [10]–[13].

However, as in any inverse problem, the atmospheric turbu-

lence problem requires a forward model that can accurately

describe the image formation process. To this end, simulating

The authors are with the School of Electrical and Computer Engineer-

ing, Purdue University, West Lafayette, IN, 47907 USA. Email: {nchimitt,

zhan3275, mao114, stanchan}@purdue.edu.

The research is based upon work supported in part by the Intelligence

Advanced Research Projects Activity (IARPA) under Contract No. 2022-

21102100004, and in part by the National Science Foundation under the

grants CCSS-2030570 and IIS-2133032. The views and conclusions contained

herein are those of the authors and should not be interpreted as necessarily

representing the ofﬁcial policies, either expressed or implied, of IARPA, or

the U.S. Government. The U.S. Government is authorized to reproduce and

distribute reprints for governmental purposes notwithstanding any copyright

annotation therein.

Fig. 1: [Top] This paper presents a new turbulence simulation

method that produces dense-ﬁeld turbulence in real-time. For

a512 ×512 image, the classical split-step propagation takes 1

second to generate a 32 ×32 grid followed by interpolation of

the ﬁeld. The proposed method takes 0.03 seconds to generate

the turbulence at dense-grid. [Bottom] Snapshot of a simulated

turbulence image from an input image with a 4K resolution

(3840 ×2160 pixels).

the turbulent effect becomes a critical step toward the goal of

designing algorithms, evaluating methods, and understanding

the limitations of imaging systems.

For decades, simulating atmospheric turbulence is most

accurately performed in the wave domain because there is no

simple intensity domain model. The “gold standard” approach

is the split-step propagation [5], [14], [15].1The idea is

to split the propagation path into segments and model the

phase distortion in each segment for every pixel individually.

However, split-step propagation is not scalable. For a 256×256

sized image, the split-step simulator reported in [5] can

evaluate a grid of size 64 ×64 where the remaining pixels

are interpolated. The reported runtime was approximately 24.6

seconds per frame on a GPU. If the size of the image grows

to 3840 ×2160 (4K resolution) and the grid is dense, i.e.

1There exist other modalities for the simulation of these effects that do not

require computationally costly numerical wave propagation [16], [16], [17],

however, split-step is presently the most theoretically justiﬁable approach.

arXiv:2210.06713v1 [eess.IV] 13 Oct 2022

without interpolation, a rough estimate is about 13.7 hours for

one image. To synthesize a training dataset containing 1000 of

these sequences where each sequence has 100 frames, this will

take 156 years. Recognizing the pressing need for an accurate

and fast turbulence simulator, we present a method that enables

turbulence simulation in real time.

A preview of our results is shown in Figure 1. While split-

step propagation is largely limited to a small grid of sprase

points, the proposed method can directly generate a dense

ﬁeld. The run-time of the proposed method is approximately

0.025 seconds for a 512 ×512 image. For a high-deﬁnition

(HD) image of size 3840×2160, the runtime is approximately

60 seconds. As can be seen in Figure 1, the new simulator

allows us to zoom in to any region of the image while the

turbulent effect is still globally correlated according to the

theoretical statistics. To our knowledge, this is the ﬁrst practi-

cal demonstration of an HD dense-ﬁeld turbulence simulation

documented in the literature.

The proposed approach, named the Dense Field Phase-to-

Space (DF-P2S) simulation, is built upon the multi-aperture

model by Chimitt and Chan [18], and the phase-to-space

(P2S) transform by Mao et al. [19]. DF-P2S overcomes a

fundamental limitation of [19] which is the size of the cross-

correlation matrix (tensor). In [19], the cross-correlation tensor

must be pre-computed, stored, and decomposed before running

the simulator. This causes some computational overhead, how-

ever, the bigger issue is memory. The largest cross-correlation

tensor that can be stored is for a spatial grid of size 32 ×32

using 36 basis coefﬁcients. To simulate an image with a higher

resolution, we need to interpolate the ﬁeld, which limits the

overall accuracy of such a simulation. Our solution to over-

come this memory bottleneck is to maintain the homogeneity

along the spatial dimensions of the correlation tensor and

perform an approximation of the cross-correlation functions

which otherwise restrict this behavior. This is based on a

new observation that the exact form of the cross-correlation

functions can be approximated without severely hurting the

tensor statistics. As a result, we can employ Fourier-based

techniques to draw dense ﬁeld samples spatially at a low

computational cost. This allows us to maintain a similar speed

as P2S [19], yet gain an increase in statistical accuracy.

To summarize, this paper offers two contributions:

1) Real-time dense ﬁeld turbulence simulation. We re-

port the ﬁrst turbulence simulator that can simulate over

a dense ﬁeld and in real-time.

2) New approximation to the cross-correlation function.

We show how certain off-diagonal blocks of the cross-

correlation matrix used in [19] can be removed to utilize

Fourier-based generation, hence enabling a signiﬁcant

resolution upscale.

II. OUTLINE OF GENERAL SIMULATION PRINCIPLES:

BUILDING BLOCKS

To keep track of the notations, we use object plane coordi-

nates x= (x, y)and image plane coordinates u= (u, v). For

a function deﬁned across the aperture, we use the coordinates

ξ= (ξ, η)or its polar form ρ= (ρ, θ). We also use the

polar coordinates s= (s, ϕ)to denote the displacement in the

Zernike space.

The turbulence effect is modeled by convolving a spatially

varying point spread function (PSF) to a diffraction-limited

clean image. In the case of incoherent light, which is the focus

of this work, the observed image I(x)is

I(x) = hx(u)~Ig(u),(1)

where hx(u)is the PSF with the subscript to emphasize that

it is spatially varying, and Ig(u)is the distortion-free image.

Note that the observed image is indexed by xwhereas the PSF

and ideal image are indexed by u. This is to emphasize that

after hx(u)is convolved with Ig(u), only the center pixel is

used to construct I(x).

The per-pixel PSF can be generated per the Fraunhofer

diffraction equation with phase error [20]. Denoting P(ξ)as

the aperture function, hx(u)is

hx(u) = 



Fourier nP(ξ)e−jφx(ξ)o



2,(2)

withholding some constants that determine the size of the PSF

according to the optical parameters. Here, φx(ξ)is the phase

distortion function that varies over ξfor coordinate xin the

image. Note that φx0(ξ)6=φx1(ξ)if x06=x1.

Given that the PSF generation (2) and the image formation

(1) is relatively standard, the central focus of a simulation

approach then falls upon the generation of the random phase

φx(ξ)in accordance with its theoretically given statistics.

There are two main categories for generating φx: (i) split-

step propagation [5], [14], [15], which numerically propagates

a wave through a random volume, and hence modeling the

medium; (ii) collapsed phase-over-aperture [18], which gener-

ates the phase function directly at the aperture, which we refer

to as the multi-aperture model. We illustrate the differences in

these two approaches in Figure 2.

A. Computational Bottleneck of Split-Step

Before we discuss the two building blocks of our simulator,

it would be useful to highlight the limitations of the split-step

method [5], [14], [15]. The split-step method directly mirrors

the physical process by which light propagates. After a point

propagates through the simulated medium, it arrives at the

aperture of the imaging system with a phase component φ(ξ).

In the case of turbulence, the statistics of φ(ξ)is determined

by the structure function

Dφ(ξ,ξ0) = E[(φ(ξ)−φ(ξ0))2].(3)

Assuming that the random function φ(ξ)is homogeneous and

isotropic, the structure function can be simpliﬁed to

Dφ(|ξ−ξ0|)=6.88(|ξ−ξ0|/r0)5/3,(4)

where r0is the Fried parameter [21].

To numerically generate the phase φ, the split-step method

uses the Kolmogorov power spectrum density (PSD) [22] (or

similarly Von Karman spectrum [5], etc) to generate discrete

planes of turbulent distortions, referred to as phase screens

[15]. This can be done directly with knowledge of the PSD and

Fig. 2: Here we show the main difference between classical approaches of modeling the medium and propagation directly,

[5], [14], and our approach based on [18], [19]. In split-step, a subset of pixels in the image have point sources propagated

through the medium which are then interpolated between to form the image at full resolution. In our approach, every single

point has its own basis vector representation, meaning a phase function is generated for each pixel without interpolation. We

emphasize that both approaches generate phase realizations for the image formation process, but it is the generation approach

that differs.

the Fourier transform based random ﬁeld generation approach.

The phase screens are placed along the propagation path as

shown in Figure 2. The size of the phase screens is generated

larger than the input image in order to model the spatial

correlations which are determined by the overlap of the phase

screen along the path.

Since the split-step models the medium directly, it is

regarded as the most theoretically justiﬁable approach. A

comprehensive discussion of this model is provided in [15].

However, the main limitation of split-step is its computational

requirements. For every point source, we need to perform

multiple fast Fourier transforms (FFTs). A simulation with M

phase screens results in M(W×H)2D FFTs for an W×H

image, with each FFT being at the size of the image. Scaling

the process of generating a large dataset is nearly impossible.

B. Building Block 1: Multi-Aperture Model

Recognizing the speed limit of the split-step simulation,

Chimitt and Chan proposed a new concept in [18] which

they named the multi-aperture model. The idea is to skip

the propagation by going to the statistical description of the

resultant phase φx(ξ)using the Zernike representation ﬁrst

proposed by Noll in 1976 [23]. The idea is to deﬁne a

radius Rand a vector ρsuch that Rρis the polar coordinate

representation of ξ. Then, the phase φxcan be represented as

φx(Rρ)

|{z }

φx(ξ)

∞

j=1

ax,j Zj(ρ)≈

j=1

ax,j Zj(ρ),(5)

with Zj(ρ)as the jth Zernike function and ax,j as its respec-

tive coefﬁcient. We emphasize that the phase φxis location

speciﬁc, i.e., the phase function at xis different from the

phase at x0if x6=x0. Therefore, the Zernike coefﬁcients ax,j

are different from ax0,j . However, the basis function Zj(ρ)is

shared for all locations. In this paper, we set N= 36, though

this can easily be increased at the cost of some speed.

The coefﬁcients ax,j are zero-mean Gaussian, and have a

correlation matrix (technically, a tensor stored in the matrix

form) which we denote the (x,x0, i, j)th element of the matrix

Aas

[A]x,x0,i,j =E[ax,i ax0,j ].(6)

This notation stresses that the matrix Ais location dependent

on the pair xand x0. In the special case when the corre-

lation matrix is spatially invarying, we write [A]x,x0,i,j as

[A]x−x0,i,j . If we further assume that x0=x, the correlation

matrix becomes [A]0,i,j =E[ax,i ax,j ], which is equivalent

to the covariance matrix given by Noll [23].

Based on the decomposition in Equation (5), we can

simulate the turbulence in three steps [18]: (i) Collapse the

screens; (ii) Draw spatially correlated Zernike vectors; (iii)

Draw inter-modally correlated Zernike vector. Therefore, we

have effectively converted the split-step propagation into a

sampling problem of the Zernike coefﬁcients.

This multi-aperture model provides roughly a 6×increase

in speed over split-step. However, the major drawback is that

the spatial correlation for the higher order (ax,i where i≥3)

Zernike terms are not computationally feasible. The generation

of the spatial and intermodal correlated vectors requires a

large covariance matrix, which exponentially increases in

size with the size of the image. Therefore, only tilts are

reasonably implementable for this method. Furthermore, the

image formation process is the same as split-step, requiring us

to generate one PSF for one pixel repeatedly over the entire

ﬁeld of view plus convolutions to produce the ﬁnal image.

文档加载中……请稍候！
如果长时间未打开，您也可以点击刷新试试。

下载文档到电脑，查找使用更方便

10 玖币 0人已下载

立即下载

摘要：

1Real-TimeDenseFieldPhase-to-SpaceSimulationofImagingthroughAtmosphericTurbulenceNicholasChimitt,StudentMember,IEEE,XingguangZhang,StudentMember,IEEE,ZhiyuanMao,StudentMember,IEEE,StanleyH.Chan,SeniorMember,IEEEAbstractNumericalsimulationofatmosphericturbulenceisoneofthebiggestbottlenecksindevelopi...

展开>> 收起<<

1 Real-Time Dense Field Phase-to-Space Simulation of Imaging through Atmospheric Turbulence.pdf

共11页,预览3页

还剩页未读，继续阅读

声明：本站为文档C2C交易模式，即用户上传的文档直接被用户下载，本站只是中间服务平台，本站所有文档下载所得的收益归上传人(含作者)所有。玖贝云文库仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对上载内容本身不做任何修改或编辑。若文档所含内容侵犯了您的版权或隐私，请立即通知玖贝云文库，我们立即给予删除！

1 Real-Time Dense Field Phase-to-Space Simulation of Imaging through Atmospheric Turbulence

相关推荐

开通VIP享超值会员特权

作者详情

相关内容

热门标签

举报选择: