In the image, a person, presumably a man, is seen standing in front of a large white paper. He is pointing at a diagram on the paper, which appears to be a complex mathematical formula or equation. The man is wearing glasses and has a focused expression on his face, suggesting that he is deeply engrossed in the information on the paper. The paper seems to be the main focus of the scene, as it occupies most of the frame. The man's posture and the angle of his arm suggest that he is carefully examining the details of the diagram, possibly trying to understand or explain it to others. Overall, the image conveys a sense of concentration and intellectual curiosity as the man explores the intricate details of the diagram on the white paper.
Text transcribed from the image:
OMNIV
Jiang*, Fangwen Tu*, Yixuan Long, Aabhaas Vaish, Bowen Zhou, Qinyi Wang, Wel
tion
onidealities, such as pixel and readout latency, significantly
ffect frame interpolation image quality.
der low-light conditions such as indoor capture, EVS response
omes slower, resulting in ghosting or blurry frame.
EVS pixel refractory period are inevitable and worsen artifacts.
Frame Reconstruction
Eqn. (6)
The pixel-wise photocurrent estimation problem is modelled as graph opti
states x = [ipd (k)]
Qipd(1)
Eqn. (7)
VFE(1)
Eqn. (8)
Eqn. (7)
Eqn. (5)
ipd(2)
ZCIS = [DN (1)]
EVS measurement ZEVS = [c(k+1)]T
time
Eqn. (5)
CIS measurement
pd(M)
Eqn. (7)
Eqn. (7)
VFE(N)
Cost function equation:
Eqn. (8)
Eqn. (8)
REVS, CIS diagonal weig
T
x = argmin (eCIS NCISеCIS +еEVSE
ecis, eЕvs: measurement e
pixel
voltage
VDC
event firing
CIS measurement:
updating Vref
charge
VON
DN
light
exposure
charge2v
ADC
VFE
Cdown
VFE
photodiode
i2v
event firing & events
readout
readout latency tri
refractory period trp
EVS measurement:
IZ
pd
in-pixel
off-pixel
time
Hybrid CIS-EVS sensor principle.
fr (ipd): maping ipd onto the time-varying coefficient of the LPF
+1) = f (ipd(k)) (fDC (ipd(k + 1)) - VFE(k))
te
DN
=
G
[ ipa(t)dt
ts
(5)
fDc (ipd): current-voltage conv
(7)
(8)
m: the slope approximation of the VFE curve
AVE(k) =m(k+1) [ton(k) - tin (k)] AVE(k) = m(k+1)trp
(12)
c(k+1) VFE(k + 1) − VFE(k)
When enabling the voltage compensation for readout
latency (RL) and refractory time (RP)
-
c(k+1) ≈ Vfe(k+1)−√FE(k) — AV(k) — ^VÉ (k)
The frame estimation problem is formulated as pixel-wise computati
adjusted CIS timing parameters.
ection,
on, explicitly
time
row exp
row exposure
row exposure
Jupled inverse problem
act removal.
outperforms state-of-the-art methods in
CIS frame zcis(-1)
wing effect with up to 4 dB improvement in PSNR
improvement in LPIPS score.
oint Framework
Frame reconstruction: Provides the
optimization methodology for solving
the coupled inverse problem.
Refinement network: Focuses on
removing noise and improving the
mage quality.
120 FPS blurry CIS input
with RS effect
10,000 FPS output
without RS effect
Frame
reconstruction
Nonideal EVS input
Final output
Refinement
network
Deblurred fraimie with the RS effect
corrected
CIS frame zcis() CIS frame zcis(+1)
Reconstruction using Zcis(-1), ZCIS() and events
| Reconstruction using Zcis(), Zcis(/+1) and events
Each row is reconstructed by merg
consecutive CIS frames and their corre:
events based on the row-specific rolling
timing characteristics.
Refinement Network
The NAFNet basic concept is adopted, and two
major modifications are made:
•
Spatial attention: Focuses on noisy motion areas
for better denoising.
Perceptual loss: Helps recover image quality and
mitigates blurriness or texture degradation.
Loss function:
--
Ls = will + WVGGLVGG + WGram Gram