OffBeatMulticoset Sampling for 4D Capture Beyond the Camera Frame Rate

Chi-Wei Lee, An-Kai Chen, Cheng-Han Chiang, Yi-Chun Chang, Po-Chih Kuo

National Tsing Hua University, Hsinchu, Taiwan

With the same cameras and frame count, staggered trigger times let OffBeat follow the motion between synchronized frames. In the orbit, time runs over the 60 test frames and back.

Abstract

For 4D reconstruction, a synchronized multi-camera rig records all views at the same timestamps and misses the motion between them. Staggered triggers spread the same frames over more timestamps. However, when the cameras along each row fire one after another, as in 4DSloMo's schedule on the Neural 3D Video (N3DV) rig, the row acts as one moving camera that confounds depth with motion. We introduce OffBeat, which models a staggered rig as a spatially distributed multicoset sampler: shifted copies of one sampling grid. From the calibration alone, OffBeat derives camera groups that fire together and their firing order. Since one group records each timestamp, a short-lived primitive can fit an error its cameras share, such as a color offset, and flicker in novel views. We show that a primitive's opacity over time weights all groups equally, for every number of groups, only if it lasts at least one camera frame interval. On Spacetime Gaussians, a minimum lifetime approximates this bound, and a closed-form smoothed box of that length satisfies it exactly, with less flicker than a Gaussian in its place at nearly equal PSNR. In simulated staggered capture on N3DV, OffBeat outperforms the full 4DSloMo pipeline with video diffusion on every test frame, by 3.04 dB in mean dynamic-region PSNR. Between the first and last synchronized frame, it outperforms the best synchronized method of each scene, with or without frame interpolation, by 1.98 dB in mean dynamic-region PSNR, while synchronized capture is better at its own frames. On PanopticSports, in this frame range, OffBeat is comparable in PSNR to synchronized capture with interpolation. On every N3DV scene, replacing 4DSloMo's schedule with ours raises the dynamic-region PSNR of Spacetime Gaussians, 4DSloMo without diffusion, and our reconstruction. Multi-camera rigs could follow faster motion at the same frame count by changing only when each camera fires.

Method

OffBeat overview: staggered capture, period window, shift-and-stack densification and rendering at any time
OffBeat overview. Colors are trigger times within the camera frame interval T.

Results

N3DV

Coffee Martini
Flame Salmon

The held-out test view over frames 100 to 159 shows the ground truth beside seven methods, with the dynamic-region PSNR of every frame below.

Dynamic-region PSNR in dB over frames 100 to 159
MethodCoffee MartiniCook SpinachCut Roasted BeefFlame SalmonFlame SteakSear Steak
Synchronized STG23.1725.8825.9519.9823.8427.16
Synchronized STG + interpolation23.8825.9526.5720.5924.7927.91
4DSloMo, full pipeline22.7925.2626.7521.6125.4627.27
STG on our schedule24.9427.8728.2423.6927.7330.08
OffBeat25.2628.7629.2524.1629.0430.92

Novel views between frames

Basketball, bullet time

In bullet time the camera moves on a novel path while time stays halfway between two synchronized shots, and the lower row freezes a synchronized shot as a control.

PanopticSports

Basketball

On frames 0 to 48, OffBeat is 1.30 dB above synchronized STG and on par with it after interpolation.

BibTeX

@article{lee2026offbeat,
  title   = {OffBeat: Multicoset Sampling for 4D Capture
             Beyond the Camera Frame Rate},
  author  = {Lee, Chi-Wei and Chen, An-Kai and Chiang, Cheng-Han
             and Chang, Yi-Chun and Kuo, Po-Chih},
  journal = {arXiv preprint arXiv:2611.26347},
  year    = {2026}
}