visionsim.emulate package

Subpackages

• visionsim.emulate.dvs package
  • Module contents

Submodules

visionsim.emulate.imu module

visionsim.emulate.imu.imu_integration(acc_pos: Iterable[npt.ArrayLike], vel_ang: Iterable[npt.ArrayLike], dt: float, gravity: npt.NDArray | None = None, pose_init: npt.NDArray | None = None, vel_init: npt.NDArray | None = None) → Iterator[npt.NDArray]

Integrate IMU measurements and estimate trajectory using forward Euler integration [1].

Parameters:

• acc_pos (Iterable[npt.ArrayLike]) – Positional acceleration as measured by the IMU. Expects an iterable of [ax, ay, az] vectors in m/s^2 (in camera-coordinates).

• vel_ang (Iterable[npt.ArrayLike]) – Rotational velocity as measured by the gyro. Expects an iterable of [wx, wy, wz] vectors in Rad/s (in camera-coordinates).

• dt (float) – Sampling period in seconds. Typically equal to 1/fps.

• gravity (npt.NDArray, optional) – Gravity vector in m/s^2 (in world-coordinates). Defaults to -9.8 m/s^2 in Z.

• pose_init (npt.NDArray, optional) – Initial pose. Defaults to identity.

• vel_init (npt.NDArray, optional) – Initial positional velocity. Defaults to the zero vector.

Yields:

Iterator[npt.NDArray] – Estimated pose

References

[1] (1,2,3)

Forster et al. (2015), “IMU Preintegration on Manifold for Efficient Visual-Inertial Maximum-a-Posteriori Estimation”.

visionsim.emulate.imu.imu_integration_step(pose: npt.NDArray, vel_pos: npt.NDArray, vel_ang: npt.NDArray, acc_pos: npt.NDArray, dt: float) → tuple[npt.NDArray, npt.NDArray]

Computes single Euler integration step [1].

While the integration is performed in world coordinates, this helper operates on velocities and accelerations in camera coordinates and perform the coordinate change internally. Poses remain in world coordinates.

Note

Angular acceleration handling is not implemented!

Parameters:

• pose (npt.NDArray) – Current camera pose in world coordinates.

• vel_pos_c (npt.NDArray) – Translational velocity in camera coordinates.

• vel_ang_c (npt.NDArray) – Angular velocity in camera coordinates.

• acc_pos_c (npt.NDArray) – Translational acceleration in camera coordinates.

• dt (float) – Sampling period in seconds.

Returns:

Camera pose at next time step (in world-coords), Translational velocity at next time step (in camera coords)

Return type:

tuple[npt.NDArray, npt.NDArray]

visionsim.emulate.imu.emulate_imu(poses: list[npt.NDArray] | npt.NDArray, *, dt: float = 0.00125, std_acc: float = 0.008, std_gyro: float = 0.0012, std_bias_acc: float = 5.5e-05, std_bias_gyro: float = 2e-05, init_bias_acc: npt.NDArray | None = None, init_bias_gyro: npt.NDArray | None = None, gravity: npt.NDArray | None = None, rng: np.random.Generator | None = None) → Iterator[dict[str, npt.ArrayLike]]

Emulate IMU measurements from a sequence of ground-truth poses.

Follows the Appendix in Crassidis (2006) [2], also see Sec. IV.B. in Leutenegger et al. (2015) [3], and Sec. IV in Forster et al. (2015) [1].

The default parameter values are taken from Table I in Leutenegger et al [3].

Parameters:

• poses (list[npt.NDArray] | npt.NDArray) – Sequence of ground-truth poses to emulate IMU from.

• dt (float, optional) – Sampling period in seconds. Defaults to 1/800.

• std_acc (float, optional) – Standard deviation for positional acceleration in m/(s^2 sqrt(Hz)). Defaults to 8e-3.

• std_gyro (float, optional) – Standard deviation for angular velocity in rad/(s sqrt(Hz)). Defaults to 1.2e-3.

• std_bias_acc (float, optional) – Bias for positional acceleration in m/(s^3 sqrt(Hz)). Defaults to 5.5e-5.

• std_bias_gyro (float, optional) – Bias for angular velocity in rad/(s^2 sqrt(Hz)). Defaults to 2e-5.

• init_bias_acc (npt.NDArray, optional) – Initial positional acceleration. Defaults to the zero vector.

• init_bias_gyro (npt.NDArray, optional) – Initial angular velocity. Defaults to the zero vector.

• gravity (npt.NDArray, optional) – Gravity vector in m/s^2 (in world-coordinates). Defaults to -9.8 m/s^2 in Z.

• rng (np.random.Generator, optional) – Random generator instance. Defaults to np.random.default_rng.

Yields:

Iterator[dict[str, npt.ArrayLike]] – Return “acc_reading”, “gyro_reading”, “acc_bias”, “gyro_bias”, and “t”.

References

[2]

Crassidis (2006), “Sigma-Point Kalman Filtering for Integrated GPS and Inertial Navigation”.

[3] (1,2)

Leutenegger et al. (2015), “Keyframe-based visual-inertial odometry using nonlinear optimization”.

visionsim.emulate.rgb module

visionsim.emulate.rgb.emulate_rgb_from_sequence(sequence: npt.ArrayLike, readout_std: float = 20.0, fwc: float = 500.0, bitdepth: int = 12, factor: float = 1.0, rng: np.random.Generator | None = None) → npt.NDArray

Emulates a conventional RGB camera from a sequence of intensity frames.

Note

Motion-blur is approximated by averaging consecutive ground truth frames, this can be done more efficiently if optical flow is available. See emulate_rgb_from_flow for more.

Parameters:

• sequence (npt.ArrayLike) – Input sequence of linear-intensity frames, can be a collection of frames, or np/torch array with time as the first dimension.

• readout_std (float, optional) – Standard deviation of zero mean Gaussian read noise. Defaults to 20.0.

• fwc (float, optional) – Full well capacity, used for normalization. Defaults to 500.0.

• bitdepth (int, optional) – Resolution of ADC in bits. Defaults to 12.

• factor (float, optional) – Scaling factor to control intensity of output RGB image. Defaults to 1.0.

• rng (np.random.Generator, optional) – Optional random number generator. Defaults to none.

Returns:

Quantized sRGB patch is returned

visionsim.emulate.rgb.emulate_rgb_from_flow()

Not (Yet) Implemented

visionsim.emulate.spc module

visionsim.emulate.spc.emulate_spc(img: npt.NDArray[np.floating], factor: float = 1.0, rng: np.random.Generator | None = None) → npt.NDArray[np.integer]

Perform bernoulli sampling on linearized RGB frames to yield binary frames.

Parameters:

• img (npt.ArrayLike) – Linear intensity image to sample binary frame from.

• factor (float, optional) – Arbitrary corrective brightness factor. Defaults to 1.0.

• rng (np.random.Generator, optional) – Optional random number generator. Defaults to none.

Returns:

Binary single photon frame

visionsim.emulate.spc.spc_avg_to_rgb(mean_binary_patch: torch.Tensor | npt.NDArray, factor: float = 1.0, epsilon: float = 1e-06, quantile: float | None = None) → torch.Tensor | npt.NDArray

Convert average of many single photon binary frames to conventional RGB image.

Invert the process by which binary frames are emulated. The result can be either linear RGB values or sRGB values depending on how the binary frames were constructed.

Assuming each binary patch was created by a Bernoulli process with p=1-exp(-factor*rgb), then the average of binary frames tends to p. We can therefore recover the original rgb values as -log(1-bin)/factor where bin is the average of many binary frames.

Parameters:

• mean_binary_patch (torch.Tensor | npt.NDArray) – Binary avg to convert to rgb, expects a np.ndarray or torch tensor.

• factor (float, optional) – Arbitrary corrective brightness factor. Defaults to 1.0.

• epsilon (float, optional) – Smallest allowed value before taking logarithm, needed for stability. Defaults to 1e-6.

• quantile (float, optional) – If supplied, normalize by this quantile. For instance, if set to 0.9 then the ninety-percent brightest value will be used as the new maximum, with brighter values being clipped. Defaults to None (no normalization/clipping).

Returns:

Conventional intensity image

Module contents