examples/nerf.py
def forward(
        self,
        ray_bundle: RayBundle,
        path = None,
        **kwargs,
    ):
        """
        The forward function accepts the parametrizations of
        3D points sampled along projection rays. The forward
        pass is responsible for attaching a 3D vector
        and a 1D scalar representing the point's
        RGB color and opacity respectively.

        Args:
            ray_bundle: A RayBundle object containing the following variables:
                origins: A tensor of shape `(minibatch, ..., 3)` denoting the
                    origins of the sampling rays in world coords.
                directions: A tensor of shape `(minibatch, ..., 3)`
                    containing the direction vectors of sampling rays in world coords.
                lengths: A tensor of shape `(minibatch, ..., num_points_per_ray)`
                    containing the lengths at which the rays are sampled.

        Returns:
            rays_densities: A tensor of shape `(minibatch, ..., num_points_per_ray, 1)`
                denoting the opacitiy of each ray point.
            rays_colors: A tensor of shape `(minibatch, ..., num_points_per_ray, 3)`
                denoting the color of each ray point.
        """
        # We first convert the ray parametrizations to world
        # coordinates with `ray_bundle_to_ray_points`.
        rays_points_world = ray_bundle_to_ray_points(ray_bundle)
        # rays_points_world.shape = [minibatch x ... x 3]

        # For each 3D world coordinate, we obtain its harmonic embedding.
        embeds = self.harmonic_embedding(
            rays_points_world
        )
        # embeds.shape = [minibatch x ... x self.n_harmonic_functions*6]

        # self.mlp maps each harmonic embedding to a latent feature space.
        features = self.mlp(embeds)
        # features.shape = [minibatch x ... x n_hidden_neurons]

        # Finally, given the per-point features,
        # execute the density and color branches.

        rays_densities = self._get_densities(features)
        # rays_densities.shape = [minibatch x ... x 1]

        rays_colors = self._get_colors(features, ray_bundle.directions)
        # rays_colors.shape = [minibatch x ... x 3]

        if path is not None:
            save_io_nerf(rays_points_world, rays_densities, rays_colors, path)

        return rays_densities, rays_colors

        The forward function accepts the parametrizations of
        3D points sampled along projection rays. The forward
        pass is responsible for attaching a 3D vector
        and a 1D scalar representing the point's
        RGB color and opacity respectively.

        Args:
            ray_bundle: A RayBundle object containing the following variables:
                origins: A tensor of shape `(minibatch, ..., 3)` denoting the
                    origins of the sampling rays in world coords.
                directions: A tensor of shape `(minibatch, ..., 3)`
                    containing the direction vectors of sampling rays in world coords.
                lengths: A tensor of shape `(minibatch, ..., num_points_per_ray)`
                    containing the lengths at which the rays are sampled.

        Returns:
            rays_densities: A tensor of shape `(minibatch, ..., num_points_per_ray, 1)`
                denoting the opacitiy of each ray point.
            rays_colors: A tensor of shape `(minibatch, ..., num_points_per_ray, 3)`
                denoting the color of each ray point.
        
The forward function accepts the parametrizations of
3D points sampled along projection rays. The forward
pass is responsible for attaching a 3D vector
and a 1D scalar representing the point's
RGB color and opacity respectively.
The forward function accepts the parametrizations of 3D points sampled along projection rays . The forward pass is responsible for attaching a 3D vector and a 1D scalar representing the point ' s RGB color and opacity respectively .
def forward(
        self,
        ray_bundle: RayBundle,
        path = None,
        **kwargs,
    ):
        rays_points_world = ray_bundle_to_ray_points(ray_bundle)
        embeds = self.harmonic_embedding(
            rays_points_world
        )
        features = self.mlp(embeds)
        rays_densities = self._get_densities(features)
        rays_colors = self._get_colors(features, ray_bundle.directions)
        if path is not None:
            save_io_nerf(rays_points_world, rays_densities, rays_colors, path)
        return rays_densities, rays_colors
def forward ( self , ray_bundle : RayBundle , path = None , ** kwargs , ) : rays_points_world = ray_bundle_to_ray_points ( ray_bundle ) embeds = self . harmonic_embedding ( rays_points_world ) features = self . mlp ( embeds ) rays_densities = self . _get_densities ( features ) rays_colors = self . _get_colors ( features , ray_bundle . directions ) if path is not None : save_io_nerf ( rays_points_world , rays_densities , rays_colors , path ) return rays_densities , rays_colors
The forward function accepts the parametrizations of
3D points sampled along projection rays.
The forward function accepts the parametrizations of 3D points sampled along projection rays .
"""
        The forward function accepts the parametrizations of
        3D points sampled along projection rays. The forward
        pass is responsible for attaching a 3D vector
        and a 1D scalar representing the point's
        RGB color and opacity respectively.

        Args:
            ray_bundle: A RayBundle object containing the following variables:
                origins: A tensor of shape `(minibatch, ..., 3)` denoting the
                    origins of the sampling rays in world coords.
                directions: A tensor of shape `(minibatch, ..., 3)`
                    containing the direction vectors of sampling rays in world coords.
                lengths: A tensor of shape `(minibatch, ..., num_points_per_ray)`
                    containing the lengths at which the rays are sampled.

        Returns:
            rays_densities: A tensor of shape `(minibatch, ..., num_points_per_ray, 1)`
                denoting the opacitiy of each ray point.
            rays_colors: A tensor of shape `(minibatch, ..., num_points_per_ray, 3)`
                denoting the color of each ray point.
        """ # We first convert the ray parametrizations to world # coordinates with `ray_bundle_to_ray_points`. # rays_points_world.shape = [minibatch x ... x 3] # For each 3D world coordinate, we obtain its harmonic embedding. # embeds.shape = [minibatch x ... x self.n_harmonic_functions*6] # self.mlp maps each harmonic embedding to a latent feature space. # features.shape = [minibatch x ... x n_hidden_neurons] # Finally, given the per-point features, # execute the density and color branches. # rays_densities.shape = [minibatch x ... x 1] # rays_colors.shape = [minibatch x ... x 3]
A tensor of shape `(minibatch, ..., num_points_per_ray, 1)`
denoting the opacitiy of each ray point.
rays_colors: A tensor of shape `(minibatch, ..., num_points_per_ray, 3)`
denoting the color of each ray point.
A tensor of shape ` ( minibatch ... num_points_per_ray 1 ) ` denoting the opacitiy of each ray point . rays_colors : A tensor of shape ` ( minibatch ... num_points_per_ray 3 ) ` denoting the color of each ray point .
rays_densities
self
ray_bundle
RayBundle
A RayBundle object containing the following variables:
origins: A tensor of shape `(minibatch, ..., 3)` denoting the
origins of the sampling rays in world coords.
directions: A tensor of shape `(minibatch, ..., 3)`
containing the direction vectors of sampling rays in world coords.
lengths: A tensor of shape `(minibatch, ..., num_points_per_ray)`
containing the lengths at which the rays are sampled.
A RayBundle object containing the following variables : origins : A tensor of shape ` ( minibatch ... 3 ) ` denoting the origins of the sampling rays in world coords . directions : A tensor of shape ` ( minibatch ... 3 ) ` containing the direction vectors of sampling rays in world coords . lengths : A tensor of shape ` ( minibatch ... num_points_per_ray ) ` containing the lengths at which the rays are sampled .
path