VisionX: Implementing PointCloudProviders

PointCloud providers in VisionX allow to send pointclouds in the ArmarX framework. The visionx::PointCloudProvider ManagedIceObject is the basic superclass for pointcloud providers. The superclass provides the following features:

• pointcloud transport via ethernet or shared memory: whether shared memory is used or not is determined by comparing the IP adresses of provider and processor
• optional support for frame per second synchronization using the VisionX::CapturingPointCloudProvider as superclass
• visualization of the pointclouds using the PointCloudMonitor plugin for ArmarXGui

Creating a pointcloudprovider

PointCloud providers abstract pointcloud components such as RGB-D sensors. How to integrate a new pointcloud source as pointcloud provider is achieved by subclassing the visionx::PointCloudProvider or visionx::CapturingPointCloudProvider in the following way:

#include <VisionX/components/pointcloud_core/CapturingPointCloudProvider.h>
 
namespace visionx
{
    class ExamplePointCloudProvider :
        virtual public visionx::CapturingPointCloudProvider
    {
    public:
        /**
         * @see armarx::ManagedIceObject::onInitComponent()
         */
        virtual void onInitCapturingPointCloudProvider();
 
        /**
         * @see armarx::ManagedIceObject::onConnectComponent()
         */
        virtual void onConnectCapturingPointCloudProvider()  { }
 
        /**
         * @see armarx::ManagedIceObject::onDisconnectComponent()
         */
        virtual void onDisconnectCapturingPointCloudProvider() { }
 
        /**
         * @see armarx::ManagedIceObject::onExitComponent()
         */
        virtual void onExitCapturingPointCloudProvider() { }
 
        /**
         * @see armarx::Component::getDefaultName()
         */
        virtual std::string getDefaultName() const
        {
            return "ExamplePointCloudProvider";
        }
    };
}

The methods that need to be implemented are derived from the armarx::ManagedIceObject :

• The visionx::PointCloudProvider::onInitCapturingPointCloudProvider method is called once the object is constructred
• The visionx::PointCloudProvider::onConnectCapturingPointCloudProvider method is called once all network dependencies are resolved
• the visionx::PointCloudProvider::onDisconnectCapturingPointCloudProvider method is called if a network dependency is lost
• the visionx::PointCloudProvider::onExitCapturingPointCloudProvider method is called once the application is terminated

As for every ManagedIceObject, the default name of the object needs to be provided using the getDefaultName() method.

Setting up an pointcloudprovider

The Pointcloud content type can be changed on the fly. However, for shared memory one need to know the maximum memory size in advance in order to allocate the buffer. Please note that the size also depends on the point cloud type. Default allocated memory size is for 640 x 480 pcl::PointXYZRGBA point clouds. In case you need more memory just override visionx::PointCloudProvider::getDefaultPointCloudFormat() and set the capacity manually:

MetaPointCloudFormatPtr ExamplePointCloudProvider::getDefaultPointCloudFormat()
{
    MetaPointCloudFormatPtr info = new MetaPointCloudFormat();
    info->capacity = 1600 * 1200 * sizeof(ColoredPoint3D);
    info->size = info->capacity;
    info->type = PointContentType::eCbrCbgCbbCba_PfxPfyPfz;
    return info;
}

Providing pointclouds

The method visionx::PointCloudProvider::providePointCloud<PointT>(pointCloudPtr) takes an arbitrary pointcloud of type PointT, the user needs to take care that the size does not exceed the allocated buffer specified for the shared memory transfer

Using the capturing pointcloud provider

Most pointcloud providers will have a capturing thread, that performs the capturing of the pointclouds. Such a capturing thread is part of the visionx::CapturingPointCloudProvider:

/**
 * Main capturing function.
 */
virtual bool capture() = 0;

The capture method needs to be overwritten in order to fill the pointcloud buffers. Thereby, the locking of the write mutex needs to be performed by the user again:

bool ExamplePointCloudProvider::capture()
{
 
    // do not forget to lock your variable pointCloudPtr 
    providePointCloud<pcl::PointXYZ>(pointCloudPtr);
 
    return true;
}

Capturing pointcloud providers have two different methods of invocing the capture method:

• eCaptureSynchronization: the capture methods itself handles the timing and synchronization
• eFpsSynchronization: the framework will call the capture method with the fps rate as specified in the CapturingPointCloudProvider::startCapture(float fps)

The synchronization type needs to be specified in the onInit method using:

void CapturingPointCloudProvider::setPointCloudSyncMode(PointCloudSyncMode pointcloudSyncMode)