VisionX: Implementing ImageProcessors

Image processors in VisionX allow to retrieve images from an ImageProvider and process them. The visionx::ImageProcessor ManagedIceObject is the basic superclass for image processors. The superclass provides the following features

• image transport via ethernet or shared memory: whether shared memory is used or not is determined by comparing the IP adresses of provider and processor
• automatic conversion to the requested image format (if possible)
• optional provision of result / visualization images using an embedded visionx::ImageProvider
• usage of multiple image providers as input

Creating an imageprocessor

Integrating a new image processor is achieved by subclassing the visionx::ImageProcessor in the following way:

#include <VisionX/core/ImageProcessor.h>
 
namespace visionx
{
    class ExampleImageProcessor :
        virtual public visionx::ImageProcessor
    {
    public:
        /**
         * @see armarx::ManagedIceObject::onInitComponent()
         */
        virtual void onInitImageProcessor();
 
        /**
         * @see armarx::ManagedIceObject::onConnectComponent()
         */
        virtual void onConnectImageProcessor()  { }
 
        /**
         * @see armarx::ManagedIceObject::onDisconnectComponent()
         */
        virtual void onDisconnectImageProcessor() { }
 
        /**
         * @see armarx::ManagedIceObject::onExitComponent()
         */
        virtual void onExitImageProcessor() { }
 
        /**
         * @see armarx::Component::getDefaultName()
         */
        virtual std::string getDefaultName() const
        {
            return "ExampleImageProcessor";
        }
 
        /**
         * Process the vision component.
         *
         * The main loop of the imageprocessor to be implemented in the
         * subclass. Do not block this method. One process should execute
         * exactly one image processing step.
         */
        virtual void process() = 0;
 
    };
}

The methods that need to be implemented are derived from the armarx::ManagedIceObject :

• The visionx::ImageProcessor::onInitImageProcessor method is called once the object is constructred
• The visionx::ImageProcessor::onConnectImageProcessor method is called once all network dependencies are resolved
• the visionx::ImageProcessor::onDisconnectImageProcessor method is called if a network dependency is lost
• the visionx::ImageProcessor::onExitImageProcessor method is called once the application is terminated

As for every ManagedIceObject, the default name of the object needs to be provided using the getDefaultName() method.

The process method should be used to implement the processing of input images. It is started in a separate thread once the ImageProcessor is fully connected.

Setting up an imageprocessor

The setup of the image providers settings such as image format and number of images needs to be performed in the onInitImageProvider method:

void ExampleImageProcessor::onInitImageProcessor()
{
    // make dependency to image provider
    usingImageProvider("DummyImageProvider");
}
 
void ExampleImageProcessor::onConnectImageProcessor()
{
    // retrieve image provider "DummyImageProvider" and request RGB type images
    ImageProviderInfo imageProvider = getImageProvider("DummyImageProvider", eRbg);
}

The image is automatically converted from the ImageProvider image type to the desired image type, if possible. Possible image types are:

• eBayerPattern
• eGrayScale
• eRgb
• eFloat1Channel
• eFloat3Channels

Multiple providers can be used by issuing multiple usingImageProvider and getImageProvider commands.

Retrieving images

In order to process the images, the pure virtual process method can be implemented and several utility methods can be used to retrieve the images. The process method is called repeatedly, exit checking is done outside the process method. Consequently, the implementation of the process method should not block or loop but only trigger one processing step.

There are essentially two types of accessing the images: polling and waiting.

Retrieving images by polling

For retrieving images by polling, the visionx::ImageProcessor::getImages can be used in the following way:

void ExampleImageProcessor::process()
{
    int numberImages = getImages("DummyImageProvider", images);
    if(numberImages > 0)
        doImageProcessing(images);
}

The getImages method will return the most recent images available and the number of images a provided by the ImageProvider. This polling method is useful, if the image processing usually takes much longer then the framerate of input images.

Retrieving images by waiting

For retrieving images by waiting, the visionx::ImageProcessor::waitForImages method can be used

void ExampleImageProcessor::process()
{
    if(!waitForImages("DummyImageProvider"))
    {
        printf("Timeout or error in wait for images\n");
    }
    else
    {
        // retrieve images
        int numberImages = getImages("DummyImageProvider", images);
        doImageProcessing(images);
    }
}

The waitForImages method waits until new images are available from the provider. This guarantees that a subsequent call to getImages will succeed in reading the images.