Image Component Library (ICL)
Public Member Functions | Private Attributes | List of all members
icl::filter::ColorSegmentationOp Class Reference

Class for fast LUT-based color segmentation. More...

#include <ColorSegmentationOp.h>

Inheritance diagram for icl::filter::ColorSegmentationOp:
icl::filter::UnaryOp icl::utils::Configurable

Public Member Functions

 ColorSegmentationOp (icl8u c0shift=2, icl8u c1shift=2, icl8u c2shift=2, core::format fmt=core::formatYUV) throw (utils::ICLException)
 Creates a new instance of this class with given segmentation parameters. More...
 
 ~ColorSegmentationOp ()
 Destructor. More...
 
virtual void apply (const core::ImgBase *src, core::ImgBase **dst)
 main apply function More...
 
icl8u classifyPixel (icl8u r, icl8u g, icl8u b)
 classifies a pixel in given rgb format More...
 
void clearLUT (icl8u value=0)
 clears the whole segmentation LUT More...
 
void setSegmentationShifts (icl8u c0shift, icl8u c1shift, icl8u c2shift)
 sets the segmentation shifts More...
 
void setSegmentationFormat (core::format fmt) throw (utils::ICLException)
 sets the internally used segmentation format More...
 
const icl8ugetSegmentationShifts () const
 returns the pointer to the 3 internally used segmentation shifts More...
 
core::format getSegmentationFormat () const
 returns the current internally used segmentation format More...
 
const core::Img8ugetSegmentationPreview ()
 returns in internally managed image of the segmentation result More...
 
void lutEntry (icl8u a, icl8u b, icl8u c, icl8u rA, icl8u rB, icl8u rC, icl8u value)
 given a,b and c in segFormat, this function fills the LUT within a sub-volume with given radii More...
 
void lutEntry (core::format fmt, int a, int b, int c, int rA, int rB, int rC, icl8u value) throw (utils::ICLException)
 given a,b and c in format fmt, this function fills the LUT within a sub-volume with given radii More...
 
void load (const std::string &filename)
 loads the segmentation LUT only (no other parameters) More...
 
void save (const std::string &filename)
 saves the segmentation LUT only (no other parameters) More...
 
const core::Img8ugetLUTPreview (int xDim, int yDim, icl8u zValue)
 this also creates a preview for the current segmentation LUT More...
 
const core::Img8ugetColoredLUTPreview (int xDim, int yDim, icl8u zValue)
 this also creates a colored preview for the current segmentation LUT More...
 
const std::vector< core::Color > & getClassMeanColors ()
 returns a mean color for all used class labels (in rgb format) More...
 
const icl8ugetLUT () const
 returs the internal lut data More...
 
icl8ugetLUT ()
 returns the internal lut data More...
 
void getLUTDims (int &w, int &h, int &t) const
 returns the lut-sizes More...
 
- Public Member Functions inherited from icl::filter::UnaryOp
 UnaryOp ()
 Explicit empty constructor. More...
 
 UnaryOp (const UnaryOp &other)
 Explicit copy constructor. More...
 
UnaryOpoperator= (const UnaryOp &other)
 Explicit declaration of the assignment operator. More...
 
virtual ~UnaryOp ()
 Destructor. More...
 
virtual ICL_DEPRECATED void applyMT (const core::ImgBase *operand1, core::ImgBase **dst, unsigned int nThreads)
 apply function for multithreaded filtering (currently even slower than using one thread) More...
 
virtual const core::ImgBaseapply (const core::ImgBase *src)
 applys the filter usign an internal buffer as output image More...
 
void operator() (const core::ImgBase *src, core::ImgBase **dst)
 function operator (alternative for apply(src,dst) More...
 
const core::ImgBaseoperator() (const core::ImgBase *src)
 function operator for the implicit destination apply(src) call More...
 
const core::ImgBaseoperator() (const core::ImgBase &src)
 reference based function operator More...
 
void setClipToROI (bool bClipToROI)
 sets if the image should be clip to ROI or not More...
 
void setCheckOnly (bool bCheckOnly)
 sets if the destination image should be adapted to the source, or if it is only checked if it can be adapted. More...
 
bool getClipToROI () const
 returns the ClipToROI status More...
 
bool getCheckOnly () const
 returns the CheckOnly status More...
 
virtual void setPropertyValue (const std::string &propertyName, const utils::Any &value) throw (utils::ICLException)
 sets value of a property (always call call_callbacks(propertyName) or Configurable::setPropertyValue) More...
 
- Public Member Functions inherited from icl::utils::Configurable
virtual ~Configurable ()
 virtual destructor More...
 
 Configurable (const Configurable &other)
 Copy constructor. More...
 
Configurableoperator= (const Configurable &other)
 Assignment operator. More...
 
void setConfigurableID (const std::string &ID) throw (ICLException)
 sets the ID of this configurable More...
 
const std::string & getConfigurableID () const
 returns the configurables static ID More...
 
bool isOrderedFlagSet () const
 returns whether the ordered flag is set More...
 
void deactivateProperty (const std::string &pattern)
 adds an additional deativation pattern More...
 
void deleteDeactivationPattern (const std::string &pattern)
 removed a formerly added deactivation pattern More...
 
std::vector< std::string > getPropertyListWithoutDeactivated () const
 this returns a filtered list of properties (using all filters added by deactivateProperty) More...
 
virtual void adaptProperty (const std::string &name, const std::string &newType, const std::string &newInfo, const std::string &newToolTip) throw (ICLException)
 this function can be used to adapt a specific property afterwards More...
 
void registerCallback (const Callback &cb)
 add a callback for changed properties More...
 
void removedCallback (const Callback &cb)
 removes a callback that was registered before More...
 
void syncChangesTo (Configurable *others, int num=1)
 this can be used to let this instance also apply property changes to others More...
 
virtual std::vector< std::string > getPropertyList () const
 returns a list of All properties, that can be set using setProperty More...
 
virtual bool supportsProperty (const std::string &propertyName) const
 base implementation for property check (seaches in the property list) More...
 
virtual void saveProperties (const std::string &filename, const std::vector< std::string > &propertiesToSkip=EMPTY_VEC) const
 writes all available properties into a file More...
 
virtual void loadProperties (const std::string &filename, const std::vector< std::string > &propertiesToSkip=EMPTY_VEC)
 reads a camera config file from disc More...
 
virtual std::string getPropertyType (const std::string &propertyName) const
 get type of property More...
 
virtual std::string getPropertyInfo (const std::string &propertyName) const
 get information of a properties valid values More...
 
virtual Any getPropertyValue (const std::string &propertyName) const
 returns the current value of a property or a parameter More...
 
virtual std::string getPropertyToolTip (const std::string &propertyName) const
 returns the tooltip description for a given property More...
 
virtual int getPropertyVolatileness (const std::string &propertyName) const
 Returns whether this property may be changed internally. More...
 

Private Attributes

core::format m_segFormat
 format, that is used for internal segmentation More...
 
core::Img8u m_inputBuffer
 internal image in depth8u and segmentation format More...
 
core::Img8u m_outputBuffer
 internal buffer holding the output image More...
 
core::Img8u m_segPreview
 internal buffer for providing a preview of the current segmentation More...
 
core::Img8u m_lastDst
 last used destination image More...
 
icl8u m_bitShifts [3]
 bit shifts for all 8-Bit channels More...
 
LUT3D * m_lut
 color classification lookup table More...
 

Additional Inherited Members

- Public Types inherited from icl::utils::Configurable
typedef Function< void, const Property & > Callback
 Function type for changed properties. More...
 
- Static Public Member Functions inherited from icl::filter::UnaryOp
static UnaryOpfromString (const std::string &definition) throw (utils::ICLException)
 Creates a UnaryOp instance from given string definition. More...
 
static std::string getFromStringSyntax (const std::string &opSpecifier) throw (utils::ICLException)
 gives a string syntax description for given opSpecifier More...
 
static std::vector< std::string > listFromStringOps ()
 returns a list of all supported OP_SPEC values for the fromString function More...
 
static void applyFromString (const std::string &definition, const core::ImgBase *src, core::ImgBase **dst) throw (utils::ICLException)
 creates, applies and releases a UnaryOp defined by given definition string More...
 
- Static Public Member Functions inherited from icl::utils::Configurable
static std::string create_default_ID (const std::string &prefix)
 this function can be used in subclasses to create a default ID More...
 
static Configurableget (const std::string &id)
 returns configurable by given ID More...
 
static void register_configurable_type (const std::string &classname, Function< Configurable * > creator) throw (ICLException)
 registers a configurable type More...
 
static std::vector< std::string > get_registered_configurables ()
 returns a list of all registered configurable classnames More...
 
static Configurablecreate_configurable (const std::string &classname) throw (ICLException)
 creates a configurable by given name More...
 
- Static Public Attributes inherited from icl::utils::Configurable
static const std::vector< std::string > EMPTY_VEC
 used as shortcut – just an empty vector of std::strings More...
 
- Protected Member Functions inherited from icl::filter::UnaryOp
bool prepare (core::ImgBase **ppoDst, core::depth eDepth, const utils::Size &imgSize, core::format eFormat, int nChannels, const utils::Rect &roi, utils::Time timestamp=utils::Time::null)
 
virtual bool prepare (core::ImgBase **ppoDst, const core::ImgBase *poSrc)
 check+adapt destination image to properties of given source image More...
 
virtual bool prepare (core::ImgBase **ppoDst, const core::ImgBase *poSrc, core::depth eDepth)
 
- Protected Member Functions inherited from icl::utils::Configurable
void addProperty (const std::string &name, const std::string &type, const std::string &info, const Any &value=Any(), const int volatileness=0, const std::string &tooltip=std::string()) throw (ICLException)
 This can be used by derived classes to store supported properties in the internal list. More...
 
void addChildConfigurable (Configurable *configurable, const std::string &childPrefix="")
 This adds another configurable as child. More...
 
void removeChildConfigurable (Configurable *configurable)
 removes the given child configurable More...
 
Propertyprop (const std::string &propertyName) throw (ICLException)
 this CAN be used e.g. to store a property value in internal property-list More...
 
const Propertyprop (const std::string &propertyName) const throw (ICLException)
 this CAN be used e.g. to store a property value in internal property-list More...
 
 Configurable (const std::string &ID="", bool ordered=true) throw (ICLException)
 create this configurable with given ID More...
 
void call_callbacks (const std::string &propertyName, const Configurable *caller) const
 calls all registered callbacks More...
 
- Protected Attributes inherited from icl::filter::UnaryOp
utils::MultiThreaderm_poMT
 
- Protected Attributes inherited from icl::utils::Configurable
std::vector< Callbackcallbacks
 internally managed list of callbacks More...
 

Detailed Description

Class for fast LUT-based color segmentation.

General Information

Color segmentation is a very common issue in computer vision applications. The ColorSegmentationOp class implements a common and very efficient LUT based segmentation algorithm. It can be used to apply any possible segmentation where the classification of one pixel is only influenced by that pixels color value

The Segmentation Algorithm

Please consider an input color image I. In a first step, I is converted into depth8u and the specified segmentation-color-format (this can be set in the constructor or with the corresponding setter-function ColorSegmentationOp::setSegmentationFormat). This conversion is skipped if the given image parameters are already correct. Now, every pixel pI = (A,B,C) (e.g. if the segmentation format is formatRGB, pI=(R,G,B)) is classified by applying a lookup transformation on it. Basically, we use a lookup table of the whole segmentation color space for this. This implies that size of the lookup table becomes 256*256*256. In order to reduce the amount of cache misses when picking elements of this 16MB LUT, it is possible to adjust the number of bits that are used for each color channel. In the example above, 8 bits (256 values) are used. If the number of bits are shifted (using the constructors channel-shift parameters or the setter function ColorSegmentationOp::setSegmentationShifts), the size of the lookup table becomes smaller and therefore, the lookup operation becomes faster (see Benchmarks:). Another advantage of using the channel-shifts in order to reduce the bits used to represent a certain image channel is that is also provides a better generalization. Consider the following example (see Example (Image Convolution))

Example (Image Convolution)

Segmentation format: YUV, Shifts = (8,0,0).
In this case, the Y (brightness) information is not used for segmentation at all. Therefore the pixel classification does not depend on the pixels brightness, which is very useful for color-segmentation. The size of the used lookup table becomes 256*256 which is 65KB. As the noise level of common cameras is still quite high, it might also be possible to use only the 7 (or even only the 6) most significant bits for the two color channels U and V without a significant reduction of the segmentation quality.

How to Define the Segmentation Lookup Table

Another important question that has not been answered yet is, the lookup table can be set in order to achieve a certain segmentation result. Consider you want to segment a yellow ball in front of a black background. Now, you simply have to set the lookup table values for all possible colors of this ball to a value that is different from 0 (so maybe 200). Then the segmentation result will be white where the ball was, and black (value 0) otherwise. But how can you set up the lookup table values? Of course, it might be quite hard to find all possible colors of the ball, but it might be easy to pick some of its color e.g. by mouse. If you once have these color prototype pixels, you can now use one of the the ColorSegmentationOp::lutEntry functions in order to set up all corresponding lookup table entries to your class label (which was 200 in the example above). The lutEntry methods also provide the functionality to not only set up the prototype-pixel's class-labels, but also the class-lables of all pixels within the vicinity of the prototype pixels. So sometimes, if your 'to-be-segmented'-yellow ball is quite homogeneously yellow, it can be sufficient to add only a single average yellow prototype pixel with high radii for the single color components.
If you don't want to implement mouse handling in order to be able to click at your objects to get color-prototypes, you can also use the ICL-example tool 'icl-color-picker'.

Using more than one Class

By adding color prototypes for different objects, but with different class labels, you can also use this class to segment several objects at once. In the next step, you will possibly use an instance of ICLCV/RegionDetector to extract the segemented image regions. Here, you can then obtain the corresponding class label by using the ImageRegion's getVal method. Please note that always the last lut-entry is used if your prototype-entries overlap.

Restrictions

Maybe it is now, if not before necessary to mention, that the ColorSegmentationOp can only be set up to classify pixels into 255 valid classes. But actually, this should not become a problem at all as the classification quality usually restricts the number of classes to a maximum of about 10.

Benchmarks:

Constructor & Destructor Documentation

icl::filter::ColorSegmentationOp::ColorSegmentationOp ( icl8u  c0shift = 2,
icl8u  c1shift = 2,
icl8u  c2shift = 2,
core::format  fmt = core::formatYUV 
)
throw (utils::ICLException
)

Creates a new instance of this class with given segmentation parameters.

Parameters
c0shiftnumber of least significant bits that are removed from channel0
c1shiftnumber of least significant bits that are removed from channel1
c2shiftnumber of least significant bits that are removed from channel2
fmtinternally used segmentation format (needs to have 3 channels)
icl::filter::ColorSegmentationOp::~ColorSegmentationOp ( )

Destructor.

Member Function Documentation

virtual void icl::filter::ColorSegmentationOp::apply ( const core::ImgBase src,
core::ImgBase **  dst 
)
virtual

main apply function

Implements icl::filter::UnaryOp.

icl8u icl::filter::ColorSegmentationOp::classifyPixel ( icl8u  r,
icl8u  g,
icl8u  b 
)

classifies a pixel in given rgb format

void icl::filter::ColorSegmentationOp::clearLUT ( icl8u  value = 0)

clears the whole segmentation LUT

const std::vector<core::Color>& icl::filter::ColorSegmentationOp::getClassMeanColors ( )

returns a mean color for all used class labels (in rgb format)

const core::Img8u& icl::filter::ColorSegmentationOp::getColoredLUTPreview ( int  xDim,
int  yDim,
icl8u  zValue 
)

this also creates a colored preview for the current segmentation LUT

the method behaves like getLUTPreview, except, it tints the resulting lut-slice with mean colors of that class

const icl8u* icl::filter::ColorSegmentationOp::getLUT ( ) const

returs the internal lut data

The data order is depth-major row-major i.e. an index(x,y,z) is estimated by (x + w*y + w*h * z)

icl8u* icl::filter::ColorSegmentationOp::getLUT ( )

returns the internal lut data

Please be careful with this method :-)

void icl::filter::ColorSegmentationOp::getLUTDims ( int &  w,
int &  h,
int &  t 
) const

returns the lut-sizes

w = 1+(0xff >> bitShift[0]) etc.

const core::Img8u& icl::filter::ColorSegmentationOp::getLUTPreview ( int  xDim,
int  yDim,
icl8u  zValue 
)

this also creates a preview for the current segmentation LUT

Here, you can choose, which dimension shall become the resulting images with and height. The remaining 3rd dimension is slices with given zValue

core::format icl::filter::ColorSegmentationOp::getSegmentationFormat ( ) const
inline

returns the current internally used segmentation format

const core::Img8u& icl::filter::ColorSegmentationOp::getSegmentationPreview ( )

returns in internally managed image of the segmentation result

The image has the dimensions 2^(8-shift0) x 2^(8-shift1) and it has 2^(8-shift3) channels

const icl8u* icl::filter::ColorSegmentationOp::getSegmentationShifts ( ) const
inline

returns the pointer to the 3 internally used segmentation shifts

void icl::filter::ColorSegmentationOp::load ( const std::string &  filename)

loads the segmentation LUT only (no other parameters)

void icl::filter::ColorSegmentationOp::lutEntry ( icl8u  a,
icl8u  b,
icl8u  c,
icl8u  rA,
icl8u  rB,
icl8u  rC,
icl8u  value 
)

given a,b and c in segFormat, this function fills the LUT within a sub-volume with given radii

void icl::filter::ColorSegmentationOp::lutEntry ( core::format  fmt,
int  a,
int  b,
int  c,
int  rA,
int  rB,
int  rC,
icl8u  value 
)
throw (utils::ICLException
)

given a,b and c in format fmt, this function fills the LUT within a sub-volume with given radii

void icl::filter::ColorSegmentationOp::save ( const std::string &  filename)

saves the segmentation LUT only (no other parameters)

void icl::filter::ColorSegmentationOp::setSegmentationFormat ( core::format  fmt)
throw (utils::ICLException
)

sets the internally used segmentation format

void icl::filter::ColorSegmentationOp::setSegmentationShifts ( icl8u  c0shift,
icl8u  c1shift,
icl8u  c2shift 
)

sets the segmentation shifts

Member Data Documentation

icl8u icl::filter::ColorSegmentationOp::m_bitShifts[3]
private

bit shifts for all 8-Bit channels

core::Img8u icl::filter::ColorSegmentationOp::m_inputBuffer
private

internal image in depth8u and segmentation format

core::Img8u icl::filter::ColorSegmentationOp::m_lastDst
private

last used destination image

LUT3D* icl::filter::ColorSegmentationOp::m_lut
private

color classification lookup table

core::Img8u icl::filter::ColorSegmentationOp::m_outputBuffer
private

internal buffer holding the output image

core::format icl::filter::ColorSegmentationOp::m_segFormat
private

format, that is used for internal segmentation

core::Img8u icl::filter::ColorSegmentationOp::m_segPreview
private

internal buffer for providing a preview of the current segmentation


The documentation for this class was generated from the following file: