- Download and install Huddle.Engine. (download link not yet available, see below how to build and run Huddle.Engine)
- Install Meteor (https://www.meteor.com)
- Further details on the Huddle JavaScript API at https://github.com/huddlelamp/meteor-huddle
- Install EmguCV (http://www.emgu.com/ or direct from SourceForge.net http://sourceforge.net/projects/emgucv/)
- Huddle.Engine was implemented with Emgu CV version 2.4.9-alpha
- Copy all dlls from %EMGU_HOME%/bin/x64 to %HUDDLE_ENGINE_HOME%/lib/OpenCV
- Open Huddle.sln in Visual Studio 2013
- Set Huddle.Engine as startup project.
- Rebuild solution!! (this step might be important if an EmguCV exception appears.
- Run Huddle.Engine by starting the project in Visual Studio 2013. (run project in DEBUG mode!!!)
- An error indicating that pipeline could not be loaded might occur. Close exception and UI will appear.
- Click open button in tool bar and open HuddleLamp.hep in %HUDDLE_ENGINE_HOME%/Pipelines/HuddleLamp.hep
- Run pipeline.
This section addresses the development a new processor and its integration into the tracking framework. It is split into the three parts Processor Logic, Processor Properties, and Processor User Interface. If each part is implemented as described, a processor can be integrated into a processing pipeline using the drag and drop user interface of the tracking framework.
A new processor can be integrated into the tracking framework by deriving it from BaseProcessor or RgbProcessor. The RgbProcessor is an abstract base class for image processing that handles basic functions such as rendering pre-process and post-process images into BitmapSource objects. The BitmapSource object can be used later to visualize images apriori and posteriori manipulation. The ProcessAndView method provides access to the image stream of either an InputSource or another processor. For more information on the usage of the BaseProcessor see MergeProcessor.
For the UI a ViewTemplateAttribute with a processor name and a view template name is required. The tracking framework uses the template name to search for the corresponding DataTemplate key within the assembly. A data binding between a processor's properties and its DataTemplate is possible. Adding properties to a processor class is explained below.
/// <summary>
///
/// </summary>
[ViewTemplate("MyProcessor", "MyViewTemplateName")]
public class MyProcessor : RgbProcessor
{
public override Image<Rgb, byte> ProcessAndView(Image<Rgb, byte> image)
{
// manipulation of image goes here
return image;
}
}
A property is a standard .NET property and works with any MVVM framework such as MVVM Light. To enable data binding, a property needs to raise property change events (see INotifyPropertyChanged).
#region FlipHorizontal
/// <summary>
/// The <see cref="FlipHorizontal" /> property's name.
/// </summary>
public const string FlipHorizontalPropertyName = "FlipHorizontal";
private bool _flipHorizontal = false;
/// <summary>
/// Sets and gets the FlipHorizontal property.
/// Changes to that property's value raise the PropertyChanged event.
/// </summary>
public bool FlipHorizontal
{
get
{
return _flipHorizontal;
}
set
{
if (_flipHorizontal == value)
{
return;
}
RaisePropertyChanging(FlipHorizontalPropertyName);
_flipHorizontal = value;
RaisePropertyChanged(FlipHorizontalPropertyName);
}
}
#endregion
IMPORTANT: The current DataContractSerializer uses an opt-in method to serialize properties. Please consult the MSDN Xml serialization website for further information. Therefore, the following is important in order to avoid exception during serialization (e.g., save or load a pipeline):
Add a IgnoreDataMemberAttribute to each property that should not be serialized.
#region PreProcessImage
/// <summary>
/// The <see cref="PreProcessImage" /> property's name.
/// </summary>
public const string PreProcessImagePropertyName = "PreProcessImage";
private BitmapSource _preProcessImage;
/// <summary>
/// Sets and gets the PreProcessImage property.
/// Changes to that property's value raise the PropertyChanged event.
/// </summary>
[IgnoreDataMember]
public BitmapSource PreProcessImage
{
get
{
return _preProcessImage;
}
set
{
if (_preProcessImage == value)
{
return;
}
RaisePropertyChanging(PreProcessImagePropertyName);
_preProcessImage = value;
RaisePropertyChanged(PreProcessImagePropertyName);
}
}
#endregion
Create a new ResourceDictionary or use an existing dictionary and add a new DataTemplate to the dictionary. A processor has a PreProcessImage and a PostProcessImage property by default. Each can be bound to an Image component's source.
Additional properties like the FlipHorizontal property can be bound in order to make them accessible and manipulable in the UI.
<ResourceDictionary xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
xmlns:util="clr-namespace:Tools.FlockingDevice.Tracking.Util"
xmlns:processor="clr-namespace:Tools.FlockingDevice.Tracking.Processor">
<DataTemplate x:Key="MyViewTemplateName" DataType="processor:MyProcessor">
<StackPanel>
<StackPanel Orientation="Horizontal" Margin="5">
<GroupBox Header="Pre Processed Image">
<Image Source="{Binding Path=PreProcessImage}" />
</GroupBox>
<GroupBox Header="Post Processed Image">
<Image Source="{Binding Path=PostProcessImage}" />
</GroupBox>
</StackPanel>
<CheckBox Content="Flip Horizontal" VerticalAlignment="Center" IsChecked="{Binding Path=FlipHorizontal}" />
</StackPanel>
</DataTemplate>
</ResourceDictionary>
- When started, the "Senz3D" node automatically determines the region that the RGB image has within the larger depth image using the UV map. The result is then sent a single time further down the pipeline and can be picked up by the "Basics" node as data of type ROI. The "Basics" node then uses just the region for which RGB and depth information is available.
- The "Basics" node tries to process every input, so that's why "Basics" needs some data filtering before the node.