Data flow programming
Object oriented languages provide the programmer with many useful
tools for abstracting problems. It is however only one way of looking
at the structure of a program, and in some cases it is useful to use
other models for writting a program. One that is often useful in
image processing is the 'Data Flow' model. This section outlines a set
of classes that allow this programming model to be easly expressed in
NOTE: For clarity the complete Data Processing system is documented here in the
core documentation even though parts of the implementation is in the RavlOS module.
Often we want to process a stream of data, not just a single
element. It is this kind of processing where we can get real benefits
from using more explicit representation of the data flow through a
program. The flow of data in a program is only linear in the simplest
cases, so to make things clearer we will adopt a diagram notation for
the data stream. The basic components of these diagrams are show in
Figures 1(a), 1(b), 1(c), 1(d) and
1(e). As stated earlier the convention of keeping the data
flow from left to right is maintained.
In the stream model we need a `source' of data to process as shown in
Figure 1(a) and a `sink' to send the data to, when it has
been processed, seen in Figure 1(b). In this library both
the source and the sink (or input and output) are passive. To drive
the flow of data we need a component drive the flow, this is the
`pump' shown in Figure 1(c). Figure 1(d) shows a
FIFO (First In First Out) buffer, in a queue of data for processing
can be stored. And finally we need to do some processing or
transformations on the data, this is done by a process, shown in
All these components have corresponding classes in the system. Sources
or data inputs are embodied by the templated class DPIPortC<x>.
Data sinks or outputs are embodied by DPOPortC<x>. There are two
basic types of `pump'. DPIOConnectC<> attempts to copy the entire
contents of the stream immediately. There is also DPMTIOConnectC<>
which starts a new thread to copy the data. For the threaded version
of the pump we use a different symbol, >>= to make it clear where in
the pipe the pump is located.
Since each input and output of all the comments are templated it is possible
to send almost any `C++' object through any part of the pipe. The only constraints,
mentioned earlier, are having a copy constructor and an assignment operator. Since
the processing pipe is built with template functions, the type of data can be
changed arbitrarily by processing operations, and as long as the input and output
types match the user need not intervene.
Data processing Components
|(a) Data source ||(b) Data sink || (c) Pump || (d) FIFO Buffer || (e) Process |
Figure 2 shows a simple process taking data in
processing it and outputting it. The circle indicates where the
'pump' for the pipe is located. In this case the pipe is driven from
its sink end, pulling data through the pipe. By default the system puts
the pump at the data sink. This choice is based on the view that you
should only process data as and when it is needed.
Simple data pipe.
Here is an example of a pipe using this type of processing. The source and sinks are
classed derived from the templated classes DPIPortC<x> and DPOPortC<x>, they
convert data too and from the standard C++ text streams. The following code takes
a series of numbers from the file `in.dat' adds an offset of 2 and scales it by 3,
calculates a running average of the data and outputs it to `out.dat'
DPIFileC<RealT>("in.dat") >> DPOffsetScale(2.0,3.0) >> DPRunningAverage(1.0,5) >> DPOFileC<RealT>("out.dat");
operator >>(const DPIOPortC &,DPIStreamOpC)
Connect(const DPIPortC &,const DPOPortC &)
operator >>(const DPIPortC &,const DPOPortC &)
operator >>(const DPIPortC &,const DPIOPortC &)
operator >>(const DPIOPortC &,const DPOPortC &)
operator >>=(const DPIPortC &,const DPOPortC &)
operator >>=(const DPIPortC &,const DPIOPortC &)
operator >>=(const DPIOPortC &,const DPOPortC &)
operator >>=(const DPIOPortC &,const DPIOPortC &)
DPMTIOConnect(const DPIPortC &,const DPOPortC &)