SUBTOPICS

Introduction - Introduction Single Items - Processing single data items Ports - Data sources and sinks IO - Specialist Input/Output ports Processes - Complex processes Split - Spliting streams Taps - Taps Buffers - Queuing buffers Multiplexer - Multiplexers Threads - Threading Hold - Last value buffer Fail Over - Fail over switch Seekable Ports Events - Event handling Seekable Ports - Random access ports Internal Extras

Data flow programming

Object oriented languages provide the programmer with many useful tools for abstracting problems. It is however only one way of looking at the structure of a program, and in some cases it is useful to use other models for writting a program. One that is often useful in image processing is the 'Data Flow' model. This section outlines a set of classes that allow this programming model to be easly expressed in C++ program.

---

NOTE: For clarity the complete Data Processing system is documented here in the core documentation even though parts of the implementation is in the RavlOS module.

Often we want to process a stream of data, not just a single element. It is this kind of processing where we can get real benefits from using more explicit representation of the data flow through a program. The flow of data in a program is only linear in the simplest cases, so to make things clearer we will adopt a diagram notation for the data stream. The basic components of these diagrams are show in Figures 1(a), 1(b), 1(c), 1(d) and 1(e). As stated earlier the convention of keeping the data flow from left to right is maintained.

In the stream model we need a `source' of data to process as shown in Figure 1(a) and a `sink' to send the data to, when it has been processed, seen in Figure 1(b). In this library both the source and the sink (or input and output) are passive. To drive the flow of data we need a component drive the flow, this is the `pump' shown in Figure 1(c). Figure 1(d) shows a FIFO (First In First Out) buffer, in a queue of data for processing can be stored. And finally we need to do some processing or transformations on the data, this is done by a process, shown in Figure 1(e).

All these components have corresponding classes in the system. Sources or data inputs are embodied by the templated class DPIPortC<x>. Data sinks or outputs are embodied by DPOPortC<x>. There are two basic types of `pump'. DPIOConnectC<> attempts to copy the entire contents of the stream immediately. There is also DPMTIOConnectC<> which starts a new thread to copy the data. For the threaded version of the pump we use a different symbol, >>= to make it clear where in the pipe the pump is located.

Since each input and output of all the comments are templated it is possible to send almost any `C++' object through any part of the pipe. The only constraints, mentioned earlier, are having a copy constructor and an assignment operator. Since the processing pipe is built with template functions, the type of data can be changed arbitrarily by processing operations, and as long as the input and output types match the user need not intervene.

Figure 1: Data processing Components

(a) Data source	(b) Data sink	(c) Pump	(d) FIFO Buffer	(e) Process

Figure 2 shows a simple process taking data in processing it and outputting it. The circle indicates where the 'pump' for the pipe is located. In this case the pipe is driven from its sink end, pulling data through the pipe. By default the system puts the pump at the data sink. This choice is based on the view that you should only process data as and when it is needed.

Figure 2: Simple data pipe.

Here is an example of a pipe using this type of processing. The source and sinks are classed derived from the templated classes DPIPortC<x> and DPOPortC<x>, they convert data too and from the standard C++ text streams. The following code takes a series of numbers from the file `in.dat' adds an offset of 2 and scales it by 3, calculates a running average of the data and outputs it to `out.dat'

DPIFileC<RealT>("in.dat") >> DPOffsetScale(2.0,3.0) >> DPRunningAverage(1.0,5) >> DPOFileC<RealT>("out.dat");

Normal classes:

DPIOConnectC	Connect some IOPorts.
DPMTIOConnectC	Connect some IOPorts.
DPMTIOConnectBaseC	Connect some IOPorts.

Normal functions:

operator >>(const DPIOPortC &,DPIStreamOpC)
Connect(const DPIPortC &,const DPOPortC &)
operator >>(const DPIPortC &,const DPOPortC &)
operator >>(const DPIPortC &,const DPIOPortC &)
operator >>(const DPIOPortC &,const DPOPortC &)
operator >>=(const DPIPortC &,const DPOPortC &)
operator >>=(const DPIPortC &,const DPIOPortC &)
operator >>=(const DPIOPortC &,const DPOPortC &)
operator >>=(const DPIOPortC &,const DPIOPortC &)
DPMTIOConnect(const DPIPortC &,const DPOPortC &)

Develop classes:

DPIOConnectBaseBodyC	IO Connector base
DPIOConnectBaseC	IO Connector base
DPIOConnectBodyC	Connect some IOPorts body.
DPMTIOConnectBaseBodyC	Connect some IOPorts body.
DPMTIOConnectBodyC	Connect some IOPorts body.