If we wish to pipeline an operation we need to place buffers in the
processing stream. We can do this with the `DPBufferC<>' class,
shown in Figure 11. We can use this together with the
`pump' operator to create a processing pipeline. The pipeline below
uses 3 threads: one for reading the data, one for processing the data
and finally one to write the data out again. All this can be setup
with a single, if long, line of code given bellow. A diagram of the
pipe created is given in figure 12. It is easy to
extent this pipe line to include any number of processing steps. Since
a pipe line is only as fast as its slowest element it is useful to
balance the processing by grouping fast processes together. This is
easy to achieve, given this syntax.
DPIFileC<RealT>("in.dat") »= DPBufferC<RealT>(2) » DPRunningAverage(1.0,5) »= DPBufferC<RealT>(2) »= DPOFileC<RealT>("out.dat");
Example of processing pipeline.
There are actually two kinds of buffer available, fixed length and
infinite buffers. In most cases a finite buffer is desirable because it
provides some flow control through the program. If there is no limit on the
buffer size, a program could take an arbitrarily large amount of memory.