Queuing buffers

Figure 11: Buffer.

If we wish to pipeline an operation we need to place buffers in the processing stream. We can do this with the `DPBufferC<>' class, shown in Figure 11. We can use this together with the `pump' operator to create a processing pipeline. The pipeline below uses 3 threads: one for reading the data, one for processing the data and finally one to write the data out again. All this can be setup with a single, if long, line of code given bellow. A diagram of the pipe created is given in figure 12. It is easy to extent this pipe line to include any number of processing steps. Since a pipe line is only as fast as its slowest element it is useful to balance the processing by grouping fast processes together. This is easy to achieve, given this syntax.

DPIFileC<RealT>("in.dat") »= DPBufferC<RealT>(2) » DPRunningAverage(1.0,5) »= DPBufferC<RealT>(2) »= DPOFileC<RealT>("out.dat");

Figure 12: Example of processing pipeline.

There are actually two kinds of buffer available, fixed length and infinite buffers. In most cases a finite buffer is desirable because it provides some flow control through the program. If there is no limit on the buffer size, a program could take an arbitrarily large amount of memory.

Normal classes:

DPBufferC	MT safe queue handle.
DPFixedBufferC	MT safe queue handle.

Develop classes:

DPBufferBodyC	MT safe queue body
DPFixedBufferBodyC	MT safe queue body