Use Pipes

This is probably the best compromise between performances and memory consumption (and for large quantity of data outperforms the copy into memory). It tends however to be complicated to implement and mantain. In order to work it is required to create a PipeIdnputStream a PipedOutputStream and a new Thread:

• The two ends (PipeIdnputStream and PipedOutputStream) must be in two different Threads.

The explanation of this is that pipes are a synchronization construct that contains a small buffer. When the thread containing the PipedOutputStream fill the buffer it is suspended by the pipe. And it is awakened again when the thread that contains the PipedInputStream has read some data making some space available in the buffer.

  PipedInputStream in = new PipedInputStream();
  PipedOUtputStream out = new PipedOutputStream(in);
  new Thread(
    new Runnable(){
      public void run(){
        //put your code that writes data to the outputstream here.
        putDataOnOutputStream(out);
      }
    }
  ).start();
  //data can be read from the pipedInputStream here.	
  processDataFromInputStream(in);

Our tests demonstrates it provides very good performances on multi-core processor systems and low memory footprint.

This code is just a little example, and pipes look easy to use. But the example code is over-simplified. If for some reason the function !processDataFromInputStream don't close the PipeIdnputStream (maybe for an internal exception), the internal buffer of the pipe fills up, the thread suspends waiting for somebody to empty the buffer, and you end up with a dangling thread that never completes. So you must take special care of exception handling and closing all the streams in finally blocks.

There is also the other way round of this example:

  PipedInputStream in = new PipedInputStream();
  PipedOUtputStream out = new PipedOutputStream(in);
  new Thread(
    new Runnable(){
      public void run(){
        //data can be read from the pipedInputStream here.	
        processDataFromInputStream(in);
      }
    }
  ).start();
//put your code that writes data to the outputstream here.
putDataOnOutputStream(out);

Here is a good discussion on the problems you might run into: http://www.coderanch.com/t/499592/Streams/java/PipedOUtputStream-PipedInputStream

File

OutputStream2Inputstream