[BUG] Yahoo Pipes Redefines The Union Operator

Yahoo has recently launched the new product called Pipes. The product name is taken from the world of UNIX where a pipe is for the transfer of data between applications. But Yahoo! Pipes goes beyond what just pipes are and what pipes do though as the application provides functions (or as they are called in the app - modules) that will perform a variety of different actions. As any product from major brands gets intial hype so did the Pipes and there servers went down for sometime just 8 hours after the launch. I was playing around with Yahoo Pipes over weekend, while there was lot of scope for improvement in terms of usability as a tester I just wanted to try out few things. I started creating a pipe which has two fetch modules both the fetch modules were given the same url to extract (http://feeds.feedburner.com/softwaretesting) the output of these pipes was given to an Union operator which supposedly Unions the output of the two fetch modules. In general the Union operator is expected to combine the output of the two modules and remove the duplicate.

So I did expect the same but the output was that each article on by blog was repeated twice in the output. This behavior of Union operator while deviates from the general definition, also does not make much sense in Pipes use case. You can try this pipe here. Though the RSS readers may detect these duplicates should'nt Yahoo pipes be handling it.
Hope that Yahoo takes a note of this bug and fixes it soon.


  1. M Ceglowski2:14 AM

    Thanks for your comments, and for checking out Pipes!

    You can get the results you want by adding a Unique module to your pipe. Union doesn't eliminate duplicates by default because there may be use cases where it's of interest to find out how many repeating items occurred across feeds.

    Best regards,

    M Ceglowski
    Pipes Dev

  2. Thanks Ceglowski for the clarification.

  3. Anonymous1:26 PM

    Mallik the image was useless...i tried it was not opening in like 2 mins. kindly ensure you put images at a better file host.
    The article was good.

  4. I am not getting time to make my hands dirty with Yahoo Pipes.

    I came to know about this post from Inferno Blog hunt.