Map (no reduce) with Ripeline
I went to MongoSF 2011 and attended a talk called “MongoDB’s New Aggregation Features - A Sneak Peek” by Chris Westin. As of writing, I don’t believe that slides are posted, but they will be soon.
In the talk, Chris said that they’re working on pipeline-based aggregation for Mongo, as well as a rich set of pre-built operators that can be applied in any order in that pipeline. Step 1: define your data (a Mongo collection), step 2: write your pipeline, step 3: profit.
In functional terms, this Mongo pipeline is functionally equivalent to applying multiple mappers to each element in a collection (ruby’s version for Arrays: http://www.ruby-doc.org/core/classes/Array.html#M000249), and streaming results from each mapper to the next. That model fits lots of problems too, and it’s simpler to conceptualize. Sometimes you just don’t need to reduce.
Turns out someone wrote a pipeline for
Continue reading →