What do you have to say for the skeptics of Hadoop who think that the ecosystem is getting too complex with too many overlapping projects doing almost similar things?
There is a truth to the point of growing complexity of the entire ecosystem but there is also a misattribution of the complexity that comes with it. Unlike many other unified single-stack architectures that came before, the Hadoop platform is built around individual layers of individual responsibilities. This is the Unix philosophy; each of these layers is built in order to perform one thing and one thing well. This not only helps in delineating responsibilities, but it also helps in a much faster evolution. Remember that several different open developer communities are working on each layer. Sometimes, this does mean there are two or more disjoint sets of developers that work on the same layer, but that's okay – either each of those projects carve out their niche or the single best project simply emerges. In a truly open community, a meritocracy, no single vendor ultimately decides the best approach.
via NoSQL databases