Massive data volumes making Hadoop hot

Jaikumar Vijayan | ComputerWorld | March 8, 2011

Unlike traditional relational database management systems, Hadoop is designed to work with multiple data types and data sources. Hadoop's Distributed File System (HDFS) technology allows large application workloads to be broken up into smaller data blocks that are replicated and distributed across a cluster of commodity hardware for faster processing

The technology is already used widely by some of the world's largest Web properties, such as Facebook, EBay, Amazon, Baidu and Yahoo. Observers note that Yahoo has been one of the biggest contributors to Hadoop.

Increasingly, Hadoop technology is used in banks, advertising companies, life science firms, pharmaceutical companies and by other corporate IT operations, said Stephen O'Grady, an analyst with RedMonk.

What's driving Hadoop is the desire by companies to leverage massive amounts of different kinds of data to make business decisions, O'Grady said. The technology lets companies process terabytes and even petabytes of complex data relatively effectively and at substantially lower cost than conventional relational database management systems, experts say.