Links to useful Big Data development websites

Enterprise Data Hub

Get Hadoop & Spark on your desktop (via a virtual machine) – Hortonworks, MapR, Cloudera

Get Hadoop & Spark on the cloud – Amazon Elastic MapReduce 

Data Integration

Stream weblogs in to a Hadoop Distributed File System (HDFS) – Apache Flume

Process streams of data – Apache Storm

Implement a publish-subscribe message queue system – Apache Kafka

Push/pull data to/from Hadoop to a relational database – Apache Sqoop

Extract data from any website –

Integrate publicly available 3rd party data via a web service – Public APIs


Data Discovery

Query your big data using standard SQL in real time – Apache Drill

Discover insights from your data – Tableau, Qlik


Plot graphs online from your data –

Statistical analysis (aka Data Science)

Data analysis with some statistical calculations – Python

Serious statistical analysis – R programming

Reporting & Dashboards

Enterprise scale reporting & dashboards – Cognos, Business Objects, Microstrategy, OBIEE, SSRS


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s