ArtsAutosBooksBusinessEducationEntertainmentFamilyFashionFoodGamesGenderHealthHolidaysHomeHubPagesPersonal FinancePetsPoliticsReligionSportsTechnologyTravel

Big Data: Understanding New Insights

Updated on June 10, 2015
Big Data Analytics
Big Data Analytics | Source

Big Data: What's that?

The term "Big Data" was introduced a decade ago. It is used to refer a massive volume of data which can be both structured and unstructured that is so large it is difficult to process using traditional software and database techniques.

Big Data is a broader classification of data sets which are large and complex. These data sets are used to analyze new insights and thus making big data good for predictive analyses. Big Data can find new correlations like 'preventing disease, spotting business trends, even predicting crimes.'


Large Doesn't Mean It's Big

It depends on a system if a data set can be called Big Data or not. If you look at the email system, the maximum size of attachments is 25 MB. 25 MB is not at all huge if you compare it with a 25 GB of data set. For the email system a file exceeding 25 MB becomes big data because it can't handle and store it easily.

Getting Started With Big Data

With the upcoming of the latest trends and technology, Big Data is becoming one of the most important technology and has the potential to change the way how information is used to enhance customer experience and creating business models.


Big Data has enabled organizations to store, manage and process vast amounts of data at the right speed and time to gain the right insights. Most organizations are at an early stage in their big data journey. They are experimenting with the techniques that allows them to collect huge amount of data and find hidden treasures which is within that data which can show an early indication to an important change.


Big Data solution requires quality infrastructure which needs to be in place to support scalability, distribution and management of data.

Which Social Media platform do you use regularly?

See results

Big Data Evolution And Characteristics

Managing and analyzing data have always offered great benefits to an organization. Traditionally companies didn't have too much data to deal with. There were customers who bought the same product in the same way, keeping things simple and straight forward. But over time, due to competition amongst the companies, new products were launched and it complicated everything.


From spreadsheets to Relational Database Management System(RDBMS) to Distributed File Systems(DFS), data storing systems have changed and so does the process to analyze and process them.

Powered By Visually
Powered By Visually | Source

Big Data is defined as any kind of data source that has at least three shared characteristics:

  • Extremely large Volumes of data: The quantity of data being generated is important in this context.
  • Extremely high Velocity of data: How fast the data is generated and processed to meet the demands and the challenges which lie ahead in the path of growth and development.
  • Extremely wide Variety of data: The data may be structured(text, xml files etc.) or unstructured(pictures, videos etc.)

Powered By Flickr
Powered By Flickr | Source

Types of Big Data

Big Data comes in various varieties, from dollar transactions to tweeting an image. Thus, this information needs to be integrated for analysis and data management.

Data management has always been around for a long time, what makes it difficult are:

  • New data sources like data generated from sensors, smartphone and tablets.
  • Previously generated data hasn't been captured because we didn't have any cost-effective way to deal with the data.

Sources of Structured Data: Data Center
Sources of Structured Data: Data Center | Source

Structured Data

In structured data, the data have a defined length and format. It accounts for almost 20% of the data that is out there. Examples of structured data include numbers, dates, and groups of
words and numbers called strings.

Structured data is the data with which we deal the most and it's usually stored in a database. The evolution in technology provides newer sources of structured data often in real time and large volumes.

Type of Structured Data

Computer or Machine Generated
Human Generated
Sensor Data
Input Data
Web Log Data
Click Stream Data
Financial Data
Gaming Related Data
Dividing Structured Data Into Two Categories: Computer/Machine And Human Generated
Sources of Unstructured Data: Social Media
Sources of Unstructured Data: Social Media | Source

Unstructured Data

Unstructured data is everywhere. In fact, most individuals and organizations
conduct their lives around unstructured data. Just as with structured data,
unstructured data is either machine generated or human generated.

Types of Unstructured Data

Computer or Machine Generated
Human Generated
Satellite images
Text internal to your company
Scientific data
Social media data
Photographs and video
Mobile data
Radar or sonar data
Website content
Dividing Structured Data Into Two Categories: Computer/Machine And Human Generated
The Cycle of Big Data Management
The Cycle of Big Data Management | Source

Setting The Architectural Foundation

Before we start talking about architecture, lets take into account the functional requirements of big data.

To begin with, the data is captured, and then organized and integrated for analysis. Analysis is based on the problem being solved. Management then takes action on the results obtained from the analysis. This corresponds to the big data cycle of management.


In addition of these functional requirements, required performance is also of utmost importance. We need right amount of computational requirement like power and speed.

Big Data Architecture
Big Data Architecture | Source

Recommended Reading

Big Data: A Revolution That Will Transform How We Live, Work, and Think
Big Data: A Revolution That Will Transform How We Live, Work, and Think

“Illuminating and very timely a fascinating — and sometimes alarming — survey of big data’s growing effect on just about everything: business, government, science and medicine, privacy, and even on the way we think.”

 

The Big Data Journey

Companies have always had to deal with lots of data to out rank their competitors. With the right technology in place, companies can solve business problems and react to opportunities.


With big data, data patterns can be analyzed to manage cities, prevent failures, manage traffic, improve customer satisfaction and the list goes on.

Have you ever dealt with Big Data?

See results

Comments

Submit a Comment

  • choosetolive profile image

    Ravi and Swastha 

    3 years ago from London, Canada

    @Zander Collisin - No problem & welcome. Oh common. For this hub itself I know there was quite a lot of effort gone in. Anyways, you still have opportunity to edit this hub and make it a big one and let me know. I will read your revised hub as well. Thanks again.

  • Zander Collision profile imageAUTHOR

    Zander Collision 

    3 years ago

    @choosetolive - Thank you for your great comment. There was so much to write about, it was really daunting to make it short.

  • choosetolive profile image

    Ravi and Swastha 

    3 years ago from London, Canada

    Good research and well conveyed about Big data in short in this hubpage.

    Thanks for sharing a wonderful hub. Voted up and Useful hub.

working

This website uses cookies

As a user in the EEA, your approval is needed on a few things. To provide a better website experience, hubpages.com uses cookies (and other similar technologies) and may collect, process, and share personal data. Please choose which areas of our service you consent to our doing so.

For more information on managing or withdrawing consents and how we handle data, visit our Privacy Policy at: https://hubpages.com/privacy-policy#gdpr

Show Details
Necessary
HubPages Device IDThis is used to identify particular browsers or devices when the access the service, and is used for security reasons.
LoginThis is necessary to sign in to the HubPages Service.
Google RecaptchaThis is used to prevent bots and spam. (Privacy Policy)
AkismetThis is used to detect comment spam. (Privacy Policy)
HubPages Google AnalyticsThis is used to provide data on traffic to our website, all personally identifyable data is anonymized. (Privacy Policy)
HubPages Traffic PixelThis is used to collect data on traffic to articles and other pages on our site. Unless you are signed in to a HubPages account, all personally identifiable information is anonymized.
Amazon Web ServicesThis is a cloud services platform that we used to host our service. (Privacy Policy)
CloudflareThis is a cloud CDN service that we use to efficiently deliver files required for our service to operate such as javascript, cascading style sheets, images, and videos. (Privacy Policy)
Google Hosted LibrariesJavascript software libraries such as jQuery are loaded at endpoints on the googleapis.com or gstatic.com domains, for performance and efficiency reasons. (Privacy Policy)
Features
Google Custom SearchThis is feature allows you to search the site. (Privacy Policy)
Google MapsSome articles have Google Maps embedded in them. (Privacy Policy)
Google ChartsThis is used to display charts and graphs on articles and the author center. (Privacy Policy)
Google AdSense Host APIThis service allows you to sign up for or associate a Google AdSense account with HubPages, so that you can earn money from ads on your articles. No data is shared unless you engage with this feature. (Privacy Policy)
Google YouTubeSome articles have YouTube videos embedded in them. (Privacy Policy)
VimeoSome articles have Vimeo videos embedded in them. (Privacy Policy)
PaypalThis is used for a registered author who enrolls in the HubPages Earnings program and requests to be paid via PayPal. No data is shared with Paypal unless you engage with this feature. (Privacy Policy)
Facebook LoginYou can use this to streamline signing up for, or signing in to your Hubpages account. No data is shared with Facebook unless you engage with this feature. (Privacy Policy)
MavenThis supports the Maven widget and search functionality. (Privacy Policy)
Marketing
Google AdSenseThis is an ad network. (Privacy Policy)
Google DoubleClickGoogle provides ad serving technology and runs an ad network. (Privacy Policy)
Index ExchangeThis is an ad network. (Privacy Policy)
SovrnThis is an ad network. (Privacy Policy)
Facebook AdsThis is an ad network. (Privacy Policy)
Amazon Unified Ad MarketplaceThis is an ad network. (Privacy Policy)
AppNexusThis is an ad network. (Privacy Policy)
OpenxThis is an ad network. (Privacy Policy)
Rubicon ProjectThis is an ad network. (Privacy Policy)
TripleLiftThis is an ad network. (Privacy Policy)
Say MediaWe partner with Say Media to deliver ad campaigns on our sites. (Privacy Policy)
Remarketing PixelsWe may use remarketing pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to advertise the HubPages Service to people that have visited our sites.
Conversion Tracking PixelsWe may use conversion tracking pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to identify when an advertisement has successfully resulted in the desired action, such as signing up for the HubPages Service or publishing an article on the HubPages Service.
Statistics
Author Google AnalyticsThis is used to provide traffic data and reports to the authors of articles on the HubPages Service. (Privacy Policy)
ComscoreComScore is a media measurement and analytics company providing marketing data and analytics to enterprises, media and advertising agencies, and publishers. Non-consent will result in ComScore only processing obfuscated personal data. (Privacy Policy)
Amazon Tracking PixelSome articles display amazon products as part of the Amazon Affiliate program, this pixel provides traffic statistics for those products (Privacy Policy)