ArtsAutosBooksBusinessEducationEntertainmentFamilyFashionFoodGamesGenderHealthHolidaysHomeHubPagesPersonal FinancePetsPoliticsReligionSportsTechnologyTravel
  • »
  • Technology»
  • Computers & Software»
  • Computer Science & Programming

Big Data what is it ?

Updated on July 3, 2017

What is Big Data?

Big Data is high-volume, high-velocity, and/or high-variety information assets that demand cost-effective, innovative forms of information processing that enable enhanced insight, decision making and process automation.

The 4 V's of Big Data

Velocity:

is the idea that data is being generated
extremely fast, a process that never stops.
Attributes include near or real-time streaming
and local and cloud-based technologies
that can process information very quickly.
Volume:

is the amount of data generated.
For example, exabytes, zettabytes, yottabytes, etc..
Drivers of volume are the increase in data sources,
higher resolution sensors and scalable infrastructure.
Veracity:

is the quality and origin of data.
Attributes include consistency, completeness, integrity,
and ambiguity.
Drivers include cost, and the need for traceability.
Variety:

is the idea that data comes from different sources,
machines, people, processes,
both internal and external to organizations.
Attributes include the degree of structure and complexity
and drivers are mobile technologies, social media,
wearable technologies, geo technologies,
video, and many, many more.
And the last V is value.

Let's look at some examples of the V's in action.

Velocity:

Every 60 seconds, hours of footage are uploaded to YouTube.
This amount of data is generated every minute.
So think about how much accumulates over hours, days,
and in years.
Volume:

Every day we create approximately 2.5 quintillion bytes of data.
That's 10 million Blu-ray DVD's every day.
The world population is approximately seven billion people, and the vast majority of people are now using digital devices.

These devices all generate, capture, and store data.
And with more than one device, for example,
mobile devices, desktop computers, laptops, et cetera,
we're seeing even more data being produced.

Variety:

Let's think about the different types of data, text, pictures, and film.
What about sound, health data from wearable devices, and many different types of data from devices connected to the internet of things.

Veracity:

80% of data is considered to be unstructured
and we must devise ways
to produce reliable and accurate insights.
The data must be categorized, analyzed and visualized.
The emerging V is value.
This V refers to our ability and need
to turn data into value.
Value isn't just profit.
It may be medical or social benefits,
or customer, employee, or personal satisfaction.
The main reasons for why people invest time to understand
Big Data is to derive value from it.

What is the Hadoop and why it is considered a great Big Data solution ?

Hadoop is an open-source software framework used to store and process huge amounts of data.

It is implemented in several distinct, specialized modules:
Storage, principally employing the Hadoop File System, or HDFS,
Resource management and scheduling for computational tasks,
Distributed processing programming models based on MapReduce,
Common utilities and software libraries necessary for the entire Hadoop platform.
Hadoop is a framework written in Java, originally developed by Doug Cutting
who named it after his son's toy elephant.
Hadoop uses Google's MapReduce technology as its foundation.

How is Big Data Used?


Companies like Amazon, Netflix and Spotify use algorithms based on big data
to make specific recommendations based on customer preferences and historical behavior.
Personal assistants like Siri on Apple devices use big data to devise answers
to the infinite number of questions end users may ask.
Google now makes recommendations based on the big data on a user's device.

Now that we have an idea of how consumers are using big data, let's take a look at how big data is impacting business.
In 2011, McKinsey & Company said that big data was going to become the key basis of competition supporting new waves of productivity growth and innovation.
In 2013, UPS announced that it was using data from customers, drivers and vehicles in a new route guidance system aimed to save time, money and fuel.

Initiatives like this one support the statement that big data will fundamentally change the way businesses compete and operate.

How Big Data Is Used In Amazon Recommendation Systems To Change Our Lives

Privacy and Security Issues in the Age of Big Data

Big data can enable “invasions of privacy, invasive marketing, decreased civil liberties, and increased state and corporate control”. The amount of information collected on each individual can be processed to provide a surprisingly complete picture. As a result, organizations that own data are legally responsible for the security and the usage policies they apply to their data.

Attempts to anonymous specific data are not successful in protecting privacy because there is so much available that some data can be used as a correlation for identification purposes.

Users' data are also constantly in transit, being accessed by inside users and outside contractors, government agencies, and business partners sharing data for research.

SOLUTION: Privacy, for legal reasons, must be preserved even at the cost – not only monetary but that of system performance. Developing approaches include “differential privacy”, a formal and proven model that comes with a great deal of systems overhead; and an emerging technology known as homomorphic encryption, which allows analytics to work with encrypted data. Older, more standard solutions include encryption of data within the database, access control, and stringent authorization policies. Keeping security patches up to date, another bit of standard wisdom, is important.

An important consideration for implementing privacy policies is that legal requirements vary from country to country, and it is necessary to comply with the policies of the countries where you are active.

5 out of 5 stars from 1 rating of Big Data what is it ?

Comments

    0 of 8192 characters used
    Post Comment

    • profile image

      naoufal el mar 11 months ago

      Very good, this article contains very important information

    working

    This website uses cookies

    As a user in the EEA, your approval is needed on a few things. To provide a better website experience, hubpages.com uses cookies (and other similar technologies) and may collect, process, and share personal data. Please choose which areas of our service you consent to our doing so.

    For more information on managing or withdrawing consents and how we handle data, visit our Privacy Policy at: "https://hubpages.com/privacy-policy#gdpr"

    Show Details
    Necessary
    HubPages Device IDThis is used to identify particular browsers or devices when the access the service, and is used for security reasons.
    LoginThis is necessary to sign in to the HubPages Service.
    Google RecaptchaThis is used to prevent bots and spam. (Privacy Policy)
    AkismetThis is used to detect comment spam. (Privacy Policy)
    HubPages Google AnalyticsThis is used to provide data on traffic to our website, all personally identifyable data is anonymized. (Privacy Policy)
    HubPages Traffic PixelThis is used to collect data on traffic to articles and other pages on our site. Unless you are signed in to a HubPages account, all personally identifiable information is anonymized.
    Amazon Web ServicesThis is a cloud services platform that we used to host our service. (Privacy Policy)
    CloudflareThis is a cloud CDN service that we use to efficiently deliver files required for our service to operate such as javascript, cascading style sheets, images, and videos. (Privacy Policy)
    Google Hosted LibrariesJavascript software libraries such as jQuery are loaded at endpoints on the googleapis.com or gstatic.com domains, for performance and efficiency reasons. (Privacy Policy)
    Features
    Google Custom SearchThis is feature allows you to search the site. (Privacy Policy)
    Google MapsSome articles have Google Maps embedded in them. (Privacy Policy)
    Google ChartsThis is used to display charts and graphs on articles and the author center. (Privacy Policy)
    Google AdSense Host APIThis service allows you to sign up for or associate a Google AdSense account with HubPages, so that you can earn money from ads on your articles. No data is shared unless you engage with this feature. (Privacy Policy)
    Google YouTubeSome articles have YouTube videos embedded in them. (Privacy Policy)
    VimeoSome articles have Vimeo videos embedded in them. (Privacy Policy)
    PaypalThis is used for a registered author who enrolls in the HubPages Earnings program and requests to be paid via PayPal. No data is shared with Paypal unless you engage with this feature. (Privacy Policy)
    Facebook LoginYou can use this to streamline signing up for, or signing in to your Hubpages account. No data is shared with Facebook unless you engage with this feature. (Privacy Policy)
    MavenThis supports the Maven widget and search functionality. (Privacy Policy)
    Marketing
    Google AdSenseThis is an ad network. (Privacy Policy)
    Google DoubleClickGoogle provides ad serving technology and runs an ad network. (Privacy Policy)
    Index ExchangeThis is an ad network. (Privacy Policy)
    SovrnThis is an ad network. (Privacy Policy)
    Facebook AdsThis is an ad network. (Privacy Policy)
    Amazon Unified Ad MarketplaceThis is an ad network. (Privacy Policy)
    AppNexusThis is an ad network. (Privacy Policy)
    OpenxThis is an ad network. (Privacy Policy)
    Rubicon ProjectThis is an ad network. (Privacy Policy)
    TripleLiftThis is an ad network. (Privacy Policy)
    Say MediaWe partner with Say Media to deliver ad campaigns on our sites. (Privacy Policy)
    Remarketing PixelsWe may use remarketing pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to advertise the HubPages Service to people that have visited our sites.
    Conversion Tracking PixelsWe may use conversion tracking pixels from advertising networks such as Google AdWords, Bing Ads, and Facebook in order to identify when an advertisement has successfully resulted in the desired action, such as signing up for the HubPages Service or publishing an article on the HubPages Service.
    Statistics
    Author Google AnalyticsThis is used to provide traffic data and reports to the authors of articles on the HubPages Service. (Privacy Policy)
    ComscoreComScore is a media measurement and analytics company providing marketing data and analytics to enterprises, media and advertising agencies, and publishers. Non-consent will result in ComScore only processing obfuscated personal data. (Privacy Policy)
    Amazon Tracking PixelSome articles display amazon products as part of the Amazon Affiliate program, this pixel provides traffic statistics for those products (Privacy Policy)