Question

Search and select any commercial Big Data Solution available in any domain and pick any one factor below to support your selection. At least 10 sentences

1. Scalability
2. Sizing
3. Performance
4. Manageability & flexibility
5. Complexity, Ease of use
6. Relevant Functionality
7. Proprietary & open source SW features & support

Answer 1

1.Big Data got the distinction as “Big” is that it became too much for traditional systems to handle. What once required gigabytes now scales up even more to terabytes and larger. A good data storage provider should offer you an infrastructure on which to run all your other big data analytics tools as well as a place to store and query your data.

scalabilty: Scaling can be difficult, but absolutely necessary in the growth of a successful data-driven company. There are a few signs that it’s time to implement a scaling platform. When users begin complaining about slow performance, or service outages, it’s time to scale. Don’t wait for the problem to turn into major source of contention in the minds of your customers. This can have a massively negative impact on retaining those customers. If possible, try to anticipate the problem before it becomes severe.

1. High CPU Usage is the most common bottleneck, and the most visible. Slowing and erratic performance is a key indicator of high CPU usage, and can often be a harbinger of other issues. User CPU means the CPU is doing productive work, but needs a server upgrade; system CPU refers to usage consumed by the operating system, and is usually related to the software; and I/O wait, which is the idling time caused by the CPU waiting for the I/O subsystem.
2. Low Memory is the next most common bottleneck. Servers without enough memory to handle an application load can slow the application completely. Low memory can require a RAM upgrade, but this can also be an indicator of a memory leak, which requires finding and repairing the leak within the application’s code.
3. High Disk Usage is another common bottleneck. This is often caused by maxed out disks, and is a huge indicator of the need for a data scale.

performance: Big data is defined as large amount of data which requires new technologies and architectures so that it becomes possible to extract value from it by capturing and analysis process. Big data due to its various properties like volume, velocity, variety, variability, value, complexity and performance put forward many challenges. Many organizations are facing challenges in facing test strategies for structured and unstructured data validation, setting up optimal test environment, working with non relational database and performing non functional testing. These challenges cause poor quality of data in production, delay in implementation and increase in cost. Map Reduce provides a parallel and scalable programming model for data-intensive business and scientific applications. To obtain the actual performance of big data applications, such as response time, maximum online user data capacity size, and a certain maximum processing capacity.

1. Manageability & flexibility :

• Big data technology based on platform-as-a-service
• Storage capacity extremely scalable (> petabytes)
• Management and analysis of structured and unstructured data
• Direct access and processing of real-time information
• End-to-end-service with established quality standards

1. Complexity, Ease of use:

2. Business operations are frequently tied to complex IT systems that have become increasingly

   difficult and costly to manage, and which can’t adequately support new ideas and changing business models.

   Few IT organizations report high levels of IT simplicity.

   Our survey asked respondents to rate themselves across six distinct areas of information

   technology. Between 15 and 35 percent of organizations gave themselves high marks

   in simplicity, depending on the IT area being examined. On average, only 19 percent

   of organizations rated their IT environments as “highly simplified.”

   •

   Less than one-quarter of organizations are ready for big data.

   The areas that enable big data analysis showed some of the highest levels of complexity.

   This is to be expected, given that big data is a relatively recent trend that many organizations

   are still struggling to get their arms around. Simplifying information management proved to be

   among the most critical priorities for IT organizations in our survey.

   •

   Integration remains a challenge.

   As more organizations migrate toward cloud applications, they face an array of integration requirements from multiple vendors. More than two-thirds of respondents cited high levels of complexity in the area of application integration. Additionally, few respondents are able to identify integration requirements during the early phase of a project, which can often lead to time and cost overruns.

2. Hadoop is an open-source big data analytics software framework for distributed storage of very large datasets on computer clusters. cloudera  have an open source element, Cloudera is mostly and enterprise solution to help businesses manage their Hadoop ecosystem.

data mining is the process of discovering insights within a database as opposed to extracting data from web pages into databases. The aim of data mining is to make predictions and decisions on the data your business has at hand.

While data mining is all about sifting through your data in search of previously unrecognized patterns, data analysis is about breaking that data down and assessing the impact of those patterns overtime. Analytics is about asking specific questions and finding the answers in big data.

R is a language for statistical computing and graphics. If the data mining and statistical software listed above doesn’t quite do what you want it to, learning R is the way forward. In fact, if you’re planning on being a data scientist, knowing R is a requirement.

popularity in the data community is Python. Created in the 1980s and named from Monty Python’s Flying Circus, it has consistently ranked in the top ten most popular programming languages in the world. Many journalists use Python to write custom scrapers if data collection tools fail to get the data

Before you can store, analyze or visualize your data, you’ve got to have some. Data extraction is all about taking something that is unstructured, like a webpage, and turning it into a structured table. Once you’ve got it structured, you can manipulate it in all sorts of ways.

1. Proprietary & open source SW features & support:

Open source data platforms like Hadoop, Cassandra, and MongoDB are core to the big data market, but vendors supporting the platforms are winning over enterprises with proprietary tools.

Commodity vs. purpose-built

In the big data space, this is usually the basic server node with its embedded storage and networking ports. Commodity servers are most often used in the scale-out model, which can become massively parallel processing (MPP), but may begin with just a few nodes. Other parts of the technology stack also may be commodity hardware, like the network switches . Hyperconvergence is starting to push these elements even closer in a single unit. The fundamental advantages of commodity hardware are the easy scalability and interchangeability of nodes, and perhaps the price negotiation power that accompanies these characteristics.