Question

You recently started working as a data analyst for the retail industry. Your company would like to invest more money into a machine learning project. You need to write at least 3-5 page executive summary that describes the benefit of using machine learning.

It should include following:

• Identify at least two business use cases for a retail industry that will benefit from Machine learning.
• Describe the importance of input data quality for machine learning.
• Describe a type of Python libraries that would be useful to fulfill machine learning requirement.

Be sure to use proper grammar and spellcheck the document before submitting.

Answer 1

First we have to know about what is machine learning so according to books Arthur Samuel coined the phrase “Machine Learning”in 1959, defining it as “the ability to learn without being explicitly programmed.” Machine Learning, at its most basic form, is the practice of using algorithms to parse data, learn from it, and then make a determination or prediction about something in the world.

The most common example for beginners is house prices. How does a site like Redfin or Zillow predict what the price of a currently-owned house is?

It’s not that complicated. Machine Learning, at its core, is really just making a line of best fit, except in many dimensions. A house price prediction model looks at a ton of data, with each data point having several dimensions like size, bedroom count, bathroom count, yard space, etc. It creates a function out of these input parameters, and then just shifts the coefficients to each of these parameters as it looks at more and more data.

This method of Machine Learning is called “Supervised Learning,” where the data given to the model includes the answer to the problem for each input set. It’s basically giving the input parameters, called features, and the outputs for each set of features, from which the model adjusts its function to match data. Then, when given any other input data, the model can execute the same function and come up with an accurate output.

Other factions of Machine Learning are Unsupervised Learning and Reinforcement Learning. Concisely, Unsupervised Learning just finds similarities in data — in our house example, the data wouldn’t include house prices (the data would only be input, it would have no output) and the model would be able to say “Hmm, well based on these parameters, House 1 is most similar to House 3” or something of the sort, but wouldn’t be able to predict the price of a given house.

Benefits of using machine learning in retail industry::

Retail is well-placed to benefit from the intersection of Artificial Intelligence, machine learning and big data. There is a need to manage and track a large number of items across various categories, track consumers’ shopping habits and above all, maintain a compelling brand that keeps consumers coming back. Today’s consumer wants to keep up with the latest trends, but also craves convenience; hence, the popularity of subscription boxes and online shopping. A recent survey of retailers worldwide identified cost savings, enhanced decision-makingand process automation as some of the main areas that AI has the potential to impact meaningfully.Of course, generating tangible insights from data collected across multiple channels requires the use of algorithms and models that can process and learn from huge datasets, and make actionable predictions and recommendations. Using data science techniques like Discriminant Analysis, the k-Nearest Neighbors algorithm, Deep Learning and Support Vector Machines, there is incredible potential for improving classification, optimization and forecasting within the retail and CPG spaces, while taking advantage of AI functionalities like Computer Vision and Natural Language Processing .So ,Here is a look at 3 applications of machine learning in retail, with examples of retailers who have already benefited from them.

1. Personalization of Recommended Products

Recommendation engines do not just bring consumers’ attention to items they are unlikely to discover on their own. When they are rightly done, they keep consumers coming back for more, and also help the retailer forecast demand and make supply decisions well in advance.

Amazon has one of the best recommendation engines on the market today, with 55% of its sales driven by its recommendations. Its recommendation algorithms take into account not just your own behaviour (what items they have clicked on, how often they place orders, what they have searched for), but also the purchasing behavior of other consumers who are similar to you. For example, if you are searching for Mother’s Day gifts, Amazon’s recommendations engine will look at your own purchasing behaviour, and the behavior of other people who are also searching for similar gifts. All of this is made possible by item-to-item collaborative filtering, which was developed by Amazon itself to solve the problem of existing algorithms being unable to scale to the massive volume its platform deals with. This method focuses on the rating distribution per item, (as opposed to per user). This allows for more stable rating distributions, which equates to the ability to scale to huge datasets.

  Such algorithms ensure significant upsell, in which the seller induces the consumer to buy more expensive or additional items to secure a more profitable sale. They also take the guesswork of a human sales assistant out of the equation, allowing for highly customized recommendations that are extremely likely to appeal to the customer. This also allows retailers to anticipate what consumers are going to buy in the future, once again taking the guesswork out of maintaining sufficient stock.

2. Supply Chain Planning

Traditionally, inventory planning has involved a significant amount of trial and error, because there are so many factors that are inherently uncertainty. How many people will shop next month? Will there be any unexpected situations that alter purchasing behaviour. Will a popular item suddenly fall out of favor, or vice versa? But with machine learning, several parameters can be studied to make informed decisions backed by extensive data analysis.

Root cause analysis, which helps to identify the reason for faults within an existing system, is one the main areas that can be gainfully automated. Machine learning algorithms can tell a more complete story of exactly what went wrong, identifying anomalies such as incomplete data or lapses in communication, and pinpoint exactly where each of these failures happened along the supply chain. Human bias and guesswork is eliminated in identifying the root cause, and this is applicable for any stage, be it warehouse processing or vendor management.

Simila algorithms can help with demand planning, by identifying top performing items that drive the most sales across multiple channels, so that best practices can be quickly identified and scaled. With the use of SKUs, there is already a significant amount of data available to retailers. The key is being able to process and analyze that data in useful ways. The sophistication afforded by machine learning algorithms allows for more segmentation, which allows for more detailed and descriptive insights. The performance of a single item can be tracked across multiple categories, and multiple parameters can be included in the analysis. Discriminant analysis can also be incorporated into machine learning algorithms addressing this area to enable and improve segmentation and classification.

The old paradigm of demand forecasting treats every SKU & transaction as an isolated event, and relies on historical data and manual decision-making (for example, how similar two items are). This often ignores factors like promotional cadence, market cannibalization and seasonal changes, which are too complex for traditional forecasting tool-sets.

Using machine learning allows retailers to combine historical and real-time data, and identify patterns that humans and traditional forecasting tools would have missed. In addition, AI can automate the process of filling out databases, particularly fixed-data fields, using Natural Language processing and Computer Vision to learn from product descriptions and images.

3.Price Optimization & Promotions

Price optimization is another area in which the number of parameters quickly overwhelms the human mind and even current software tools. Machine learning algorithms are able to create multiple decision trees based on a variety of sub-groups, before combining everything into a comprehensive predictive model that outputs sophisticated analysis and insights. After being sufficiently trained, the model can be trusted to make accurate predictions. This can be particularly useful for retailers seeking to study the potential impact of sales promotions.

For a comprehensive look into the impact of a potential promotion, there are several variables that need to be taken into account, such as the amount of the discount, the type of product, price elasticity and competition from other retailers and promotions. There are also other factors like shelf placement and marketing techniques that are difficult to input into traditional price optimization software or spreadsheets. Using a combination of AI to collect the relevant information and machine learning to process that data has already been proven to yield more accurate predictions than traditional methods. A significant amount of risk is removed from promotions, allowing retailers to only implement promotions that are very likely to deliver a satisfactory ROI. There is also no need to try out potentially risky promotions, as the algorithm already simulates the potential outcomes of each promotion, and recommends the promotions that are extremely likely to boost sales and profits.

The use of real-time data also enables the implementation of a truly agile pricing framework, while the automation afforded by AI ensures errorless data collection. Dynamic pricing is the way forward for digital platforms that expect to deal with significant volume. For example, Airbnb recommends prices to potential hosts, taking into account multiple variables that have different weightings applied to them. The assumptions behind the original algorithm were tested against prior transaction data to model against actual outcomes. The classifier technique is used by the algorithm to calculate the likelihood of a property being booked based on the property’s attributes and real-time market data. Today, Airbnb’s machine learning algorithm categorizes properties based on factors like the photos used in the listing, and even adjusts prices on its own based on changes in neighborhood boundaries. Airbnb has now launched an open-source machine learning package called Aerosolve that includes geo-based features and emphasizes user interaction.

  Inventory management and demand planning are also made easier, as machine learning enables retailers to make more informed decisions when ordering stock. As machine learning becomes more prevalent in price optimization, the next step would be to customize promotions by offering personalized promotions to individual customers based on their prior shopping behaviour and a variety of other factors.

Importance of data quality for machine learning

It is true that poor data quality is enemy number one to the widespread, profitable use of machine learning. A significant issue in Enterprise Data Management today is Data Quality, because business data requires thorough cleansing and preparation to be used as input to any Analytics or Business Intelligence system.

To properly train a predictive model, historical data must meet exceptionally broad and high quality standards. First, the data must be right: It must be correct, properly labeled, de-deduped, and so forth. But you must also have the right data — lots of unbiased data, over the entire range of inputs for which one aims to develop the predictive model. Most data quality work focuses on one criterion or the other, but for machine learning, you must work on both simultaneously.

Yet today, most data fails to meet basic “data are right” standards. Reasons range from data creators not understanding what is expected, to poorly calibrated measurement gear, to overly complex processes, to human error. To compensate, data scientists cleanse the data before training the predictive model. It is time-consuming, tedious work (taking up to 80% of data scientists’ time), and it’s the problem data scientists complain about most. Even with such efforts, cleaning neither detects nor corrects all the errors, and as yet, there is no way to understand the impact on the predictive model. What’s more, data does not always meets “the right data” standards, as reports of bias in facial recognition and criminal justice attest.

Increasingly-complex problems demand not just more data, but more diverse, comprehensive data. And with this comes more quality problems. For example, handwritten notes and local acronyms have complicated IBM’s efforts to apply machine learning (e.g., Watson) to cancer treatment.

Data quality is no less troublesome in implementation. Consider an organization seeking productivity gains with its machine learning program. While the data science team that developed the predictive model may have done a solid job cleaning the training data, it can still be compromised by bad data going forward. Again, it takes people — lots of them — to find and correct the errors. This in turns subverts the hoped-for productivity gains. Further, as machine learning technologies penetrate organizations, the output of one predictive model will feed the next, and the next, and so on, even crossing company boundaries. The risk is that a minor error at one step will cascade, causing more errors and growing ever larger across an entire process.

These concerns must be met with an aggressive, well-executed quality program, far more involved than required for day-in, day-out business. It requires the leaders of the overall effort to take all of the following five steps.

First, clarify your objectives and assess whether you have the right data to support these objectives. Consider a mortgage-origination company that wishes to apply machine learning to its loan process. Should it grant the loan and, if so, under what terms? Possible objectives for using machine learning include:

• Lowering the cost of the existing decision process. Since making better decisions is not an objective, existing data may well be adequate.
• Removing bias from the existing decision process. This bias is almost certainly reflected in its existing data. Proceed with caution.
• Improving the decision-making process — granting fewer loans which default and approving previously-rejected loans that will perform. Note that while the company has plenty of data on previously-rejected mortgages, it does not know whether these mortgages would have performed. Proceed with extreme caution.

When the data fall short of the objectives, the best recourse is to find new data, to scale back the objectives, or both.

Second, build plenty of time to execute data quality fundamentals into your overall project plan. For training, this means four person-months of cleaning for every person-month building the model, as you must measure quality levels, assess sources, de-duplicate, and clean training data, much as you would for any important analysis. For implementations, it is best to eliminate the root causes of error and so minimize ongoing cleaning. Doing so will have the salutary effect of eliminating hidden data factories, saving you time and money in operations as well. Start this work as soon as possible and at least six months before you wish to let your predictive model loose.

Third, maintain an audit trail as you prepare the training data. Maintain a copy of your original training data, the data you used in training, and the steps used in getting from the first to the second. Doing so is simply good practice (though many unwisely skip it), and it may help you make the process improvements you’ll need to use your predictive model in future decisions. Further, it is important to understand the biases and limitations in your model and the audit trail can help you sort it out.

Fourth, charge a specific individual (or team) with responsibility for data quality as you turn your model loose. This person should possess intimate knowledge of the data, including its strengths and weaknesses, and have two foci. First, day-in and day-out, they set and enforce standards for the quality of incoming data. If the data aren’t good enough, humans must take over. Second, they lead ongoing efforts to find and eliminate root causes of error. This work should already have started and it must continue.

Finally, obtain independent, rigorous quality assurance. As used here, quality assurance is the process of ensuring that the quality program provides the desired results. The watchword here is independent, so this work should be carried out by others — an internal QA department, a team from outside the department, or a qualified third party.

Even after taking these five steps, you will certainly find that your data is not perfect. You may be able to accommodate some minor data quality issues in the predictive model, such as a single missing value among the fifteen most important variables. To explore this area, pair data scientists and your most experienced businesspeople when preparing the data and training the model. It will help to improve the data quality and if your data quality is good you can tap the power of machine learning.

Python library and it's importance

Python is an interpreted, object-oriented, high-level programming language with dynamic semantics. Its high-level built in data structures, combined with dynamic typing and dynamic binding, make it very attractive for Rapid Application Development, as well as for use as a scripting or glue language to connect existing components together. Python's simple, easy to learn syntax emphasizes readability and therefore reduces the cost of program maintenance.

Python is becoming popular day by day and has started to replace many popular languages in the industry. The main reason for Python’s popularity is because of the following reasons:

• Python is popularly known as Beginner’s language because of it’s simplicity.
• Python helps developers to be more productive from development to deployment and maintenance.
• Python’s syntax is very simple and high level when compared to Java, C and C++, therefore applications can be built with less number of lines.
• Python has a large collection of libraries.
• Portable ( Of course, Probability is the main feature of Java too).

Type of Python library

1. Tensorflow :

If you are working or interested about Machine Learning, then you might have heard about this famous Open Source library known as Tensorflow. It was developed at Google by Brain Team. Almost all Google’s Applications use Tensorflow for Machine Learning. If you are using Google photos or Google voice search then indirectly you are using the models built using Tensorflow.

Tensorflow is just a computational framework for expressing algorithms involving large number of Tensor operations, since Neural networks can be expressed as computational graphs they can be implemented using Tensorflow as a series of operations on Tensors. Tensors are N-dimensional matrices which represents our Data.The major advantage of tensorflow is Parallelism, means your computational graphs are executed in parallel, you have the full control over execution and you can schedule different operations on different processors like CPU, GPU etc.Tensorflow is basically written in C and C++, but has a sophisticated Frontend for Python. Your Python code gets compiled and then runs on Tensorflow distributed execution engine developed using C and C++. Tensorflow is optimized for speed, it can make use of techniques like XLA for faster linear algebra operations.

2. Numpy:

Numpy is of course one of the greatest Mathematical and Scientific computing library for Python. Tensorflow and other platforms use Numpy internally for performing several operations on Tensors. One of the most important feature of Numpy is it’s Array interface.

This interface can be used to express images, sound waves or any other raw binary streams as arrays of real numbers with N dimensions. Knowledge of Numpy is very much important for Machine Learning and Data Science.

3. Keras :

Keras is one of the coolest Machine learning library.

It provides a easier way to express Neural networks. It also provides some of the utilities for processing datasets, compiling models, evaluating results, visualization of graphs and many more.

Keras internally uses either Tensorflow or Theano as backend. Some other pouplar neural network frameworks like CNTK can also be used.Keras is slow when compared to other libraries because it constructs a computational graph using the backend infrastructure and then uses it to perform operations. Keras models are portable (HDF5 models) and Keras provides many preprocessed datasets and pretrained models like Inception, SqueezeNet, Mnist, VGG, ResNet etc.

4. Theano :

Theano is a computational framework for computing multidimensional arrays. Theano is similar to Tensorflow , but Theano is not as efficient as Tensorflow because of it’s inability to suit into production environments. Theano can be used on a prallel or distributed environments just like Tensorflow.

These all library are important and very useful to achieve effectiveness of maching learning process.