Customer Product Search Aid Using Text Analytics and Web Scraping

The primary purpose of this project is to develop a data science product to aid consumers who look to make a retail purchase, to make an informed decision. Reviews are opinions expressed by the current owners of the products and the experts in the industry on the product features. These online reviews replace the traditional Word of Mouth recommendations and are termed as eWOMs. The consumption of these eWOMs is significantly impacted by the age, technical knowledge, and time availability of the user. The main objective of this project is to consolidate these eWOMs from various platforms like Expert Review sites (TechRadar) and social media platforms like Amazon and Twitter using Web Scraping techniques and platform provided APIs. The secondary objective is to develop a dashboard that can provide the output to the end-user and design a data collection strategy that will enable the tool to be used for different products. Two options of data collection are discussed, and ‘on the fly’ data collection method is considered for this project owing to the dynamic nature of the query, which will allow us to search for more products. Sentiment analysis is performed on the extracted reviews, and aggregated sentiment scores are computed using the python TextBlob package. These aggregated sentiment scores lessen the effects of effectively worded fake reviews and spams, which are more pronounced when users manually scan the reviews.