I am glad to have you for the fourth part of my complete guide on data cleaning. #1: Tidying Messy Data #2: Dealing with Missing Data #3.1: A discussion on the Nature of Outliers #3.2: The Origin of Outliers & Detection Techniques #4.1: Where does Data Duplication come from? #4.2…

There are six things the Lord hates, seven that are detestable to him: haughty eyes, a lying tongue, hands that shed innocent blood, a heart that devises wicked schemes, feet that are quick to rush into evil, a false witness who pours out lies and a person who stirs up…

Unlike the Queen in Snow White, we data analysts do not possess a Magic Mirror that reveals the location of duplicated data — Queen: Mirror, Mirror, on the wall, now, who is the fairest of us all? Magic Mirror: Over the seven jewelled hills, beyond the seventh fall, in the the cottage of the Seven Dwarfs, dwells Snow White, fairest of us all. Ah, the story of Snow White and the Evil Queen…

Everything that exists has an explanation for its existence — The larger the volume of the dataset we are handling, the higher the probability of encountering flawed values arising from measurement inaccuracies, data input errors, or similar sources. Why do outliers exist? Mother Nature is bountiful to all species, which is the reason why genetic variation exists and so do the outliers in actual…

I like weird people. The black sheep, the odd ducks, the rejects, the eccentric, the loners, the lost and forgotten. More often than not, these people have the most beautiful souls. — Unknown Outliers, abnormality, anomalies, irregularities, and sundry other terms are prevalent subjects you have undoubtedly encountered at least…

When you are missing someone, time seems to move slower, and when I’m falling in love with someone, time seems to be moving faster. — Taylor Swift Once you realize that your dataset has missing values, you will comprehend the considerable amount of time that you may spend dealing with…

Data cleaning has always been a nightmare that every single analyst has to walk through (like through fire and flames). Although it is a hard pill to swallow, we cannot simply skip this step since the avoidance of data cleaning only brings more miseries to the analyst. This article aims…

These days, the need of understanding paying users among companies, especially those who operate online solutions, has been arising noticeably. As a result, the business intelligence unit should provide a accurate and insightful segmentation analysis to business units as a required course of action. …