How Does Correlation Analysis Reveal Data Patterns?

Back in the 1800s, Francis Galton was measuring sweet pea seeds and ran into an odd pattern. Big seeds had big offspring, but the offspring never quite matched their parents. Everything kept drifting back toward the average.

That’s where correlation analysis began. Galton put his thinking into a paper in 1888. Pearson followed in 1896 with the formula we still use today. And 130 years on, people are still reaching for the same little number, r, and trusting it. Most never look at the chart. Here’s the problem. r takes seconds to calculate and about two seconds to believe. Two lines rising together on a chart feel like proof, and few people want to be the one asking awkward questions.

In this guide we’ll cover what the number really means, the different types, and the Pearson formula, with a small example you can work out on paper. After that comes Excel and Python, and then the mistakes that sneak into reports when nobody’s looking.

Google Preferred Sources

Don’t Let Our Best Advice Get Buried

Add Imarticus as a Preferred Source — see more of what we publish, every time you search.

A quick aside first. You can now paste a spreadsheet into an AI tool and get a correlation back in a few seconds. That’s handy, but it makes understanding the number more important, because a quick answer isn’t always the right one. If you work in finance, marketing, HR or operations and want to use those tools without writing code, take a look at Imarticus Learning’s data analytics course. It’s a 90-day weekend course built around exactly that. Okay, back to the stats.   

What Is Correlation Analysis?

Correlation analysis is a statistical method for checking how closely two variables move together, and in which direction. You get a number between -1 and +1. A plus sign means they rise and fall together; a minus sign means one goes up while the other comes down, and the closer the number is to 1 or -1, the tighter the link.

Notice what’s missing? It doesn’t say why they’re related. It doesn’t say which one moves first. And it has no idea whether a third thing is pulling both strings. That’s on purpose.

Correlation analysis isn’t magic; it’s just arithmetic on how far each value sits from its average. It answers one small question: do these two move together? Trouble starts when people ask it something bigger, like “so what’s causing this?” No coefficient, however large, can answer that.

Correlation analysis meaning in plain words

Picture two columns: hours studied and marks scored. If the kids who study more also score higher, the columns move together. That’s the correlation analysis meaning in one line: a single number for how tightly they move.

The number is called the correlation coefficient, r. It has no units, so swap rupees for dollars or hours for minutes and r doesn’t budge. Handy, but a bit dangerous too. A number with no units travels easily, and people carry it further than it should go.

What is the major attribute of correlation analysis?

The major attribute of correlation analysis is that it answers two questions at once: how strong, and which way. How strong is how far r sits from zero. Which way is the sign. So -0.9 is a tighter link than +0.4, even though it looks gloomier. It’s also symmetric. Correlate x with y or y with x and you get the same number. Remember that one, because it’s the easiest way to tell correlation from regression later.

Correlation analysis in statistics

In statistics, correlation is the first look. You run it early on a new dataset to see which variables are even worth putting side by side. The r from your sample is an estimate of ρ (rho), the true value for the whole population, and you’ll almost never get to see that. So r always comes with a bit of doubt.

The history is short. Galton published “Co-relations and their measurement” in 1888, and Pearson gave the coefficient its solid maths in 1896. Pearson got his name on the formula. Galton did plenty of the groundwork.  

Did you know?
Four datasets can have the same correlation and tell four very different stories. A correlation coefficient can look convincing while concealing the pattern that produced it. Always inspect the scatter plot. (Source)

Types of Correlation Analysis

People mix up the types of correlation analysis all the time, mostly because there are two ways to sort them. One is by direction. The other is by method.

By direction: positive, negative and zero

The direction of correlation shows how two variables move in relation to each other. Understanding these patterns helps you interpret correlation coefficients and avoid drawing the wrong conclusions from data. 

• Positive correlation means both variables go up and down together, like ad spend and units sold (usually).
• Negative correlation means one goes up while the other drops, like price and demand (usually).
• Zero correlation means there’s no straight-line pattern at all.

That last one’s a trap. Zero correlation doesn’t mean no relationship. Try it yourself. Take x from -5 to 5 and let y equal x squared. They’re perfectly linked, but Pearson’s r comes out at exactly zero, because the left half of the curve cancels the right half. Perfect relationship, correlation of nothing. r only sees straight lines.

By method: Pearson, Spearman, Kendall and others

There’s one more family, canonical correlation analysis, which compares two whole groups of variables instead of two single ones. It gets its own section below.

Which test is used for correlation analysis?

My rule is simple. Both variables are numbers, and the scatter plot looks roughly like a line? Pearson. Data in ranks, or a plot that curves but keeps heading one way? Spearman. Small sample with lots of ties? Kendall. Whichever you pick, test the result for significance. More on that in a minute. 

Correlation Analysis Formula (Pearson)

Correlation analysis in statistics leans on one formula more than any other, and it’s Pearson’s. Here’s the Pearson correlation analysis formula:

Read it in two halves. The top multiplies how far each x sits from its average by how far the matching y sits from its own average, then adds everything up.

• If x is above average right when y is above average, you get mostly positive products and a big sum.
• If one’s high while the other’s low, the products turn negative and the sum shrinks.

The bottom divides by how spread out each variable is. That’s what squeezes the answer into the range -1 to +1 and wipes out the units.

• If you ever see this formula with just (n−1) underneath and no standard deviations, that’s covariance, not r.
• In correlation vs covariance, covariance has units and no fixed limits, so it can’t tell you whether a link is strong.

Pearson correlation analysis: the assumptions

Four things should be true. The relationship should be roughly a straight line, so plot it before you calculate anything. Both variables should be continuous. Observations should be independent of each other. And there shouldn’t be extreme outliers, because outliers can affect r way more than you’d expect.

You’ll often read that both variables must be normally distributed. That mostly matters for the significance test on small samples, and published work says Pearson’s r holds up fairly well on skewed data. The straight-line assumption is the one that really bites.

How to read the number

Take that table as a rough sketch. What counts as strong depends on the field. If two instruments are supposed to measure the same thing, 0.9 can be embarrassing. In a survey about human behaviour, 0.3 can be worth acting on.

Square r and it gets easier to picture. An r of 0.9 gives r² = 0.81, so roughly 81% of the ups and downs in one variable line up, in a straight-line way, with the other.

Is the correlation significant?

A coefficient from a sample could be a fluke, so you test it. The test statistic is:

Your software gives you the p-value, so the real skill is reading it. With fewer data points, you need a much bigger r to reach significance. With a huge sample, even a tiny correlation turns “significant”. An r of 0.1 across 1,000 observations clears the usual 0.05 bar easily and explains about one per cent of the variation. Significant and important aren’t the same thing.  

Correlation Analysis Example: A Worked Calculation

Let’s do one by hand with the correlation analysis formula above. The numbers are made up: five months of a small shop’s ad spend (₹ thousand a month) and units sold (in hundreds).

Both means are 6. Plug the totals into the formula:

r = 34 / √(40 × 34) = 34 / √1360 ≈ 0.92

That’s a strong positive correlation. Square it and r² = 0.85, so about 85% of the month-to-month ups and downs in units line up with ad spend in this tiny dataset. The t-statistic is about 4.12 on 3 degrees of freedom, which gives a p-value near 0.026. Spearman on the ranks comes out at 0.90, so it doesn’t matter much which method you pick. (Run =CORREL on the same five pairs in Excel and you’ll get 0.922.)

So what would I actually conclude?

Not that ads drive sales. Five points is almost nothing. And say those five months run up to Diwali, when ad budgets and shopping both jump for reasons that have little to do with each other. The season alone could produce this exact table. The number is a reason to look closer. It’s not a verdict.

A negative correlation analysis example

Here’s another correlation analysis example, also made up. A product priced at ₹10, 12, 14, 16 and 18 sells 90, 82, 75, 60 and 55 units. The correlation lands near −0.99. Price goes up, demand goes down, and the points sit almost on a line. Strong, negative, and still not proof that price alone caused the drop.

The same data, ranked (Spearman)

Rank each column from smallest to largest. Ad spend ranks 1, 2, 3, 4, 5. Units sold (3, 5, 4, 8, 10) rank 1, 3, 2, 4, 5. The rank gaps are 0, −1, 1, 0, 0, so the squared gaps add up to 2. Put that into ρ = 1 − 6Σd² / (n(n² − 1)) and you get 1 − 12/120 = 0.90.

That formula assumes no tied ranks. Ranking is also why Spearman shrugs off outliers. A wild value can only ever be first or last, however huge it is.

When the numbers lie: Anscombe’s quartet

In 1973, the statistician Francis Anscombe published four tiny datasets, eleven points each, to make the case for always looking at a graph. All four have almost the same average, spread, best-fit line and correlation, about 0.82.

Plot them, and they look nothing alike. One is a normal scatter around a line. One is a smooth curve. One is a perfect line thrown off by a single outlier. One is a vertical stack of identical x-values with one stray point holding up the whole coefficient.

A correlation squeezes a cloud of points into one number, and plenty gets lost in the squeeze. So look at the scatter plot first. Every time.  

How to Perform Correlation Analysis in Excel and Python

The steps barely change between tools. Define the two variables and make sure the observations are paired (same month, same customer, same student). With data visualization techniques, plot a scatter chart and look for a line, outliers and odd clusters. Pick the method. Calculate the coefficient. Check significance and note your sample size. Then write the result in careful language, as an association.

Correlation analysis in Excel

Pearson correlation analysis in Excel takes one function for two columns. For several variables at once:

• Switch on the Analysis ToolPak (File → Options → Add-ins)
• Go to Data → Data Analysis → Correlation
• Select your range and tick “Labels in first row”.
• You get a correlation matrix.

Correlation analysis in Python

Pearson correlation analysis in Python is a few lines with pandas and scipy:

import pandas as pd
from scipy import stats

data = pd.DataFrame({
    "ad_spend": [2, 4, 6, 8, 10],
    "units": [3, 5, 4, 8, 10]
})

r, p = stats.pearsonr(data["ad_spend"], data["units"])
print(round(r, 3), round(p, 3))

print(data.corr(method="spearman"))

import seaborn as sns
import matplotlib.pyplot as plt

sns.heatmap(data.corr(), annot=True, vmin=-1, vmax=1)
plt.show()

With a bigger dataset, data.corr() hands you the full matrix. Run it through sns.heatmap(data.corr(), annot=True, vmin=-1, vmax=1) (after import seaborn as sns) and you get a colour grid that's a lot easier to read than rows of decimals. In SPSS it's Analyze → Correlate → Bivariate. 

Correlation Analysis in Data Mining

Correlation analysis in data mining mostly earns its keep in three ways. First, it shows you which input variables move with the thing you're trying to predict. That's feature selection. Second, it catches repeats. If two inputs correlate at 0.95 they're basically telling you the same story, and dropping both into a regression model gives you multicollinearity (the model can't tell which one deserves the credit). Third, it helps you decide what to clean up or merge before you start modelling.

Then there's the problem data people learn the hard way.

Try correlating every variable against every other one and the pairs pile up fast. With 1,000 variables that's 499,500 pairs. Even if nothing in the data were truly related, a 5% cut-off would still throw up around 25,000 "significant" results, purely by luck. More data means more impressive-looking nonsense. That's why data miners adjust for multiple comparisons, hold back a test set, and ask "does this actually make sense?" before getting excited.

Pearson's r only works with numbers. So what do you do with yes/no things, like whether someone bought tea or bought biscuits? Data mining textbooks lean on the chi-square test for categories like these, and on a measure called lift when they're looking at shopping baskets:

A lift above 1 means the two items turn up together more often than luck alone would explain, and that's a positive correlation. Below 1 is negative. Exactly 1 means independence. Say tea shows up in 40% of baskets, biscuits show up in 40%, and 30% of baskets have both. Then lift is 0.30 / (0.40 × 0.40) = 1.875. They travel together. Should they sit on the same shelf, then? That's a business decision, and the number can't make it for you.   

Also Read: How Do You Perform Cluster Analysis Step by Step? 

Correlation and Regression Analysis: What's the Difference?

Plenty of people use "correlation" and "regression" as if they were the same thing. They're not. Correlation and regression analysis show up side by side in textbooks because regression is the natural next step once correlation says "these two are linked".

Difference between correlation and regression analysis

The difference between correlation and regression analysis comes down to the question each one asks.

How r, R² and the slope connect

They aren't rivals. In simple linear regression, R² is just r squared, and the slope equals r times the spread of y divided by the spread of x. On our shop data the slope comes to 0.85 and the intercept to 0.9, so the fitted line is units = 0.9 + 0.85 × ad spend. Every extra ₹1,000 of monthly spend goes with roughly 85 more units sold.

Correlation told us the link is strong. Regression told us how steep it is.

My rule of thumb: use correlation to decide whether it's worth digging, and regression once you need a prediction or an effect size you can plug into a plan. That's the difference between correlation and regression analysis in one line.  

Canonical Correlation Analysis

Ordinary correlation compares one variable with one variable. Canonical correlation analysis compares a whole group with another whole group. Harold Hotelling introduced it in 1936. It builds a weighted blend from each set, called a canonical variate, picked so the pair is as strongly correlated as possible, then repeats for the next-best pair.

Here's a made-up example. Set one is a candidate's scores in quantitative, verbal and logical tests. Set two is four job-performance ratings after hiring. Canonical correlation analysis asks which blend of test scores lines up best with which blend of ratings.

It's powerful. It's also hard to interpret and usually wants a decent sample size, so only reach for it when single-pair correlations really can't answer your question.  

Pro Tip: Plot the data before you interpret the coefficient. Calculate Pearson’s r, then inspect the scatter plot. Look for curves, clusters, outliers and changes in spread. A single coefficient cannot tell you which pattern is present.

Where Correlation Analysis Is Used

What is correlation analysis used for day to day? Mostly screening, and the correlation analysis meaning gets clearer once you see who leans on it.

• In finance, assets that move together give you less diversification. And those relationships aren't fixed. They tend to tighten when markets get stressed, which is exactly when you want them loose.
• Marketers set ad spend against sales and email opens against purchases, with seasonality as the usual trap. HR teams compare engagement scores with attrition, and training hours with performance ratings.
• In machine learning, correlation screens features before a model gets built, which is correlation analysis in data mining at its simplest.
• Medical researchers use it when a controlled experiment would be impractical or unethical.
• Banks and fintech risk teams lean on correlation matrices constantly, partly because they're cheap and quick to read. That's also the danger.
• A matrix keeps looking authoritative long after the conditions that produced it have changed.

Limitations and Common Mistakes

Correlation analysis in statistics is easy to run and hard to interpret. Here's where it bites. Correlation is not causation. That's the golden rule, and it's worth being blunt about. If your report says "X drives Y" and the only evidence is r, rewrite the sentence. Three things can produce the same number.

• X causes Y.
• Y causes X.
• Or something else pushes both.

Tyler Vigen has built a whole website of absurd pairings. One of the best-known sets Maine's divorce rate next to the country's margarine consumption, and the two lines move together so well it became a standard example of a spurious correlation. Nobody thinks margarine ends marriages. But the chart looks every bit as solid as the ones we do believe, and that's the uncomfortable part.

What to do about confounders. You can't wish the third variable away, but you can account for it. Partial correlation measures the link between x and y after removing the part of each that the third variable explains, and multiple regression does something similar with several at once. For our shop, that means adding a "festive month" flag and checking whether the ad spend link survives. If it shrinks a lot, the season was doing most of the work.

Only straight lines count. The y = x² example gave r = 0 for a perfect relationship. A clearly curved exponential pattern can still return a middling r, around 0.5 in the examples published in the medical statistics literature, which hides how strong the link really is.

Outliers. One extreme point can create a correlation that isn't there or wipe out one that is. Anscombe's quartet is the standing proof.

Range restriction. A paper in the Clinical Kidney Journal shows the same data giving r = 0.87 across the full range and r = 0.57 when only the lower half is analysed. So you can't compare coefficients across groups or studies unless the ranges are similar, and you can inflate r just by adding extreme values.

Correlation is not agreement. If one measuring method always reads exactly twice the other, the two correlate perfectly and disagree on every single reading. To compare methods, use limits of agreement or the intraclass coefficient instead.

Small samples and fishing. Five points can't carry a business decision; a thousand variables will always hand you something to "find", and every coefficient is only as trustworthy as its data.   

Also Read: Why Is Statistical Analysis Important in Data Science? 

How Imarticus Learning Helps Professionals Put Data Analysis Tools to Work

Knowing how to ask ChatGPT a question is useful. Knowing how to turn its output into a reliable report, automate a recurring task or build a workflow that saves time is far more valuable at work. The data analytics course by Imarticus Learning, with PwC Academy, is built around that practical shift. Instead of stopping at tool demonstrations, the programme challenges you to apply AI to real work problems and show what your solutions can do.

• 18 AI tools in 90 days: Work with tools including ChatGPT, Gemini, Claude, Microsoft Copilot, Zapier, Gamma, CrewAI and Canva AI. The focus is on using the right tool for the task, rather than collecting tool names.
• No coding required: Build your AI skills from the ground up, even without a technical background. The programme covers prompting, workflow automation, AI agents and no-code app development.
• Live weekend classes led by industry practitioners: Attend live sessions on Saturdays and Sundays, typically three hours a day, with guided self-practice and project work during the week.
• Build workflows around your actual job: Use tools such as Google Sheets, ChatGPT, Notion AI, Zapier and Make to automate a weekly task. The workflow-building challenge sets a goal of saving at least one hour each week.
• Progress from prompts to AI agents: The curriculum moves from generative AI fundamentals and productivity to function-specific applications, workflow automation, AI agents, no-code website development, and AI ethics and risks.
• Joint certification: The programme offers joint certification from Imarticus Learning and PwC Academy.

FAQs About Correlation Analysis

From interpreting a weak correlation to spotting hidden variables, these answers address the frequently asked questions that arise when Correlation Analysis meets real-world data. 

What are the four types of correlation analysis?

Correlation analysis usually gets grouped into positive, negative, zero and non-linear (curvilinear) correlation. By method, you'd pick from Pearson, Spearman, Kendall's tau and point-biserial, depending on whether your data is continuous, ranked or yes/no.

Which test is used for correlation analysis?

The test used for correlation analysis is Pearson's r test. It is the usual pick for two continuous, roughly linear variables, with a t-test to check significance. For ranked data or steady curved trends, go with Spearman's rho or Kendall's tau. Correlation analysis in Excel starts with the CORREL function.

What is the golden rule of correlation analysis?

The golden rule is that Correlation doesn't imply causation. A strong r tells you two variables move together, not that one causes the other. Confounders, coincidence or reverse causality can produce the same number, so treat any correlation analysis result as a lead to test.

What are examples of correlation analysis?

Common examples of correlation analysis are ad spend versus sales, hours studied versus exam marks, price versus demand, and the daily returns of two stocks. Each correlation analysis example pairs two numeric variables and reports r, showing how closely and in which direction they move.

How to perform a correlation analysis test?

To perform a correlation analysis test, plot the two variables, pick Pearson or Spearman, calculate the coefficient, then check its p-value. In Excel, use CORREL; in Python, use scipy's pearsonr. Report r, the sample size and the p-value together.

Is there any formula for correlation analysis?

Yes, the formula for correlation analysis includes Pearson's formula is r = Σ(x−x̄)(y−ȳ) / √[Σ(x−x̄)² × Σ(y−ȳ)²]. It divides the shared movement of x and y by their individual spreads, giving a value between −1 and +1 for correlation analysis.

What is the main purpose of correlation analysis?

The main purpose of correlation analysis is to measure how strongly, and in which direction, two variables move together. It helps analysts spot relationships worth a closer look, screen variables before regression, and drop repeat features, though it can't prove cause.

What is the law of correlation analysis?

There's no formal law for correlation analysis, but two things always hold: the coefficient sits between −1 and +1, and correlation never proves causation. Treat any correlation analysis result as a description of the data you have, not a rule of nature.   

Ask Better Questions of Your Data With Correlation Analysis

If you only remember three things, make them these. Plot the data before you trust the number. Read r as the strength and direction of a straight-line link and nothing else. And never let a correlation write the word "because" for you.

So, what is correlation analysis in the end? A quick, honest first look at how two variables move together. And if anyone asks what is the major attribute of correlation analysis, it's that it gives you strength and direction in one number while saying nothing about cause. Done properly, it earns its place at the start of almost every analysis. Done carelessly, it's a very official-looking way to be wrong. The gap between the two is mostly habit, not maths.

Galton's sweet peas taught him something statisticians took a century to fully absorb: patterns are easy to spot and hard to explain. If you'd like to put data analysis tools to work on the reports, summaries and dashboards this kind of number ends up in, the data analytics course offered by Imarticus Learning is worth a look. It isn't a statistics course.

Liked This? Follow Us Where You Already Search.

Preferred Sources is Google’s built-in follow button — no app, no login, just more Imarticus in your results.