The correlation coefficient is an indicator that shows the degree of linear relationship between two measured values.
In chemistry experiments, it is used to evaluate calibration curves, reaction rate analysis, comparisons of physical properties, relationships between concentration and measurement signals, and other data.
Particularly in analytical chemistry experiments, it is important for determining whether relationships such as concentration and absorbance, concentration and peak area, or concentration and potential are linear.
However, a high correlation coefficient or coefficient of determination does not necessarily mean that the experiment is completely correct.
A high correlation may still be obtained when the number of measurement points is small, the measurement range is narrow, outliers are conveniently excluded, or systematic errors are present.
Therefore, when discussing correlation coefficients, it is necessary to consider not only the numerical value but also the shape of the graph, variation among measurement points, outliers, intercept, slope, and correspondence with the theoretical equation.
This article clearly explains, as examples of discussions that can be used in laboratory reports on correlation coefficients, the correlation coefficient r, coefficient of determination R2, linearity of calibration curves, handling of outliers, number of measurement points, errors in preparing standard solutions, concentration range, deviations at low and high concentrations, conditions under which outliers may be excluded, causes of error, and points for improvement.
Note:
This article is a reference intended to assist with discussions of calibration curves and correlation coefficients prepared in basic chemistry experiments, analytical chemistry experiments, and physical chemistry experiments at universities and similar institutions.
For the actual handling of correlation coefficients, coefficients of determination, outliers, criteria for adopting calibration curves, and statistical processing, always follow the instructions in your university’s laboratory manual and those given by your instructor or TA.
- What Is a Correlation Coefficient?
- Main Items to Include in the Results
- What Is the Coefficient of Determination R2?
- Difference Between the Correlation Coefficient r and the Coefficient of Determination R2
- What Is Linearity of a Calibration Curve?
- Discussion When R2 Is High
- Discussion When R2 Is Low
- What Is an Outlier?
- When an Outlier May Be Excluded
- When an Outlier Should Not Be Excluded
- Effect of Outliers on R2
- Precautions When the Number of Measurement Points Is Small
- Precautions When the Concentration Range Is Narrow
- When Linearity Breaks Down at High Concentrations
- When Variation Is Large at Low Concentrations
- Errors in Preparing Standard Solutions
- Errors in Measurement Signals
- Relationship Between the Intercept and Correlation Coefficient
- Relationship Between the Slope and Correlation Coefficient
- When Error Is Large Despite High Correlation
- Problems with Quantification Outside the Calibration Curve Range
- How to Discuss Results Including an Outlier
- How to Write About Excluding an Outlier
- Difference Between Correlation Coefficient and Causality
- Causes of Error Affecting the Correlation Coefficient
- When the Results Can Be Considered Good
- Example Discussions When the Experiment Did Not Go Well
- How to Write Points for Improvement
- Difference Between a Superficial Discussion and a Good Discussion
- Examples of Expressions That Can Be Used in Reports
- Points to Check When Discussing Correlation Coefficients
- Summary
What Is a Correlation Coefficient?
A correlation coefficient is a numerical value that represents the degree of linear relationship between two variables.
In general, the correlation coefficient r ranges from -1 to +1.
The closer r is to +1, the stronger the positive linear relationship; the closer r is to -1, the stronger the negative linear relationship; and the closer r is to 0, the weaker the linear relationship is considered to be.
In chemistry experiments, a positive correlation is expected in relationships where absorbance or peak area increases as concentration increases.
However, the correlation coefficient indicates the “strength of a linear relationship” and does not directly prove a cause-and-effect relationship.
In addition, when the relationship is curved, the correlation coefficient alone may not provide an appropriate evaluation.
Example Discussion:
Because the correlation coefficient r was close to 1, a strong positive linear relationship was considered to exist between the concentration of the standard solutions and the measurement signal.
This indicates that the measurement signal increased at an approximately constant rate as the concentration increased.
However, the correlation coefficient is an indicator of linearity and does not directly indicate that the measured values are free from systematic error.
Main Items to Include in the Results
To discuss the correlation coefficient, it is necessary to organize not only the numerical value of the correlation coefficient but also the calibration curve equation, coefficient of determination, number of measurement points, presence or absence of outliers, and measurement range.
Showing only the correlation coefficient without examining the graph does not sufficiently explain the characteristics of the data.
Main Items to Include in the Results
- Values plotted on the horizontal and vertical axes
- Units of each value
- Number of measurement points
- Equation of the regression line
- Correlation coefficient r
- Coefficient of determination R2
- Slope
- Intercept
- Variation among measurement points
- Presence or absence of outliers
- Whether outliers were excluded
- Concentration range used for the calibration curve
- Whether the unknown sample is within the calibration curve range
- Range in which linearity was lost
- Causes of error
- Points for improvement
Example of How to Write the Results:
When a calibration curve was prepared using the concentration of the standard solutions on the horizontal axis and absorbance on the vertical axis, the measurement points were distributed almost along the regression line, and the coefficient of determination R2 was close to 1.
This indicates that there was a good linear relationship between concentration and absorbance within the measurement range.
On the other hand, some measurement points deviated slightly from the regression line, suggesting that errors in concentration preparation or measurement procedures may have affected the results.
What Is the Coefficient of Determination R2?
The coefficient of determination R2 is an indicator of how well the regression line explains the variation in the measured values.
R2 ranges from 0 to 1, and the closer it is to 1, the better the measurement points are considered to agree with the regression line.
In a calibration curve, a higher R2 is considered to indicate better linearity.
However, a high R2 does not necessarily mean that the measured values are accurate.
For example, even if all concentrations are systematically prepared incorrectly, R2 may still be high if the measurement points lie on a straight line.
It is important to note that R2 is an indicator of linearity and does not directly indicate accuracy or trueness.
Example Discussion:
Because the coefficient of determination R2 was close to 1, the regression line was considered to represent the trend of the measurement points well.
Therefore, this calibration curve showed good linearity within the measurement range.
However, even when R2 is high, systematic errors such as errors in preparing the concentrations of standard solutions or deviations in blank correction may still be present, so the graph intercept and measurement procedures must also be checked.
Difference Between the Correlation Coefficient r and the Coefficient of Determination R2
The correlation coefficient r indicates the strength and direction of the linear relationship between two variables.
A positive r indicates a positive correlation, while a negative r indicates a negative correlation.
The coefficient of determination R2, on the other hand, indicates how well the regression line explains the measured values.
In simple regression, R2 corresponds to the square of the correlation coefficient r.
In calibration curve reports, the coefficient of determination R2 is often displayed more frequently than the correlation coefficient r.
However, R2 does not indicate the positive or negative direction of the relationship.
When a positive relationship is expected, such as between concentration and signal, it is also necessary to confirm that the slope of the graph is positive.
| Indicator | Meaning | Point for Discussion |
|---|---|---|
| Correlation coefficient r | Strength and direction of a linear relationship | Also shows whether the correlation is positive or negative |
| Coefficient of determination R2 | Goodness of fit of the regression line | The closer to 1, the better the linearity |
Example Discussion:
The correlation coefficient r indicates the direction and strength of a linear relationship, while the coefficient of determination R2 indicates how well the regression line explains the measured values.
In a calibration curve, in addition to confirming that R2 is close to 1, it is necessary to confirm that the slope is positive and that the measurement signal increases as the concentration increases.
Therefore, it is important to discuss the entire regression equation rather than considering only R2.
What Is Linearity of a Calibration Curve?
Linearity of a calibration curve means that a linear relationship exists between the concentration of the standard solution and the measurement signal.
In absorbance measurements, according to the Beer-Lambert law, absorbance is expected to be proportional to concentration within a certain range.
In HPLC and GC, concentration and peak area may show a linear relationship within a certain range.
With a calibration curve that has good linearity, it is easier to determine the concentration of an unknown sample from its measurement signal.
However, at low concentrations, the effect of noise becomes large, while at high concentrations, the detector response may become saturated or the proportional relationship may break down.
Therefore, a calibration curve must be used within the concentration range in which linearity is maintained.
Example Discussion:
Because the measurement points of the calibration curve were distributed almost along the regression line, good linearity was considered to exist between the concentration of the standard solutions and the measurement signal.
Within the range in which this linearity holds, the concentration of an unknown sample can be determined by substituting its measured value into the regression equation.
However, extrapolation outside the range in which linearity was confirmed may result in large quantitative errors.
Discussion When R2 Is High
When R2 is high, the measurement points agree well with the regression line, and the linearity of the calibration curve is considered good.
The standard solutions may have been prepared and measured relatively consistently, and the relationship between concentration and signal may have been stable.
The calibration curve is therefore easier to use for quantifying unknown samples.
However, a high R2 alone is not sufficient.
It is necessary to check whether the intercept is greatly shifted, whether the concentration range of the standard solutions is appropriate, whether outliers were excluded, and whether the unknown sample lies within the calibration curve range.
R2 is only one of the criteria used to evaluate a calibration curve.
Example Discussion:
Because R2 was close to 1, the measurement points agreed extremely well with the regression line.
This indicates a good linear relationship between the concentration of the standard solutions and the measurement signal.
However, to judge the validity of the calibration curve, it is necessary to check not only R2 but also the intercept, outliers, concentration range, and measurement range of the unknown sample.
Discussion When R2 Is Low
When R2 is low, the measurement points show large variation from the regression line, and the linear relationship is considered weak.
Possible causes include errors in preparing standard solutions, pipetting errors, insufficient sample mixing, instrument noise, changes in measurement conditions, outliers, and saturation at high concentrations.
If an unknown sample is quantified using a calibration curve with a low R2, the reliability of the calculated concentration decreases.
In such a case, improvements such as preparing the standard solutions again, repeating the measurements, restricting the range to one in which linearity holds, and checking the cause of outliers are necessary.
Example Discussion:
Because R2 was low, the linearity between the concentration of the standard solutions and the measurement signal was considered insufficient.
Possible causes include errors in dilution of the standard solutions, contamination of the measurement cell, instrument noise, or signal saturation on the high-concentration side.
Because quantitative values of unknown samples obtained using this calibration curve contain large uncertainty, the measurement conditions and concentration range must be reviewed.
What Is an Outlier?
An outlier is a value that deviates greatly from the trend of the other measurement points.
In a calibration curve, if only one point is far from the regression line, that measurement point may be an outlier.
Outliers may be caused by errors in concentration preparation, measurement errors, contamination of the cell, insufficient sample mixing, bubbles, reading errors, and other factors.
Outliers greatly affect the slope, intercept, and R2 of the regression line.
Particularly when the number of measurement points is small, a single outlier may greatly change the entire calibration curve.
Therefore, when an outlier is present, its cause must be carefully considered.
Example Discussion:
Because one measurement point deviated greatly from the regression line, that point may have contained an error in concentration preparation or measurement procedures.
An outlier greatly affects the slope, intercept, and R2 of the regression line.
Therefore, when an outlier is found, it is important to investigate its cause and, if necessary, repeat the measurement.
When an Outlier May Be Excluded
An outlier may be excluded when a clear abnormality in the measurement or operation can be identified.
Examples include preparing the standard solution with the wrong dilution, bubbles being present in the cell, an error occurring while reading the measuring instrument, or forgetting to mix the sample.
When a specific cause can be confirmed in this way, a basis for excluding the point from the calibration curve can be explained.
However, when excluding an outlier, it is useful to show how the regression line and R2 change before and after exclusion.
In addition, rather than deleting the outlier, it is desirable to repeat the measurement for confirmation.
The reason for exclusion must be clearly stated in the report.
Example Discussion:
Bubbles were observed in the cell during measurement at the point identified as an outlier, so the absorbance may have been measured as larger than the actual value.
A clear operational abnormality of this kind provides a basis for treating the point as an outlier.
However, when excluding an outlier, the reason for exclusion must be clearly stated, and if possible, the result should be confirmed by remeasurement.
When an Outlier Should Not Be Excluded
An outlier must not be excluded simply because you want to increase R2, make the regression line look cleaner, or because the point is inconvenient.
A deviating measurement point is also part of the experimental results, and deleting it without knowing the cause amounts to modifying the results for convenience.
Scientific and operational justification is required to exclude an outlier.
For an outlier whose cause is unknown, first repeat the measurement or check the procedural records.
If remeasurement is not possible, honestly discuss the results including the outlier and the uncertainty caused by its presence.
An outlier is not something that should simply be deleted but something whose cause should be considered.
Example Discussion:
It is not appropriate to exclude a measurement point solely because it deviates from the regression line.
An outlier is also part of the experimental results, and clear evidence such as an error in concentration preparation or an abnormality during measurement is required for exclusion.
If the cause is unknown, the point should be included in the discussion and treated as a factor contributing to variation in the measured values and reduced reliability of the calibration curve.
Effect of Outliers on R2
When an outlier is present, R2 tends to decrease.
This is because a point located far from the regression line reduces the degree to which all measurement points agree with the line.
Depending on the position of the outlier, the slope and intercept of the regression line itself may also change greatly.
Particularly when a point at the high- or low-concentration end deviates, it strongly affects the slope of the entire regression line.
As a result, errors may also occur in the calculated concentration of the unknown sample.
An outlier may not only reduce R2 but also shift the entire quantitative result.
Example Discussion:
The presence of the outlier may have reduced R2 and changed the slope and intercept of the regression line.
In particular, when a measurement point at the end of the calibration curve deviates, it has a large effect on the slope of the entire regression line.
Therefore, because outliers also affect quantitative values of unknown samples, checking the cause and repeating the measurement are important.
Precautions When the Number of Measurement Points Is Small
When the number of measurement points is small, the correlation coefficient and R2 may be overestimated.
For example, if a straight line is drawn through only two points, it will always be a perfect straight line.
However, this alone does not demonstrate that the calibration curve truly has good linearity.
It is important to prepare a sufficient number of standard points.
When there are few measurement points, an error in a single point has a large effect on the regression line.
For a calibration curve, multiple standard solutions should be prepared from low to high concentrations, and linearity should be confirmed over the entire measurement range.
Not only the R2 value but also the number and placement of standard points should be discussed.
Example Discussion:
When the number of measurement points is small, even a high R2 does not mean that the linearity of the calibration curve has been sufficiently confirmed.
With only a few points, a measurement error in one point has a large effect on the slope and intercept.
Therefore, to prepare a reliable calibration curve, multiple standard points must be established within the measurement range and linearity must be confirmed.
Precautions When the Concentration Range Is Narrow
When the concentration range is narrow, measurement points may appear more linear, and R2 may also tend to become high.
However, even if linearity is good over a narrow range, the same relationship does not necessarily hold over a wider range.
If the unknown sample is outside that range, the calibration curve must be extrapolated, which increases the error.
As a basic rule, a calibration curve should be prepared over a range that includes the concentration of the unknown sample.
If the unknown sample concentration is too high, it should be diluted so that it falls within the calibration curve range.
When discussing linearity, it is necessary to check whether the measurement range was appropriate.
Example Discussion:
Although the R2 of the calibration curve was high, the concentration range of the standard solutions was narrow, so the linear relationship may not necessarily hold over a wider concentration range.
If the concentration of the unknown sample lies outside the calibration curve range, quantification requires extrapolation and the error becomes larger.
Therefore, the calibration curve must be prepared over a range that includes the concentration of the unknown sample.
When Linearity Breaks Down at High Concentrations
In a calibration curve, linearity may break down on the high-concentration side.
Possible causes include absorbance becoming too high, saturation of the detector response, deviation from the Beer-Lambert law, association or precipitation occurring in the sample, and turbidity.
If measurement points on the high-concentration side fall below the regression line, signal saturation may be suspected.
If linearity is poor on the high-concentration side, the calibration curve should not be forcibly fitted as a straight line including those points.
Instead, the calibration curve should be prepared using only the range in which linearity is maintained.
If the unknown sample has a high concentration, it must be diluted so that it falls within the measurement range.
Example Discussion:
Possible reasons why measurement points on the high-concentration side deviated from the regression line include saturation of the measurement signal or deviation from the Beer-Lambert law.
At high concentrations, absorbance may become too large, making it difficult for the proportional relationship between concentration and signal to hold.
Therefore, only the concentration range in which linearity can be confirmed should be used for the calibration curve, and high-concentration unknown samples must be appropriately diluted before measurement.
When Variation Is Large at Low Concentrations
On the low-concentration side, because the measurement signal is small, the relative effects of instrument noise and blank values become large.
As a result, the measurement points tend to vary, and the linearity of the calibration curve may appear poor.
Errors on the low-concentration side are also related to the detection limit and quantification limit.
To accurately measure low-concentration samples, blank correction, increasing the number of measurements, concentration of the sample, and selection of highly sensitive measurement conditions may be necessary.
If variation is large on the low-concentration side, the large uncertainty in quantitative values within that range should be discussed.
Example Discussion:
Possible reasons why the measurement points on the low-concentration side showed variation include the small measurement signals and their susceptibility to instrument noise and blank correction.
At low concentrations, even slight operational errors result in large relative errors.
Therefore, when quantifying low-concentration samples, it is important to perform sufficient blank measurements and, when necessary, take multiple measurements and use the mean.
Errors in Preparing Standard Solutions
Errors in preparing standard solutions have a large effect on the linearity of a calibration curve.
Because the concentrations of the standard solutions become the values on the horizontal axis, errors in pipetting or using volumetric flasks, errors in calculating dilution factors, or insufficient mixing of the solutions cause the measurement points to deviate from the regression line.
Particularly in serial dilution, an error in one solution may be carried over to the next solution.
Errors in preparing standard solutions can cause a decrease in R2 and produce outliers.
To prepare an accurate calibration curve, the equipment must be used correctly, the calibration mark must be read accurately, and the solution must be mixed thoroughly.
Measuring the prepared standard solutions multiple times is also effective.
Example Discussion:
A possible reason why some points on the calibration curve deviated from the straight line is an error in preparing the concentrations of the standard solutions.
If there is an error in pipetting or dilution, the concentration value on the horizontal axis differs from the actual concentration, causing the measurement point to deviate from the regression line.
Therefore, to improve the linearity of the calibration curve, it is important to prepare the standard solutions accurately and mix them thoroughly.
Errors in Measurement Signals
Errors in the measurement signal also affect the correlation coefficient and R2.
In absorbance measurements, possible causes include contamination of the cell, bubbles, fingerprints, sample turbidity, insufficient blank correction, and incorrect wavelength settings.
In chromatography, variation in injection volume, errors in peak integration, and baseline shifts may have an effect.
If errors in the measurement signal are large, the values on the vertical axis vary, reducing the goodness of fit of the regression line.
In addition, if all signals are shifted in the same direction, the intercept and slope are systematically affected.
Calibration of the measuring instrument and standardization of measurement procedures are important.
Example Discussion:
Possible causes of variation in the measurement signal include contamination of the cell, bubbles, insufficient blank correction, and instrument noise.
These factors affect the values on the vertical axis and cause the measurement points to deviate from the regression line.
Therefore, before measurement, it is necessary to clean the cell, remove bubbles, perform blank correction, and confirm that the instrument has stabilized.
Relationship Between the Intercept and Correlation Coefficient
In a calibration curve, the intercept may be greatly shifted even when R2 is high.
In this case, although the measurement points are well aligned on a straight line, the entire set of points may be shifted along the vertical axis.
A shifted intercept may indicate insufficient blank correction, zero-point drift of the instrument, or background signals from reagents or solvents.
In other words, a high R2 and a theoretically reasonable intercept are separate issues.
If the intercept is large in a calibration curve that should pass through the origin, it affects the quantitative value of the unknown sample.
When evaluating a calibration curve, the intercept must always be checked in addition to R2.
Example Discussion:
Although R2 was high, the intercept deviated greatly from 0.
This suggests that although the measurement points were aligned linearly, a background signal may have been present even at concentration 0.
Therefore, when judging the validity of the calibration curve, not only R2 as an indicator of linearity but also the magnitude of the intercept and the appropriateness of blank correction must be considered.
Relationship Between the Slope and Correlation Coefficient
The slope indicates how much the value on the vertical axis changes when the value on the horizontal axis changes by one unit.
While the correlation coefficient and R2 represent linearity, the slope represents measurement sensitivity or the magnitude of the proportionality constant.
Even if R2 is high, if the slope differs greatly from the theoretical or literature value, there may be a problem with the measurement system.
For example, even if the concentrations of all standard solutions are prepared incorrectly by the same proportion, the measurement points may still lie on a straight line and R2 may remain high.
However, the slope may deviate from the correct value.
The validity of the slope must therefore be evaluated in addition to the correlation coefficient.
Example Discussion:
Even when R2 is high, if the slope of the regression line differs greatly from the theoretical value, errors in preparing the standard solutions or deviations in measurement sensitivity may be present.
The correlation coefficient indicates the quality of linearity but does not directly guarantee the validity of the slope.
Therefore, when evaluating a calibration curve, R2, slope, and intercept must all be checked.
When Error Is Large Despite High Correlation
Even when the correlation is high, the actual quantitative value may contain a large error.
This can occur when the measurement points lie neatly on a straight line but systematic errors are present, such as incorrect standard concentrations, shifted blank correction, or uncalibrated instrument sensitivity.
The correlation coefficient indicates how little the measurement points vary from a straight line and does not indicate whether the values are close to the true value.
Therefore, when evaluating a calibration curve, it is necessary to check not only the correlation coefficient but also the accuracy of the standard substances, equipment calibration, blank correction, measurement range, and reproducibility.
It is important not to assume that “high correlation = accurate.”
Example Discussion:
Even when R2 is high, if the concentrations of the standard solutions were systematically prepared incorrectly, the calculated unknown concentration may contain a large error.
The correlation coefficient indicates the degree to which measurement points align on a straight line but does not guarantee the accuracy of the concentration values.
Therefore, even when the correlation is high, the validity of standard solution preparation and blank correction must be confirmed.
Problems with Quantification Outside the Calibration Curve Range
If the measured value of an unknown sample lies outside the calibration curve range, its concentration must be determined by extrapolating the regression line.
However, the same linear relationship does not necessarily hold outside the calibration curve range.
Because signal saturation becomes important on the high-concentration side and noise becomes more influential on the low-concentration side, quantification by extrapolation produces larger errors.
If the unknown sample lies outside the calibration curve range, it is desirable to dilute or concentrate the sample so that the measured value falls within the calibration curve range.
In a report, check whether the unknown sample lies within the range of the standard solutions, and if it lies outside the range, discuss the increased uncertainty.
Example Discussion:
If the measured value of the unknown sample lies outside the calibration curve range, the concentration must be determined by extrapolating the regression line, reducing the reliability of the quantitative value.
Outside the calibration curve range, the linear relationship between concentration and measurement signal does not necessarily hold.
Therefore, the unknown sample should be diluted or concentrated and measured within the linear range confirmed using the standard solutions.
How to Discuss Results Including an Outlier
Even when an outlier cannot be excluded, it can still be included in the discussion.
For example, it can be written that R2 decreased because of the outlier, that the slope changed because of the outlier, or that possible causes of the outlier include errors in concentration preparation or measurement procedures.
It is important not to hide the outlier but to treat it as uncertainty in the results.
If an outlier is present, indicate which point deviates on the graph and list possible causes.
In addition, if remeasurement is possible, repeat the measurement; if it is not possible, state that the reliability of the quantitative result is reduced.
An outlier is not evidence of failure but material for considering the causes of error.
Example Discussion:
In this calibration curve, some measurement points deviated from the regression line, and the presence of these points was considered to have reduced R2.
Possible causes of the outliers include errors in dilution of the standard solutions or bubbles in the cell during measurement.
Because no clear operational error could be confirmed, these points were not excluded and were instead discussed as factors that increased uncertainty in the calibration curve.
How to Write About Excluding an Outlier
If an outlier is excluded, the reason for exclusion must always be clearly stated.
In addition, explaining how R2 and the regression equation changed before and after exclusion makes the judgment more transparent.
However, in a report, the instructions of the instructor must be followed, and outliers should not be removed arbitrarily.
A discussion of outlier exclusion must not end simply with “R2 increased after exclusion.”
It is important to explain why the point deviated, whether remeasurement is necessary, and how much the quantitative value changes as a result of exclusion.
Example Discussion:
At the measurement point identified as an outlier, bubbles had been observed in the cell during measurement, so the absorbance was highly likely to have been overestimated.
Therefore, when the regression line was calculated after excluding this point, R2 improved and agreed well with the trend of the other measurement points.
However, because excluding an outlier greatly affects the result, the reason for exclusion must be clearly stated, and if possible, the result should be confirmed by remeasurement.
Difference Between Correlation Coefficient and Causality
The correlation coefficient is an indicator that shows the tendency of two values to change together.
However, the existence of a correlation does not directly show that one variable causes the other.
In chemistry experiments, when a clear theoretical relationship exists, such as in a calibration curve, the relationship can be treated as one between concentration and signal, but in general data analysis, correlation and causality must be distinguished.
For example, even if temperature and reaction rate are correlated, catalyst activity or solubility may also be changing at the same time.
Rather than drawing conclusions from the correlation coefficient alone, it is important to discuss causality based on experimental conditions and theoretical equations.
Example Discussion:
A high correlation coefficient indicates that there is a linear relationship between two measured values, but it does not directly prove that one causes the other.
In a case such as a calibration curve where a theoretical relationship exists, the result can be interpreted as a relationship between concentration and signal, but for general experimental data, the influence of other factors must also be considered.
Therefore, it is important to interpret the correlation coefficient together with theory and experimental conditions.
Causes of Error Affecting the Correlation Coefficient
Causes of error that affect the correlation coefficient and R2 include errors in preparing standard solutions, variation in measurement signals, outliers, an insufficient number of measurement points, an inappropriate concentration range, instrument noise, insufficient blank correction, and changes in temperature or pH.
When these effects are large, measurement points deviate from the regression line and R2 decreases.
On the other hand, if the measured values are systematically shifted, R2 may remain high while the slope or intercept becomes incorrect.
When discussing the correlation coefficient, it is useful to distinguish between errors caused by variation and systematic shifts.
Example Discussion:
Possible causes of the decrease in R2 include errors in preparing the standard solutions, contamination of the cell during measurement, instrument noise, and the effect of outliers.
These factors cause the measurement points to vary from the regression line and reduce the linearity of the calibration curve.
On the other hand, even if R2 is high, if the concentration preparation is systematically shifted, the slope may be incorrect, so accuracy cannot be judged from the correlation coefficient alone.
When the Results Can Be Considered Good
From the perspective of the correlation coefficient and R2, results can be considered good when the measurement points agree well with the regression line, R2 is high, there are few outliers, and the intercept and slope are theoretically reasonable.
Furthermore, if the measured value of the unknown sample lies within the calibration curve range, the calibration curve can be considered easier to use.
However, a good calibration curve is not simply one with a high R2.
It is important that there are enough standard points, the concentration range is appropriate, blank correction has been performed, and the variation in the measured values is small.
When the calibration curve is used to calculate yield or concentration, its reliability must be evaluated comprehensively.
Example Discussion:
In this calibration curve, the measurement points were distributed almost along the regression line, and R2 was close to 1.
In addition, no outliers were observed and the intercept was small, so good linearity was judged to have been obtained within the measurement range.
Therefore, this calibration curve was considered generally appropriate as a reference line for quantifying unknown samples.
Example Discussions When the Experiment Did Not Go Well
When a good correlation coefficient or calibration curve linearity is not obtained, consider the causes based on variation among measurement points, outliers, low R2, shifts in the intercept, abnormal slopes, saturation on the high-concentration side, and noise on the low-concentration side.
Organizing the causes into standard solutions, measurement procedures, instruments, concentration range, and data processing makes the discussion easier to write.
Example Discussion:
A possible reason why R2 was low is an error in preparing the standard solutions.
If there is an error in the dilution procedure, the concentration values on the horizontal axis differ from the actual values, causing the measurement points to deviate from the regression line.
As a result, the linearity of the calibration curve decreases, and the quantitative values of unknown samples may also contain large uncertainty.
Another Example Discussion:
Because the measurement points on the high-concentration side deviated from the straight line, the R2 of the entire calibration curve was considered to have decreased.
At high concentrations, the detector response may become saturated or the proportional relationship between absorbance and concentration may break down.
In this case, the calibration curve must be prepared using the low- to medium-concentration range in which linearity is maintained, and unknown samples must be diluted when necessary before measurement.
Another Example Discussion:
Possible reasons why only one point deviated greatly from the regression line include bubbles entering the cell during measurement or insufficient mixing of the standard solution.
This outlier changed the slope and intercept of the regression line and also reduced R2.
Before excluding an outlier, its cause must be checked, and if possible, its validity should be confirmed by remeasurement.
How to Write Points for Improvement
In a discussion of correlation coefficients and calibration curves, it is important not only to point out that R2 was low or that an outlier was present, but also to explain how the experiment can be improved.
Points for improvement can be organized into preparation of standard solutions, measurement procedures, concentration range, handling of outliers, and data processing.
Improvements to Standard Solutions and Measurement Procedures
- Prepare standard solutions accurately
- Check the calculation of dilution factors
- Use pipettes and volumetric flasks correctly
- Mix the solutions thoroughly
- Perform blank correction before measurement
- Check the cell for contamination and bubbles
- Measure multiple times under the same conditions
- Perform measurements after the instrument has stabilized
Improvements to Calibration Curves and Data Processing
- Use a sufficient number of standard points
- Prepare the calibration curve over a concentration range that includes the unknown sample
- If linearity breaks down on the high-concentration side, narrow the range
- On the low-concentration side, check the effects of noise and the blank
- If an outlier is excluded, provide clear justification
- Compare the regression equation and R2 before and after exclusion
- Check not only R2 but also the slope and intercept
- Avoid extrapolation outside the calibration curve range
Example of How to Write Points for Improvement:
To improve the linearity of the calibration curve, it is necessary to accurately prepare the concentrations of the standard solutions and check blank correction and the condition of the cell before measurement.
In addition, if an outlier is observed, possible causes such as errors in concentration preparation or bubbles during measurement should be checked, and remeasurement should be performed if possible.
Furthermore, it is important to select a concentration range in which linearity is maintained and measure the unknown sample within that range.
Difference Between a Superficial Discussion and a Good Discussion
In a discussion of correlation coefficients, simply writing that “R2 was high” or “there was an outlier” results in a superficial discussion.
A good discussion explains what R2 indicates, why the outlier occurred, and how it affects the quantitative values obtained from the calibration curve.
| Superficial Discussion | Good Discussion |
|---|---|
| R2 was high. | Because R2 was close to 1, the measurement points agreed well with the regression line, indicating good linearity within the measurement range. |
| R2 was low. | Errors in preparing the standard solutions or variation in measurement signals may have caused the measurement points to deviate from the regression line, reducing the linearity of the calibration curve. |
| It improved after the outlier was deleted. | Excluding an outlier requires clear operational evidence such as bubbles or a dilution error, and the change in the regression equation before and after exclusion must be checked. |
| It is correct because there is a correlation. | A high correlation coefficient indicates linearity but does not guarantee the absence of systematic errors in standard concentrations or deviations in blank correction. |
| The points were scattered. | On the low-concentration side, because the measurement signal is small, the relative effects of instrument noise and blank values become large, making variation more likely. |
Examples of Expressions That Can Be Used in Reports
The following expressions can be used when writing the results and discussion of correlation coefficients and calibration curves.
Adjust the necessary parts according to your own experimental results.
- Because the correlation coefficient r was close to 1, a strong positive linear relationship was considered to exist between the horizontal and vertical axes.
- Because the coefficient of determination R2 was high, the regression line represented the trend of the measurement points well.
- R2 is an indicator of linearity and does not directly indicate the accuracy of measured values.
- Errors in preparing standard solutions can cause variation and outliers in the calibration curve.
- Outliers greatly affect the slope, intercept, and R2 of the regression line.
- If an outlier is excluded, clear operational justification must be provided.
- If linearity breaks down on the high-concentration side, saturation of the measurement signal may be a possible cause.
- On the low-concentration side, the relative effects of blank values and instrument noise become large.
- Quantification of unknown samples must be performed within the linear range of the calibration curve.
- The validity of a calibration curve must be evaluated using not only R2 but also the slope, intercept, outliers, and measurement range.
Points to Check When Discussing Correlation Coefficients
Checking the following points before writing the report makes the discussion easier to write.
- Is the meaning of the correlation coefficient r or coefficient of determination R2 explained?
- Is the judgment based only on the R2 value?
- Is the equation of the regression line shown?
- Have the validity of the slope and intercept been checked?
- Has it been confirmed that the concentration range of the calibration curve is appropriate?
- Has it been confirmed that the unknown sample lies within the calibration curve range?
- Has the presence or absence of outliers been checked?
- Have the causes of outliers been discussed?
- Is there justification for excluding outliers?
- Have errors in preparing standard solutions been considered?
- Has linearity on the low- and high-concentration sides been checked?
- Do the points for improvement correspond to the causes of error?
Summary
The correlation coefficient is an indicator that shows the degree of linear relationship between two variables.
In calibration curves, the correlation coefficient r and coefficient of determination R2 are used to evaluate the linearity between the concentration of standard solutions and the measurement signal.
The closer R2 is to 1, the better the regression line is considered to fit the measurement points, but this alone does not indicate that the experimental values are accurate.
When evaluating a calibration curve, it is necessary to check not only R2 but also the slope, intercept, outliers, number of standard points, concentration range, and whether the unknown sample lies within the range.
Because outliers greatly affect the regression line and quantitative values, clear justification is required if they are excluded.
Outliers whose causes are unknown should not be readily deleted but should instead be discussed as uncertainty in the measured values.
In a report, rather than simply writing that “R2 was high” or “there was an outlier,” organize and discuss the meaning of the correlation coefficient, linearity of the calibration curve, causes of outliers, whether exclusion is appropriate, concentration range, errors in preparing standard solutions, variation in measurement signals, and points for improvement.
Discussion of correlation coefficients is important for determining whether a calibration curve can be trusted for quantifying unknown samples.
