Modified Linear Regression to work on OLS, fixes #8847 #11311
Conversation
for more information, see https://pre-commit.ci
smruthi-sumanth
changed the title
There was a problem hiding this comment.
Good stuff! Apart from the changes that I've requested in my other comments, could you also rewrite the explanation at the top of the file to reflect your new implementation? In particular, make sure your new explanation does the following:
- Briefly explain what linear regression is
- Explain the OLS regression formula,
$(\mathbf{X}^{\top} \mathbf{X})^{-1} \mathbf{X} \mathbf{y}$ - Note possible issues with your implementation (i.e. inefficiency, numerical instability, etc.)
- Cite a source that readers can refer to for further info (Wikipedia or similar is fine)
I wrote a similar explanation for weighted regression in machine_learning/local_weighted_learning/local_weighted_learning.py a while back. If you'd like, you're welcome to use that explanation as a reference (though your explanation doesn't need to be nearly as long).
| """Implement Linear regression over the dataset | ||
| :param data_x : contains our dataset | ||
| :param data_y : contains the output (result vector) | ||
| def run_linear_regression_ols(data_x, data_y): |
There was a problem hiding this comment.
| def run_linear_regression_ols(data_x, data_y): | |
| def ols_linear_regression(data_x: np.ndarray, data_y: np.ndarray) -> np.ndarray: |
- Shortened the function name a bit (make sure you change the name elsewhere as well)
- Added type hints
| :param data_x : contains our dataset | ||
| :param data_y : contains the output (result vector) | ||
| def run_linear_regression_ols(data_x, data_y): | ||
| """Implement Linear regression using OLS over the dataset |
There was a problem hiding this comment.
| """Implement Linear regression using OLS over the dataset | |
| """Implement OLS linear regression over a given dataset |
Slight rewording for clarity
| error = sum_of_square_error(data_x, data_y, len_data, theta) | ||
| print(f"At Iteration {i + 1} - Error is {error:.5f}") | ||
| # Use NumPy's built-in function to solve the linear regression problem | ||
| theta = np.linalg.inv(data_x.T.dot(data_x)).dot(data_x.T).dot(data_y) |
There was a problem hiding this comment.
| theta = np.linalg.inv(data_x.T.dot(data_x)).dot(data_x.T).dot(data_y) | |
| theta = np.linalg.inv(data_x.T @ data_x) @ data_x.T @ data_y |
Instead of using .dot() for matrix multiplication, we can use numpy's @ operator, which does the same thing and is more readable
| :return : feature for line of best fit (Feature vector) | ||
| """ |
There was a problem hiding this comment.
The OLS regression function needs doctests—make sure you verify the outputs of your tests with a calculator that can do linear regression (e.g., Wolfram Alpha)
tianyizheng02
added
awaiting changes
Describe your change:
Checklist: