Assignment 6

We aim to explore various surrogate models in this assignment. We will only consider a one dimensional example to keep things simple. Lets take the classic example of

\[y = (6x-2)^2 \sin{(12x-4)}, \ x \in [0,1]\]

The function looks like this

Lets try to construct various surrogate models for this function. We will consider

• Linear regression with various basis functions
• Non-linear regression
• Piecewise cubic spline interpolation

and compare their performance.

Sampling

Even though the functional form of the true function is known, we will assume that it is unknown from this point onwards. Therefore we need to sample the function.

Task 1

Implement following sampling techniques

1. Random sampling (uniform) with 5, 10 and 15 samples
2. Equi-spaced sampling with 5, 10 and 15 samples
3. Latin hypercube sampling with 5, 10 and 15 samples

Linear Regression

Task 2

Implement linear regression with following basis functions

1. \((1, x, x^2)\)
2. \((1, x, x^2, x^3)\)
3. \((1, x, x^2, x^3, x^4)\)

Task 3

Implement linear regression with Chebyshev basis functions

1. \((1, T_1(x), T_2(x))\)
2. \((1, T_1(x), T_2(x), T_3(x), T_4(x))\)
3. \((1, T_1(x), T_2(x), T_3(x), T_4(x), T_5(x))\)

Task 4

Implement linear regression with sine basis functions

1. \((1, \sin(\pi x), \sin(2\pi x))\)
2. \((1, \sin(\pi x), \sin(2\pi x), \sin(3\pi x))\)
3. \((1, \sin(\pi x), \sin(2\pi x), \sin(3\pi x), \sin(4\pi x))\)

Nonlinear Regression

Task 5

Implement nonlinear regression for

\(\hat{y} = \beta_1 + \beta_2 x^2*\sin(\beta_3 x + \beta_4)\)

Then try

\(\hat{y} = (\beta_1 x + \beta_2)^2 \sin(\beta_3 x + \beta_4)\)

You will notice that the second model gives better results. This is obvious. But remember that we need a deep intuition to pick the correct functional form in the general case. This is usually not possible. Also you will notice that finding \(\beta\)’s is much more difficult in the case of nonlinear regression that the earlier linear case.

Piecewise Cubic Spline Interpolation

Task 6

This task is a bit more involved. You will need to learn about the cubic spline interpolation. Then use a suitable library to get the interpolation. The number of spline intervals should be 5, 10 and 15.

Deliverables

You should deliver a pdf report and a zip file containing all the codes.

In each case, you should report the mean squared error of the model with respect to the true function. You should also plot the true function and the model for each case.

Your general conclusions about the advantages and disadvantages of the various sampling techniques and regression models should be included in the report.

Fit each model with all the sampling techniques and compare the results.

Submission

Submission deadline is 23:59 on 15th April, 2025. Email all the assignments to ramkumars.24@res.iist.ac.in. There will be extensions for whatsoever reason apart from medical emergencies.