Hello, thanks for visiting. Recently, I wrote a small piece on neural net and maybe you can check the part where I wrote about how Relu can represent non-linearity. Maybe that will clear your doubt.

You need to reduce overfitting, always. You don’t want your model to perform well just on training data. That’s what overfitting means. So you can increase training data, reduce complexity in model or take some other steps to generalize your model.