Why do I get very big (or near) y_pred value for regression tasks?

[querbesd]

Hello,

First of all, congrats for making this project open-source and serving as food for thoughts for similar projects.

In this context, I tried to replicate the same architecture for a regression task at the beginning of the customer journey (predicting ltv after the first purchase).

Unfortunately, with my training set (own project) which consists of 20k rows, it seems my model's weights are too big as I get big y_pred values on my test set. I don't face this issue on the kaggle dataset.

My features are scaled and I also tried l1 l2 regul + dropout on my layers but it's not enough. My guess is my dataset is too small and the model is too sensitive to noise.

My question: is there something I can amend in the loss function to limit this issue?

PS: Editing the loss function using the mode instead of the mean of the log normal distribution function as described here helped me to get more consistant pred values (it removed inf). However, this is still far away from what I can get with a more traditional loss function (mae or mse).