Potential bug in gradient descent?

[michaelpollmann]

In the gradient descent code, if S0 is small (in absolute value) such that the coefficient is set to 0, why isn't s (= XX %*% beta) updated?

BDcocolasso/R/lasso_covariance.R

Lines 63 to 72 in e883bb3

	if (S0 > lambda){
	beta[j] <- (lambda - S0)/XX[j, j]
	s <- s + XX[,j]*( beta[j] - beta_old[j])
	}
	if (S0 < -1 * lambda){
	beta[j] <- (-1 * lambda - S0)/XX[j, j]
	s <- s + XX[,j]*( beta[j] - beta_old[j])
	}
	if (abs(S0) <= lambda)
	beta[j] <- 0

If I understand the code correctly, when beta differed from 0 in the previous iteration but is then reduced to 0 in this iteration, s will permanently be incorrect.
Hence, if beta and beta_old are not equal, one would want to update

s <- s + XX[,j]*( beta[j] - beta_old[j])

(if this is indeed a bug, one might hope that it doesn't affect results much if it is relatively rare for coefficients to go from non-zero to zero)

But it might be difficult to detect numerically if beta and beta_old are equal, such that maybe one always wants to update, e.g.

      if (S0 > lambda)
        beta[j] <- (lambda - S0)/XX[j, j]
      if (S0 < -1 * lambda)
        beta[j] <- (-1 * lambda - S0)/XX[j, j]
      if (abs(S0) <= lambda) 
        beta[j] <- 0
      s <- s + XX[,j]*( beta[j] - beta_old[j])

(and maybe it would be ever-so-slightly faster to use if ... else if ... else ..., with the first check being the case abs(S0) <= lambda)