Find and Replace first NA in each column without for loops - performance

Question

Trying to do this without For Loop but can't figure it out.
I want to replace the first NA in a column with a default value of 0.0000001.
I am doing Last Observation Carried Forward (LOCF) imputation but want to give it a default value.
If I have the following data.frame:
> Col1 Col2 Col3 Col4
> 1 NA 10 99
> NA NA 11 99
> 1 NA 12 99
> 1 NA 13 NA
I want it to look like this:
> Col1 Col2 Col3 Col4
> 1 0.0000001 10 99
> 0.0000001 NA 11 99
> 1 NA 12 99
> 1 NA 13 0.0000001
This is the code I haev that works but is very slow...
#Temporary change for missing first observation
for (u in 1:ncol(data.frame))
{
for (v in 1:nrow(data.frame))
{
#Temporary change the first observations in a row to 0.0000001 until it encounters a value that isn't NA
if(is.na(temp_equity_df_merge2[v,u]))
{
temp_equity_df_merge2[v,u]=0.0000001
}
else break
}
I want to use apply or some variant that will be faster. I am looping over 20 columns and 1 million rows.
Thanks ahead of time for the help.

Answer 1

you can apply a function to each column:
myfun <- function(x) {
x[which(is.na(x))[1]] <- 0.1
return(x)
}
> data.frame(apply(dat, 2, myfun))
v1 v2 v3 v4
1 1.0 0.1 10 99.0
2 0.1 NA 11 99.0
3 1.0 NA 12 99.0
4 1.0 NA 13 0.1
>

Answer 2

Based on the comments, you can use apply to apply a function to each column. The function will replace the first NA with 0.0000001 and return a matrix. Then you can use na.locf to fill-in the remaining NAs. Finally, I wrapped it all in data.frame since you asked for a data.frame instead of a matrix
data.frame(na.locf(apply(dat, 2, function(x) {
firstNA <- head(which(is.na(x)), 1) #position of first NA
x[firstNA] <- 0.0000001
x
})))
Col1 Col2 Col3 Col4
1 1e+00 1e-07 10 9.9e+01
2 1e-07 1e-07 11 9.9e+01
3 1e+00 1e-07 12 9.9e+01
4 1e+00 1e-07 13 1.0e-07

Answer 3

Given you have such a large data set, I would use data.table and set to avoid copying the data. Both the apply solutions copy the data at least once.
The solution involves a for loop, but an efficient one ( doing length(valid_replace) things each of which are instantaneous)
library(data.table)
DT< -as.data.table(dat)
replacing <- lapply(DT, function(x)which(is.na(x))[1])
valid_replace <- Filter(Negate(is.na), replacing)
replace_with <- 0.0001
for(i in seq_along(valid_replace)){
set(DT, i = valid_replace[i], j = names(valid_replace)[i], value = replace_with)
}