mnist png recognition

[DanikKamilov]

Thank you for help with starting application. Now it works fine, but I have one more trouble.

I use this image(28-28) to recognize by LeNet

and pyCaffe works fine and give me answer = 3
output = {6.72078215e-08, 6.08612163e-06, 8.28973225e-06, 9.99530911e-01, 6.53917276e-09, 4.10966662e-04, 1.80607959e-10, 2.87060775e-05, 2.26623479e-06, 1.27552030e-05}

cnnDroid returns to me float[1][10] = {0,NaN,NaN,NaN,0,NaN,0,0,0,0}

what I did wrong? (and one more question: can I use AbsVal activation function , not ReLU?)

[latifisalar]

Oh, that's great.
If you have converted your Caffe model with the script available in the git and also have used the NetFile in the git, I can only think of the input format of the network as a reason to this problem.
Have you made your input values binary? And make sure that you're not applying any normalization to the input data.
About the activation function, currently we only have support for the ReLU.

[DanikKamilov]

Yes, with binary values it works fine. Thank you very much.

Will it be supporting AbsVal activation function in future?

[latifisalar]

That's perfect.
We'll certainly extend the layer support but can't assure the exact time of adding support for the AbsVal activation function.
Meanwhile, you can change the implementation of ReLU layer to act as a AbsVal layer, in order to do so:

1. Do not change the NetFile, and leave the ReLU in place, because you're going to only change the functionality of this layer to act as an AbsVal function.
2. In the layers package open Convolution.java and search for "nonlinear" text
3. Change the following code appearances of the search result in order to change the functionality of ReLU:

if (nonLinear) {
    if (outputBlob[n - 1][i][j][k] < 0)
        outputBlob[n - 1][i][j][k] = 0;
}

4. Follow step 2, 3 and 4 for the FullyConnected.java

Note that with this approach, all of the ReLU layers will be converted to AbsVal, so you won't have the option to have both ReLU and AbsVal in the same network.

[DanikKamilov]

I need this , because I have 99.54% recognition for MNIST in my specific structure of CNN, so I need AbsVal activation function for this. ok, I will try. Thank you

[DanikKamilov]

I have tried this one. I've changed all search results

if (nonLinear) {
    if (outputBlob[n - 1][i][j][k] < 0)
        outputBlob[n - 1][i][j][k] = 0;
}

and now all of them like

if (nonLinear) {
    if (outputBlob[n - 1][i][j][k] < 0)
        outputBlob[n - 1][i][j][k] = Math.abs(outputBlob[n - 1][i][j][k]);
}

is it right?

after changes 'output' array returns to me {0,0,NaN,0,0,0,0,0,0} again. But NAN at right index. So if I write 3 to CNN - it returns {0,0,NaN,0,0,0,0,0,0}, if I write 5 - returns {0,0,0,0,NaN,0,0,0,0,0}, so I can work with it, but I think something wrong))

[latifisalar]

Note that for Fully Connected layers the code you need to change is slightly different:

if (nonLinear) {
    switch (nonLinearType) {
        case RectifiedLinearUnit:
            if (outputBlob[n][c] < 0)
                outputBlob[n][c] = 0;
            break;
    }
}

Usually the NAN value problem comes up when there is a bug in the inputs of the network or the network parameters (such as weights) are not matched with the actual CNN weights that you've trained in Caffe.

[DanikKamilov]

Yes, I have changed Fully Connected in right way.

Now cnndroid works fine, but returns NaN (at right index). Thank you for your help.

I will try to find mistakes in InputArray

[latifisalar]

Do you have AbsVal activation function just after the FC layer?
If you also have it after Pooling layer, you will need to also make the similar changes to the nonlinear.java.

[DanikKamilov]

Oh. Yes I have it. Ok, will try

[DanikKamilov]

I have changed nonlinear.java , but cnn returns me the same array {0,0,NaN,0,0,0,0,0,0}.

[cmd10]

I have the same problem. Did you have resolved it?

[DanikKamilov]

I just use .isNan() to understand what answer is correct.