Reporting per-class accuracy with MatCaffe

With Caffe, the per-class accuracy output as specified in prototxt files seems to be buggy. Namely, if a particular class has no sample in a particular batch, the accuracy for that class will be set to zero for that particular batch. This is not a problem itself, but it seems these zero values will be counted towards calculating the average per-class accuracy, which is very misleading. See this PR for more information.

Use the following MATLAB code (you must compile MatCaffe first) for computing per-class accuracy on a validation or test set, instead.

% global parameters
DATA_ROOT = '/home/twang/data/flower/flower_531_crop_256/';
TEST_LIST_FILE = '/home/twang/data/flower/flower_531_meta/val.txt';
NUM_CLASSES = 531;

% caffe initialization
gpu_id = 0;
model = '/home/twang/caffe/models/resnet_flower531/deploy.prototxt';
weights = '/home/twang/caffe/models/resnet_flower531/resnet_flower531_iter_120000.caffemodel';

caffe.set_mode_gpu();
caffe.set_device(gpu_id);

net = caffe.Net(model, weights, 'test');

mean_data = caffe.io.read_mean('/home/twang/caffe/models/resnet_flower531/flower_mean.binaryproto');

% read files

[files,labels] = textread(TEST_LIST_FILE, '%s %d\n');
accuracy = zeros(2, NUM_CLASSES);

for ii = 0 : NUM_CLASSES-1
    class_files = files(labels == ii); % all files for current class
    accuracy(1, ii+1) = length(class_files);
    for jj = 1 : accuracy(1, ii+1)
        im_data = caffe.io.load_image([DATA_ROOT class_files{jj}]);
        input_data = {imresize(im_data - mean_data, [224 224])};
        scores = net.forward(input_data);
        [~, predict] = max(scores{1});
        if (predict == ii+1)
            accuracy(2, ii+1) = accuracy(2, ii+1) + 1;
        end
    end
    fprintf('Class #%03d accuracy = %.2f.\n', ii+1, accuracy(2, ii+1) / accuracy(1, ii+1));
end

caffe.reset_all();