Reporting per-class accuracy with MatCaffe

With Caffe, the per-class accuracy output as specified in prototxt files seems to be buggy. Namely, if a particular class has no sample in a particular batch, the accuracy for that class will be set to zero for that particular batch. This is not a problem itself, but it seems these zero values will be counted towards calculating the average per-class accuracy, which is very misleading. See this PR for more information.

Use the following MATLAB code (you must compile MatCaffe first) for computing per-class accuracy on a validation or test set, instead.

% global parameters
DATA_ROOT = '/home/twang/data/flower/flower_531_crop_256/';
TEST_LIST_FILE = '/home/twang/data/flower/flower_531_meta/val.txt';
NUM_CLASSES = 531;

% caffe initialization
gpu_id = 0;
model = '/home/twang/caffe/models/resnet_flower531/deploy.prototxt';
weights = '/home/twang/caffe/models/resnet_flower531/resnet_flower531_iter_120000.caffemodel';

caffe.set_mode_gpu();
caffe.set_device(gpu_id);

net = caffe.Net(model, weights, 'test');

mean_data = caffe.io.read_mean('/home/twang/caffe/models/resnet_flower531/flower_mean.binaryproto');

% read files

[files,labels] = textread(TEST_LIST_FILE, '%s %d\n');
accuracy = zeros(2, NUM_CLASSES);

for ii = 0 : NUM_CLASSES-1
    class_files = files(labels == ii); % all files for current class
    accuracy(1, ii+1) = length(class_files);
    for jj = 1 : accuracy(1, ii+1)
        im_data = caffe.io.load_image([DATA_ROOT class_files{jj}]);
        input_data = {imresize(im_data - mean_data, [224 224])};
        scores = net.forward(input_data);
        [~, predict] = max(scores{1});
        if (predict == ii+1)
            accuracy(2, ii+1) = accuracy(2, ii+1) + 1;
        end
    end
    fprintf('Class #%03d accuracy = %.2f.\n', ii+1, accuracy(2, ii+1) / accuracy(1, ii+1));
end

caffe.reset_all();

Infinite error loop when using MATLAB with nohup

If you run MATLAB scripts remotely with ssh and nohup (to keep it running in the background), you may find MATLAB entering an infinite loop of the following errors when your script or function finishes:

Warning: Error reading character from command line

and

Bad file descriptor

This has nothing to do with your code, just that you will have to add "</dev/null" before the "&" symbol which is causing the error. For example, you can run a function like this:

nohup matlab -nodisplay -r "myfunc_in_current_dir('params')" > ./mylog.txt </dev/null &

Also, people suggested using the "-nojvm" option. If you are plotting in your script then this option should be avoided.

MATLAB error message during startup on Linux

If you install your MATLAB at /usr/local/, you may not be able to start MATLAB (getting a long error message saying that MATLAB is exiting with a fatal error and no GUI shows up).

Alternatively MATLAB may start with the error message "The desktop configuration was not saved successfully".

In these cases you could try to change the owner of your ~/.matlab folder (change twang to your Linux username):

sudo chown -R twang ~/.matlab

This should fix the problem.

Probing system memory info in MATLAB with Linux

I came across a MATLAB library lately which checks for available memory before proceeding, so that the algorithm knows how much memory it should use (e.g., use up to 10% of available memory).

On Windows the function memory() comes in handy for this need, while in Linux we will have to write a few lines of code.

function [ totalmem, freemem ] = linuxMemory()
% Usage: No input arguments needed.
% Returns total and free memory in Kilobytes.

    % probe system memory information
    [~,meminfo] = system('cat /proc/meminfo');

    % get total memory
    tokens = regexpi(meminfo,'^MemTotal:\s*(\d+)\s', 'tokens');
    totalmem = str2double(tokens{1}{1});

    % get available memory
    tokens = regexpi(meminfo,'^*MemFree:\s*(\d+)\s','tokens');
    freemem = str2double(tokens{1}{1});

end