Making CPU/GPU enabled functions

From Jacket Wiki

Code Structure

Most Jacket/MATLAB users rely on functions when some computations are to be done. The use of functions make it easier to have an overview of the code and ensure that it work as intended. It is of course possible to make each function in two versions: one for running MATLAB on the CPU, and one for running Jacket on the GPU. However, it is high risk having two versions as it is very easy to forget to update both functions when modifications is made to one of them. Therefore, it is usually better to only have one function, which can handle both types of input signal - one cast to the GPU by use of Jacket and the other by standard MATLAB. It can be illustrated as below for the function called myfunction:

function [ out1 ] = myfunction( in1, GPU )
% User defined function which can be used for both CPU and GPU
%
% Input;
%   in1:   Input array
%   GPU:   -1 to use CPU, positive to use GPU
%
% Output;
%   out1:  Output array
 
% Pre-allocate output variable
if GPU < 0,
    out1 = ones(size(in1));      % If the CPU is used ...
    tmp1 = rand(size(in1));
else
    out1 = gones(size(in1));     % Pre-allocate variables ...
    tmp1 = grand(size(in1));
end
 
% Perform computations
out1 = tmp1 .* in1;
 
end

One way to have one function, which can adapt to both Jacket and MATLAB is to pass a variable to the function indicating if it is for CPU or GPU computation. A negative variable could indicate CPU usage (standard MATLAB), and a zero or positive integer could be used to indicate GPU usage. Then an if-else-end statement is used to declare/pre-allocate the variables. Once that is done the computation part is exactly the same. The only disadvantage of this method is the slight overhead with the if-else-end statement. This seems like a small price to pay compared to the advantages. If possible the function should of course be made in such a way that the if-else-end statement is negligible compared to the computation part.

It is also possible to use the isa function as in the following:

function [ out ] = myfunction( in )
if strcmp(class(in),'gsingle'),
    fprintf('DO THIS IF in IS A GSINGLE');
elseif strcmp(class(in),'gdouble')
    fprintf('DO THIS IF in IS A GDOUBLE');
else
    fprintf('DO THIS IF in IS A DOUBLE');
end
 
end

In this case we don't need to pass any variable to the MATLAB/Jacket function, which in some cases is an advantage. More advanced possibilities should be possible in upcoming Jacket versions.

Speed Issues

The use of if/strcmp above is perfectly fine in many cases. But the function where it is used should of course be made in such a way that the if/strcmp only takes a fraction of the total time needed for a call to the function. I have done a large number of different trials on this and even a function as simple as:

function [ out ] = myfunction_check( in )
if strcmp(class(in),'gsingle'),
    tmp = gsingle(pi);
elseif strcmp(class(in),'gdouble')
    tmp = gdouble(pi);
else
    tmp = pi;
end
out = tmp .* in;
end

gives an added overhead which will not matter in in almost all cases. Should it happen that if/strcmp takes too much time, then rewrite the function such that there is one front-end function, which is only called once (or a few times) and has the input validation. This function should then also distribute to the compute part of the function where these functions (one for gsingle, one for gdouble, one for ...) do not have the if/strcmp included. Thereby any additional overhead is avoided. The structure is then something like:

function [ out ] = myfunction_frontend(in)
if strcmp(class(in),'gsingle'),
    % CALL GSINGLE OPTIMIZED FUNCTION WITHOUT INPUT DATA CHECKING
    out = myfunction_gsingle( in );
elseif strcmp(class(in),'gdouble')
    % CALL GDOUBLE OPTIMIZED FUNCTION WITHOUT INPUT DATA CHECKING
    out = myfunction_gdouble( in );
elseif strcmp(class(in),'gdouble')
    % CALL DOUBLE OPTIMIZED FUNCTION WITHOUT INPUT DATA CHECKING
    out = myfunction_double( in );
end
 
end

This is a generally applicable method to use. Have one function to do the input data validation and distribute computations to the optimized functions. But always try to avoid having input data validation in a function being called many times - this is also the way to handle MATLAB functions as well. This is not particularly aimed at Jacket.

Go Home: Torben's Corner