FAQ: Troubleshooting
From Jacket Wiki
Back to FAQ
MATLAB® crashes
Usually this error causes MATLAB to exit with a stack trace that looks somewhat like the one below. This means that Jacket hit an exception it cannot handle. Possible exceptions include:
- Untested Operating Systems
- Systems configured incorrectly
- Missing, out-of-date, or incorrectly installed development packages
If you are encountering such an error, please take note of the header and the 'Stack Trace' from the message thrown by MATLAB just before it crashes. For instance, the following error message was thrown because Jacket was run on a system that is unable to perform floating point calculations using C code:
------------------------------------------------------------------------ Floating point exception detected at Thu Nov 19 07:17:19 2009 ------------------------------------------------------------------------ Stack Trace: [0] ld-linux-x86-64.so.2:0x00002aaaaaab368f(1, 0, 0x2aaadc84e2f0, 0x2aaad70ae170) [1] ld-linux-x86-64.so.2:0x00002aaaaaab5028(0, 0xffffffff80000002, 0x407f9200, 0x80000002) [2] ld-linux-x86-64.so.2:0x00002aaaaaabb2a5(0x407f9200, 0x2aaaaaabaed0, 0x407f925f, 0x407f9248)
If you are encountering this error, please contact Support with your complete system configuration, along with the contents of the error message.
How do I give you reproducible code?
If you encounter serious errors, such as GPU Failures, Segmentation Faults, out-of-memory errors or Blue-Screens-of-Death, we need to take a peek at your code and run it with customized builds of Jacket to see what's going on behind the scenes.
If you see such an error, make a note of all of the following:
- The OS, architecture and MATLAB version
- GPU
- Jacket code that was running when the error message occurred
- Driver version
- (For Windows) Whether you were running in Remote Desktop
To help us reproduce the error effectively, please cut your code down to the smallest chunk that gets the error, as follows:
- Keep your system state intact! This means you do not remove GPUs, install or update GPU drivers, update MATLAB or Jacket, or change your OS.
- Make a backup of your project.
- Restart MATLAB and run the code to verify that it fails at the same line. Make a note (with screenshots if needed) of the failure.
- Remove lines of code that you think are non-essential to the failure. Run the code after each modification and verify that it fails. You can try removing:
- comments, printout statements, save-to-disk statements
- Function definitions that were not called at the point of failure
- Variables that were not used at the point of failure
- Iterate the above line as much as possible to come up with a small code chunk. Depending on your project, this may be either 10 or a 100 lines long.
- Send that code to us, with all required inputs, so that we can run it with no modifications on our side.
Email Support with your code package and error screenshots.
Error 112: Out-of-memory errors
Over the last few versions of Jacket, the memory subsystem has improved greatly; however, there are still situations where you can run out of memory. In general you might see something like this:
tried to allocate 103mb (15mb free / 512mb total) Warning: device memory is low ??? Error using ==> times Unable to allocate memory (http://accelereyes.com/faq?q=112)
The most likely cause is that the data needed for your computation is beyond the capacity of your card, and Jacket was unable to move memory around to make room. In this situation, you could adjust your algorithm to work on a subset of the data at a time. You could also try using CLEAR to explicitly free unnecessary variables earlier in the function.
You might also encounter a situation where there seems to be enough memory, yet it still reports an error. The message might look like this:
tried to allocate 765mb (1353mb free / 4096mb total)
This is fragmentation: among all the free memory left, Jacket is unable to find a large enough contiguous space open. Same as above, try adjusting your algorithm to work in batches.
Another common cause is GFOR attempting to replicate out your data to many tiles. See GFOR Memory Considerations for more on this.
Invalid MEX File Errors
A typical Invalid MEX File error is of the form:
??? Invalid MEX-file 'C:\Program Files\AccelerEyesJacket\engine\gpu_entry.mexw32': [MESSAGE]
Most invalid-MEX-file errors occur with older versions of Jacket. Before reading ahead, please check if the following tips fix your issue:
- Upgrade your NVIDIA driver
- Update your NVIDIA driver
Invalid MEX file: Not a valid Win32 application
??? Invalid MEX-file 'C:\Program Files\AccelerEyesJacket\engine\gpu_entry.mexw32': C:\Program Files\AccelerEyesJacket\engine\gpu_entry.mexw32 Not a valid Win32 application.
This problem is most commonly observed on systems that run Jacket on a 32-bit MATLAB and a 64-bit OS.
Jacket recommends that your MATLAB bit-count match your OS bit count. When you attempt to run any command on a 64-bit OS, it looks for libraries specific to the architecture (64-bit in your case). Since you are running the command from a 32-bit MATLAB, there is a mismatch.
Typically, workarounds are possible by checking that all dependent libraries are present in your system. You can do this in Windows as follows:
- Download Dependency Walker as shown below:
- Check your MATLAB architecture
- Download the correct version of Dependency Walker for free
- Run it on gpu_entry.mexw32, present in the engine folder of Jacket
- Save the results using File > Save As , as a *.DWI file.
Workarounds are available for this situation, such as one listed here. If you require assistance with getting your system to run Jacket, [Support|contact us].
NOTE: Make sure you send us the DWI file from your Dependency Walker analysis, to enable us to help you faster.
Invalid MEX-file ... gpu_entry.mexa64' ..:undefined symbol: _ZSt16__ostream_ .... "
This issue is due to library incompatibilities in older versions of MATLAB. As a workaround, the following can be done:
- Download and untar the file for your OS: 32-bit or 64-bit.
- Place them in the directory <matlab_root>/sys/os/glnx*. Suitable permissions may be required, depending on the location of MATLAB.
- Make sure that the libstdc++.so.6 points to the 6.0.9 file (ls -l should confirm that).
- Restart Jacket
Invalid MEX file: The specified module could not be found.
Note: This problem typically does not occur in later versions of Jacket. Please download the latest version here.
If you get this problem with the latest Jacket version, please contact AccelerEyes Support.
Make sure you have installed the CUDA Driver and Toolkit (on pre-Jacket 1.3 versions) as administrator. If you haven't or are not sure, reinstalling the driver and toolkit as administrator should solve the problem.
The .NET framework is needed on some Windows systems, especially when using an older version of MATLAB or if your Windows Operating System doesn't have it pre-installed. This is especially applicable to people who are getting "Invalid MEX file" errors and who have already verified that they have the CUDA drivers and toolkit properly installed.
The freely downloadable program, Dependency Walker, is a great tool for narrowing down errors related to missing or incorrect shared libraries.
Invalid MEX file: The application configuration is incorrect.
If you get this problem with the latest Jacket version, please contact AccelerEyes Support.
"Invalid MEX file: The application has failed to start because the application configuration is incorrect. Reinstalling the application might fix the problem."
Jacket's MEX files are linked to a number of shared libraries. This error is usually encountered when Jacket cannot find a shared library file on your system, or if the shared library it finds belongs to a different architecture. MEX files for Jacket versions 1.3 and above are also linked with dynamic libraries from the Microsoft Visual C++ 2008 Redistributable package. Jacket 1.3 tries to search for the DLLs on your system, and throws this error if it fails to find them.
- For Jacket 1.3 or higher, one solution to this problem is to install the free Microsoft Visual C++ 2008 Redistributable package (64bit or 32bit). Note that MS Visual C++ is different from MS Visual C++ Runtime.
- Often, a shared library that belongs to a different architecture is also the cause (For instance, the file, <cuda_root>/lib64/cudart.dll built for 64-bit systems cannot be used with 32-bit systems.
The freely downloadable program, Dependency Walker, is a great tool for narrowing down errors related to missing or incorrect shared libraries.
Invalid MEX-file: libgomp.so: No such file or directory.
Note: This problem typically does not occur in later versions of Jacket. Please download the latest version here.
If you get this problem with the latest Jacket version, please contact AccelerEyes Support.
The shared library file, libgomp.so is shipped with GCC version 4.2 or later. Systems that have older versions of GCC will face this error. Jacket recommends that your system have GCC 4.3.2 or higher. This error should be resolved if a later version of GCC is installed.
Some sample systems on which this and similar errors were observed include Ubuntu 8.04 and RHEL 5.x.
To ensure that you do not face this or similar issues, please verify that your version of GCC (as indicated by gcc --version) is 4.3.2 or higher. Typically, later versions of Linux distributions - Ubuntu 9+, Fedora 10+ for instance - include an up-to-date version of GCC.
Detailed instructions for installing GCC are given at: http://gcc.gnu.org/install/
More discussion on the issue can be found on the following forum link: Crash in Gactivate
Invalid MEX-file: Library not loaded
Note: This problem typically does not occur in later versions of Jacket. Please download the latest version here.
If you get this problem with the latest Jacket version, please contact AccelerEyes Support.
The error might look like this:
>> grand(4) ??? Invalid MEX-file '/usr/local/jacket/engine/grand.mexmaci': dlopen(/usr/local/jacket/engine/grand.mexmaci, 1): Library not loaded: @rpath/libjacket.dylib Referenced from: /usr/local/jacket/engine/grand.mexmaci Reason: image not found.
Basic Errors
Error 131: Undefined behavior: CPU variables subscripted with GFOR iterators
See GFOR Usage: Subscripting into CPU variables
Error 132: Licensing does not allow Remote Desktop
Remote Desktop is not compatible with some older Jacket versions. With Jacket 1.7 and newer, you should not see this error. See Jacket Over Remote Connections.
Windows Remote Desktop Connection
See Jacket Over Remote Connections
Warning: Only storing result from first GFOR iteration into MATLAB variable
See GFOR Usage: Pulling back to MATLAB
Warning: Detected possible nested GFOR
Error: GFOR not supported with logical indexing
See GFOR Usage: No logical indexing
"FATAL: error finding class" followed by an invalid license error
If you use Jacket 1.2.2 or earlier, running GINFO and GACTIVATE on MATLAB 2007A or 2007B gives a license-related error, usually of the form:
Jacket:123(ready_preexisting_jvm) FATAL: error finding class java.lang.NoClassDefFoundError: licensing/ClassPathMod Jacket:331(mexFunction) FATAL: Could not attach to a preexisting JVM or create a new one to verify license. Please contact sales@accelereyes.com for assistance. Jacket internal error. Please visit the FAQ page (http://www.accelereyes.com/faq) for more information. If you cannot find a solution please post a message on the AccelerEyes Forums (http://www.accelereyes.com/forums) or send an email to support@accelereyes.com.
This is a known incompatibility between MATLAB and Jacket and most commonly manifests itself in MATLAB 2007A or 2007B. With Jacket 1.3 and higher versions, Jacket will move from a Java-based licensing system to a C++ licensing system, and therefore the easiest solution to this is to upgrade your version of Jacket to post-1.2.2. If you are unable to upgrade for some reason, you may use the following solutions, based on your version of MATLAB:
- For MATLAB 2007A:
- In MATLAB type
edit classpath.txt
- In the opened text file append the path to <jacket_root>/engine directory at the bottom.
- Restart Matlab. You may also need to add the following paths to classpath.txt if the above doesnt work. <jacket_root>/flexlm.jar
- <jacket_root>/EccpressoAll.jar
- For MATLAB 2007B:
- Please read this forum post. At the end of the forum post, there is an attachment called config_classpath.zip.
- Download the new P-file (config_classpath.p).
- Paste this file in <jacket_root>/engine after creating a back-up of your original P-file, just in case.
- Restart MATLAB.
On some systems, the above solutions may not work, or may manifest themselves with different error messages, on versions of MATLAB that are not 2009A or 2009B. In such cases, please download Sun Java 1.6.0 or higher from The Sun Java website. After installing Sun Java 1.6.0, edit the MATLAB_JAVA variable from MATLAB to point to the location of the 'jre' folder.
Error: Requires DLA license
Several linear algebra functions require a Jacket DLA license addon for all or part of their features. For the full list, see Jacket DLA.
Error: Requires SLA license
Several sparse linear algebra functions require a Jacket SLA license addon for all or part of their features. For the full list, see Jacket SLA.
Error: GFOR does not support COLON
See GFOR Usage: Iterator not allowed in colon expressions
Uninstallation of previous version failed. Installer Aborting.
This is a Windows installer error that results in failure to install Jacket properly. The typical situation where this is encountered is that the installer spots an old version of Jacket to delete, but fails to uninstall it. The most common reason for this is manual deletion of the "AccelerEyes\Jacket" directory without updating the Windows registry. To enable proper Jacket installation, the following steps are necessary:
- Run the Windows Registry editor program (To reach this, hit the Windows Key + "R" .. and type "regedit" in the box and hit enter)
- In the regedit window, navigate to:
** 64-bit: "HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\{B0CC6472-A084-4190-886E-976691F21543}"
** 32-bit: "HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{B0CC6472-A084-4190-886E-976691F21543}"
Note: The key listed here (BOCC*) corresponds to an installation of Jacket-1.8. Older keys may be found in the registry using a search for "Jacket".
If further assistance is required, please contact Support.
- Verify the details in the right pane that this key corresponds to AccelerEyes Jacket.
- Delete the key (Right click on the key in the left pane and select Delete)
Not Recognizable as p-file
This is an error specific to older versions of MATLAB and may appear as below when you try to execute .p files shipped with Jacket.
>> ginfo ??? Corrupt P-file "C:\Program Files\AccelerEyes\Jacket\engine\ginfo.p". >> gactivate ??? Corrupt P-file "C:\Program Files\AccelerEyes\Jacket\engine\gactivate.p".
The Jacket installer asks you to choose which version of MATLAB you are running, enabling it to install p-files that are compatible with your version of MATLAB. The error above likely indicates you made a mistake in selecting that option.
To fix, please uninstall Jacket and reinstall again to get the right p-file installation.
Alternatively, you can also manually copy files from the <jacket>\.Alternate directory into the root Jacket installation directory. On Windows, drag and drop the files and let Windows merge them and overwrite the existing improper files. On Linux and OSX, this command does the same:cp -fvr <jacket>/.Alternate/* <jacket>/
Error 110: Issues with License File
Error 110: license file not readable. Please see http://www.accelereyes.com/faq?q=110
Error 110 indicates a problem with licensing or GPU entry. This error can be replicated by trying to run the following Jacket command in MATLAB:
>> gpu_entry(8)
To fix this problem, verify all of the following steps:
- Did you copy the license file, jlicense.dat to the correct location? The correct location is <jacket_root>/engine for a license that will be used by a single user on a single machine. Note that the location may vary with the type of license purchased. Please refer to the instructions on the Jacket Installation page.
- Open jlicense.dat in a text editor. Do you see a version number that corresponds to your Jacket version? For example, if this is what you see:
FEATURE jacket jacket 1.30
Then it indicates that your license file is for Jacket 1.3. Verify that this is the Jacket version you installed. (Note that license files for version 1.X will work for minor versions 1.X.Y that follow 1.X)
- Verify that your machine has a MAC address, and that the MAC address of your machine exactly matches the field, "HOSTID" in your license file, jlicense.dat. (To get the MAC address, in Windows, run ipconfig /all and in Linux, ifconfig and observe the alphanumeric characters next to the "Phy Addr" or "HW Addr" field.)
- Did you try to make changes to the contents of the jlicense.dat file? For security reasons, editing certain fields of the license file is not allowed by the licensing mechanism.
- Are you trying to run Jacket over a remote connection? Why is this important?
- Do you have the right CUDA driver version installed, and from the correct source? See Jacket Installation instructions.
If you are still not able to solve the issue, please contact Support with all of the above questions answered. We'd be happy to help.
Error 111: System command "nvcc --version" could not be executed.
This error message is observed on Jacket versions pre-1.3:
The system command "nvcc --version" could not be executed. Either the CUDA toolkit is not installed, not properly configured, or faulty.
This error is caused when Jacket tries to execute the system() command from MATLAB.
- To verify this, try to run the following command and make a note of the output variables:
[status, result] = system( 'nvcc --version' );
A non-zero value for the status variable indicates that executing the command failed. Try re-installing the CUDA driver and toolkit and trying again.
- On Linux, if you are using MATLAB on a screen manager such as "screen", try running MATLAB from the terminal without screen.
- On Linux, try starting MATLAB using the command matlab -nojvm.
If you are still not able to solve the issue, please contact Support with all of the above questions answered. We'd be happy to help.
Error: "Undefined function or variable gpu_entry"
This error is most likely because Jacket is not in your MATLAB path. Please follow the instructions on the Jacket Installation page to add Jacket to your path.
Error: "Unexpected unknown exception from MEX file."
Note: This problem typically does not occur in later versions of Jacket. Please download the latest version here.
If you get this problem with the latest Jacket version, please contact AccelerEyes Support.
Typically this error message is encountered upon running any operation that requires compilation; such as an element-wise multiplication of matrices, for example.
One known cause of this problem is an incomplete or missing install of a development package. Please make sure you have all development packages necessary for successful compilation and execution of simple C code. For example, on Ubuntu Linux, the command to do this is:
sudo apt-get install build-essential
This may also happen if you encountered errors in your installation of CUDA.
Error 124: Variable too large
The computation produced a variable that's too large to store, either on the GPU or even back on the CPU.
>> gones(1024,1024,4096); ??? Error using ==> gones Variable too large. Please see http://accelereyes.com/faq?q=a124
It also shows up sometimes in GFOR loops, where the tiling may expand computations out to produce large intermediate variables. See GFOR Usage for more details and possible workarounds.
Error 125: Unsupported subscript expression
With Jacket v1.5 we upgraded GFOR to allow expressions in subscripts that involve some basic arithmetic around the iterator (See GFOR and GFOR Usage). In the first version, we only support "affine expressions" because we can easily pre-calculate their effect on the iterator (shifting, scaling) back on the CPU before executing. For example, assume i is the iterator, we support expressions of the form a*i+b where a and b are regular MATLAB scalars. Here's an example of how this might look:
A = grand(n,n,30); gfor i = 1:30 a = A(:,:,floor(i + 0.2)); ??? Error using ==> subsref Unsupported subscript expression. Please let us know about this: http://accelereyes.com/faq?q=a125 a = A(:,:,floor(i + 0.2));
If you run into error messages like this, please let us know by contacting Support and we'll aim to get fixes out to you in the minor releases that follow.
Error 126: GPU Entry not found
This error is thrown by gmex.m, which is part of the Jacket SDK.
gpu_entry not found on the path. Did you add the <JACKET_PATH>/ engine to the path? For more information on this error, read the FAQ entry for Jacket Error A126: http://accelereyes.com/faq?q=a126
It appears that an installation of Jacket could not be found. Ensure that Jacket is installed properly and present in the MATLAB Path, according to the instructions present in this link.
Driver and Runtime Errors
Error 3: Unspecified Driver Error
Typically this error message is observed on running the first Jacket command in a new MATLAB session.
Jacket does a verification of the system environment (CUDA driver and Runtime versions) before it is initialized. Part of this involves getting the CUDA driver version. Sometimes, an incorrect environment manifests itself in this verification failing.
The error message looks like this:
??? Error using ==> gpu_entry An error occurred while verifying the driver version to run Jacket: Jacket Error A3: Unspecified Driver Error (ERROR STRING) For more information on this error, visit http://www.accelereyes.com/faq?q=a3
The string "ERROR STRING" in the message above represents the error returned by the driver.
This message is usually caused by problems with the driver installation or issues encountered during driver initialization.
Solution:
Note that some or all of the steps below require you to have root access. If you are unsure, contact a System Administrator or a person with root access who is experienced with Linux usage. Also note that the solution may vary according to your Linux distribution. These steps will definitely work for Fedora.
Linux Only
- Verify that the Linux NVIDIA drivers were installed only from sources listed in the Jacket Installation page. Other sources may work for every-day use, but are usually insufficient for CUDA-based Jacket programming.
- Verify that the drivers are the latest version as required by the Jacket version you are using. The driver and toolkit versions for each Jacket release are listed in the Release_Notes.
- Verify that you are running Jacket in a runlevel that loads the driver (such as 5). You can do this by running the command:runlevel
The output of this typically indicates the default run-level first, and the current run-level last.
- If you are not in the desired run-level, change your runlevel as:init 5
- Verify that changing the run-level worked by typing in a MATLAB prompt (with Jacket paths loaded):ginfo
Windows Only
- Verify that the Windows drivers were installed only from sources listed in the Jacket Installation page. Other sources may work for every-day use, but are usually insufficient for CUDA-based Jacket programming.
- Verify that the drivers are the latest version as required by the Jacket version you are using. The driver and toolkit versions for each Jacket release are listed in the Release_Notes.
Mac Only
- Verify that the Mac drivers were installed only from sources listed in the Jacket Installation page. Other sources may work for every-day use, but are usually insufficient for CUDA-based Jacket programming.
- Verify that the drivers are the latest version as required by the Jacket version you are using. The driver and toolkit versions for each Jacket release are listed in the Release_Notes.
If you are using the correct driver, it might be that the CUDA driver has locked itself into a weird state. We suggest re-installing the driver and rebooting the computer. If none of this works, please contact Support and provide system details such as driver version, operating system, 32- or 64-bit, and any (minimal) code that might help reproduce the problem.
Error 4: CUDA Runtime Error
??? Error using ==> gpu_entry CUDA runtime error: <Error String> For more information on this error, read the FAQ entry for Error 4: http://accelereyes.com/faq?q=4
The CUDA runtime that powers Jacket may fail during execution. At such times, it returns an error code. Jacket returns this to the user in the form seen above, where <error string> corresponds to the error returned by the CUDA Runtime.
A CUDA runtime error has many causes and there are numerous error codes that could be returned. A list of all possible CUDA runtime errors can be found here.
If you see errors that are not listed below, please contact us for assistance.
CUDA Runtime Error: Invalid Device Ordinal: This indicates that the "device ordinal" supplied by Jacket to the CUDA runtime does not correspond to a valid CUDA device.
One way this could happen is if you install a driver on a system with one GPU (say, an 8400), then put in another GPU later (say, a Tesla C1060). On our system, the output of cat /proc/driver/nvidia/cards/1 shows an invalid Video BIOS.
Model: Tesla C1060
IRQ: 16
Video BIOS: ??.??.??.??.??Typically, a driver reinstall (with both cards present) will fix this issue.
CUDA Runtime Error: no CUDA-capable device is detected: This indicates thats there are no CUDA-capable devices found in the system.
Windows Only
- Verify that the Windows drivers were installed only from sources listed in the Jacket Installation page. Other sources may work for every-day use, but are usually insufficient for CUDA-based Jacket programming.
- Verify that the drivers are the latest version as required by the Jacket version you are using. The driver and toolkit versions for each Jacket release are listed in the Release_Notes.
Typically, if the system has a CUDA-capable GPU and the latest drivers installed, Jacket should detect the GPU. If you are trying to access the Windows machine remotely, please visit Can I use Jacket over remote connections?.
Error 5: Incompatible CUDA runtime version detected
??? Error using ==> gpu_entry Incompatible CUDA runtime version detected: Found x.y, Expected a.b http://accelereyes.com/faq?q=5
Jacket All versions of Jacket post-1.3 package the CUDA toolkit, and the environment is set so that Jacket finds and uses this toolkit. You're likely seeing this error because Jacket is picking up an older toolkit instead of the one we package. Double check that you haven't set LD_LIBRARY_PATH to an old version, or deleted the libraries we package.
ArrayFire At run-time, ArrayFire is picking up an older version of the CUDA Toolkit than we built against. Double check that your LD_LIBRARY_PATH doesn't point to the old one, and that you've installed the expected CUDA Toolkit. On Mac OSX, you may need to also check your $HOME/.MacOSX/environment.plist settings.
If you would like assistance with this, please contact us.
Errors with the word "failure"
If you are getting any of the following errors:
CUDPP failure CUBLAS failure CUFFT failure CULA failure GPU failure OpenGL failure JIT failure CUSPARSE
This set of error messages point to system failures that prevent Jacket from performing GPU operations without compromising the stability of the system. The underlying cause may be one of many, including an incorrect CUDA installation or corrupted state on the GPU from previous GPU failures.
To eliminate an incorrect CUDA driver installation as the cause, please verify the requirements in the Jacket Installation page and follow the instructions given in the Jacket Installation page. If you are still encountering this error, please try to document the exact set of operations you ran to produce the error, including:
- M-code that you ran
- The way you started MATLAB (matlab -nojvm, matlab -nodesktop etc)
- Whether you were sitting at the machine or remotely logged in
Next, try to condense the M-code down to a small piece of code that has all variables declared. Verify that it reproduces the failure message reliably. Send this information to AccelerEyes Support. We'd be glad to help you fix this error.
Error 115: Internal CUFFT Error
You have encountered an internal error with the CUFFT library that Jacket's FFT functions are based on. This error may be encountered while running Jacket code using any of the functions FFT, FFT2, IFFT, IFFT2, FFTN, IFFTN. As of Jacket 1.4.1, we have found workarounds in partnership with NVIDIA for most of the use-cases in which this error happens. However, if you are still encountering this error, please send a small piece of code (with all variables declared, and inputs replaced with rands if necessary) to Support. We would be glad to help you.
Error 121: CUDA driver version is insufficient for CUDA runtime version
When the driver is correctly installed and the driver module has been loaded, but there is a version mismatch between the driver and the toolkit, the following error is observed:
??? Error using ==> gpu_entry Jacket Error A121: CUDA driver version is insufficient for CUDA runtime version. The CUDA driver you have installed (v3.0) is not compatible with this version of Jacket (v3.1). Please update your driver to the latest CUDA-compatible version.
Solution:
This solution applies to Jacket 1.3 or higher only. Ensure that you have Jacket 1.3 or higher before following the steps below.
- Verify that your NVIDIA driver is compatible with the CUDA toolkit shipped along with the Jacket installation as follows:
- Check the version of the CUDA toolkit shipped with the Jacket installation by viewing the Release Notes.
- Check the NVIDIA driver version on your system as shown here.
- Download the latest driver from NVIDIA.
Error 122: NVIDIA driver not installed or not initialized.
Jacket has detected that there is no CUDA driver installed on the machine. Please make sure that an NVIDIA driver is installed from links provided in the Jacket Installation page. If you do have a driver installed already, please contact Support for more assistance with this problem. As always, we would be glad to help.
CUDA failed to initialize
The error messages below are returned when the GPU driver cannot initialize properly.
The most common cause of this is a missing or incorrect CUDA driver. To test this assumption, please install the latest CUDA drivers from http://www.nvidia.com/drivers.
This problem is also caused by attempting to run Jacket over Windows Remote Desktop. To run Jacket over remotely in Windows, please use a non-RDP solution such as TightVNC.
Please read the FAQ entries below for solutions specific to your issue.
Error 119: CUDA failed to initialize (no device)
Jacket Error A119: CUDA failed to initialize (no device). For more information, see Jacket FAQ entry: http://www.accelereyes.com/faq?q=a119 (gpgpu.cu:285)
This error occurs when your machine enters a state that prevents the drivers from accessing a CUDA-capable GPU.
Specifically, on Linux, under certain conditions, the device nodes corresponding to your CUDA-capable GPU can become inaccessible or absent. This will cause the driver to return an error mentioning that no device was found.
Solutions:
Note that some or all of the steps below require you to have root access. If you are unsure, contact a System Administrator or a person with root access who is experienced with Linux usage. Also note that the solution may vary according to your Linux distribution. These steps will definitely work for Fedora.
- Common to all OSes:
- Ensure that you have CUDA-capable NVIDIA GPUs on your machine.
- Ensure that you have the latest NVIDIA driver installed from the NVIDIA drivers page.
- Windows Only:
- If you are running Jacket over Windows Remote Desktop or similar software, be aware that these might interfere with Jacket's attempts to access the GPU. Read this article for more information.
- Linux Only:
runlevel
The output of this typically indicates the default runlevel first, and the current runlevel last.
2. If you are not in the desired run-level, change your runlevel as:init 5ginfo
4. If changing the runlevel did not work, verify that you have device nodes present, using the command ls /dev/nvidia*. Verify that these nodes have read and write permissions for the user you are logged in as.
Error 120: CUDA failed to initialize (invalid device)
Jacket Error A120: CUDA failed to initialize (invalid device). For more information, see Jacket FAQ entry: http://www.accelereyes.com/faq?q=a120 (gpgpu.cu:285)
This error occurs when your machine enters a state that prevents the drivers from accessing a CUDA-capable GPU.
Specifically, on Linux, under certain conditions, the device nodes corresponding to your CUDA-capable GPU can be incorrectly created.
The following error message may mean that device nodes were present, but could not be accessed.
NVIDIA: could not open device /dev/nvidia0 (permission denied)
The following message may indicate that the nodes were incorrectly created:
NVIDIA: could not open device /dev/nvidia0 (no such file/directory)
The following message may that some or all device control files are absent.
NVIDIA: could not open device /dev/nvidia0 (no such device)
Solutions:
- Common to all OSes:
- Ensure that you have CUDA-capable NVIDIA GPUs on your machine.
- Ensure that you have the latest NVIDIA driver installed from the NVIDIA drivers page.
- Windows Only:
- If you are running Jacket over Windows Remote Desktop or similar software, be aware that these might interfere with Jacket's attempts to access the GPU. Read this article for more information.
- Linux Only:
Note that some or all of the steps below require you to have root access. If you are unsure, contact a System Administrator or a person with root access who is experienced with Linux usage. Also note that the solution may vary according to your Linux distribution. These steps will definitely work for Fedora.
- Verify that you are running Jacket in a runlevel that loads the driver (such as 5 on Fedora systems). You can do this as:
runlevel
The output of this typically indicates the default runlevel first, and the current runlevel last.
- If you are not in the desired run-level, change your runlevel as:
init 5- Verify that changing the runlevel worked by typing in a MATLAB prompt (with Jacket paths loaded):
ginfo
- If changing the runlevel did not work, verify that you have device nodes present, using the command ls /dev/nvidia*. Verify that these have execute permissions.
Error 123: CUDA failed to initialize (other messages)
If you encounter an error message as shown:
Jacket Error A123: CUDA failed to initialize (ERROR STRING). For more information, see Jacket FAQ entry: http://www.accelereyes.com/faq?q=a123 (gpgpu.cu:285)
If ERROR STRING is not NO DEVICE or INVALID DEVICE, please contact Support. We will help you track down the problem.
Error 127: CUDA_LIB_PATH or CUDA_INC_PATH not found
This error is thrown by gmex.m, which is part of the Jacket SDK.
CUDA_LIB_PATH environment variable is not defined. Is the CUDA toolkit properly installed? For more information on this error, read the FAQ entry for Jacket Error A127: http://accelereyes.com/faq?q=a127
This error indicates that the environment variables CUDA_LIB_PATH or CUDA_INC_PATH were not defined.
Please ensure that the CUDA_LIB_PATH environment variable points to the lib directory of your CUDA installation, and CUDA_INC_PATH points to the include directory.
Error 128: Cannot locate CUDA <version> at <location>
Note: This error does not occur with Jacket 1.7 and later. If you see this error and are using an older Jacket version, please download the latest Jacket version today to take advantage of feature upgrades and bugfixes.
If you wish to use the older version for any reason, we can help you with that. Please contact Support.
Error 143: An existing CUDA context was found. Aborting initialization
This means that the current MATLAB session already has the GPU initialized by a program other than Jacket. This will result in one or both of Jacket and the external program to work incorrectly in an undefined manner. For example, with PCT MATLAB, something like the following will fail with the mentioned error:
>> a = gpuArray(10); % PCT code >> ginfo % Jacket code
The solution is to use Jacket in a MATLAB session on its own.
Warning 144: CUDA GL context creation failed
Note: This error does not occur with Jacket 1.8.1 and later. If you see this error and are using an older Jacket version, please download the latest Jacket version today to take advantage of feature upgrades and bugfixes.
This error occurs when CUDA is unable to interoperate properly with OpenGL. This is common to systems which use a non-CUDA GPU for display purposes.
Laptops using Optimus Technology are especially vulnerable to this error. To fix this problem on Optimus laptops, please follow the instructions posted here:
Error_145: Jacket For Windows Environment PATH Not Set
While installing Jacket for Windows, a problem with updating the Windows PATH environment variable was encountered. The PATH variable is required to reference the Jacket installed directory so that Jacket can find the necessary libraries it requires. You'll now manually update the PATH variable as follows:
- Right-click My Computer, and then click Properties.
- Click the Advanced (System Settings) tab.
- Click Environment variables.
- Find the PATH System variable and click Edit.
- Append the directory of where you installed Jacket to the start of the value:
- i.e. C:\Program Files\Accelereyes\Jacket\engine\bin; for 32-bit or C:\Program Files\Accelereyes\Jacket\engine\bin64; for 64-bit Windows.
- Make sure the semi-colon is included above and do NOT accidentally delete anything that was already set in the PATH.
- Make sure not to forget the engine\bin part.
- Click Ok and exit out of the Environment Variables settings.