How to solve "Blas GEMM launch failed" in TensorFlow

Sometimes you see Blas GEMM launch failed error in TensorFlow.
It is a little difficult to understand.
So this introduce how to solve "Blas GEMM launch failed" in TensorFlow.

What is TensorFlow ?

TensorFlow is very famous machine learning library.
In machine learning field, everyone know it.
It is useful for neural network processing.
In TensorFlow you has detailed settings like hidden layer or activation function.

TensorFlow is a free and open-source software library for dataflow and differentiable programming across a range of tasks. It is a symbolic math library, and is also used for machine learning applications such as neural networks.
TensorFlow - Wikipedia

Cause of "Blas GEMM launch failed"

When you execute TensorFlow process in Python, you may see this error.

InternalError: Blas GEMM launch failed

A reason of this error is memory overflow.
If you try to use too much memory, the error happens.

In order to check memory, it is easy to check GPU memory usage.
In case of Windows10, you can run task manager with using taskmgr command.
The you can see GPU memory usage of GPU0 or GPU1 .
If GPU memory usage goes to 100% when you run TensorFlow prrogram, it means memory overflow.

Check GPU memory usage. In Windows10 you can check it on task manager.
[Tensorflow]InternalError: Blas SGEMM launch failed | CodeLab

Is shared GPU memory available ?

On task manager, you can see private GPU memory and shared GPU memory.
So you may estimate that shared GPU memory is available.
But it is not possible to use shared GPU memory for TensorFlow.

Shared GPU memory is not on GPU. It is PC memory that is shared for GPU.
CUDA can use only GPU memory. So it can't use shared GPU memory.

Shared memory is reserved memory of main RAM.
It is not on your NVIDIA GPU. CUDA can't use it.
Is shared GPU memory available in TensorFlow ?

How to avoid "Blas GEMM launch failed"

There are some solutions.

• Replace GPU with bigger memory
• Modify your program to use less memory
• Use GPU1 instead of GPU0

I introduce how to use another GPU (GPU1).

How to use another GPU

If you have 2 GPUs, you can use seccond GPU.
In order to choose GPU on Python, you can use config before run TensorFlow process.

import keras
import tensorflow as tf
tf.Session(config=tf.ConfigProto(device_count = {'GPU': 1}))

Reference：
Select GPU on Jupyter - Qiita

Finally

• Cause of Blas GEMM launch failed is GPU memory overflow
• Shared GPU memory is not available because it is not on GPU
• One of the solution is choosing another GPU with config.