While running some LSTM experiments in Keras the code ran into memory leak, which turned out to be an issue of Theano and BLAS hook up. Here is the solution I found.

In case you don’t want the default BLAS with Theano (due to various reasons such as memory leak or speed), you can install OpenBLAS as an alternative.

Below is a summary from here for Windows:

• Download mingw64_dll.zip.

• Download OpenBLAS-v0.2.14-Win64-int32.zip.

• Create a directory to put OpenBLAS, C:\openblas for instance.

• Copy the DLLs from mingw64_dll and OpenBlas/bin into C:\openblas.

• Add to following lines to your .theanorc file:

    [blas]
    ldflags = -LC:\openblas -lopenblas
  

• Add C:\OpenBlas to the PATH environment variable.

• Run check_blas.py for verification:

    import os, theano
  
    total_path = os.path.join(os.path.dirname(theano.__file__), "misc", "check_blas.py")
  
    os.system("python " + total_path)
  

If there is a more principled way to do this (e.g., via conda or pip install), please let me know.