I have a GUI that allows the user to classify the images and train the CNN model or run predictions with the existing model. I don’t have utilized GPUs on the machine. So, I have set up a CPU multiprocessing module to accelerate the performance and parallel the work. I know that TensorFlow modules must be defined inside functions in order to use in multiple cores.
def create_model(list_of_params_list): import tensorflow as tf # Model is defined and starts training here. def multicore_controller(list_of_params_list): import multiprocessing with multiprocessing.Pool() as p: p.starmap_async(create_model, list_of_params_list) p.close() p.join()
create_function creates multiple CNN models with different parameters and hyperparameters (such as the size of kernels, loss function, etc…). However, on `import tensorflow as tf line, the script stops responding (throws no errors nor visible output) and stays at that line forever. What is the problem and how can I fix it? Thanks in advance.
Edit: I tried on tensorflow version 2.3.X, 2.6.X, 2.7.X, 2.8.X, and 2.9.1 but no changes.
I still couldn’t find the reason for the bug. However, you can way around the problem when you spawn your processes instead of forking:
import multiprocessing ctx = multiprocessing.get_context("spawn") with ctx.Pool() as p: p.starmap_async(create_model, list_of_params_list) p.close() p.join()
It is still very welcome to acquaint with the problem.
Answered By – justRandomLearner