Getting same runtime for cpu and gpu on Google Colab

Issue

So I’ve been experiencing with Colab in order to conduct my deep learning project for my Bachelor’s. When I run the provided example on colab to test the comparison speed between cpu and gpu it works fine, however when I try with my own code, I get the same run time for both. The task that I was conducting was simply converting 1000 jpg images to RGB values using the PIL.Image package. Shouldn’t the runtime when using a gpu be much faster? Or is that only the case when running deep learning models? Please find the code I used below:

import pandas as pd
import tensorflow as tf
import timeit

device_name = tf.test.gpu_device_name()
if device_name != '/device:GPU:0':
  raise SystemError('GPU device not found')
print('Found GPU at: {}'.format(device_name))

def cpu():
  dataset = pd.read_excel("/content/drive/My Drive/BachelorProject/labels/user_01.xlsx")
  classes = pd.read_excel("/content/drive/My Drive/BachelorProject/labels/Classes.xlsx")
  classes = list(np.array(classes).flatten())
  labels = np.array(dataset["Class"])
  labels = [classes.index(x) for x in labels]

  '''
  All the photos filename in an array
  '''
  files = np.array(dataset["Photo"])
  imgDir = "/content/drive/My Drive/BachelorProject/images_routine_users/user_01/user1_01/"
  with tf.device('/cpu:0'):
    files = np.array([convertToRGB(imgDir+x) for x in files[0:100]])

  img_width, img_height = 32, 32
  input_shape = (img_width, img_height, 3)

def gpu():
  dataset = pd.read_excel("/content/drive/My Drive/BachelorProject/labels/user_01.xlsx")
  classes = pd.read_excel("/content/drive/My Drive/BachelorProject/labels/Classes.xlsx")
  classes = list(np.array(classes).flatten())
  labels = np.array(dataset["Class"])
  labels = [classes.index(x) for x in labels]

  '''
  All the photos filename in an array
  '''
  files = np.array(dataset["Photo"])
  imgDir = "/content/drive/My Drive/BachelorProject/images_routine_users/user_01/user1_01/"
  with tf.device('/device:GPU:0'):
    files = np.array([convertToRGB(imgDir+x) for x in files[0:100]])

  img_width, img_height = 32, 32
  input_shape = (img_width, img_height, 3)

cpu()
gpu()

print('CPU (s):')
cpu_time = timeit.timeit('cpu()', number=10, setup="from __main__ import cpu")
print(cpu_time)
print('GPU (s):')
gpu_time = timeit.timeit('gpu()', number=10, setup="from __main__ import gpu")
print(gpu_time)
print('GPU speedup over CPU: {}x'.format(int(cpu_time/gpu_time)))

The output I get is the following:

Found GPU at: /device:GPU:0
CPU (s):
167.21270494400005
GPU (s):
166.9953728999999
GPU speedup over CPU: 1x

Which is essentially saying that the runtime for cpu and gpu is the same. Hope to hear what you have to say about this. Thanks

Solution

Tensorflow device setting won’t affect non-Tensorflow operations.

That said, Tensorflow has now it’s own numpy API that can use the device that is set up for Tensorflow in e.g. with tf.device('/device:GPU:0').

The new api can be used similarly as numpy with

import tensorflow.experimental.numpy as tnp

Here is also a nice blog post about the new api with performance comparison to normal (cpu) numpy.

And here is a related issue about using gpu on colab with plain python.

Answered By – hilipati

This Answer collected from stackoverflow, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0