Tensorflow single output node

Issue

The problem I have is the same as Neural network with a single out with tensorflow

I have a problem classifying between two classes. My data is labeled by 0s and 1s. I want to use Tensorflow neural network with one node at the output, so the result will be the probability between 0 and 1 in the example from the ones. Here is my code attempt:

example_size = 100
X = tf.random.normal((example_size,2))
y = tf.constant([[int(x)] for x in (X[:,0] > X[:,1])])
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Flatten(input_shape = (2,)))
model.add(tf.keras.layers.Dense(units = 1 , activation = "sigmoid")) # output layer
loss = tf.keras.losses.SparseCategoricalCrossentropy()
optim = tf.keras.optimizers.Adam(learning_rate = 0.01)
metrics = ["accuracy"]
model.compile(loss = loss, optimizer = optim, metrics = metrics)
model.fit(X, y, batch_size = example_size, epochs = 100, shuffle = True, verbose =1)

The error code:

---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
Input In [38], in <cell line: 11>()
      9 metrics = ["accuracy"]
     10 model.compile(loss = loss, optimizer = optim, metrics = metrics)
---> 11 model.fit(X, y, batch_size = example_size, epochs = 100, shuffle = True, verbose =1)

File c:\python\python39\lib\site-packages\keras\utils\traceback_utils.py:67, in filter_traceback.<locals>.error_handler(*args, **kwargs)
     65 except Exception as e:  # pylint: disable=broad-except
     66   filtered_tb = _process_traceback_frames(e.__traceback__)
---> 67   raise e.with_traceback(filtered_tb) from None
     68 finally:
     69   del filtered_tb

File c:\python\python39\lib\site-packages\tensorflow\python\eager\execute.py:54, in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     52 try:
     53   ctx.ensure_initialized()
---> 54   tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
     55                                       inputs, attrs, num_outputs)
     56 except core._NotOkStatusException as e:
     57   if name is None:

InvalidArgumentError: Graph execution error:

Detected at node 'sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits' defined at (most recent call last):
    File "C:\Users\Lior\anaconda3\lib\runpy.py", line 197, in _run_module_as_main
      return _run_code(code, main_globals, None,
    File "C:\Users\Lior\anaconda3\lib\runpy.py", line 87, in _run_code
      exec(code, run_globals)
    File "C:\Users\Lior\anaconda3\lib\site-packages\ipykernel_launcher.py", line 16, in <module>
      app.launch_new_instance()
    File "C:\Users\Lior\anaconda3\lib\site-packages\traitlets\config\application.py", line 846, in launch_instance
      app.start()
    File "C:\Users\Lior\anaconda3\lib\site-packages\ipykernel\kernelapp.py", line 677, in start
      self.io_loop.start()
    File "C:\Users\Lior\anaconda3\lib\site-packages\tornado\platform\asyncio.py", line 199, in start
      self.asyncio_loop.run_forever()
    File "C:\Users\Lior\anaconda3\lib\asyncio\base_events.py", line 601, in run_forever
      self._run_once()
    File "C:\Users\Lior\anaconda3\lib\asyncio\base_events.py", line 1905, in _run_once
      handle._run()
    File "C:\Users\Lior\anaconda3\lib\asyncio\events.py", line 80, in _run
      self._context.run(self._callback, *self._args)
    File "C:\Users\Lior\anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 471, in dispatch_queue
      await self.process_one()
    File "C:\Users\Lior\anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 460, in process_one
      await dispatch(*args)
    File "C:\Users\Lior\anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 367, in dispatch_shell
      await result
    File "C:\Users\Lior\anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 662, in execute_request
      reply_content = await reply_content
    File "C:\Users\Lior\anaconda3\lib\site-packages\ipykernel\ipkernel.py", line 360, in do_execute
      res = shell.run_cell(code, store_history=store_history, silent=silent)
    File "C:\Users\Lior\anaconda3\lib\site-packages\ipykernel\zmqshell.py", line 532, in run_cell
      return super().run_cell(*args, **kwargs)
    File "C:\Users\Lior\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2863, in run_cell
      result = self._run_cell(
    File "C:\Users\Lior\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2909, in _run_cell
      return runner(coro)
    File "C:\Users\Lior\anaconda3\lib\site-packages\IPython\core\async_helpers.py", line 129, in _pseudo_sync_runner
      coro.send(None)
    File "C:\Users\Lior\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3106, in run_cell_async
      has_raised = await self.run_ast_nodes(code_ast.body, cell_name,
    File "C:\Users\Lior\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3309, in run_ast_nodes
      if await self.run_code(code, result, async_=asy):
    File "C:\Users\Lior\anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3369, in run_code
      exec(code_obj, self.user_global_ns, self.user_ns)
    File "C:\Users\Lior\AppData\Local\Temp\ipykernel_27496\3786262486.py", line 11, in <cell line: 11>
      model.fit(X, y, batch_size = example_size, epochs = 100, shuffle = True, verbose =1)
    File "c:\python\python39\lib\site-packages\keras\utils\traceback_utils.py", line 64, in error_handler
      return fn(*args, **kwargs)
    File "c:\python\python39\lib\site-packages\keras\engine\training.py", line 1409, in fit
      tmp_logs = self.train_function(iterator)
    File "c:\python\python39\lib\site-packages\keras\engine\training.py", line 1051, in train_function
      return step_function(self, iterator)
    File "c:\python\python39\lib\site-packages\keras\engine\training.py", line 1040, in step_function
      outputs = model.distribute_strategy.run(run_step, args=(data,))
    File "c:\python\python39\lib\site-packages\keras\engine\training.py", line 1030, in run_step
      outputs = model.train_step(data)
    File "c:\python\python39\lib\site-packages\keras\engine\training.py", line 890, in train_step
      loss = self.compute_loss(x, y, y_pred, sample_weight)
    File "c:\python\python39\lib\site-packages\keras\engine\training.py", line 948, in compute_loss
      return self.compiled_loss(
    File "c:\python\python39\lib\site-packages\keras\engine\compile_utils.py", line 201, in __call__
      loss_value = loss_obj(y_t, y_p, sample_weight=sw)
    File "c:\python\python39\lib\site-packages\keras\losses.py", line 139, in __call__
      losses = call_fn(y_true, y_pred)
    File "c:\python\python39\lib\site-packages\keras\losses.py", line 243, in call
      return ag_fn(y_true, y_pred, **self._fn_kwargs)
    File "c:\python\python39\lib\site-packages\keras\losses.py", line 1860, in sparse_categorical_crossentropy
      return backend.sparse_categorical_crossentropy(
    File "c:\python\python39\lib\site-packages\keras\backend.py", line 5238, in sparse_categorical_crossentropy
      res = tf.nn.sparse_softmax_cross_entropy_with_logits(
Node: 'sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits'
Received a label value of 1 which is outside the valid range of [0, 1).  Label values: 0 0 1 1 1 0 1 0 0 0 0 0 0 0 1 1 0 1 0 1 0 1 1 0 1 1 1 1 0 1 1 0 1 1 1 1 1 0 1 0 1 1 0 1 1 1 1 1 0 0 1 0 0 1 0 1 0 0 0 1 0 0 1 1 0 0 1 0 0 1 1 1 1 0 0 1 0 0 0 0 1 0 1 0 1 0 0 1 0 0 0 0 0 1 1 1 1 1 1 0
     [[{{node sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits}}]] [Op:__inference_train_function_15984]

How I can fix it?

I also read the comment to Neural network with a single out with tensorflow witch is led me to How to choose cross-entropy loss in TensorFlow? , but that question is not about my topic at all. And I already used the sigmoid cost that he recommended in the binary case.

Solution

Use the binary cross entropy loss function for binary classification tasks:

import tensorflow as tf

example_size = 100
X = tf.random.normal((example_size,2))
y = tf.constant([[int(x)] for x in (X[:,0] > X[:,1])])
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Flatten(input_shape = (2,)))
model.add(tf.keras.layers.Dense(units = 1 , activation = "sigmoid")) # output layer
loss = tf.keras.losses.BinaryCrossentropy()
optim = tf.keras.optimizers.Adam(learning_rate = 0.01)
metrics = ["accuracy"]
model.compile(loss = loss, optimizer = optim, metrics = metrics)
model.fit(X, y, batch_size = example_size, epochs = 100, shuffle = True, verbose =1)

If you really want to use tf.keras.losses.SparseCategoricalCrossentropy, change your output layer to:

model.add(tf.keras.layers.Dense(units = 2 , activation = "softmax")) # output layer

See this post for more information.

Answered By – AloneTogether

This Answer collected from stackoverflow, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0

Leave a Reply

(*) Required, Your email will not be published