# What are "Innermost" and "Outermost" indices in a dataset?

## Issue

I’m trying to understand how to constuct datasets for tensorflow, but I’m confused by this tutorial’s description:

``````All shapes are: (batch, time, features)
Window shape: (3, 7, 19)
Inputs shape: (3, 6, 19)
Labels shape: (3, 1, 1)
``````

Typically, data in TensorFlow is packed into arrays where the
outermost index is across examples (the "batch" dimension). The middle
indices are the "time" or "space" (width, height) dimension(s). The
innermost indices are the features.

The code above took a batch of three 7-time step windows with 19
features at each time step. It splits them into a batch of 6-time step
19-feature inputs, and a 1-time step 1-feature label.

Source: Time series forecasting

I am confused by many of the terms here, which is making it difficult to understand what is being said:

• What is "innermost" and "outermost"? My intuition tells me that index 0 is the innermost, but that contradicts them describing the batch index as "outermost". What is "outer" about index 0?
• What are the "middle indices"? Since the phrase is a plural, I would assume there are 2 middle indices, implying 4 dimensions in total, but there are only 3.
• The middle indices are described as representing "time or space". What is meant by that in this context?

## Solution

I’ve been working with these datasets for a while now and I think I can have a go at answering this myself:

Innermost vs outermost:

I think the intuition behind innermost and outermost lies in where you would put the indexing brackets. Take this 2D array:

``````x = [[1, 2], [3, 4], [5, 6]]
``````

This array has 3 items in the outermost dimension because you access those with the first bracket/parenthesis (it is not inside the other dimensions). ie. if you wanted to access the 0th list in the outermost dimension you would call `x[0]`

``````>>> x[0]
[1, 2]
``````

This could also be called accessing the 0th row. I think this is by convention, nothing more. To access the inner dimension we’d have to get to it first, by chosing the outer dimension. So if we want the 1st element in the inner dimension of the 0th element in the outer dimension, we would call `x[0][1]`:

``````>>> x[0][1]
2
``````

Time or space:

The outermost dimension is the batch, and the innermost is the feature. The middle indices are therefore the aspects of the feature that we’re presenting to the model. These could be `x` and `y` coordinates, or perhaps `time` vs `amplitude` of a soundwave.