What does the bracket mean in XLA HLO's tensor shape?

Issue

I am confused with the number in {}. For example, the below code is sample XLA HLO, and I am having trouble understanding what %bitcast.26 does.

%gather.347 = f16[8192,768]{1,0} gather(f16[30522,768]{1,0} %convert.339, s32[8192]{0} %bitcast.25), offset_dims={1}, collapsed_slice_dims={0}, start_index_map={0}, index_vector_dim=1, slice_sizes={1,768}
%bitcast.26 = f16[16,512,768]{2,1,0} bitcast(f16[8192,768]{1,0} %gather.347)

Of course, I can see that the dimension changed from f16[8192,768]{1,0} to f16[16,512,768]{2,1,0}, but I do not get what {1,0} and {2,1,0} represents.

Solution

Square bracket([]): logical dimension.

Curly bracket({}): physical dimension layout that is going to be stored on the device (GPU, CPU, or TPU, whatever).

Answered By – Jueon Park

This Answer collected from stackoverflow, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0

Leave a Reply

(*) Required, Your email will not be published