stock data to fit for Conv2D – Python tensorflow

Issue

I am trying to feed stock data to Conv2D. But ran into dimension problem. I have no idea how to solve it and need help. Below are detailed steps that I have implemented.

I have attached data and code in the following link:
https://drive.google.com/drive/folders/1snsQ-96AeRn521oc0aQVlTTd9nHbtyjO?usp=sharing

by the code itself should run. It will download the data automatically. but ive taken out the featuers to simplify the run. So it will have 5 features in the attached code.

but to give you quick glance of
The Problem I had———————–

1. Got stock data and generated some features, it looks like:
enter image description here

2. Add time step to it by using:

def reshape_data(X, y, period=28):

n_past = period # number of days to look back in the past and compile into a time series
trainX = []
trainY = np.array(y.iloc[n_past:])
trainY = trainY[..., np.newaxis]
for i in range(n_past, len(X)):
    trainX.append(X[i - n_past:i, 0:X.shape[1]])
trainX = np.array(trainX)
return trainX, trainY

enter image description here

Note:

data can be found here

https://drive.google.com/drive/folders/1snsQ-96AeRn521oc0aQVlTTd9nHbtyjO?usp=sharing

I have applied pca on it. But simply convert it into numpy and apply reshap_data() on trainX should work

trainX, trainY = reshape_data(X_train_pca, y_train, period=30)

3. shape

trainX (5768, 30, 30) # 5768-rows, 30- time steps, 30- # of features

trainY (5768,1)

4. Add 1 axis after train X

trainX = trainX[...,np.newaxis]

trainX is now (5768, 30, 30, 1)

5. Build model

enter image description here

6. fit and run

model.compile(optimizer=Adam(learning_rate=0.01) , metrics="mse", loss='binary_crossentropy')

reduce_lr  = tf.keras.callbacks.ReduceLROnPlateau(monitor='val_loss',factor=0.5,patience=10,verbose=0,mode='auto',min_delta=0.0002,cooldown=0,min_lr=0.0001)


early_stop = tf.keras.callbacks.EarlyStopping(monitor="val_loss", patience=80, mode="min",  restore_best_weights = True)

history = model.fit(trainX, trainY, epochs=300,
               batch_size= 512, shuffle=False, verbose = 1,
               # validation_data=(testX, testY),
               validation_split=0.2,
               callbacks=[early_stop, reduce_lr] )

7. ERROR

enter image description here

I thought since I have convered the stock into 30,30,1 should looks like a image dataset, which would enable tensorflow to work. But somehow it doesn’t

Solution

Add two layers after your convolution layer:

model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(1, activation='sigmoid'))

And do not mix up tensorflow.keras and keras. Rather just use tensorflow.keras for everything.

Answered By – AloneTogether

This Answer collected from stackoverflow, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0

Leave a Reply

(*) Required, Your email will not be published