I am trying to feed stock data to Conv2D. But ran into dimension problem. I have no idea how to solve it and need help. Below are detailed steps that I have implemented.
I have attached data and code in the following link:
by the code itself should run. It will download the data automatically. but ive taken out the featuers to simplify the run. So it will have 5 features in the attached code.
but to give you quick glance of
The Problem I had———————–
2. Add time step to it by using:
def reshape_data(X, y, period=28):
n_past = period # number of days to look back in the past and compile into a time series trainX =  trainY = np.array(y.iloc[n_past:]) trainY = trainY[..., np.newaxis] for i in range(n_past, len(X)): trainX.append(X[i - n_past:i, 0:X.shape]) trainX = np.array(trainX) return trainX, trainY
data can be found here
I have applied pca on it. But simply convert it into numpy and apply reshap_data() on trainX should work
trainX, trainY = reshape_data(X_train_pca, y_train, period=30)
trainX (5768, 30, 30) # 5768-rows, 30- time steps, 30- # of features trainY (5768,1)
4. Add 1 axis after train X
trainX = trainX[...,np.newaxis] trainX is now (5768, 30, 30, 1)
5. Build model
6. fit and run
model.compile(optimizer=Adam(learning_rate=0.01) , metrics="mse", loss='binary_crossentropy') reduce_lr = tf.keras.callbacks.ReduceLROnPlateau(monitor='val_loss',factor=0.5,patience=10,verbose=0,mode='auto',min_delta=0.0002,cooldown=0,min_lr=0.0001) early_stop = tf.keras.callbacks.EarlyStopping(monitor="val_loss", patience=80, mode="min", restore_best_weights = True) history = model.fit(trainX, trainY, epochs=300, batch_size= 512, shuffle=False, verbose = 1, # validation_data=(testX, testY), validation_split=0.2, callbacks=[early_stop, reduce_lr] )
I thought since I have convered the stock into 30,30,1 should looks like a image dataset, which would enable tensorflow to work. But somehow it doesn’t
Add two layers after your convolution layer:
model.add(tf.keras.layers.Flatten()) model.add(tf.keras.layers.Dense(1, activation='sigmoid'))
And do not mix up
keras. Rather just use
tensorflow.keras for everything.
Answered By – AloneTogether