Memory Allocation of Solution 4

[xhh232018]

Hi~ Thanks for your team's great work. I implemented the solution 4 in a local machine. However, I found something quite annoying in the loop of cross-validation. Here are 2 pictures about the memory problem:


The first one is the statistics of memory using during the fold_0. The second one is "fold_1". It is obvious that it will cost 3GB RAM after each training processing:

I started this training process last night and went to bed. I saw the error of "cannot allocate memory" this morning. Can u give me some tip so how to solve this problem?

[jakubczakon]

Hi @xhh232018,

I think what is happening here, is the out of fold predictions on train and test are appended to the list. Those will later be averaged (for test) and dumped on disk.

I think you could add a snipped that dumps those every iteration and then in a separate script (or notebook) you could average those and make submission.

Also, you could test it by running the train_evaluate_cv() instead of train_evaluate_predict_cv() . I often run train_evaluate_cv() for quick(ish) experiments and train_evaluate_predict_cv() later.