Skip to content

Inconsistency in pretrained mae vitb16 weights #35

@jafarinia

Description

@jafarinia

Hi
I have a question about your pretreined mae vitb16 weights.
When I use "https://github.com/ShoufaChen/AdaptFormer/releases/download/v0.1/mae_pretrain_vit_b.pth" which is the one on your GitHub repo and train on Cifar100 for 1 epoch I generally get
Acc@1 37.330 Acc@5 67.820 loss 2.768
(the command is python main_image.py --batch_size=128 --cls_token --epochs=1 --finetune=mae_pretrain_vit_b.pth --dist_eval
--data_path=cifar100 --output_dir=output --num_workers=16 --drop_path=0.0 --blr=0.1 --dataset=cifar100 --ffn_adapt)
But
When I use "https://dl.fbaipublicfiles.com/mae/pretrain/mae_pretrain_vit_base.pth" which is the claimed weight on your paper's text and train on Cifar100 for 1 epoch I generally get
Acc@1 5.660 Acc@5 21.350 loss 4.295.
(the command is python main_image.py --batch_size=128 --cls_token --epochs=1 --finetune=mae_pretrain_vit_base.pth --dist_e
val --data_path=cifar100 --output_dir=output --num_workers=16 --drop_path=0.0 --blr=0.1 --dataset=cifar100 --ffn_adapt)
These results are consistent over multiple runs.
My questions are, what is the difference between these two pretrained weights? And what causes this huge difference in results?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions