The following is the blog for reference in this article. You can refer to it if you need it:
1,Swing transformer segmentation source code (passed)
2,Swing transformer classification source code (passed)
3,Understanding points of swin transformer

1, Summary

In the recent swin transformer fire, the code has been open source, and girhub has soared to 4.5k directly. It is estimated that the next paper about various network structures combined with swin transformer will come out. Many leaders on the Internet have explained the principle of swing transformer, and some blogs will be put below to help understand it. This article mainly shares how to run through the source code on colab. Bloggers also stepped on a lot of holes in the process of tossing and turning.

2, Text

1. Related blog sharing

A. Code related explanation
(1),Swin Transformer of CV+Transformer , this is the first blog about swin transformer code explanation shared by a big man. The code here is not official open source code, but wild code. But it is also helpful to understand swin transformer, which you can refer to.
(2),Graphic Swin Transformer , this article is about the official source code, mainly about the structure of swin transformer.
B. Explanation related to the principle of the paper
Detailed derivation of 2021 swing transformer attention mechanism , this blog is quite detailed and discovered only recently. It actually pushes the calculation formula of complexity and some calculation of attention mask. You can refer to a wave. If you want to understand swin transformer well, it is really recommended that you read the paper and code together and confirm each other, so that you can better understand it and facilitate your magic change and various applications.

2. Official swin transformer source code

👉 Stamp right: Swing transformer source code
By the way, I mainly share the code about classification application. The classification problem is relatively simple, and it is most appropriate to use this task to understand swin transformer.

The following steps are given in Chinese:
2.1 clone this code to your server or local, and I will clone it to collab

git clone https://github.com/microsoft/Swin-Transformer.git
cd Swin-Transformer

As shown in the figure:
2.2. Create an operation environment and enter the environment
Since colab does not have conda installed, install conda first. Execute the following commands directly in the colab terminal:

%%bash
MINICONDA_INSTALLER_SCRIPT=Miniconda3-4.5.4-Linux-x86_64.sh
MINICONDA_PREFIX=/usr/local
wget https://repo.continuum.io/miniconda/$MINICONDA_INSTALLER_SCRIPT
chmod +x $MINICONDA_INSTALLER_SCRIPT
./$MINICONDA_INSTALLER_SCRIPT -b -f -p $MINICONDA_PREFIX

After installation, you can use conda -V to check the conda version to determine that the installation is successful.
Then use conda to create and activate the swin g virtual environment

conda create -n swin python=3.7 -y
conda activate swin

2.3 environment required for installation

conda install pytorch==1.7.1 torchvision==0.8.2 cudatoolkit=10.1 -c pytorch

Pay attention to your own environment here. My cuda is 10.1, so you can directly follow the one given by the official. There are many ways to look at your cuda environment. The most reliable one is this:
cat /usr/local/cuda/version.txt

Installation timm==0.3.2

pip install timm==0.3.2

Well, here comes the big head. Keng dad's apex is quite uncomfortable to install.
I reported the error directly according to the official operation on colab, and then I How to install nvidia apex on Google Colab Found an effective method on. See below:

%%writefile setup.sh

export CUDA_HOME=/usr/local/cuda-10.1
git clone https://github.com/NVIDIA/apex
pip install -v --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" ./apex

The above command is valid through personal test, but executing PIP install - V -- no cache dir -- global option = "-- cpp_ext" -- global option = "-- cuda_ext". / apex takes a long time and needs to wait patiently.
The installation process is as follows:

Installation succeeded

2.4 residual environment

pip install opencv-python==4.4.0.46 termcolor==1.1.0 yacs==0.1.8

If the above is successful, then this environment has been configured!!!

3. Data set

The official data set matching the code is imagenet, but imagenet is too large. I just want to run through it and see what's going on with the source code. There's no need to download such large data. The data set used in this paper is from Swing transformer classification source code (passed)，
Network disk link Extraction code: uwew
After uploading the data set compressed package to colab, you also need to decompress it

# zip file decompression
!pip install pyunpack
!pip install patool
from pyunpack import Archive
Archive('/content/drive/MyDrive/Swin-Transformer/imagenet.zip').extractall('/content/Swin-Transformer/imagenet')

4. Training

With data and code, it runs

The first is the operation mode of training:

python -m torch.distributed.launch --nproc_per_node 1 --master_port 12345 main.py --cfg configs/swin_tiny_patch4_window7_224.yaml --data-path imagenet --batch-size 64

Then there is how the test runs:

python -m torch.distributed.launch --nproc_per_node 1 --master_port 12345 main.py --eval --cfg configs/swin_tiny_patch4_window7_224.yaml --resume /pth/swin_tiny_patch4_window7_224.pth --data-path imagenet

Swin transformer file directory
There are also links in the network disk of pth folder. The apex in the link can be ignored. You can install it with the apex installation command I mentioned above.

3, Summary

I hope I can help you. If you think this article is helpful to you, please point a praise and support it! If you have any questions, you can also comment at the bottom of the article. Let's communicate and solve the problems together! I wish you all can run through!!!