Introduction

MindCV is an open source toolbox developed based on MindSpore and dedicated to the research and development of computer vision related technologies. It provides a large number of classic models and SoTA models in the field of computer vision and their pre-trained weights and training strategies. At the same time, it also provides SoTA algorithms such as automatic enhancement to improve model performance. With decoupled module design, you can easily apply MindCV to your own CV tasks.

>>> import mindcv
# 创建数据集
>>> dataset = mindcv.create_dataset('cifar10', download=True)
# 创建模型
>>> network = mindcv.create_model('resnet50', pretrained=True)

# 配置和启动迁移学习任务
python train.py --model swin_tiny --pretrained --opt=adamw --lr=0.001 --data_dir=/path/to/dataset

performance results

The summary results of model implementation and retraining based on MindCV can be found in benchmark_results.md. The training strategies used and the weights of the trained models can be obtained through the links in the table.

See the configs directory for each model explanation and training instructions.

Install

rely

• mindspore >= 1.8.1
• numpy >= 1.17.0
• pyyaml ​​>= 5.3
• wxya
• openmpi 4.0.3 (required for distributed mode)

Run the following script to install related dependencies.

pip install -r requirements.txt

Users can follow the official guidance and choose the most suitable version of MindSpore according to their own hardware platform to install. If you need to use it under distributed conditions, you also need to install openmpi.

The following instructions will assume that the user has correctly installed the relevant dependencies.

PyPI installation

Released versions of MindCV can be installed via PyPI.

Source installation

The latest MindCV on Git can be installed with the following command.

pip install git+https://github.com/mindspore-lab/mindcv.git

Note: MindCV can be installed on Linux and Mac systems, but it cannot be installed on Windows systems yet.

quick start

Getting Started Tutorial

Before getting started with MindCV, you can read MindCV’s migration learning tutorial, which can help users quickly understand the various important components of MindCV and the training, verification, and testing processes.

Here are some code samples for your quick experience.

>>> import mindcv
# 列出满足条件的预训练模型名称
>>> mindcv.list_models("swin*", pretrained=True)
['swin_tiny']
# 创建模型
>>> network = mindcv.create_model('swin_tiny', pretrained=True)
# 验证模型的准确率
>>> !python validate.py - -model = swin_tiny - -pretrained - -dataset = imagenet - -val_split = validation
{'Top_1_Accuracy': 0.808343989769821, 'Top_5_Accuracy': 0.9527253836317136, 'loss': 0.8474242982580839}

Image classification example

Inference is performed on an image using a SoTA model loaded with pre-trained parameters.

>>> !python infer.py - -model = swin_tiny - -image_path = './tutorials/data/test/dog/dog.jpg'
{'Labrador retriever': 0.5700152, 'golden retriever': 0.034551315, 'kelpie': 0.010108651,
 'Chesapeake Bay retriever': 0.008229004, 'Walker hound, Walker foxhound': 0.007791956}

The top 1 predictor is the Labrador Retriever, which is the breed of dog in this picture.

model training

passtrain.pyusers can easily train models on standard datasets or custom datasets, and users can set training strategies (such as data enhancement and learning path strategies) through external variables or yaml configuration files.

# 单卡训练
python train.py --model resnet50 --dataset cifar10 --dataset_download

The above code is an example of training ResNet on a single card (CPU/GPU/Ascend) on the CIFAR10 dataset, throughmodelanddatasetThe parameters specify the model and dataset to be trained respectively.

For large datasets like ImageNet, it is necessary to train in a distributed fashion across multiple devices.Based on MindSpore’s good support for distributed related functions, users can usempirunfor distributed training of the model.

# 分布式训练
# 假设你有4张GPU或者NPU卡
mpirun --allow-run-as-root -n 4 python train.py --distribute \
	--model densenet121 --dataset imagenet --data_dir ./datasets/imagenet

A complete list of parameters and descriptions are inconfig.pydefined in, can runpython train.py --helpQuick look.

To resume training, please specify--ckpt_pathand--ckpt_save_dirparameter, the script will load the model weights and optimizer state from the path, and resume the interrupted training process.

• Hyperparameter configuration and pre-training strategy

You can write yaml files or set external parameters to specify configuration data, models, optimizers and other components and their hyperparameters. The following is an example of model training using a preset training strategy (yaml file).

mpirun --allow-run-as-root -n 4 python train.py -c configs/squeezenet/squeezenet_1.0_gpu.yaml

Predefined Training Strategies MindCV currently advances more than 20 model training strategies and achieved SoTA performance on ImageNet.For specific parameter configuration and detailed accuracy performance summary, please refer toconfigsfolder. You can easily use these training strategies in your model training to improve performance (just reuse or modify the corresponding yaml file)

• Training on ModelArts/OpenI platform

To train on ModelArts or OpenI cloud platform, you need to do the following:

1、在云平台上创建新的训练任务。
2、在网站UI界面添加运行参数`config`，并指定yaml配置文件的路径。
3、在网站UI界面添加运行参数`enable_modelarts`并设置为True。
4、在网站上填写其他训练信息并启动培训任务。

model validation

usevalidate.pyThe trained model can be easily verified.

# 验证模型
python validate.py --model=resnet50 --dataset=imagenet --data_dir=/path/to/data --ckpt_path=/path/to/model.ckpt

• Validation during training

When you need to track the change of the accuracy of the model on the test set during the training process, please enable the parameter--val_while_train,as follows

python train.py --model=resnet50 --dataset=cifar10 \
		--val_while_train --val_split=test --val_interval=1

The training loss and test accuracy for each epoch will be saved in{ckpt_save_dir}/results.logmiddle.

• Static and dynamic graph modes

By default, model training (train.py) runs on MindSpore in graph mode, which is optimized for performance and parallel computing using static graph compilation. In contrast, the advantages of pynative mode are flexibility and ease of debugging.For easier debugging, you can pass the parameter--modeSet to 1 to set the run mode to debug mode.

The mixed mode based on ms_function is a mixed mode that takes into account the efficiency and flexibility of MindSpore.Users can use thetrain_with_func.pyfile to train with that blend mode.

python train_with_func.py --model=resnet50 --dataset=cifar10 --dataset_download --epoch_size=10

Note: This is an experimental training script and is still being improved. Using this mode on MindSpore version 1.8.1 or earlier is currently not stable.

tutorial

We provide a series of tutorials to help users learn how to use MindCV.

model list

Currently, MindCV supports the following models.

support algorithm

log

renew

1. Update the ResNet network pre-training weights, now the pre-training weights have higher Top1 accuracy
   • ResNet18 accuracy increased from 70.09 to 70.31
   • ResNet34 accuracy increased from 73.69 to 74.15
   • ResNet50 accuracy increased from 76.64 to 76.69
   • ResNet101 accuracy increased from 77.63 to 78.24
   • ResNet152 accuracy increased from 78.63 to 78.72
2. Update the pre-training weight name and the corresponding download URL link according to the rule (model_scale-sha256sum.ckpt).

1. Add Lion (EvoLved Sign Momentum) optimizer, paper https://arxiv.org/abs/2302.06675
   • The learning rate used by Lion is generally 3 to 10 times smaller than Adamw, and the weight decay (weigt_decay) is 3 to 10 times larger.
2. Add 6 models and their training strategies, pre-training weights:
3. Support gradient clip

1. MindCV v0.1 released! Supports installation via PyPI (pip install mindcv).
2. Added pre-training weights and strategies for 4 models: googlenet, inception_v3, inception_v4, xception

1. Support adding learning rate warm-up operation to all learning rate strategies, except cosine decay strategy.
2. supportRepeated Augmenationoperation, through--aug_repeatsSet it to a value greater than 1 (usually 3 or 4).
3. EMAs are supported.
4. The BCE loss function is further optimized by supporting mixup and cutmix operations.

1. Supports visualization of model loss and accuracy.
2. Support the learning rate warm-up operation of the cosine decay strategy in the cosine dimension (previously only supported the step dimension).

1. Supports 2 ViT pre-trained models.
2. Support RandAugment augmentation operation.
3. Improved usability of CutMix operations, CutMix and Mixup can now be used together.
4. Solved the bug of learning rate drawing.

1. Both BCE and CE loss functions currently support class-weight config operations, label smoothing operations, and auxiliary logit input operations (applicable to similar Inception models).

1. Support for Adan optimizer (trial version).

way of contribution

Developers and users are welcome to raise issues or submit code PRs, or contribute more algorithms and models to make MindCV better together.

See CONTRIBUTING.md for contributing guidelines. Please follow the rules stated in the model writing guidelines for contributing model interfaces 🙂

license

This project follows the Apache License 2.0 open source agreement.

thank you

MindCV is an open source project jointly developed by the MindSpore team, Xidian University, and Xi’an Jiaotong University. Sincere thanks to all involved researchers and developers for their hard work on this project. I am very grateful for the computing resources provided by the OpenI platform.

quote

If you find MindCV helpful to your project, please consider citing:

@misc{MindSpore Computer Vision 2022,
    title={{MindSpore Computer  Vision}:MindSpore Computer Vision Toolbox and Benchmark},
    author={MindSpore Vision Contributors},
    howpublished = {\url{https://github.com/mindspore-lab/mindcv/}},
    year={2022}
}