mmaction2 / Vibevoice /en /notes /changelog.md

niobures

mmaction2

d3dbf03 verified 3 months ago

preview code

raw

history blame contribute delete

69.5 kB

Changelog

1.2.0 (10/12/2023)

Highlights

Support the Training of ActionClip
Support VindLU multi-modality algorithm
Support MobileOne TSN/TSM

New Features

Support the Training of ActionClip (2620)
Support video retrieval dataset MSVD (2622)
Support VindLU multi-modality algorithm (2667)
Support Dense Regression Network for Video Grounding (2668)

Improvements

Support Video Demos (2602)
Support Audio Demos (2603)
Add README_zh-CN.md for Swin and VideoMAE (2621)
Support MobileOne TSN/TSM (2656)
Support SlowOnly K700 feature to train localization models (2673)

Bug Fixes

Refine ActionDataSample structure (2658)
Fix MPS device (2619)

1.1.0 (7/3/2023)

Highlights

Support HACS-segments dataset(ICCV'2019), MultiSports dataset(ICCV'2021), Kinetics-710 dataset(Arxiv'2022)
Support rich projects: gesture recognition, spatio-temporal action detection tutorial, and knowledge distillation
Support TCANet(CVPR'2021)
Support VideoMAE V2(CVPR'2023), and VideoMAE(NeurIPS'2022) on action detection
Support CLIP-based multi-modality models: ActionCLIP(Arxiv'2021) and CLIP4clip(ArXiv'2022)
Support Pure Python style Configuration File and downloading datasets by MIM

New Features

Support HACS-segments dataset (2224)
Support TCANet (2271)
Support MultiSports dataset (2280)
Support spatio-temporal action detection tutorial (2428)
Support knowledge distillation based on MMRazor (2458)
Support VideoMAE V2 (2460)
Support ActionCLIP (2470)
Support CLIP4clip (2489)
Support Kinetics-710 dataset (2534)
Support gesture recognition project (2539)
Support VideoMAE on action detection (2547)
Support downloading datasets by MIM (2465)
Support new config (2542)

Improvements

Refactor TSM init_weights (2396)
Add unit test for Recognizer 2D (2432)
Enhance inference APIs (2472)
Support converting ST-GCN and PoseC3D to ONNX (2543)
Support feature extraction head (2525)

Bug Fixes

Fix CircleCI (2351)
Fix demo skeleton script (2380)
Fix docker file branch (2397)
Fix NTU pose extraction script (2402)
Rename typing and enhance collect_env script (2420)
Fix multi-label classification (2425, 2466, 2532)
Fix lfb configs (2426)
Fix a warning caused by torch.div (2449)
Fix incompatibility of ImgAug and latest Numpy (2451)
Fix MViT with_cls_token argument (2480)
Fix timm BC-breaking for TSN (2497)
Close FileHandler in Windows to make the temporary directory can be deleted (2565)
Update minimum PyTorch version to 1.8.1 (2568)

Documentation

Fix document links in README (2358, 2372, 2376, 2382)
Update installation document (2362)
Update upstream library version requirement (2383)
Fix Colab tutorial (2384, 2391, 2475)
Refine documents (2404)
Update outdated config in readme (2419)
Update OpenMMLab related repo list (2429)
Fix UniFormer README and metafile (2450)
Add finetune document (2457)
Update FAQ document (2476, 2482
Update download datasets document (2495)
Translate Chinese document (2516, 2506, 2499)
Refactor model zoo and dataset zoo (2552)
Refactor Chinese document (2567)

1.0.0 (4/6/2023)

Highlights

Support RGB-PoseC3D(CVPR'2022).
Support training UniFormer V2(Arxiv'2022).
Support MSG3D(CVPR'2020) and CTRGCN(CVPR'2021) in projects.
Refactor and provide more user-friendly documentation.

New Features

Support RGB-PoseC3D (2182)
Support training UniFormer V2 (2221)
Support MSG3D and CTRGCN in projects. (2269, 2291)

Improvements

Use MMEngine to calculate FLOPs (2300)
Speed up LFB training (2294)
Support multiprocessing on AVA evaluation (2146)
Add a demo for exporting spatial-temporal detection model to ONNX (2225)
Update spatial-temporal detection related folders (2262)

Bug Fixes

Fix flip config of TSM for sth v1/v2 dataset (#2247)
Fix circle ci (2336, 2334)
Fix accepting an unexpected argument local-rank in PyTorch 2.0 (2320)
Fix TSM config link (2315)
Fix numpy version requirement in CI (2284)
Fix NTU pose extraction script (2246)
Fix TSM-MobileNet V2 (2332)
Fix command bugs in localization tasks' README (2244)
Fix duplicate name in DecordInit and SampleAVAFrame (2251)
Fix channel order when showing video (2308)
Specify map_location to cpu when using _load_checkpoint (2252)

Documentation

Refactor and provide more user-friendly documentation (2341, 2312, 2325)
Add README_zh-CN (2252)
Add social networking links (2294)
Fix sthv2 dataset annotations preparation document (2248)

1.0.0rc3 (2/10/2023)

Highlights

Support Action Recognition model UniFormer V1(ICLR'2022), UniFormer V2(Arxiv'2022).
Support training MViT V2(CVPR'2022), and MaskFeat(CVPR'2022) fine-tuning.

New Features

Support UniFormer V1/V2 (#2153)
Support training MViT, and MaskFeat fine-tuning (#2186)
Support a unified inference interface: Inferencer (#2164)

Improvements

Support load data list from multi-backends (#2176)

Bug Fixes

Upgrade isort to fix CI (#2198)
Fix bug in skeleton demo (#2214)

Documentation

Add Chinese documentation for config.md (#2188)
Add readme for Omnisource (#2205)

1.0.0rc2 (1/10/2023)

Highlights

Support Action Recognition model VideoMAE(NeurIPS'2022), MViT V2(CVPR'2022), C2D and skeleton-based action recognition model STGCN++
Support Omni-Source training on ImageNet and Kinetics datasets
Support exporting spatial-temporal detection models to ONNX

New Features

Support VideoMAE (#1942)
Support MViT V2 (#2007)
Support C2D (#2022)
Support AVA-Kinetics dataset (#2080)
Support STGCN++ (#2156)
Support exporting spatial-temporal detection models to ONNX (#2148)
Support Omni-Source training on ImageNet and Kinetics datasets (#2143)

Improvements

Support repeat batch data augmentation (#2170)
Support calculating FLOPs tool powered by fvcore (#1997)
Support Spatial-temporal detection demo (#2019)
Add SyncBufferHook and add randomness config in train.py (#2044)
Refactor gradcam (#2049)
Support init_cfg in Swin and ViTMAE (#2055)
Refactor STGCN and related pipelines (#2087)
Refactor visualization tools (#2092)
Update SampleFrames transform and improve most models' performance (#1942)
Support real-time webcam demo (#2152)
Refactor and enhance 2s-AGCN (#2130)
Support adjusting fps in SampleFrame (#2157)

Bug Fixes

Fix CI upstream library dependency (#2000)
Fix SlowOnly readme typos and results (#2006)
Fix VideoSwin readme (#2010)
Fix tools and mim error (#2028)
Fix Imgaug wrapper (#2024)
Remove useless scripts (#2032)
Fix multi-view inference (#2045)
Update mmcv maximum version to 1.8.0 (#2047)
Fix torchserver dependency (#2053)
Fix gen_ntu_rgbd_raw script (#2076)
Update AVA-Kinetics experiment configs and results (#2099)
Add joint.pkl and bone.pkl used in multi-stream fusion tool (#2106)
Fix lint CI config (#2110)
Update testing accuracy for modified SampleFrames (#2117), (#2121), (#2122), (#2124), (#2125), (#2126), (#2129), (#2128)
Fix timm related bug (#1976)
Fix check_videos.py script (#2134)
Update CI maximum torch version to 1.13.0 (#2118)

Documentation

Add MMYOLO description in README (#2011)
Add v1.x introduction in README (#2023)
Fix link in README (#2035)
Refine some docs (#2038), (#2040), (#2058)
Update TSN/TSM Readme (#2082)
Add chinese document (#2083)
Adjust document structure (#2088)
Fix Sth-Sth and Jester dataset links (#2103)
Fix doc link (#2131)

1.0.0rc1 (10/14/2022)

Highlights

Support Video Swin Transformer

New Features

Support Video Swin Transformer (#1939)

Improvements

Add colab tutorial for 1.x (#1956)
Support skeleton-based action recognition demo (#1920)

Bug Fixes

Fix link in doc (#1986, #1967, #1951, #1926,#1944, #1944, #1927, #1925)
Fix CI (#1987, #1930, #1923)
Fix pre-commit hook config (#1971)
Fix TIN config (#1912)
Fix UT for BMN and BSN (#1966)
Fix UT for Recognizer2D (#1937)
Fix BSN and BMN configs for localization (#1913)
Modeify ST-GCN configs (#1913)
Fix typo in migration doc (#1931)
Remove Onnx related tools (#1928)
Update TANet readme (#1916, #1890)
Update 2S-AGCN readme (#1915)
Fix TSN configs (#1905)
Fix configs for detection (#1903)
Fix typo in TIN config (#1904)
Fix PoseC3D readme (#1899)
Fix ST-GCN configs (#1891)
Fix audio recognition readme (#1898)
Fix TSM readme (#1887)
Fix SlowOnly readme (#1889)
Fix TRN readme (#1888)
Fix typo in get_started doc (#1895)

1.0.0rc0 (09/01/2022)

We are excited to announce the release of MMAction2 v1.0.0rc0. MMAction2 1.0.0beta is the first version of MMAction2 1.x, a part of the OpenMMLab 2.0 projects. Built upon the new training engine.

Highlights

New engines. MMAction2 1.x is based on MMEngine](https://github.com/open-mmlab/mmengine), which provides a general and powerful runner that allows more flexible customizations and significantly simplifies the entrypoints of high-level interfaces.
Unified interfaces. As a part of the OpenMMLab 2.0 projects, MMAction2 1.x unifies and refactors the interfaces and internal logics of train, testing, datasets, models, evaluation, and visualization. All the OpenMMLab 2.0 projects share the same design in those interfaces and logics to allow the emergence of multi-task/modality algorithms.
More documentation and tutorials. We add a bunch of documentation and tutorials to help users get started more smoothly. Read it here.

Breaking Changes

In this release, we made lots of major refactoring and modifications. Please refer to the migration guide for details and migration instructions.

0.24.0 (05/05/2022)

Highlights

Support different seeds

New Features

Add lateral norm in multigrid config (#1567)
Add openpose 25 joints in graph config (#1578)
Support MLU Backend (#1608)

Bug and Typo Fixes

Fix local_rank (#1558)
Fix install typo (#1571)
Fix the inference API doc (#1580)
Fix zh-CN demo.md and getting_started.md (#1587)
Remove Recommonmark (#1595)
Fix inference with ndarray (#1603)
Fix the log error when IterBasedRunner is used (#1606)

0.23.0 (04/01/2022)

Highlights

Support different seeds
Provide multi-node training & testing script
Update error log

New Features

Support different seeds(#1502)
Provide multi-node training & testing script(#1521)
Update error log(#1546)

Documentations

Update gpus in Slowfast readme(#1497)
Fix work_dir in multigrid config(#1498)
Add sub bn docs(#1503)
Add shortcycle sampler docs(#1513)
Update Windows Declaration(#1520)
Update the link for ST-GCN(#1544)
Update install commands(#1549)

Bug and Typo Fixes

Update colab tutorial install cmds(#1522)
Fix num_iters_per_epoch in analyze_logs.py(#1530)
Fix distributed_sampler(#1532)
Fix cd dir error(#1545)
Update arg names(#1548)

ModelZoo

0.22.0 (03/05/2022)

Highlights

Support Multigrid training strategy
Support CPU training
Support audio demo
Support topk customizing in models/heads/base.py

New Features

Support Multigrid training strategy(#1378)
Support STGCN in demo_skeleton.py(#1391)
Support CPU training(#1407)
Support audio demo(#1425)
Support topk customizing in models/heads/base.py(#1452)

Documentations

Add OpenMMLab platform(#1393)
Update links(#1394)
Update readme in configs(#1404)
Update instructions to install mmcv-full(#1426)
Add shortcut(#1433)
Update modelzoo(#1439)
add video_structuralize in readme(#1455)
Update OpenMMLab repo information(#1482)

Bug and Typo Fixes

Update train.py(#1375)
Fix printout bug(#1382)
Update multi processing setting(#1395)
Setup multi processing both in train and test(#1405)
Fix bug in nondistributed multi-gpu training(#1406)
Add variable fps in ava_dataset.py(#1409)
Only support distributed training(#1414)
Set test_mode for AVA configs(#1432)
Support single label(#1434)
Add check copyright(#1447)
Support Windows CI(#1448)
Fix wrong device of class_weight in models/losses/cross_entropy_loss.py(#1457)
Fix bug caused by distributed(#1459)
Update readme(#1460)
Fix lint caused by colab automatic upload(#1461)
Refine CI(#1471)
Update pre-commit(#1474)
Add deprecation message for deploy tool(#1483)

ModelZoo

Support slowfast_steplr(#1421)

0.21.0 (31/12/2021)

Highlights

Support 2s-AGCN
Support publish models in Windows
Improve some sthv1 related models
Support BABEL

New Features

Support 2s-AGCN(#1248)
Support skip postproc in ntu_pose_extraction(#1295)
Support publish models in Windows(#1325)
Add copyright checkhook in pre-commit-config(#1344)

Documentations

Add MMFlow (#1273)
Revise README.md and add projects.md (#1286)
Add 2s-AGCN in Updates(#1289)
Add MMFewShot(#1300)
Add MMHuman3d(#1304)
Update pre-commit(#1313)
Use share menu from the theme instead(#1328)
Update installation command(#1340)

Bug and Typo Fixes

Update the inference part in notebooks(#1256)
Update the map_location(#1262)
Fix bug that start_index is not used in RawFrameDecode(#1278)
Fix bug in init_random_seed(#1282)
Fix bug in setup.py(#1303)
Fix interrogate error in workflows(#1305)
Fix typo in slowfast config(#1309)
Cancel previous runs that are not completed(#1327)
Fix missing skip_postproc parameter(#1347)
Update ssn.py(#1355)
Use latest youtube-dl(#1357)
Fix test-best(#1362)

ModelZoo

Improve some sthv1 related models(#1306)
Support BABEL(#1332)

0.20.0 (07/10/2021)

Highlights

Support TorchServe
Add video structuralize demo
Support using 3D skeletons for skeleton-based action recognition
Benchmark PoseC3D on UCF and HMDB

New Features

Support TorchServe (#1212)
Support 3D skeletons pre-processing (#1218)
Support video structuralize demo (#1197)

Documentations

Revise README.md and add projects.md (#1214)
Add CN docs for Skeleton dataset, PoseC3D and ST-GCN (#1228, #1237, #1236)
Add tutorial for custom dataset training for skeleton-based action recognition (#1234)

Bug and Typo Fixes

Fix tutorial link (#1219)
Fix GYM links (#1224)

ModelZoo

Benchmark PoseC3D on UCF and HMDB (#1223)
Add ST-GCN + 3D skeleton model for NTU60-XSub (#1236)

0.19.0 (07/10/2021)

Highlights

Support ST-GCN
Refactor the inference API
Add code spell check hook

New Features

Support ST-GCN (#1123)

Improvement

Add label maps for every dataset (#1127)
Remove useless code MultiGroupCrop (#1180)
Refactor Inference API (#1191)
Add code spell check hook (#1208)
Use docker in CI (#1159)

Documentations

Update metafiles to new OpenMMLAB protocols (#1134)
Switch to new doc style (#1160)
Improve the ERROR message (#1203)
Fix invalid URL in getting_started (#1169)

Bug and Typo Fixes

Compatible with new MMClassification (#1139)
Add missing runtime dependencies (#1144)
Fix THUMOS tag proposals path (#1156)
Fix LoadHVULabel (#1194)
Switch the default value of persistent_workers to False (#1202)
Fix _freeze_stages for MobileNetV2 (#1193)
Fix resume when building rawframes (#1150)
Fix device bug for class weight (#1188)
Correct Arg names in extract_audio.py (#1148)

ModelZoo

Add TSM-MobileNetV2 ported from TSM (#1163)
Add ST-GCN for NTURGB+D-XSub-60 (#1123)

0.18.0 (02/09/2021)

Improvement

Add CopyRight (#1099)
Support NTU Pose Extraction (#1076)
Support Caching in RawFrameDecode (#1078)
Add citations & Support python3.9 CI & Use fixed-version sphinx (#1125)

Documentations

Add Descriptions of PoseC3D dataset (#1053)

Bug and Typo Fixes

Fix SSV2 checkpoints (#1101)
Fix CSN normalization (#1116)
Fix typo (#1121)
Fix new_crop_quadruple bug (#1108)

0.17.0 (03/08/2021)

Highlights

Support PyTorch 1.9
Support Pytorchvideo Transforms
Support PreciseBN

New Features

Support Pytorchvideo Transforms (#1008)
Support PreciseBN (#1038)

Improvements

Remove redundant augmentations in config files (#996)
Make resource directory to hold common resource pictures (#1011)
Remove deprecated FrameSelector (#1010)
Support Concat Dataset (#1000)
Add to-mp4 option to resize_videos.py (#1021)
Add option to keep tail frames (#1050)
Update MIM support (#1061)
Calculate Top-K accurate and inaccurate classes (#1047)

Bug and Typo Fixes

Fix bug in PoseC3D demo (#1009)
Fix some problems in resize_videos.py (#1012)
Support torch1.9 (#1015)
Remove redundant code in CI (#1046)
Fix bug about persistent_workers (#1044)
Support TimeSformer feature extraction (#1035)
Fix ColorJitter (#1025)

ModelZoo

Add TSM-R50 sthv1 models trained by PytorchVideo RandAugment and AugMix (#1008)
Update SlowOnly SthV1 checkpoints (#1034)
Add SlowOnly Kinetics400 checkpoints trained with Precise-BN (#1038)
Add CSN-R50 from scratch checkpoints (#1045)
TPN Kinetics-400 Checkpoints trained with the new ColorJitter (#1025)

Documentation

Add Chinese translation of feature_extraction.md (#1020)
Fix the code snippet in getting_started.md (#1023)
Fix TANet config table (#1028)
Add description to PoseC3D dataset (#1053)

0.16.0 (01/07/2021)

Highlights

Support using backbone from pytorch-image-models(timm)
Support PIMS Decoder
Demo for skeleton-based action recognition
Support Timesformer

New Features

Support using backbones from pytorch-image-models(timm) for TSN (#880)
Support torchvision transformations in preprocessing pipelines (#972)
Demo for skeleton-based action recognition (#972)
Support Timesformer (#839)

Improvements

Add a tool to find invalid videos (#907, #950)
Add an option to specify spectrogram_type (#909)
Add json output to video demo (#906)
Add MIM related docs (#918)
Rename lr to scheduler (#916)
Support --cfg-options for demos (#911)
Support number counting for flow-wise filename template (#922)
Add Chinese tutorial (#941)
Change ResNet3D default values (#939)
Adjust script structure (#935)
Add font color to args in long_video_demo (#947)
Polish code style with Pylint (#908)
Support PIMS Decoder (#946)
Improve Metafiles (#956, #979, #966)
Add links to download Kinetics400 validation (#920)
Audit the usage of shutil.rmtree (#943)
Polish localizer related codes(#913)

Bug and Typo Fixes

Fix spatiotemporal detection demo (#899)
Fix docstring for 3D inflate (#925)
Fix bug of writing text to video with TextClip (#952)
Fix mmcv install in CI (#977)

ModelZoo

Add TSN with Swin Transformer backbone as an example for using pytorch-image-models(timm) backbones (#880)
Port CSN checkpoints from VMZ (#945)
Release various checkpoints for UCF101, HMDB51 and Sthv1 (#938)
Support Timesformer (#839)
Update TSM modelzoo (#981)

0.15.0 (31/05/2021)

Highlights

Support PoseC3D
Support ACRN
Support MIM

New Features

Support PoseC3D (#786, #890)
Support MIM (#870)
Support ACRN and Focal Loss (#891)
Support Jester dataset (#864)

Improvements

Add metric_options for evaluation to docs (#873)
Support creating a new label map based on custom classes for demos about spatio temporal demo (#879)
Improve document about AVA dataset preparation (#878)
Provide a script to extract clip-level feature (#856)

Bug and Typo Fixes

Fix issues about resume (#877, #878)
Correct the key name of eval_results dictionary for metric 'mmit_mean_average_precision' (#885)

ModelZoo

Support Jester dataset (#864)
Support ACRN and Focal Loss (#891)

0.14.0 (30/04/2021)

Highlights

Support TRN
Support Diving48

New Features

Support TRN (#755)
Support Diving48 (#835)
Support Webcam Demo for Spatio-temporal Action Detection Models (#795)

Improvements

Add softmax option for pytorch2onnx tool (#781)
Support TRN (#755)
Test with onnx models and TensorRT engines (#758)
Speed up AVA Testing (#784)
Add self.with_neck attribute (#796)
Update installation document (#798)
Use a random master port (#809)
Update AVA processing data document (#801)
Refactor spatio-temporal augmentation (#782)
Add QR code in CN README (#812)
Add Alternative way to download Kinetics (#817, #822)
Refactor Sampler (#790)
Use EvalHook in MMCV with backward compatibility (#793)
Use MMCV Model Registry (#843)

Bug and Typo Fixes

Fix a bug in pytorch2onnx.py when num_classes <= 4 (#800, #824)
Fix demo_spatiotemporal_det.py error (#803, #805)
Fix loading config bugs when resume (#820)
Make HMDB51 annotation generation more robust (#811)

ModelZoo

Update checkpoint for 256 height in something-V2 (#789)
Support Diving48 (#835)

0.13.0 (31/03/2021)

Highlights

Support LFB
Support using backbone from MMCls/TorchVision
Add Chinese documentation

New Features

Support LFB (#553)
Support using backbones from MMCls for TSN (#679)
Support using backbones from TorchVision for TSN (#720)
Support Mixup and Cutmix for recognizers (#681)
Support Chinese documentation (#665, #680, #689, #701, #702, #703, #706, #716, #717, #731, #733, #735, #736, #737, #738, #739, #740, #742, #752, #759, #761, #772, #775)

Improvements

Add slowfast config/json/log/ckpt for training custom classes of AVA (#678)
Set RandAugment as Imgaug default transforms (#585)
Add --test-last & --test-best for tools/train.py to test checkpoints after training (#608)
Add fcn_testing in TPN (#684)
Remove redundant recall functions (#741)
Recursively remove pretrained step for testing (#695)
Improve demo by limiting inference fps (#668)

Bug and Typo Fixes

Fix a bug about multi-class in VideoDataset (#723)
Reverse key-value in anet filelist generation (#686)
Fix flow norm cfg typo (#693)

ModelZoo

Add LFB for AVA2.1 (#553)
Add TSN with ResNeXt-101-32x4d backbone as an example for using MMCls backbones (#679)
Add TSN with Densenet161 backbone as an example for using TorchVision backbones (#720)
Add slowonly_nl_embedded_gaussian_r50_4x16x1_150e_kinetics400_rgb (#690)
Add slowonly_nl_embedded_gaussian_r50_8x8x1_150e_kinetics400_rgb (#704)
Add slowonly_nl_kinetics_pretrained_r50_4x16x1(8x8x1)_20e_ava_rgb (#730)

0.12.0 (28/02/2021)

Highlights

Support TSM-MobileNetV2
Support TANet
Support GPU Normalize

New Features

Support TSM-MobileNetV2 (#415)
Support flip with label mapping (#591)
Add seed option for sampler (#642)
Support GPU Normalize (#586)
Support TANet (#595)

Improvements

Training custom classes of ava dataset (#555)
Add CN README in homepage (#592, #594)
Support soft label for CrossEntropyLoss (#625)
Refactor config: Specify train_cfg and test_cfg in model (#629)
Provide an alternative way to download older kinetics annotations (#597)
Update FAQ for
- 1). data pipeline about video and frames (#598)
- 2). how to show results (#598)
- 3). batch size setting for batchnorm (#657)
- 4). how to fix stages of backbone when finetuning models (#658)
Modify default value of save_best (#600)
Use BibTex rather than latex in markdown (#607)
Add warnings of uninstalling mmdet and supplementary documents (#624)
Support soft label for CrossEntropyLoss (#625)

Bug and Typo Fixes

Fix value of pem_low_temporal_iou_threshold in BSN (#556)
Fix ActivityNet download script (#601)

ModelZoo

Add TSM-MobileNetV2 for Kinetics400 (#415)
Add deeper SlowFast models (#605)

0.11.0 (31/01/2021)

Highlights

Support imgaug
Support spatial temporal demo
Refactor EvalHook, config structure, unittest structure

New Features

Support imgaug for augmentations in the data pipeline (#492)
Support setting max_testing_views for extremely large models to save GPU memory used (#511)
Add spatial temporal demo (#547, #566)

Improvements

Refactor EvalHook (#395)
Refactor AVA hook (#567)
Add repo citation (#545)
Add dataset size of Kinetics400 (#503)
Add lazy operation docs (#504)
Add class_weight for CrossEntropyLoss and BCELossWithLogits (#509)
add some explanation about the resampling in slowfast (#502)
Modify paper title in README.md (#512)
Add alternative ways to download Kinetics (#521)
Add OpenMMLab projects link in README (#530)
Change default preprocessing to shortedge to 256 (#538)
Add config tag in dataset README (#540)
Add solution for markdownlint installation issue (#497)
Add dataset overview in readthedocs (#548)
Modify the trigger mode of the warnings of missing mmdet (#583)
Refactor config structure (#488, #572)
Refactor unittest structure (#433)

Bug and Typo Fixes

Fix a bug about ava dataset validation (#527)
Fix a bug about ResNet pretrain weight initialization (#582)
Fix a bug in CI due to MMCV index (#495)
Remove invalid links of MiT and MMiT (#516)
Fix frame rate bug for AVA preparation (#576)

ModelZoo

0.10.0 (31/12/2020)

Highlights

Support Spatio-Temporal Action Detection (AVA)
Support precise BN

New Features

Support precise BN (#501)
Support Spatio-Temporal Action Detection (AVA) (#351)
Support to return feature maps in inference_recognizer (#458)

Improvements

Add arg stride to long_video_demo.py, to make inference faster (#468)
Support training and testing for Spatio-Temporal Action Detection (#351)
Fix CI due to pip upgrade (#454)
Add markdown lint in pre-commit hook (#255)
Speed up confusion matrix calculation (#465)
Use title case in modelzoo statistics (#456)
Add FAQ documents for easy troubleshooting. (#413, #420, #439)
Support Spatio-Temporal Action Detection with context (#471)
Add class weight for CrossEntropyLoss and BCELossWithLogits (#509)
Add Lazy OPs docs (#504)

Bug and Typo Fixes

Fix typo in default argument of BaseHead (#446)
Fix potential bug about output_config overwrite (#463)

ModelZoo

Add SlowOnly, SlowFast for AVA2.1 (#351)

0.9.0 (30/11/2020)

Highlights

Support GradCAM utils for recognizers
Support ResNet Audio model

New Features

Automatically add modelzoo statistics to readthedocs (#327)
Support GYM99 (#331, #336)
Add AudioOnly Pathway from AVSlowFast. (#355)
Add GradCAM utils for recognizer (#324)
Add print config script (#345)
Add online motion vector decoder (#291)

Improvements

Support PyTorch 1.7 in CI (#312)
Support to predict different labels in a long video (#274)
Update docs bout test crops (#359)
Polish code format using pylint manually (#338)
Update unittest coverage (#358, #322, #325)
Add random seed for building filelists (#323)
Update colab tutorial (#367)
set default batch_size of evaluation and testing to 1 (#250)
Rename the preparation docs to README.md (#388)
Move docs about demo to demo/README.md (#329)
Remove redundant code in tools/test.py (#310)
Automatically calculate number of test clips for Recognizer2D (#359)

Bug and Typo Fixes

Fix rename Kinetics classnames bug (#384)
Fix a bug in BaseDataset when data_prefix is None (#314)
Fix a bug about tmp_folder in OpenCVInit (#357)
Fix get_thread_id when not using disk as backend (#354, #357)
Fix the bug of HVU object num_classes from 1679 to 1678 (#307)
Fix typo in export_model.md (#399)
Fix OmniSource training configs (#321)
Fix Issue #306: Bug of SampleAVAFrames (#317)

ModelZoo

Add SlowOnly model for GYM99, both RGB and Flow (#336)
Add auto modelzoo statistics in readthedocs (#327)
Add TSN for HMDB51 pretrained on Kinetics400, Moments in Time and ImageNet. (#372)

v0.8.0 (31/10/2020)

Highlights

Support OmniSource
Support C3D
Support video recognition with audio modality
Support HVU
Support X3D

New Features

Support AVA dataset preparation (#266)
Support the training of video recognition dataset with multiple tag categories (#235)
Support joint training with multiple training datasets of multiple formats, including images, untrimmed videos, etc. (#242)
Support to specify a start epoch to conduct evaluation (#216)
Implement X3D models, support testing with model weights converted from SlowFast (#288)
Support specify a start epoch to conduct evaluation (#216)

Improvements

Set default values of 'average_clips' in each config file so that there is no need to set it explicitly during testing in most cases (#232)
Extend HVU datatools to generate individual file list for each tag category (#258)
Support data preparation for Kinetics-600 and Kinetics-700 (#254)
Use metric_dict to replace hardcoded arguments in evaluate function (#286)
Add cfg-options in arguments to override some settings in the used config for convenience (#212)
Rename the old evaluating protocol mean_average_precision as mmit_mean_average_precision since it is only used on MMIT and is not the mAP we usually talk about. Add mean_average_precision, which is the real mAP (#235)
Add accurate setting (Three crop * 2 clip) and report corresponding performance for TSM model (#241)
Add citations in each preparing_dataset.md in tools/data/dataset (#289)
Update the performance of audio-visual fusion on Kinetics-400 (#281)
Support data preparation of OmniSource web datasets, including GoogleImage, InsImage, InsVideo and KineticsRawVideo (#294)
Use metric_options dict to provide metric args in evaluate (#286)

Bug Fixes

Register FrameSelector in PIPELINES (#268)
Fix the potential bug for default value in dataset_setting (#245)
Fix multi-node dist test (#292)
Fix the data preparation bug for something-something dataset (#278)
Fix the invalid config url in slowonly README data benchmark (#249)
Validate that the performance of models trained with videos have no significant difference comparing to the performance of models trained with rawframes (#256)
Correct the img_norm_cfg used by TSN-3seg-R50 UCF-101 model, improve the Top-1 accuracy by 3% (#273)

ModelZoo

Add Baselines for Kinetics-600 and Kinetics-700, including TSN-R50-8seg and SlowOnly-R50-8x8 (#259)
Add OmniSource benchmark on MiniKineitcs (#296)
Add Baselines for HVU, including TSN-R18-8seg on 6 tag categories of HVU (#287)
Add X3D models ported from SlowFast (#288)

v0.7.0 (30/9/2020)

Highlights

Support TPN
Support JHMDB, UCF101-24, HVU dataset preparation
support onnx model conversion

New Features

Support the data pre-processing pipeline for the HVU Dataset (#277)
Support real-time action recognition from web camera (#171)
Support onnx (#160)
Support UCF101-24 preparation (#219)
Support evaluating mAP for ActivityNet with CUHK17_activitynet_pred (#176)
Add the data pipeline for ActivityNet, including downloading videos, extracting RGB and Flow frames, finetuning TSN and extracting feature (#190)
Support JHMDB preparation (#220)

ModelZoo

Add finetuning setting for SlowOnly (#173)
Add TSN and SlowOnly models trained with OmniSource, which achieve 75.7% Top-1 with TSN-R50-3seg and 80.4% Top-1 with SlowOnly-R101-8x8 (#215)

Improvements

Support demo with video url (#165)
Support multi-batch when testing (#184)
Add tutorial for adding a new learning rate updater (#181)
Add config name in meta info (#183)
Remove git hash in __version__ (#189)
Check mmcv version (#189)
Update url with 'https://download.openmmlab.com' (#208)
Update Docker file to support PyTorch 1.6 and update install.md (#209)
Polish readsthedocs display (#217, #229)

Bug Fixes

Fix the bug when using OpenCV to extract only RGB frames with original shape (#184)
Fix the bug of sthv2 num_classes from 339 to 174 (#174, #207)

v0.6.0 (2/9/2020)

Highlights

Support TIN, CSN, SSN, NonLocal
Support FP16 training

New Features

Support NonLocal module and provide ckpt in TSM and I3D (#41)
Support SSN (#33, #37, #52, #55)
Support CSN (#87)
Support TIN (#53)
Support HMDB51 dataset preparation (#60)
Support encoding videos from frames (#84)
Support FP16 training (#25)
Enhance demo by supporting rawframe inference (#59), output video/gif (#72)

ModelZoo

Update Slowfast modelzoo (#51)
Update TSN, TSM video checkpoints (#50)
Add data benchmark for TSN (#57)
Add data benchmark for SlowOnly (#77)
Add BSN/BMN performance results with feature extracted by our codebase (#99)

Improvements

Polish data preparation codes (#70)
Improve data preparation scripts (#58)
Improve unittest coverage and minor fix (#62)
Support PyTorch 1.6 in CI (#117)
Support with_offset for rawframe dataset (#48)
Support json annotation files (#119)
Support multi-class in TSMHead (#104)
Support using val_step() to validate data for each val workflow (#123)
Use xxInit() method to get total_frames and make total_frames a required key (#90)
Add paper introduction in model readme (#140)
Adjust the directory structure of tools/ and rename some scripts files (#142)

Bug Fixes

Fix configs for localization test (#67)
Fix configs of SlowOnly by fixing lr to 8 gpus (#136)
Fix the bug in analyze_log (#54)
Fix the bug of generating HMDB51 class index file (#69)
Fix the bug of using load_checkpoint() in ResNet (#93)
Fix the bug of --work-dir when using slurm training script (#110)
Correct the sthv1/sthv2 rawframes filelist generate command (#71)
CosineAnnealing typo (#47)

v0.5.0 (9/7/2020)

Highlights

MMAction2 is released

New Features

Support various datasets: UCF101, Kinetics-400, Something-Something V1&V2, Moments in Time, Multi-Moments in Time, THUMOS14
Support various action recognition methods: TSN, TSM, R(2+1)D, I3D, SlowOnly, SlowFast, Non-local
Support various action localization methods: BSN, BMN
Colab demo for action recognition