This is an official implementation of the ECCV 2022 paper AutoTransition: Learning to Recommend Video Transition Effects.
We release the videos with annotated transitions extracted from the video editing template on online video editing platforms. The dataset can be downloaded from here: HF Dataset
To speed up the training, we convert videos to JPEG image and extract audio features before training. Run the following commands to finish these steps:
python3 tools/convert_video_folder.py ./path/to/template_root
python3 tools/extract_audio_features.py ./path/to/template_root path/to/annotation.json --model_path /path/to/audio/model.pth --cudaThe pretrained Harmonic CNN model could be downloaded from this link.
To train transition embeddings:
python3 tools/run_net.py --cfg configs/base/train_transition_embedding.yaml \
DATASET.TRANSITION_CLASSIFICATION.JSON_ANNOTATION /path/to/annotation.json \
DATASET.TRANSITION_CLASSIFICATION.TEMPLATE_ROOT /path/to/template_rootThe transition embeddings can be found in ./log directory after training.
To train transition recommendation:
python3 tools/run_net.py --cfg configs/base/train_transition_recommendation.yaml \
MODEL.TRANSITION_TRANSFORMER.EMBEDDING.PRETRAINED_EMBEDDING /path/to/pretrained/transition/embedding.pth \
DATASET.TRANSITION_DATASET.JSON_ANNOTATION /path/to/annotation.json \
DATASET.TRANSITION_DATASET.TEMPLATE_ROOT /path/to/template_root
tensorboard --logdir=./log