mayu-ot / Rethinking Evs
Labels
Projects that are alternatives of or similar to Rethinking Evs
rethinking-evs
Scripts of our CVPR'19 paper "Rethinking the Evaluation of Video Summaries" [arXiv]
Setup
- Create an environment.
$ conda env create -f environment.yml
- Activate the new environment.
$ conda activate vsum_eval
Data
SumME
The data can be downloaded from the project page.
Copy the files in GT/
to data/raw/summe/GT/
.
TVSum
Follow the steps described in the TVSum Github page.
Copy ydata-tvsum50.mat
to data/raw/tvsum/
Optional: For evaluate video summaries using KTS segmentation, we use KTS segmentation results provided here.
Download shot_SumMe.mat
and shot_TVSum.mat
and copy it to data/raw/summe(or tvsum)/
.
Project Organization
.
├── AUTHORS.md
├── LICENSE
├── README.md
├── data
│ ├── interim
│ ├── processed
│ └── raw # please see "Data" description above
│ ├── summe
│ │ └── GT/
│ │ └── shot_SumMe.mat
│ ├── tvsum
│ │ └── ydata-tvsum50.mat
│ │ └── shot_TVSum.mat
│ └── example.json
├── notebooks
└── src
Evaluate your video summaries on SumMe
We provide an evaluation script that also computes baseline scores with 100 trials. For evaluating your own video summaries on SumMe, please use the following format and save the results in a JSON file.
{
"video name":
{
'summary': [x1, x2, ... xn] # frame-level 0/1 labels
'segment': [s1, s2, ... sm] # segmentation results
},
...
}
xi=1 when i-th frame is in an output summary, otherwise 0.
s1, s2, ... sm are indices of frames corresponding to shot boundaries.
An example is in data/raw/example.json
.
To evaluate the summarization results, run src/summe_eval.py
as
python src/summe_eval.py path/to/json_file
The evaluation results will be saved to data/processed/json_file.eval.csv
.