Noagarcia/context-art-classification - GitHub

Skip to content Dismiss alert {{ message }} noagarcia / context-art-classification Public

• Notifications You must be signed in to change notification settings
• Fork 4
• Star 22

Automatic art classification with deep learning and knowledge graphs.

22 stars 4 forks Branches Tags Activity Star Notifications You must be signed in to change notification settings

• Code
• Issues 2
• Pull requests
• Actions
• Projects
• Security
• Insights

Additional navigation options

noagarcia/context-art-classification  masterBranchesTagsGo to fileCodeOpen more actions menu

Folders and files

Name		Name	Last commit message	Last commit date
Latest commit   History 10 Commits10 Commits
Data		Data
examples		examples
Readme.md		Readme.md
attributes.py		attributes.py
dataloader_kgm.py		dataloader_kgm.py
dataloader_mtl.py		dataloader_mtl.py
main.py		main.py
model_kgm.py		model_kgm.py
model_mtl.py		model_mtl.py
params.py		params.py
test.py		test.py
train.py		train.py
utils.py		utils.py
View all files

Repository files navigation

• README

2022/01/05: Downloading links for the pre-trained models have been updated. Sorry for the wait.

Context Embeddings for Art Classification

Pytorch code for the classification part of our ICMR 2019 paper Context-Aware Embeddings for Automatic Art Analysis. For the retrieval part, check this other repository.

Setup

1. Download dataset from here.

2. Clone the repository:

   git clone https://github.com/noagarcia/context-art-classification.git

3. Install dependencies:

   • Python 2.7
   • pytorch (conda install pytorch=0.4.1 cuda90 -c pytorch)
   • torchvision (conda install torchvision)
   • visdom (check tutorial here)
   • pandas (conda install -c anaconda pandas)
   • gensim (conda install -c anaconda gensim)

4. For the KGM model, download the pre-computed graph embeddings from here, and save the file into the Data/ directory.

Train

• To train MTL multi-classifier run:

  python main.py --mode train --model mtl --dir_dataset $semart

• To train KGM classifier run:

  python main.py --mode train --model kgm --att $attribute --dir_dataset $semart

Where $semart is the path to SemArt dataset and $attribute is the classifier type (i.e. type, school, time, or author).

Test

• To test MTL multi-classifier run:

  python main.py --mode test --model mtl --dir_dataset $semart

• To test KGM classifier run:

  python main.py --mode test --model kgm --att $attribute --dir_dataset $semart --model_path $model-file

Where $semart is the path to SemArt dataset, $attribute is the classifier type (i.e. type, school, time, or author), and $model-file is the path to the trained model.

You can download our pre-trained models from:

• MTL
• KGM Type
• KGM School
• KGM Timeframe
• KGM Author

Results

Classification results on SemArt:

Model	Type	School	Timeframe	Author
VGG16 pre-trained	0.706	0.502	0.418	0.482
ResNet50 pre-trained	0.726	0.557	0.456	0.500
ResNet152 pre-trained	0.740	0.540	0.454	0.489
VGG16 fine-tuned	0.768	0.616	0.559	0.520
ResNet50 fine-tuned	0.765	0.655	0.604	0.515
ResNet152 fine-tuned	0.790	0.653	0.598	0.573
ResNet50+Attributes	0.785	0.667	0.599	0.561
ResNet50+Captions	0.799	0.649	0.598	0.607
MTL context-aware	0.791	0.691	0.632	0.603
KGM context-aware	0.815	0.671	0.613	0.615

Examples

Paintings with the highest scores for each class:

Citation

@InProceedings{Garcia2017Context, author = {Noa Garcia and Benjamin Renoust and Yuta Nakashima}, title = {Context-Aware Embeddings for Automatic Art Analysis}, booktitle = {Proceedings of the ACM International Conference on Multimedia Retrieval}, year = {2019}, }

About

Automatic art classification with deep learning and knowledge graphs.

Topics

art computer-vision classification

Resources

Readme

Uh oh!

There was an error while loading. Please reload this page.

Activity

Stars

22 stars

Watchers

1 watching

Forks

4 forks Report repository

Releases

No releases published

Packages

No packages published

Languages

• Python 100.0%

You can’t perform that action at this time.