Compare commits

...

10 Commits

Author SHA1 Message Date
junfeng tu 79bd5e6ac8
Update README.md
Notice! It will never  be updated!
2024-03-11 17:27:54 +08:00
junfeng tu de9d942267
Update README.md
Notice! never update!
2024-03-11 17:27:01 +08:00
junfeng tu 706ede3945
Update README.md 2023-03-01 20:09:40 +08:00
junfeng tu c3cb64cc7a
Update README.md 2022-10-16 15:24:05 +08:00
junfeng tu 5c26972d9c
Update README.md 2022-10-16 15:22:56 +08:00
junfeng tu 97b346ffba
Update README.md 2022-10-16 15:22:27 +08:00
junfeng tu f532ac85cf
Update README.md 2022-10-16 15:19:29 +08:00
junfeng tu faef37dddf
Add files via upload 2022-07-13 21:11:18 +08:00
junfeng tu 30b09ba889
Update README.md 2022-07-04 08:45:40 +08:00
junfeng tu 7e392f417d
Update README.md 2022-07-01 18:00:09 +08:00
2 changed files with 22 additions and 3 deletions

View File

@ -1,10 +1,11 @@
# Differentiable Cross Modal Hashing via Multimodal Transformers
# Differentiable Cross Modal Hashing via Multimodal Transformers [paper](https://dl.acm.org/doi/abs/10.1145/3503161.3548187)
# This project has been moved to [clip-based-cross-modal-hash](https://github.com/kalenforn/clip-based-cross-modal-hash/tree/main)
## Framework
The main architecture of our method.
![framework](./data/structure.jpg)
We propose a selecting mechanism to generate hash code that will transfor the discrete space into a continuous space. Hash code will be encoded as a $2D$ vector.
We propose a selecting mechanism to generate hash code that will transfor the discrete space into a continuous space. Hash code will be encoded as a seires of $2D$ vectors.
![hash](./data/method.jpg)
## Dependencies
@ -24,7 +25,7 @@ then use the "data/make_XXX.py" to generate .mat file
For example:
> cd COCO_DIR # include train val images and annotations files
>
> make mat
> mkdir mat
>
> cp DCMHT/data/make_coco.py mat
>
@ -64,6 +65,18 @@ After the dataset has been prepared, we could run the follow command to train.
## Result
![result](./data/result.png)
## Citation
```
inproceedings{10.1145/3503161.3548187,
author = {Tu, Junfeng and Liu, Xueliang and Lin, Zongxiang and Hong, Richang and Wang, Meng},
title = {Differentiable Cross-Modal Hashing via Multimodal Transformers},
year = {2022},
booktitle = {Proceedings of the 30th ACM International Conference on Multimedia},
pages = {453461},
numpages = {9},
}
```
## Acknowledegements
[CLIP](https://github.com/openai/CLIP)
@ -76,3 +89,9 @@ After the dataset has been prepared, we could run the follow command to train.
[DADH](https://github.com/Zjut-MultimediaPlus/DADH)
[deep-cross-modal-hashing](https://github.com/WangGodder/deep-cross-modal-hashing)
## Apologize:
*2023/03/01*
I find figure 1 with the wrong formula for the vartheta, the right one is the function (10). It has been published, so I can't fix it.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 271 KiB

After

Width:  |  Height:  |  Size: 294 KiB