Compare commits
10 Commits
694ea9015d
...
79bd5e6ac8
| Author | SHA1 | Date |
|---|---|---|
|
|
79bd5e6ac8 | |
|
|
de9d942267 | |
|
|
706ede3945 | |
|
|
c3cb64cc7a | |
|
|
5c26972d9c | |
|
|
97b346ffba | |
|
|
f532ac85cf | |
|
|
faef37dddf | |
|
|
30b09ba889 | |
|
|
7e392f417d |
25
README.md
25
README.md
|
|
@ -1,10 +1,11 @@
|
|||
# Differentiable Cross Modal Hashing via Multimodal Transformers
|
||||
# Differentiable Cross Modal Hashing via Multimodal Transformers [paper](https://dl.acm.org/doi/abs/10.1145/3503161.3548187)
|
||||
# This project has been moved to [clip-based-cross-modal-hash](https://github.com/kalenforn/clip-based-cross-modal-hash/tree/main)
|
||||
|
||||
## Framework
|
||||
The main architecture of our method.
|
||||

|
||||
|
||||
We propose a selecting mechanism to generate hash code that will transfor the discrete space into a continuous space. Hash code will be encoded as a $2D$ vector.
|
||||
We propose a selecting mechanism to generate hash code that will transfor the discrete space into a continuous space. Hash code will be encoded as a seires of $2D$ vectors.
|
||||

|
||||
|
||||
## Dependencies
|
||||
|
|
@ -24,7 +25,7 @@ then use the "data/make_XXX.py" to generate .mat file
|
|||
For example:
|
||||
> cd COCO_DIR # include train val images and annotations files
|
||||
>
|
||||
> make mat
|
||||
> mkdir mat
|
||||
>
|
||||
> cp DCMHT/data/make_coco.py mat
|
||||
>
|
||||
|
|
@ -64,6 +65,18 @@ After the dataset has been prepared, we could run the follow command to train.
|
|||
## Result
|
||||

|
||||
|
||||
## Citation
|
||||
```
|
||||
inproceedings{10.1145/3503161.3548187,
|
||||
author = {Tu, Junfeng and Liu, Xueliang and Lin, Zongxiang and Hong, Richang and Wang, Meng},
|
||||
title = {Differentiable Cross-Modal Hashing via Multimodal Transformers},
|
||||
year = {2022},
|
||||
booktitle = {Proceedings of the 30th ACM International Conference on Multimedia},
|
||||
pages = {453–461},
|
||||
numpages = {9},
|
||||
}
|
||||
```
|
||||
|
||||
## Acknowledegements
|
||||
[CLIP](https://github.com/openai/CLIP)
|
||||
|
||||
|
|
@ -76,3 +89,9 @@ After the dataset has been prepared, we could run the follow command to train.
|
|||
[DADH](https://github.com/Zjut-MultimediaPlus/DADH)
|
||||
|
||||
[deep-cross-modal-hashing](https://github.com/WangGodder/deep-cross-modal-hashing)
|
||||
|
||||
|
||||
## Apologize:
|
||||
*2023/03/01*
|
||||
|
||||
I find figure 1 with the wrong formula for the vartheta, the right one is the function (10). It has been published, so I can't fix it.
|
||||
|
|
|
|||
BIN
data/result.png
BIN
data/result.png
Binary file not shown.
|
Before Width: | Height: | Size: 271 KiB After Width: | Height: | Size: 294 KiB |
Loading…
Reference in New Issue