Skip to content

Conversation

@sfallah
Copy link
Contributor

@sfallah sfallah commented Nov 20, 2025

Feature Request: #16676

Make sure to read the contributing guidelines before submitting a PR

@github-actions github-actions bot added model Model specific examples python python script changes labels Nov 20, 2025
@sfallah sfallah marked this pull request as draft November 20, 2025 09:12
@sfallah
Copy link
Contributor Author

sfallah commented Nov 21, 2025

@Acly
I thought it might interest you!
I have implement add_rel_pos_inplace and get_rel_pos (for DeepseekOCR SAM-Vit) without using the cpu only ggml ops (ggml_get_rel_pos etc.) and it works on Metal.
I will also test on CUDA tomorrow, and hopefully the CI checks will later show if the implementation also works for other devices.

You will find the code here:
https://github.com/sfallah/llama.cpp/blob/sf/deepseek-ocr/tools/mtmd/clip.cpp#L2473
https://github.com/sfallah/llama.cpp/blob/sf/deepseek-ocr/tools/mtmd/clip.cpp#L2502

Disclaimer:
My implementation is very rudimentary but @bluebread reviewed and already improved it a bit in separate branch.
I have been using ChatGPT to translate from original pytorch code but have debugged the code and got it to run.
The functions have also been tested with and compared with ggml/examples/sam (sam example from ggml repo), the results align well.

bluebread and others added 5 commits November 22, 2025 02:09
Fixed get_rel_pos & add_rel_pos_inplace operator
- issues fixed mainly related wrong config like n_patches etc.
- configs need to be corrected in the converter
@ngxson
Copy link
Collaborator

ngxson commented Nov 22, 2025

I have implement add_rel_pos_inplace and get_rel_pos (for DeepseekOCR SAM-Vit) without using the cpu only ggml ops (ggml_get_rel_pos etc.) and it works on Metal.

Just wondering, which ops are currently CPU-only (apart from ggml_arange?)

Beside, it's better introduce new op in a follow-up PR, to avoid adding too much complexity to the review process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

examples model Model specific python python script changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants