Telegram Group & Telegram Channel
LLMs can see and hear without any training

30 Jan 2025 · Kumar Ashutosh, Yossi Gandelsman, Xinlei Chen, Ishan Misra, Rohit Girdhar ·

We present MILS: Multimodal Iterative LLM Solver, a surprisingly simple, training-free approach, to imbue multimodal capabilities into your favorite LLM. Leveraging their innate ability to perform multi-step reasoning, MILS prompts the LLM to generate candidate outputs, each of which are scored and fed back iteratively, eventually generating a solution to the task. This enables various applications that typically require training specialized models on task-specific data. In particular, we establish a new state-of-the-art on emergent zero-shot image, video and audio captioning. MILS seamlessly applies to media generation as well, discovering prompt rewrites to improve text-to-image generation, and even edit prompts for style transfer! Finally, being a gradient-free optimization approach, MILS can invert multimodal embeddings into text, enabling applications like cross-modal arithmetic.

Paper: https://arxiv.org/pdf/2501.18096v1.pdf

Code: https://github.com/facebookresearch/mils

https://www.tg-me.com/deep_learning_proj
Please open Telegram to view this post
VIEW IN TELEGRAM



tg-me.com/LLM_learning/33
Create:
Last Update:

LLMs can see and hear without any training

30 Jan 2025 · Kumar Ashutosh, Yossi Gandelsman, Xinlei Chen, Ishan Misra, Rohit Girdhar ·

We present MILS: Multimodal Iterative LLM Solver, a surprisingly simple, training-free approach, to imbue multimodal capabilities into your favorite LLM. Leveraging their innate ability to perform multi-step reasoning, MILS prompts the LLM to generate candidate outputs, each of which are scored and fed back iteratively, eventually generating a solution to the task. This enables various applications that typically require training specialized models on task-specific data. In particular, we establish a new state-of-the-art on emergent zero-shot image, video and audio captioning. MILS seamlessly applies to media generation as well, discovering prompt rewrites to improve text-to-image generation, and even edit prompts for style transfer! Finally, being a gradient-free optimization approach, MILS can invert multimodal embeddings into text, enabling applications like cross-modal arithmetic.

Paper: https://arxiv.org/pdf/2501.18096v1.pdf

Code: https://github.com/facebookresearch/mils

https://www.tg-me.com/deep_learning_proj

BY Github LLMs




Share with your friend now:
tg-me.com/LLM_learning/33

View MORE
Open in Telegram


telegram Telegram | DID YOU KNOW?

Date: |

Dump Scam in Leaked Telegram Chat

A leaked Telegram discussion by 50 so-called crypto influencers has exposed the extraordinary steps they take in order to profit on the back off unsuspecting defi investors. According to a leaked screenshot of the chat, an elaborate plan to defraud defi investors using the worthless “$Few” tokens had been hatched. $Few tokens would be airdropped to some of the influencers who in turn promoted these to unsuspecting followers on Twitter.

Telegram Auto-Delete Messages in Any Chat

Some messages aren’t supposed to last forever. There are some Telegram groups and conversations where it’s best if messages are automatically deleted in a day or a week. Here’s how to auto-delete messages in any Telegram chat. You can enable the auto-delete feature on a per-chat basis. It works for both one-on-one conversations and group chats. Previously, you needed to use the Secret Chat feature to automatically delete messages after a set time. At the time of writing, you can choose to automatically delete messages after a day or a week. Telegram starts the timer once they are sent, not after they are read. This won’t affect the messages that were sent before enabling the feature.

telegram from us


Telegram Github LLMs
FROM USA