We present Audio Flamingo 3 (AF3), a fully open state-of-the-art (SOTA) large audio-language model that advances reasoning and understanding across speech, sound, and music. AF3 introduces: (i) AF-Whisper, a unified audio encoder trained using a novel strategy for joint representation learning across all 3 modalities of speech, sound, and music; (ii) flexible, on-demand thinking, allowing the model to do chain-of-thought-type reasoning before answering; (iii) multi-turn, multi-audio chat; (iv) long audio understanding and reasoning (including speech) up to 10 minutes; and (v) voice-to-voice interaction. To enable these capabilities, we propose several large-scale training datasets curated using novel strategies, including AudioSkills-XL, LongAudio-XL, AF-Think, and AF-Chat, and train AF3 with a novel five-stage curriculum-based training strategy. Trained on only open-source audio data, AF3 achieves new SOTA results on over 20+ (long) audio understanding and reasoning benchmarks, surpassing both open-weight and closed-source models trained on much larger datasets.
I think I used a bit too much sarcasm. I wanted to take a spin on the idea how the AI industry simultaneously uses copyright, and finds ways to “circumvent” the traditional copyright that was written before we had large language models. An AI is neither a telephone-book, nor should every transformative work be Fair Use, no questions asked. And this isn’t really settled as of now. We likely need some more court cases and maybe a few new laws. But you’re right, law is complicated, there is a lot of nuance to it and it depends on jurisdiction.
There is a lot of disinformation being spread on copyright because major rights-holders hope to gain a lot of money for nothing.
US fair use has always worked like this. Other countries without fair use had to make laws to enable AI training. I know about Japan and the EU.
It is precisely because of these new laws that AI training in the EU is possible at all (eg by Mistral AI or by various universities/research institutions). But because of lobbying by rights-holders, this is quite limited. It’s not possible to train AIs in the EU that are as capable as those from the US, where Fair Use comes directly from the constitution and can’t be easily lobbied aside by monied interests.
I think I used a bit too much sarcasm. I wanted to take a spin on the idea how the AI industry simultaneously uses copyright, and finds ways to “circumvent” the traditional copyright that was written before we had large language models. An AI is neither a telephone-book, nor should every transformative work be Fair Use, no questions asked. And this isn’t really settled as of now. We likely need some more court cases and maybe a few new laws. But you’re right, law is complicated, there is a lot of nuance to it and it depends on jurisdiction.
There is a lot of disinformation being spread on copyright because major rights-holders hope to gain a lot of money for nothing.
US fair use has always worked like this. Other countries without fair use had to make laws to enable AI training. I know about Japan and the EU.
It is precisely because of these new laws that AI training in the EU is possible at all (eg by Mistral AI or by various universities/research institutions). But because of lobbying by rights-holders, this is quite limited. It’s not possible to train AIs in the EU that are as capable as those from the US, where Fair Use comes directly from the constitution and can’t be easily lobbied aside by monied interests.