Today's selected cutting-edge AI news, welcome to read 👇
🎥 Runway launches Gen-3 Alpha video-to-video feature, transforming videos into any style with one click, and adding special effects!
🎬 Alibaba Tongyi Wanxiang releases a new AI video generation model with multi-style cinematic quality, supporting multiple languages and variable resolutions!
🎨 Alibaba's EchoMimic introduces the ComfyUI plugin, easily generating lip-synced character portrait animation videos!
🖌️ Alibaba open-sources the Flux local redrawing ControlNet, allowing precise image editing with simple strokes!
✂️ AI YouTube Shorts Generator automatically clips exciting moments from long videos to generate engaging short videos, enhancing creative efficiency!
Latest News
1. Runway launches Gen-3 Alpha video-to-video feature, converting any style with one click.
This feature can convert the style of a video to any desired style with stable visuals. You can even add special effects to the video.
Try it online: https://app.runwayml.com
Currently available only to paid members. As shown in the example below, you can freely change the environment, season, weather, and even add snow effects to the video.
2. Alibaba Tongyi Wanxiang releases a new AI video generation model.
Apply for a trial: https://tongyi.aliyun.com/wanxiang/wanxvideo
- Powerful dynamic visual generation capability;
- Skilled in conceptual understanding and compositional generation, easily mastering various artistic styles;
- Optimized performance for Chinese elements, delivering cinematic-quality visuals;
- Supports multiple languages and variable resolution generation.
AI Art
1. Alibaba’s audio-to-video project EchoMimic now has a ComfyUI plugin.
You only need to provide a character image and audio to generate a realistic and lip-synced character portrait animation video. Additionally, with adjustable markers, the animation can be made more vivid and natural.
GitHub: https://github.com/smthemex/ComfyUI_EchoMimic
2. Flux local redrawing ControlNet.
Alibaba open-sourced a local redrawing ControlNet based on Flux. You can paint over the areas to be modified, input prompts, and make precise changes.
GitHub: https://github.com/alimama-creative/FLUX-Controlnet-Inpainting
Below is a demonstration from left to right: input image --> masked image --> SDXL repair --> FLUX repair
Open Source Projects
1. An automated tool for clipping exciting parts of long videos and generating short videos: AI YouTube Shorts Generator.
The main function is to extract interesting and engaging clips from long videos, then edit, add background music, generate titles and descriptions, and finally produce new short videos.
GitHub: https://github.com/SamurAIGPT/AI-Youtube-Shorts-Generator
Workflow:
- Video download: Given a YouTube video link, the tool automatically downloads the video;
- Transcription: Uses Whisper to transcribe the video content;
- Highlight extraction: Utilizes GPT-4 to identify the most engaging parts of the video;
- Speaker detection: Detects speakers in the video;
- Vertical clipping: Clips the highlights into vertical videos and creates short videos.
Personally, I think this is quite suitable for automating the clipping of speeches by celebrities. If you're interested, feel free to explore it.
