🚀 Core v2.0.0 - FFmpeg for the era of AI 🎉

🔥 Today we released @diffusionstudio/core@2.0.0, a major update that completely removes PixiJS and Pixi Filters, replacing them with a custom 2D context rendering implementation.

🌟 What's New?

• 💡 Tiny bundle size — Just 49KB gzip, with only one remaining dependency 📦
• ⚡ Faster composition mutations — Fewer abstractions = more speed 🚀
• 🧠 Fixes major GPU memory leaks — Enjoy a more stable experience 🧹💪
• 🌐 Better browser compatibility — Works seamlessly across more environments 🌐

📖 Migration Guide: If you’re upgrading to V2, check out our guide 👉 Migration Guide: V1 to V2

Happy hacking! 🎉

• Implemented ability to insert clips into a stacked track at a given index e.g. track.add(new Clip(), 2)
• Ensured clips within stacked tracks can be split without the track being reordered

Example

const composition = new core.Compostion();
const track = composition.createTrack('base').stacked();

await track.add(new core.Clip({ name: 'foo' });
await track.add(new core.Clip({ name: 'bar' });

// now add a clip in between
await track.add(new core.Clip({ name: 'pong' }), 1);


console.log(track.clips[0].name); // foo
console.log(track.clips[1].name); // pong
console.log(track.clips[2].name); // bar

Implemented a method to efficiently retrieve the waveform of a clip and display it on the timeline.

• Implemented that the currently visible clip will be removed from the scene graph when the parent track gets disabled

• Ensured large videos with lots of cuts can be rendered by demuxing them on demand

• added partial video loading support
• increased test coverage to >90%

Basic usage of the new opus encoder

import { CanvasEncoder } from '@diffusionstudio/core';

const encoder = new OpusEncoder({
  output: (chunk, meta) => {
    // mux
  },
  error: console.error,
});

await encoder.configure({
  numberOfChannels: 1,
  sampleRate: 48000,
});

encoder.encode({
  data: new Int16Array(24000),
  numberOfFrames: 24000,
});

The new opus encoder replaces the WebCodecs AudioEncoder enabling Diffusion Studio to run in all major browsers.

• Integrated Motion Canvas inspired animation API

Example

const text = await composition.add(
  new core.TextClip({
    text: 'Hello World',
    position: 'center',
    fontSize: 34
  })
);

// begin with calling the animate method
text.animate()
  .rotation(243).to(360 * 2, 15) // start at 243 deg, and animate to 2 * 360 deg in 15 frames
  .scale(0.3).to(1, 10) // scale from 0.3 to 1 in 10 frames

• Improved render performance by ~24%
• Implemented new clip lifecycle with the following phases:
  • constructor: invoked first, during the initialization of the clip. This is where the initial state and values should be set up.
  • init: called asynchronously before the Clip is added to a track/composition.
  • enter: triggered right before the Clip is drawn to the canvas.
  • update: called on every redraw of the clip.
  • exit: called after the Clip has been drawn to the canvas for the last time.

Functional Properties

We introduced functional properties for advance animations. Instead of a clip value/ Keyframe you can now assign a function.

Example

await composition.add(
 new core.ImageClip(new File(...), {
   x: (time: core.Timestamp) => time.seconds * 500,
   y: (time: core.Timestamp) => time.seconds * 200,
 })
);

When used in combination with regular functions, the this context will be the Clip itself.