doodlechaos

Black MIDI, but every note spawns a physics object... (4k, 60fps, ray traced)

Added 2021-07-07 18:49:03 +0000 UTC

I created a program in Unity that can read Black MIDI files and generate a Synthesia like visualization. However, each time a piano key is pressed, a corresponding tetromino physics object is spawned (since it plays the Tetris Theme A). The high note density areas created some interesting effects.

I had a simpler version of this almost done, but for some reason I always feel the need to do a bunch of extra modifications that people probably won't even notice (60fps + 4k, I still don't have a 4k monitor, so I don't even know what it truly looks like, lol). After figuring out how to use Unity's HDRP, I couldn't resist setting up path tracing. Unfortunately, the Unity Recorder package isn't capable of waiting for the ray tracing paths to converge before moving to the next frame yet. To circumvent this, my code paused the simulation for about 200 frames for path convergence stalling, took a 4k screenshot, then repeated the process for about 72 hours straight. I was very sad when a bug in my code glitched out the physics at the very end of the simulation. Surprisingly, after fixing it and skipping a bunch of frames to set up a pick up shot where it failed, the physics behaved deterministically so the transition was unnoticeable.

Keep a lookout for your name on the outro screen. The blocks are ordered and sized in terms of lifetime contribution, as well as color coded by tier. I'm not making any money off this video, so thank you for the support!

Best,

- Mark