TomboFry

Last week, I uploaded the first YouTube video of a chiptune cover in a completely different style in 7 years (and even then that was one single exception - otherwise, it's been more like 10!). During this time, I've been uploading music via screen recordings of FL Studio. The reason I did this for so long is that growing up, I always loved to see how people made songs, and loved seeing all the notes and recordings scroll by as the music played out. I wanted people to experience the same fascination I did, even if you might consider the method to be a little cheap/lazy from a production point of view (as opposed to making music videos, animations, or other artwork to display on screen instead).

Hopefully, the new video style captures the same idea as it did before - I still wanted people to be able to see what each channel looks like, and now this is done by displaying the waveforms of each track instead. This is not to say that I plan on moving away from FL Studio any time soon - that would be silly given how much time I've spent attempting to master it (har har, music production pun) - but I suppose I want a unique visual style that I've created myself, and something that feels familiar too.

How does it work?

I've done this by creating a piece of software from scratch, which takes a bunch of audio files you provide, along with a label and a colour, and loads the waveforms and displays them on the screen. The video is also pixelated by default, which matches the chiptune aesthetic. If you want to go even more in-depth, I've got a GitHub repository set up online, where you can view the source code and use it for yourself.

First, you'll need a song.json file. This contains all the tracks, their labels, and their colours. You can also specify if you want to align each waveform display to the start of notes, so they look nicer when playing melodies (but I've found that it doesn't look good for bass notes or drums).

Once that's set up, the software will attempt to load each of the audio files you've specified in song.json, and see if it can decode it. If it can, it'll start loading enough samples from each track to fill the duration of at least one frame on screen (which is usually about 700 samples when played at 30 frames per second). Next, if skip over some details of the extra calculations it performs (for alignment and such), it will essentially loop through each sample and draw a pixel for whatever amplitude that sample has at that particular time throughout the frame. For big jumps in volume, it'll also draw a line from one sample to another, so it looks like one continuous waveform.

At the end of each frame, the software will resize the image to scale we specify (4x by default), and render that to the end of a video file. This is actually the slowest part of the software, and I'm looking to optimise this if possible, because currently it takes just about as long to render the video as it does to play the song.

When any of the audio tracks have run out samples to load, the software will be notified and the video will finish rendering. While all of this is happening, it displays a progress bar on the screen which is estimated based on the total duration of the first audio track loaded in.

Anything to add?

Well, since I uploaded the first video in this style, I've added the ability to use gradient backgrounds, which I think look a lot nicer than the flat colours you see in the image at the top, although I genuinely don't think there's anything else to add.

Something I would like to attempt is loading an FL Studio project file, taking all the note data for each associated track, and display them on screen somehow, but - a) that would essentially require creating a new library from scratch which parses FLP files and all their note data, and b) tie me in to FL Studio forever which, again, is not ever likely to change, but the current method literally just uses audio files - I appreciate the simplicity.

What does it look like?

After all that, if you'd like to see what it actually looks like in motion, here's the video I uploaded - I hope you like it!