Background

This post concerns a DRM system used in a proprietary JavaScript-based music player. The music is sequenced locally in the client based on instrument and note data, à la MIDI. The music player does not have any export capabilities, but like the previous instalment in our DRM escapades, it should be possible to leverage the JavaScript player to extract the music data and convert it to MIDI ourselves.

As always, the particular DRM system is not relevant and will not be identified.

First steps, investigating the JavaScript

The music player, as its role would suggest, features a ‘play’ button, which we determine calls into some JavaScript. Tracing the execution, we find a call to the interesting-looking function below:

c.prototype.f = function() {
	var f, d, e = this;
	e.I = 0;
	if (e.Q) {
		e.P += 2.5;
		while (f = e.D.Upto(true, e.I > 0 ? e.P : 9999)) {
			e.H = (f.Time * e.J) + e.N;
			for (d = 0; d < f.Events.length; d++) {
				f.Events[d].Play(e)
			}
		}
	}
};

Although the code is obfuscated, we can make some reasonably-educated guesses about its function. It would be reasonable to surmise that e.P is some kind of counter (probably the time?), various functions are called, and then a for loop loops through various ‘Events’ and plays each individually.

Where do these ‘Events’ come from? The code tells us they come from the Upto function. This function is defined as follows:

i.prototype.Upto = function(o, p) {
	var k = this,
		l = o ? k.a : k.b,
		j = k.X,
		m = null,
		n;
	while (l < j.length && !j[l]) {
		l++
	}
	if (l < j.length) {
		n = k.Y[l];
		if (n <= p) {
			m = j[l++]
		}
	}
	if (o) {
		k.a = l
	} else {
		k.b = l
	}
	return m ? {
		Time: n,
		Events: m
	} : m
};

This time, the obfuscation makes the code significantly harder to determine what exactly is going on (based on the name, perhaps it fetches a list of ‘Events’ ‘up to’ a certain point), but we can still gather some important information by tracing backwards systematically. We can see that the Events come from m, which is derived by indexing j, which is a reference to a k.X array. We also see that some Time-related data comes from n, which is related in some way to k.Y.

Diving in to the data

The preceding snippets are only two small pieces of code in a very large chunk of obfuscated code, and it would take a long time to work through it all. Perhaps we can get a better idea of where to look by examining these referenced data structures.

Using the Firefox Developer Console, we can examine the k.X array, revealing that it is an array of arrays. Each sub-array has a variable number of elements, an example of which is shown below:

[
  Object { Page: 0, Low: 15840, Needed: 0, … },
  Object { Rate: 0.008680555555555556 },
  Object { M: 3280, Low: 6300 },
  Object { P: 2, N: 76, O: 0.5669291338582677, Lasts: 0.6163194444444444, … },
  Object { P: 2, N: 69, O: 0.7244094488188977, Lasts: 0.18229166666666669, … },
  Object { P: 2, N: 61, O: 0.5669291338582677, Lasts: 0.6163194444444444, … }
]

It looks like the data contained within each sub-array is quite heterogeneous. The first one above seems to have something to do with a Page, and the next to do with Rate (tempo?). The third looks obscure, but the final three look quite interesting, with a lot of payload data. In particular, Lasts sounds like something to do with duration, which could indicate a note being played.

What about the k.Y array? This turns out to be a one-dimensional array, with the same number of elements as the k.X array. As an example:

[0, 0.18229166666666669, 0.20833333333333334, 0.390625, …]

Recall that k.Y had something to do with timing, so it seems reasonable to hypothesise that these are time values (in seconds?), and each corresponds with one sub-array in the k.X array, which stores data about music events at that time.

Surmising the purpose of the obfuscated members

Returning to the JavaScript code, we identify the following constructor code relating to the interesting-looking objects (with P, N, O and Lasts variables):

var a = (function() {
	function i(n, m, j, l, o, p) {
		var k = this;
		k.BBox = n;
		k.P = m;
		k.N = j;
		k.O = Math.min(l, 127) / 127;
		k.Lasts = o;
		k.Id = p
	}
	// ...

We know from earlier that a Play function will be called on this object, and looking at the definition of that function in relation to this kind of object, we see it calls the following function:

c.prototype.Note = function(k, m, i, h) { /* P, N, O, Lasts */
	var n = this,
		g = n.H,
		f = n.L[k],
		l = n.Foundry.Font(f).Note(n.Z ? m : m + c.instrumentTranspositions[f]),
		e = c.context,
		d = e.createBufferSource(),
		j = e.createGain();
	j.gain.value = i;
	// ...
	d.buffer = l.Audio;
	// ...
	if (h > l.Lasts + 0.2 && l.From) {
		d.loop = true;
		d.loopStart = l.To;
		d.loopEnd = l.From
	}
	d.connect(j);
	d.start(g, 0, h)
};

The function is actually very long, but the important parts are reproduced above. This looks very promising, there is something to do with notes and instruments, audio buffers, and playing audio.

Working backwards, we can now build some hypotheses about what the P, N, O and Lasts variables refer to.

Lasts (in this function passed as h) seems to have something to do with looping the audio buffer, consistent with our suggestion that this has to do with duration.

O (passed as i) becomes gain.value, straightforwardly suggesting this is a volume parameter.

N (passed as m) is passed as the parameter to a Note function, and optionally has something to do with transposition, so it sounds like this may be to do with the pitch of the note. Indeed, looking at the values of N in the examples (76, 69, 61), these look like quite reasonable values for MIDI pitch numbers (E5, A4 and D4, respectively).

P (passed as k) is used to index an array n.L to derive a value f, which then appears to select a Foundry.Font. The n.L array in this case has the value [54, 54, 0], which look like MIDI instrument (program) numbers (2x Voice Oohs, and Acoustic Grand Piano). It would be reasonable to surmise, then, that n.L is an array with one element for each MIDI channel, specifying the instrument in use, and P is therefore the channel number.

Liberating the data from JavaScript

We can now write a simple script to dump the relevant data so we can process it further into a useable format:

var script = document.createElement('script');
script.src = 'https://cdn.jsdelivr.net/npm/file-saver@2.0.2/dist/FileSaver.min.js';
document.body.appendChild(script);

var data = {
	'D': {'X': player.Q.D.X, 'Y': player.Q.D.Y},
	'L': player.Q.L,
}

var blob = new Blob([JSON.stringify(data)], {type: "application/json;charset=utf-8"});
saveAs(blob, "musicdata.json");

Processing the data into MIDI

We now ‘simply’ need to turn this music information into a MIDI file. To do so, we will make use of the Python Mido library:

import json
import sys

import mido

with open(sys.argv[1], 'r') as f:
	musicdata = json.load(f)

mid = mido.MidiFile(ticks_per_beat=480)

midi_events_by_track = [[] for _ in range(len(musicdata['L']))]

for i, timestamp in enumerate(musicdata['D']['Y']):
	if not musicdata['D']['X'][i]:
		continue
	
	for event in musicdata['D']['X'][i]:
		if 'Rate' in event:
			for midi_events in midi_events_by_track:
				midi_events.append((timestamp, mido.MetaMessage('set_tempo', tempo=mido.bpm2tempo(1/event['Rate']))))
		
		if 'Lasts' in event:
			if event['O'] <= 0:
				continue
			
			midi_events_by_track[event['P']].append((timestamp, mido.Message('note_on', note=event['N'], velocity=int(event['O']*127), channel=event['P'])))
			midi_events_by_track[event['P']].append((timestamp + event['Lasts'], mido.Message('note_off', note=event['N'], velocity=int(event['O']*127), channel=event['P'])))

for i, midi_events in enumerate(midi_events_by_track):
	track = mido.MidiTrack()
	mid.tracks.append(track)
	
	track.append(mido.Message('program_change', program=musicdata['L'][i], channel=i, time=0))
	
	midi_events.sort(key=lambda x: x[0])
	
	tempo = mido.bpm2tempo(120)
	last_timestamp = 0
	
	for timestamp, midi_event in midi_events:
		midi_event.time = int(mido.second2tick(timestamp - last_timestamp, mid.ticks_per_beat, tempo))
		if midi_event.time < 0:
			midi_event.time = 0
		
		track.append(midi_event)
		
		last_timestamp += mido.tick2second(midi_event.time, mid.ticks_per_beat, tempo)
		
		if midi_event.type == 'set_tempo':
			tempo = midi_event.tempo

mid.save(sys.argv[2])

Running the script, we cross our fingers, and we indeed wind up with a MIDI file that correctly represents the music played by the proprietary player!