Loops in Database

[Francisco_Vila]

Engine DJ Database - Loops BLOB Format Investigation

Summary

I’m developing a third-party application to read Engine DJ database information and display loops in a waveform visualization. I’ve successfully implemented hotcues parsing using the libdjinterop format (compressed ZLIB, big-endian doubles), but I’m struggling to correctly parse the loops BLOB format.

What I Know

Database Details

• Database: Engine DJ v2 format (m.db)
• Table: PerformanceData
• Column: loops (BLOB type)
• Reference: Using libdjinterop schema as reference ( GitHub - xsco/libdjinterop: C++ library for access to DJ record libraries )

Test Case

• Track: “Gimme Gimme Your Love On Me (Avenore Remix)” by “Abba”
• Loop: Named “pre drop”
• Expected time range: 01:03 to 01:05 (63-65 seconds)
• Expected sample range (at 44.1 kHz): ~2,778,300 to ~2,866,500 samples

BLOB Structure Observed

The loops BLOB is NOT compressed (no ZLIB header 0x78 0x9C), unlike hotcues.

Hexadecimal dump of the Abba track loop BLOB (200 bytes):

Offset  00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F  ASCII
------  -----------------------------------------------  ----------------
0x0000: 08 00 00 00 00 00 00 00 08 70 72 65 20 64 72 6F  .........pre dro
0x0010: 70 00 00 00 80 11 77 45 41 00 00 00 40 91 18 46  p.....wEA...@..F
0x0020: 41 01 01 FF F4 D3 38 00 00 00 00 00 00 00 F0 BF  A.....8.........
0x0030: 00 00 00 00 00 00 F0 BF 00 00 00 00 00 00 00 00  ................
0x0040: 00 00 00 00 00 F0 BF 00 00 00 00 00 00 F0 BF 00  ................
0x0050: 00 00 00 00 00 00 00 00 00 00 00 00 F0 BF 00 00  ................
0x0060: 00 00 00 00 F0 BF 00 00 00 00 00 00 00 00 00 00  ................
0x0070: 00 00 00 F0 BF 00 00 00 00 00 00 F0 BF 00 00 00  ................
0x0080: 00 00 00 00 00 00 00 00 00 00 F0 BF 00 00 00 00  ................
0x0090: 00 00 F0 BF 00 00 00 00 00 00 00 00 00 00 00 00  ................
0x00A0: 00 F0 BF 00 00 00 00 00 00 F0 BF 00 00 00 00 00  ................
0x00B0: 00 00 00 00 00 00 00 00 F0 BF 00 00 00 00 00 00  ................
0x00C0: F0 BF 00 00 00 00 00 00                          ........

Byte pattern F0 BF = -1.0 as double (empty loop marker)

Expected Format (according to libdjinterop)

According to the libdjinterop header files:

// Per loop structure:
- uint8_t: Label length
- char[]: Label string (variable length)
- double (8 bytes): Start sample offset
- double (8 bytes): End sample offset
- uint8_t: Is start set flag
- uint8_t: Is end set flag
- uint8_t[4]: Color (ARGB - alpha, red, green, blue)

My Interpretation

Based on the hex dump, I’m parsing as follows:

• Offset 0x00-0x07: 08 00 00 00 00 00 00 00 = Header (8 loops maximum, as int64)
• Offset 0x08: 08 = Label length (8 bytes)
• Offset 0x09-0x10: 70 72 65 20 64 72 6F 70 = “pre drop” (8 bytes)
• Offset 0x11: 00 = Null terminator
• Offset 0x12-0x19: 00 00 80 11 77 45 41 00 = Start sample offset (8 bytes)?
• Offset 0x1A-0x21: 00 00 40 91 18 46 41 01 = End sample offset (8 bytes)?
• Offset 0x22: 01 = Is start set flag
• Offset 0x23: 01 = Is end set flag
• Offset 0x24-0x27: FF F4 D3 38 = Color (ARGB)
• Offset 0x28+: Multiple -1.0 doubles (empty loops 2-8)

The Problem

I cannot correctly decode the start/end sample offsets. I’ve tried:

1. Little-endian double (8 bytes)

Bytes: 00 00 80 11 77 45 41 00
Result: 1.9215012566983543E-307 (essentially 0)

2. Big-endian double (8 bytes, like hotcues)

Bytes reversed: 00 41 45 77 11 80 00 00
Result: ~0 (still incorrect)

3. Float (4 bytes, various offsets)

Bytes: 80 11 77 45
Result: 3953.094 samples = 0.09 seconds (not 63 seconds)

Bytes: 77 45 41 00
Result: 5.994227E-39 (incorrect)

4. Int64/UInt64 (8 bytes)

Little-endian: 18,372,251,183,218,688 (too large)
Big-endian: 140,812,503,826,688 (still too large)

Questions for the Community

1. Is the libdjinterop documentation correct for loops? Should loops use the same format as documented in their header files?

2. Are loops stored differently than hotcues? Hotcues are ZLIB-compressed and use big-endian doubles for sample positions. Loops appear uncompressed - do they use a different encoding?

3. What is the correct byte interpretation? Looking at the bytes 00 00 80 11 77 45 41 00, how should these be decoded to get approximately 2,778,300 samples (63 seconds at 44.1 kHz)?

4. Is there padding or a different structure? I notice 41 appears multiple times (offsets 0x18 and 0x20), which is common in IEEE-754 float representations. Could the structure be different than documented?

5. Has anyone successfully parsed loops from Engine DJ v2 databases? Any working code examples or documentation would be greatly appreciated!

Additional Context

• Hotcues work perfectly using ZLIB decompression + big-endian doubles
• Other fields work correctly: track metadata, BPM, key, playlists all parse correctly
• Environment: C# application reading SQLite database directly
• Goal: Display loop markers in a waveform visualization similar to Engine DJ

What I Need

The specific byte-to-number conversion formula/method for the start and end sample offsets in the loops BLOB. Even a single working example would help me understand the correct format!

Thank you for any help or insights!

---

Repository reference: GitHub - xsco/libdjinterop: C++ library for access to DJ record libraries Specific file: include/djinterop/engine/v2/loops_blob.hpp

[Pasha]

@Francisco_Vila

display loops in a waveform visualization

Where…. are you trying to display loops

[Francisco_Vila]

As I stated at the beginning of my post, I am trying to graphically display the loops in a wayform display developed in C#

[Pretski]

Maybe you can try chatgpt to convert C++ andgive you some hints like this :

long corresponds to C++ int64_t.

• uint corresponds to C++ uint32_t.

• Strings are encoded in UTF-8, prefixed by their length (standard, portable pattern).

• The ToBlob() and FromBlob() methods mirror binary serialization of primitive fields.

• ToString() matches your C++ operator<< output.

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;

public struct LoopsBlob : IEquatable<LoopsBlob>
{
    /// <summary>
    /// List of loops.
    /// </summary>
    public List<LoopBlob> Loops { get; set; }

    /// <summary>
    /// Extra data (if any) found in a decoded blob.
    /// </summary>
    public List<byte> ExtraData { get; set; }

    /// <summary>
    /// Encode this struct into binary blob form.
    /// </summary>
    /// <returns>Returns a byte array.</returns>
    public byte[] ToBlob()
    {
        using var ms = new MemoryStream();
        using var bw = new BinaryWriter(ms);

        // Example encoding (you’ll want to define LoopBlob’s binary serialization logic)
        bw.Write(Loops.Count);
        foreach (var loop in Loops)
        {
            var loopBytes = loop.ToBlob(); // assume LoopBlob has ToBlob()
            bw.Write(loopBytes.Length);
            bw.Write(loopBytes);
        }

        bw.Write(ExtraData.Count);
        bw.Write(ExtraData.ToArray());

        return ms.ToArray();
    }

    /// <summary>
    /// Decode an instance of this struct from binary blob form.
    /// </summary>
    /// <param name="blob">Binary blob.</param>
    /// <returns>Returns a decoded instance of this struct.</returns>
    public static LoopsBlob FromBlob(byte[] blob)
    {
        var result = new LoopsBlob
        {
            Loops = new List<LoopBlob>(),
            ExtraData = new List<byte>()
        };

        using var ms = new MemoryStream(blob);
        using var br = new BinaryReader(ms);

        int loopCount = br.ReadInt32();
        for (int i = 0; i < loopCount; i++)
        {
            int len = br.ReadInt32();
            var loopBytes = br.ReadBytes(len);
            result.Loops.Add(LoopBlob.FromBlob(loopBytes)); // assume LoopBlob has FromBlob()
        }

        int extraLen = br.ReadInt32();
        result.ExtraData.AddRange(br.ReadBytes(extraLen));

        return result;
    }

    public bool Equals(LoopsBlob other)
    {
        return Loops.SequenceEqual(other.Loops) &&
               ExtraData.SequenceEqual(other.ExtraData);
    }

    public override bool Equals(object obj) => obj is LoopsBlob other && Equals(other);

    public override int GetHashCode()
    {
        unchecked
        {
            int hash = 17;
            foreach (var loop in Loops)
                hash = hash * 31 + loop.GetHashCode();
            foreach (var b in ExtraData)
                hash = hash * 31 + b.GetHashCode();
            return hash;
        }
    }

    public static bool operator ==(LoopsBlob lhs, LoopsBlob rhs) => lhs.Equals(rhs);
    public static bool operator !=(LoopsBlob lhs, LoopsBlob rhs) => !lhs.Equals(rhs);

    public override string ToString()
    {
        var loopsStr = string.Join(", ", Loops);
        return $"LoopsBlob{{loops=[{loopsStr}]}}";
    }
}

Notes: LoopBlob should also define ToBlob() and FromBlob() for binary serialization. List is used instead of std::vectorstd::byte, but you can easily switch to byte if you prefer fixed arrays. Equality and ToString() mimic the C++ friend operator overloads.

Hope this helps ?

[Pasha]

@Francisco_Vila

I am trying to graphically display the loops in a wayform display

Yessss but…. On what? A computer monitor? A projector? And to who are you trying to display it to?

[Francisco_Vila]

The application will run on a computer and be displayed on a monitor. But that’s not what’s important; I’m interested in knowing how it stores the loop information in the database so I can read it and represent it graphically in the application.

[MrWilks]

@djliquidice has done some cool stuff involving waveforms from what I remember?

[Francisco_Vila]

@MrWilks

has done some cool stuff involving waveforms from what I remember?

I need to understand how the loop information for songs is stored in the database (m.db) used by Engine DJ. I’ve already figured out the waveform display and cue points, as you can see in the image.

[djliquidice]

What you are looking for is in libdjinterop. I was able to display loop info easily.

	// LOOPS
	nlohmann::json loopsJSON = nlohmann::json::array();
	std::vector<std::optional<djinterop::loop>> loops = track.loops();

	for (auto& loop : loops) {
		nlohmann::json loopJSON = nlohmann::json::object();

		// Debug from here
		if (loop.has_value()) {
			djinterop::loop innerLoop = loop.value();

			std::string label = innerLoop.label;
			double sso = loop.value().start_sample_offset;
			double eso = loop.value().end_sample_offset;

			djinterop::pad_color color = loop.value().color;

			nlohmann::json pad_colorJSON = nlohmann::json::array();
			pad_colorJSON.push_back(color.r);
			pad_colorJSON.push_back(color.g);
			pad_colorJSON.push_back(color.b);
			pad_colorJSON.push_back(color.a);

			loopJSON["label"] = label;
			loopJSON["start_sample_offset"] = sso;
			loopJSON["end_sample_offset"] = eso;
			loopJSON["colors"] = pad_colorJSON;
			// nlohmann::json loopJSON =
		}


		loopsJSON.push_back(loopJSON);
	}
	resultObj["loops"] = loopsJSON;