Hello everyone,
I’m having issues with the sync mode of Engine DJ Desktop for Windows. First, Let me explain my workflow with engineDJ , and then the errors I’ve encountered.
I store my entire music collection on an external HDD, mainly organized by record labels:
LABELS/RECORD_LABEL/[ID]RELEASE.
Every time I buy a new release, I save it to the hard drive. In Engine DJ, I maintain a similar structure and import the folder from the disk into a playlist hierarchy:
LABELS/RECORD_LABEL/[ID]RELEASE.
Once the files are analyzed, I prepare my additional playlists using the imported tracks. These are the playlists I actually use for DJing, under a structure like:
ORDERED_LISTS/my_live_gig.
These playlists are then exported via Sync Manager to one or more USB sticks, which I use on my Denon SC Live 4.
I believe this is a fairly standard workflow. However, as the number of playlists has grown (I have one playlist per label release just for organizing purpose), I’ve noticed that syncing to USB sticks has become increasingly slow. Initially, when my collection had only a few playlists, syncing took seconds. Now, syncing a playlist of around 30–40 tracks can take hours.
I’ve read on the forum in several threads that there is indeed a known issue related to the number of playlists in the collection when syncing and exporting.
That’s the first part of the problem, I’m not sure if there’s a workarround or a better way of working with lots of playlist, but I’m glad to ear about.
Second part: While running tests to improve sync speed—and probably due to not fully understanding how Engine DJ handles its database—I accidentally deleted some playlists from the collection and then synced the external drive. As a result, a large number of playlists were deleted from all devices. Backup restore did not recover them properly, as it did not work as expected.
I lost about a year of music curation, thousands of hours of playlist preparation. Fortunately, the analyzed tracks are still accessible in the database, and I hope to gradually rebuild the playlists without having to reanalyze the files and recreate all cue points, these are still in the DDBB tables.
To prevent this from happening again, I started investigating what went wrong in order to find a safe and correct workflow. During this process, I discovered several interesting things, one of which is likely a bug, It´s described below.
Let’s see; Engine DJ stores a copy of the database on every device connected through the software. For example, if I use a USB stick to add tracks, a copy of the database is created on that stick.
In my case and with my workflow, there is one database on the PC, another on the external HDD, and a third one on the USB stick. The results shown on EngineDJ are a data blend from all sources connected. Let’s look at how Engine DJ and its database behave.
If I create a playlist, it is automatically added to all my three databases: PC, HDD, and USB.
Furthermore, in the Playlist table, a new entry is created with a field called isPersisted, which determines whether the playlist is visible or not in engineDJ. This is important because when a playlist is created, it is immediately added to the databases of all connected devices. This means that a USB stick that has not yet been synced already will contain an entry for that playlist, even if you don’t sync.
If I then remove the USB stick without syncing, and afterwards delete the playlist, it is removed from the databases of the currently connected devices (the PC and the external HDD).
However, when I reconnect the USB stick, Engine DJ automatically merges the Playlist table data from the USB into the other databases, causing a “ghost” entry to appear in all databases, the entry from the deleted playlist wich was not present anymore in the PC neither in the HDD , but still in the USB. And this entry cannot be removed from the tables. The playlist has the isPersisted flag set to 0 in all three databases and does not appear in any collection. The entry remains stuck in the database and cannot be deleted. The database clean function does not clear this field either.
As a result, the Playlist table gradually fills up with broken playlists that are impossible to remove. This process is invisible to the user because the deleted playlists never appear in the UI, even though they remain in the database. In mi case, the Playlist table stored, more than a thousand entries broken (yes I created and removed a lot of playlists). However I think this is a bug related to Playlist management and it would be useful to know whether my sync performance issues are related to this poor handling of playlist data or not.
Thanks for reading and sorry about the brick, I’m glad to ear a better, safe and efficient way to manage my collection.