Database corrupt

After accessing a specific playlist on the SC live 4 i got the message my database got corrupted.

Engine DJ 5.0.1

My database is on a Samsung T7 shield. All music is stored in folders and these haven’t been moved or altered. I can acces the music files and the drive doesnt have any faults.

Did a library backup from 5 days ago but this also seems corrupted.

I can see Playlist but not all sublists and none of them contain tracks.

Any help would be apreciated

Greetz

Hi @GILA ,

Sorry this happened to you.

Is your T7 Shield the primary location for music or is it a performance volume? If your music is stored primarily on your computer, do you have a real backup from something like TimeMachine or Acronis perhaps?

If your music is primarily stored on the Shield, a “library backup” (in quotes because it has limited functionality) doesn’t do what people expect it to do, and there are quite a few expectations from that labeled feature.

The Library Backup function only makes a copy of the database on your computer. Full stop. So, if your T7 is where your music is primarily stored, executing the Library Backup in Engine DJ won’t do anything to that external volume.

The absolute best way to do a backup of your database is manually. I’ve been doing this since 2004 now and it’s saved my butt a few times. Actually, i do this every time i make a change to my music collection, because I’ve been bitten by this issue before.

So here’s what i do before making ANY changes to the databases:

  • With Engine DJ closed: Manually zip up Engine Library/ :open_file_folder: on my laptop and my external SSD. I name them something like Engine Library - YY.MM.DD - # Change type.zip
  • Then I make my changes, run database cleanup, close Engine DJ & make backup zip files again with the same naming convention.
  • All backups (laptop and my SSD) are copied to my NAS for safe keeping.

Call me crazy, but I can easily roll back after an update by doing this.

I know this won’t help you today, but perhaps this can help you and anyone else in the future who is looking for one of the best ways to ensure they can roll back from a corruption event.

Moin @djliquidice ,

Tnx for your detailed awnser, very much apreciated.

the analyzing didn’t take me that long fortunally, 1,5hr for 17000 files (700gb). sorting it again will take a little longer but i have the time for that.

Just a quick question, so I understand correctly. I Have all the music on the USB drive an create my library in Engine Desktop. (I’m a bedroom DJ, pls dont judge. The future might bring me a NAS one day :slight_smile: ) After every change do I copy the backup thats created on that drive or do I copy the backup thats created in my /music folder on the pc?

Thanks in advance

Glad you don’t have a ton of heartburn about re-analyzing. This happened to me in 2023 and I was heartbroken about it. I didn’t understand how Engine DJ’s database architecture worked, and that corruption inspired me to dive into how things operate and then get more active in this community, sharing what I know. This has always been my model for working in tech, and is what’s known in the wider tech spaces as the Open Source Mindset.

You’ve asked a lot of very good questions, so I’m going to be a bit verbose in my answer in hopes that someone in the future might have a similar setup, needing the same advice.

Make sure you have a backup of that entire USB drive if that’s the only place you store your music.. This is something that so many people don’t do at all. That backup can be in the form of your data living on your future NAS and there are plenty of apps that can manage the backup process (only copying the deltas between changes for example).

Second, be aware that Engine won’t work with network mounted volumes, in case that’s where your mind was going. In other words, if you try to register music that lives on any network mounted volume inside of Engine, it will ignore it.

The most important database is that in which resides on your USB drive. Being that it’s where your music is stored, the database on that USB drive’s Engine Library/ :open_file_folder: contains all of your mission-critical data, so I would certainly prioritize that. Your USB Drive’s database dir will contain:

  • Registered music, cue points, loops, sample registries, etc.. (m.db)
  • If that same USB drive is used for performances, your performance histories are stored there (hm.db)
  • Other databases (stubs for Serato, etc..)

The Engine Library/:open_file_folder: on your PC will contain a copy of your history database, as well as copies of the playlists for your USB drive. Every time you connect your USB Drive, Engine will sync that data to the main PC databases.

Why?

Engine uses some what of a “hub and spoke” architecture when it comes to databases. The Hub is your PC, and the spokes are any and all USB storage devices (often called performance drives) that you’ve used with Engine. This allows you the freedom to have different performance drives for different use cases and maintain some consistency across them when it comes to history management, etc..

In my Mac’s History database (hm.db), there are 15 distinct performance volumes registered. Here’s a query that shows each of the database IDs for each performance volume (the type of ID is called a UUID or Universally Unique IDentifier).

This flexibility comes at a cost and creates frustration for so many people who aren’t very computer savvy, as Engine’s approach really assumes that the music is stored locally on your PC and you only use external volumes strictly as performance volumes.

One example of the source of frustration: When a DJ is searching for something that doesn’t seem to exist in Engine, they blame Engine, not realizing that they didn’t connect the external source of music (USB drive).

Here’s what Engine DJ looks like running with my music SSD unmounted. All of the playlists are here, but no music. If I was a non-tech savvy DJ i would be pretty upset seeing this at first glance, thinking Engine screwed up. But no, it wasn’t Engine, it was me (in this hypothetical case).

Hope the above helps. I’ve spent the past ~3 years diving into Engine’s database schema to learn how it works in order to develop my own tools.

Feel free to send any more questions you have and either myself or anyone else in the community will jump in =)

moin @djliquidice ,

thanks a lot for this post. Very informative.

To be a DJ nowadays you must be multi skilled.

Brgds BeatMaster

IDK about that. These days, all you need is some waveforms on a display, a video of a 5 second mix to go viral and you’re a super star. :rofl:

Just kidding.

Always happy to share what i know about this stuff. :slight_smile:

I just need to do a better job coding my tools so I can make them Open source.

Moin @djliquidice ,

pretty good idea.

Phps. Denon or some company like that is interested in your skills and all issues may be solved roughly.

To be a DJ nowadays you have to act like an octopus :innocent::clinking_glasses::smiling_face_with_sunglasses::bottle_with_popping_cork::radioactive:

In earlier times the crowd attended an evening at the discotheque to learn about new records and asking the DJ for the artist and title.

Nowadays the guests appear at DJ’ table, presenting their mobil / smart phone, playing any track and request from the DJ to play this immediately.

And that’S not a joke

Enjoy the rest of the Sunday brgds from HAM BeatMaster