I was working on preparing playlists on the way to a gig in the van when in a slightly stronger bump my usb dock disconnected the pen drive with which I was preparing the session. The result was that it deleted all the playlists that were on the USB drive (a Sandisk xtreme pro) the computer I was working with was not my main computer that I had at home so I found myself arriving at the gig just over an hour before the start and all the playlists had been deleted. the songs were on the pen drive as the histories were still there and so I was able to save the night, creating some old session playlists.
But has this happened to you? how could it be solved if I had the main computer? I was happy since version 4.0 nothing failed me and now this. I really want to keep using Denon but it’s not easy for me with these things.
Your’e blaming engine for a data corruption issue if the storage was disconnected improperly when this is part of modern computing - storage needs to be dismounted properly.
But has this happened to you? how could it be solved if I had the main computer?
Sudden removal of a USB drive while it’s being accessed by any software can lead to unpredictable outcomes, and this applies to the Engine DJ software as well. When a USB is disconnected without properly ejecting it, files, including playlists and databases, may not be fully saved or updated, leading to corruption or loss … it’s a risk with any software, not just Engine DJ.
Always have two USB drives with you, better three or more. USB drives can fail and they have a limited life span … and tomorrow I will make a fresh backup too.
If the playlists are on your main computer, you can export them again via the SYNC manager, … but the edits that you made during the drive are probably lost …
Hi, I have been using mac for 20 years and I have never had a problem like this, also the usb has other files which work correctly, I have not lost the songs but engine deleted everything from my collection, I reset the colletion, it is not that the usb drive is screwed up.
That’s exactly what happened.
I always keep a backup, unfortunately I didn’t have it updated with all the work I was doing these days.
I also have an emergency backup in dropbox.
You can be lucky for a long time … but if you disconnect a USB drive often enough, problems will eventually show up. While your situation was an accidental disconnect, it is not different from disconnecting the USB without proper unmounting or ejecting from the point of view of the system. Anything bad can happen, missing files, corrupted files, data loss … it is unpredictable.
I’m just saying that it was accidentally disconnected by a bump in the road (I always eject the usb when I finish my work on engine). I found it curious that it erased all the information from the lists. as if it was new. But the files were there and the cues. but I could not access because there was not even a list. Do you see this as normal?
I was also asking for a solution in case it happens again.
There is no solution. If things like that happen there may be a complete data loss or corruption of the drive. Modern computer systems are quite advanced, but not every desaster can be fixed. Yes, there may be partially written files, 0s and 1s, on the drive, but fixing this is a job for data recovery, if it is possible … in any case it can be quite expensive.
Audio files and database files are different things. If the drive is disconnected during a writing process, something will be lost. The file system keeps track of the files. If it cannot finish updating, information will be lost or corrupted. This is normal.
The way out is to use stable connections, make many backups, sort out faulty drives quickly at the first sight of problems. So the answer is: In the worst case scenario, there is no way to restore lost data. Be prepared for this!
I realize you’re upset and I do empathize with you. However, your lack of understanding of how computers work seems to be causing you to blame the wrong thing.
You happen to be in the middle of a database write when your drive disconnected. This caused data that was meant to be made permanent to get lost.
It’s not Engine’s fault that this happened, it’s yours. Furthermore, I’m going to flag that all of the database files (mostly m.db) needs to have an integrity check run against them (outside of engine DJ) to ensure they are not malformed.
Here’s an explainer for non-computer nerds on why one must unmount a storage device and how data can get corrupted.
First of all, I thank you very much for taking the time to answer me.
I was just asking if anyone has ever had this happen to them and if they knew of a solution. I don’t need people saying the program works awesome it’s all your fault that you don’t know how to eject a disc.
I confirm that when I finish a job in any activity that I do as a professional, I disconnect the drives where I work whether SSD or USB …
I was saying that in more than 20 years using Mac (not windows) many times the usb have been disconnected unintentionally or by mistake and never in any program, and for work I use many type davinci, or resolume arena or after effects, nothing has ever happened. I must be the luckiest person on earth as another comment said. And just working with engine dj casually deleted my playlists (not the songs and histories). Not the playlists I was editing at the time, which can be understandable. But all the lists (it has reset the usb collection). I would be grateful if you could show me a video about why it is important to disconnect the usb. I am a telecommunications engineer.
I was just asking if anyone has ever had this happen to them and if they knew of a solution. Anyway I send you a big hug.
So that I can better understand the process of operation.
if i am working on the usb (remember that the computer i was working on at that moment was not my usual computer, and the songs and playlists were on the usb), the computer did not have the updated playlists and songs. i was just creating new playlists and sorting and creating cues.
Is the computer also synchronising things with the usb?
just to know how the program works
How would you have solved it?
Thank you very much for your time.
No, wrong. You don’t understand how computers work.
Engine just does an sqlite call to the database with every change you make. Then SQLite asks the OS to write a new collection to disk, or a part thereof, with every change you make. The OS will start overwriting your collection bit by bit with the new version, and when it is done it will update the file allocation table. Maybe it used a couple of extra free sectors and mark others as free, maybe it wont. If during this write the drive gets disconnected you end up with 2 collections that are both incomplete (for example the beginning of the new one, the end of the old one, but in practice a whole lot more complicated than that), and a file allocation table that points to a random combination of both. You cant expect any software to read such a corrupted file. Not Engine, not Rekordbox, not Photoshop, none…
On top of that you have things like write buffers. In consumer hardware the storage medium will take data in its write buffer, report to the OS the data has been written, and then start to write its write buffer to the actual permanent storage, be it a spinning disk or a bunch of NAND modules… See where this can go wrong? Asking your drive to eject will ask the drive to empty its write buffer to actual disk, among other things… and then there is the non-ECC RAM story, which means that a consumer computer can and will corrupt your data at any time or place anyway. If you dont want this you have to invest in very expensive server hardware with expensive SAS drives, ECC RAM, etc etc, and especially dont use removeable storage. The next best thing is doing incremental backups regularly, in at least 2 different places on 2 different media types. Even on servers this is the advised strategy because hardware can and will fail, and even a server has a few single points of failure, albeit fewer. On top, you can RAID what you want, there is a chance of user error too…
Bottom line, if your drive ejected during a write proces, thats very bad luck, and I understand you being mad, but you should be mad at Newtons laws, not InMusic. Or at Apple because it integrated such a flimsy USB-C connector that ejects with the slightest bit of force…
PS: your music files are fine because they weren’t being written to while updating playlists.
I feel engine dj needs to include performance drive db in the back up process. It should back up the db on the individual drive to the individual drive…just like it does to the main db.
And the issue @Juan_Manuel_Villanue ran into may have been saved by restoring a slightly older back db
Yes, I have lost data in the past by accidental disconnects. Not with Engine DJ, but with other software, among other things with DaVinci Resolve. As I said, the solution is to accept that sometimes there is no solution - be prepared for the worst case scenario: data loss. And have a plan B. B as in Backup.
Please understand that nobody here claimed that Engine DJ is 100% super and you are not. Engine DJ has its problems and many of us report them here in this forum. You have had an accident, your USB was disconnected, nobody said that you did it on purpose.
Just because something hasn’t gone wrong for 20 years doesn’t mean the risk wasn’t there - it simply means you’ve been lucky so far. It only takes one mistake or an accident to realize that good habits are about minimizing risk, not about assuming past success guarantees future safety - which it does not.
I must be the luckiest person on earth as another comment said.
Yes, believe me.
And just working with engine dj casually deleted my playlists
No, your playlists were lost during a data-critical accident. It is not something that Engine DJ did - as in did on purpose or did out of spite. This can happen with any software, not just with Engine DJ. Try the same with DaVinci, pick some operation that takes some time and switch off power or disconnect the hard disk or whatever. You will loose something. A hard USB disconnect is not just working. If you were using Engine DJ in a normal setting without interruptions and your playlists were lost, you could file a bug report. But disconnects are not a normal mode of operation.
Thank you very much guys for your wisdom and teachings, you are right, it was not the fault of the program, it was an accident, in any case the fault is mine.
Thanks for wasting your valuable time helping me and giving me solution
