Corrupted Database

Engine DJ Software Engine DJ (Mac & PC)
, ,
1

Hi all,

Having an issue with a USB.

Brief background, this is a USB that lives in my Prime 4, i have three, one gets taken out and updated with new music, two live in the P4 all the time.

I’ve been having an issue recently with this one where a couple of times a night it ‘ejects’ itself when searching for a track, and i either have to wait for it to give me the error message that its ejected, or pull it out and put it back in.

I have backed it up to an external hard drive, formatted the USB, and copied all of the folders EXCEPT the Engine Library folder back on to it, and re-analysed it in Engine DJ.

After about 30 minutes, a dialogue box has popped up telling me that the database on that USB is corrupted, and cancelled the analysing process.

Does anyone know what the issue might be? The USB itself is a Samsung FIT 3.1 256gb which are pretty bombproof so i’m not convinced the USB is faulty, and running first aid on it throws up no issues.

N.B. I have a similar issue on my P4 with my other USBs, although this one is the worst offender. The problems started when i started using Mac instead of Windows but that being said they were sporadically doing it before, so i think that is just a coincidence.

Thanks all,

Gav

2

If you’ve switched platforms id say its worth formatting the USB, ensuring Engine desktop is up to date then reload all your music back onto it.

1 Like
3

Hi Stu,

Yeah, well thats what i thought the issue might be which is why i did the format. I’m up to date on both Engine DJ and OS as well.

Currently trying again, one folder at a time, to see if any particular folder presents an issue.

Going to be a long day :joy:

Cheers!

1 Like
4

Update,

Putting folders on one-by-one didn’t work, the collection folder just remained blank. So, deleted everything off the USB again and now performing the slowest, most secure seven-pass format on it. If theres a dodgy sector hopefully that will either pick it up or reset it so hard that it fixes it…

1 Like
5

Did you say you were storing the music directly on the USB? won’t the file path be different for any new music added using the Mac rather than Windows? and that will make the DB on the flash drive fall over?

6

Why are you copying folders direct to the usb? Don’t you use engine DJ to load files to the usb?

  1. Format the usb
  2. Sync manager to transfer your files to it

If you chose to do things any other way, then you need to know what you are doing.

7

Thanks mufasa.

This is the way i have always done it, load folders onto the USB, then drag and drop folders from the USB into the relevant folder in Engine DJ. I’m not saying it’s right, i’m saying it works!

Although apparently with this one now it isn’t, im not sure its the method thats the problem though but once this format has completed i will try doing it the proper way and see if that yields better results.

8

When i moved to Mac (about three months ago?) i installed a fresh copy of Engine (obviously), deleted the Engine Library folders on the USB and reanalysed everything on the Mac, so in theory the file paths should be right and have for the most part worked fine, music loads fine in OS up until the USB ejects itself.

My thinking is if i can get this one right and working i will do the other two as well, so i will follow Mufasas advice and use the sync manager and see if that works.

An hour later an we are on pass 2 of 7 of the format, so this USB is getting CLEANED!

9

Ive made a pact with myself to avoid getting involved/embroiled in any discussions about library management from now on Gav(too many arguments), but i agree that the sync manager is probably the best step, or at the very least worth a try to see if its better.

You have done the right thing after the move by the looks of it though with the fresh install of everything.

10

I also don’t get into arguments, just not worth it! Plenty of people know more about this than i do, and i’m happy to take others advice!

Yeah, i’ll give Sync Manager a go. I never really used it before because my way of doing it wasn’t broke, so i didn’t fix it but now… its broke!

The more i think about it the more I’m starting to think it may be the USB itself, with a corrupted sector or something. It seems like the data itself was fine as everything worked, but it was the searching and therefore reading process that ‘ejected’ it, maybe when its search took it thorugh the bad sector.

A quick format earlier would have just erased the files (but doesn’t really erase them, as im sure you know, just opens the area up for rewriting) but this 7 pass format rewrites a random string of 0s and 1s… so may reset a bad sector by essentially brute force.

Only on pass 3 now so its going to be a while til i find out! Any and all further advice welcome!

11

I guess for the price of a new USB3.0 stick it might be worth just buying one and trying that? i think the little Sandisk ones i use are only about 15 quid for the high volume option.

12

I was thinking that. I’ve got a Sandisk 256gb at home, alternatively a new samsung one (which are excellent by the way! very fast, and water and heat resistant) is only about £30.

I’ll let this do its thing and see what happens.

1 Like
13

Now before you use sync manager Make sure the source collection is in top shape.

14

Hi, I make regular copies of the database2 directory as backup, both in Folder Music on C: Drive and on USB drive. I don’t like how the tracks and up the usb drive using sync manager. Therefore I put the structure of files on the USB drive an import the files from there.

15

Hi all, Update to the story. I have emailed support today to try and get the issue resolved but thought i would post a bit more here in the hopes that someone may have a solution and/or to help others.

So the three-USB issue was kind of resolved, in that i managed to get the vast majority of the music on the three USBs analysed except for one Pomo Only folder that i chickened out of doing, basically because i found that at a certain point it would just corrupt. I don’t know if this folder was to blame but i left it at that.

I carried on having less, but still some, issues, so my solution to that was to clear my internal SSD which i never used, and to put the music from the three USBs onto the internal drive. So i connected it all, formatted the internal SSD, and copied the files across.

Then, with my Mac Air M2, i tried adding the files on the SSD to my collection, to analyse and then sync back to the drive, and again, at a certain point, i get the ‘Engine Library Is Corrupt’ message.

I then deleted all existing Engine Libraries on the SSD, and tried again, and got the same.

I don’t think I understand WHAT is corrupt. Is it a file? I thought that the library being corrupt was basically damaged code or a damaged structure in the library file itself, which shouldnt be the case as i am trying to create a new one from scratch, so is it a file that Engine can’t read?

I’d love to go through it file-by file to try and locate the problem but theres about 600gb of music so not going to be possible. Also, i can’t narrow it down to a specific folder because i keep changing the order of how i try to analyse them which doesn’t seem to affect the outcome.

TL:DR My problem seems to be that something is corrupt meaning Engine can’t create a library?

16

Hopefully one of the staff reply here and you can send your DB and log files over to them to investigate Gav. I think ive seen them doing it before.

17

Thanks Stu. Yeah, i hope so too, unfortunately I just don’t know enough about it to know what might be causing it. Just weird that its almost after a certain amount of data it throws a fit, rather than the actual data itself. I can continue using my USBs this weekend so its not an emergency but i’d like the issue to just stop happening!

Cheers mate!

18

Fairly limited myself but try and look for any tracks with weird characters in, stuff like this pipe character | are a classic DB issue… if you can access the logs it should tell you at what point its corrupting and it’ll have what is causing it.

19

Thats a good shout, my music is all legit so i don’t know why that character would crop up but i know sometimes theres code in places of things like apostrophes, ampersands etc so its a possibility. Can i view logs, or do i need support/devs to look into that?

20

I notice you said you had some from a Promo pool? it could be those files that are causing it.

I think you can view them yourself, let me have a mess around when ive finished my day job and ill try and find out where they are etc… it’ll be in the Engine Library folder somewhere.