So I have an SC Live 4, a 256GB SD card, and my music library (about 180GB) completely analyzed in engine 4.0 on Windows 11.
I have tried 3 times now to get this to work.
•In Engine, I use sync manager to sync my entire library to the SD card. This takes almost an entire day off processing, like 12-18 hours.
•I eject the card in Engine after it’s done, not even immediately, sometimes it’s a few hours after it I see it done in the morning. It is not shown in Windows either, so I remove the SD card from the card reader.
•I then put it in the SC Live 4. I have done this both before powering it up (even with it unplugged) and while it is on. Either way I get the message that the SD card was not ejected properly and the database is damaged.
This most recent time, when I went to browse it said “updating” I left it like that for a good 10 hours and it never changed or completed its update. Restart the unit, same thing.
Now the music is physically there and analyzed, but I have to browse manually by folder, by individual artist names. Search does not work at all. Nothing is in the collection at all.
So having thousands of artist names alphabetically without any kind of search or sorting by title, bpm, etc. is just about as useless as not having music on there to begin with.
It will analyze a song on the fly and add it to collection when I load it to a deck.
This is about as useful, but less organized as putting all my music on the SD card in the folders I have on my PC and browsing it manually.
Is there some kind of trick or special procedure for this to work, or does it just ■■■■ and not work?
Am I better off syncing with rekordbox and having the SC Live analyze them on the fly, but at least have some type of structure and search to it and cue points set?
What model of SD card are you using? and what speed rating is it?
Are you using an external SD reader and if so which port is it plugged into your computer? Is your music on an internal drive on the PC? or is it on a separate external drive?
I feel like sorting the issues around how long it takes to export will assist with the DB corruption issue. To copare, i have around 180gb of music myself and it takes around 15 minutes to do a full clean export to the internal SSD on the Prime 2.
FOr the library, have you added everything into the ‘collection’ section of Engine desktop? and have you then used playlists to sort your music further? i personally mirror all my folder names in Engine using playlists, so my ‘House Music’ folder has a ‘House Music’ playlist.
Engine is partially to blame and your process is as well. Even though these devices have SD Cards, they are not really designed for the type of work that you’re looking to do.
Your Process & Storage Medium
SD Cards are cheap flash storage that are slow for the IOPS (input/output per second) that Engine requires. These flash devices are best for short bursts (photos) at best. Their failure rate is pretty high also. The Embedded / single board computer (SBC) community has been moving on from SD cards for years now.
This video that was recently published explains why.
I would suggest a faster SSD device that has a USB interface over an SD card.
Engine DJ | It’s all about how it’s architected!
Even if you replaced the SD Card with a much faster SSD, you’re going to run into some speed issues due to Engine’s architecture, in addition to the amount of writes it has to do. My music is on an external NVME that has a potential for sequential reads and writes at +800MB/s:
All that said, Engine’s architecture seems to be a limiter, even on this pretty fast M2 Max laptop with three Thunderbolt four ports. I imagine it doesn’t help slow SD card speeds at all.
The transfer speed is dependant on multiple variables from the class of SD card to the card read/write port/adapter. 10 hours sync time for 180GB would suggest an average write speed of 40mbit/s Amazon.co.uk <-this card has the “potential” to write at 90mbit/s (in theory, again dependant on hardware) which could speed the sync up to around 4.5 hours. - I would suggest testing sync with just one playlist of about 100 tracks just to see how/if the card performs.
Edit: Apologises, the linked SD card has the potential to sync at 90MB/s - 180GB in approx 30mins.
I happened to have a pretty similar experience yesterday with such a quite fast (?) 1TB Sandisk Extreme Pro micro SD-card (Class 10 V30 U3 A2, theoretically 200MB/s - not Mbit! - reading speed, and 140 MB/s writing speed, according to the data sheet).
Approx. 32000 tracks with a little more than 590GB (most of them FLAC, well maintained track names, artists, albums, genres etc.) were transferred with Engine DJ from the PC to the card in ~ 6 hours. Which is not particularly fast.
After inserting the card into my Prime 2, the database including the search function worked for a few minutes. But verrrrry sloooooowly. Then the message “database corrupt” appeared. And indeed, the original 370MB database on the card (\Engine Library\Database2) shrank to less than 10MB at that very moment and the card was thus worthless for database access. The tracks could only be addressed via the directory structure. I reproduced the same behavior again in three attempts, just like the TO…
I’m pretty sure that with the 32000 tracks I’m not yet exhausting the capacity of the Prime 2. I seem to remember that I had already stored more tracks (mp3) than this on the internal drive and was still able to use the database without any problems and with acceptable delays.
So it’s obviously an issue with the card. I have therefore given up and just ordered a new and larger internal SATA SSD which will - hopefully - deliver the required performance.
I am fighting this very issue and am working on a tool to help figure out ■■■ is going on. I use an external SSD for storage and export to SSDs inside of my SC6000Ms.
The source SSD gets corrupted as well as the destination.
Since the database management worked fine in earlier versions of Engine DJ and Denon DJ, would a downgrade really be a good idea, as suggested by @mufasa?
I’m using a Sandisk Ultra 256GB SDXC Class 10, U1card formatted to exFAT. 150MB/s read, 80MB/s write.
I’m using an external USB C 3.0 SD card reader plugged directly into the USB C port of my laptop, a Dell XPS 15 with 10th gen i7 8-core and 16GB of RAM.
The music is stored on the computer via internal micro SD card reader. The micro SD card for my music is a PNY 512GB PRO Elite Class 10 U3 V30. 100MB/s read, 90MB/s write.
I use the same card reader to dump my 16GB GoPro card in seconds. Writing all of my music library to the 256GB Sandisk card in explorer, without using engine takes about 4 hours, so 4-5x longer using Engine.
Yes, everything is added in collection. There are some playlists for some genres, but not for everything, but everything shows up in collection on the PC.
I read on here that someone else had the same issue with Engine 4.0, so I downgraded the SC Live and the PC software to 3.4 and am trying again. I’m still waiting for everything to sync from last night before I try again. Hopefully it fixes it, but I would definitely like it to sync faster.
I think storing your music on that micro SD card could be what is causing your bottle neck, it’s likely the built in port it’s plugged into is only USB 2.0 and the media itself isn’t that fast.
If you had your music collection either on your computers hard drive or a thunderbolt compatible external SSD drive connected to a relevant USB-C port I would guess the sync would be much quicker, therefore reducing corruption errors.
I have done 100 songs that are on average 3-4 minutes each and roughly 10mbps because 320kbps mp3s, for a load of about 1GB to test and it takes 3-4 seconds per song.
~18,000 songs and 3 seconds per song = 54,000 seconds / 60 = 900 minutes / 60 = 15 hours.
~18,000 songs and 4 seconds per song = 72,000 seconds / 1,200 minutes/ 60 = 20 hours.
So 18 hours is pretty much on target given varying song length and file size.
The PC has 2x USB C thunderbolt ports for connectivity. They are both USB 3.0. There is no USB 2.0 anywhere in the computer unless I add on a hub that steps it down.
An external drive isn’t an option because , you know, portability, and the ssd is one that’s built into the motherboard of the PC, so it’s not upgradable and is smaller than my music collection.
The Micro SD slot is THE only way outside of an USB option to expand storage and the only way to do it internally.
This doesn’t bottleneck anything else and I have zero issues with speed exporting rekordbox, using it for storage for video editing, etc. It has no speed or data issues or errors anywhere except exporting to an SD card with Denon Engine for the SC Live to read it.
Honestly, the SD card in the computer reads and writes faster than anything plugged into USB.
I feel @STU-C is on the ball as usual with his suggestion
The onboard SD card reader on many laptops may be on 2.0 bus speed. But I see that your are using an external card reader. Is it a named brand? What is it rated as?
Get a Samsung T7 for your music
I have been using externals since HD externals for my storage. Hell at some point when I was doing videos I even had the big externals with their own power supply.
You could Velcro a T7 to a the lid of the laptop , I have seen folks do that.
100% not doing an external, period. If it’s just for extra storage, I have 4TB on a dedicated server for backup purposes. For on the go, which is what I use my laptop for, an extrrnal is not going to happen.
The speed isn’t the issue here. I’ve done the math on transfer rates and they’re in acceptable range, even though Engine slows it down a lot more than what it does natively in Explorer.
The issue is the SC Live 4 reading a corrupted database every single time. I brought up the time, because it’s frustrating, engine taking so long and then the SD reads corrupt database on the SC Live. It does not read corrupt database if I plug it back into the PC.
Firstly I’ll address the Rekordbox point, that program is just as slow, if not slower than any export I’ve ever done in Engine. And that is using the same library, same playlists and same brand of good quality USB 3.0 flash drives.
On your other point, the speed could well be the issue, if you’re leaving a computer alone for 15hrs how do you know what other processes it is or isn’t carrying out? If it’s not going to sleep and impacting the export. Especially given it sounds like it’s running Windows, anything could happen with that OS…… the only solution I can suggest to you is format the SD card, run the engine desktop clean up action then re-sync the SD card again. Also making sure there is no conflicting engine DB file on your source Micro SD.
Anyway, yet another reason why I avoid ‘library chat’, so I’m out.
