Extremally long "Aggregating Drives" task when i start Engine DJ or insert USB stick

Engine DJ Software Engine DJ (Mac & PC)
1

Hi

As title says - I’m experiencing a very long startup time of Engine DJ, due to “Aggregating Drives” task. My collection is quite big (~70k of tracks) - all of those are in one drive that is installed in my PC (not laptop). I do not understand, why every time I start damn Engine DJ software it need to aggregate drives. During this task I cant do anything with the software. Of course I can cancel this job, but when I do that I have i.e. difficulties to add tracks to my collection (software acts like adding them, but in reality it not)

Same is when I alrady pass this “aggregate drives” at start and then i put USB stick (or connect my SC6000 in computer mode with USB in) it starts aggregating dives again. Right now, it aggregates my SanDisk 256GB USB stick (Fat32 format) with 2000 tracks in it for 1,5h and it is still going… What the hell ?

I start recordbox with same collection and same USB stick and it is almost instant.

Please do something with this (or maybe I can do something ?), because it is ridiculous that I need to wait 2 or even 3h to start working with my music library, connect my players via Wi-Fi to the software or syncronize usb drive…

2

Some plain English or even some useful “alt text” or “hover text” wouldn’t go amiss instead of those rather cryptic messages on Engine DJ

3

I’ve never seen this, but it seems you’re not alone:

https://www.reddit.com/r/DenonPrime/comments/1bawpv0/engine_dj_on_start_up_takes_ages_aggregating/?rdt=59887

What are the specs of your computer? What type of storage are using internal and external? Solid state or spinning hard disk? What are the makes & models? Are you using USB 3.0+ devices?

6

Here are my specs:

AMD Ryzen 3700x ; Kingston 32GB ram DDR4 3600 ; 2x 1TB Samsung evo SSD drives (no music on them) and Segate 8TB HDD SATA3 internal drive with 4,5TB of music ; Windows 11 Pro

For USB I have USB3 devices formated in FAT32

7

This guy – have you ruled out this being too slow? Perhaps open task manager and see if you can detect if the disk is thrashing?

I don’ have nearly as many tracks as you and run on mac os, and use all solid state devices.

keen@CommanderKeen:DJSSD$ sqlite3 'Engine Library/Database2/m.db' 'select count(path) from Track'
8476
8

For performance reasons (among others) I would suggest formatting these as ExFAT.

9

I would agree with you if:

  1. This was a old drive, but it is new (one month old and without errors (checked); same was with previous drive (4TB one) or when I have music on my M.2 SSD (but moved due to capacity)
  2. this also for usb drives. Today my SanDisk 256GB usb drive with 2000 tracks was aggregated for 2,5h and it is the same with my other sticks from Samsung or Kingston.

This drives me crazy, thats why I made this topic after few months of dealing with it. Now I have enough… thinking about sellin my 2*SC6000 and x1850 and switch back to alfabetaomegasmth

1 Like
10

I recognize your frustration. Even new drives can have problems, and unless you’re running a tool to literally read, write, read again, write again each sector you cannot be 100% certain that the new drive doesn’t have issues. I’ve had new drives die that were months old and had to RMA them.

It also depends on what type of drive you bought. Is it CMR or SMR? These details all matter when doing loads of quick reads/writes.

I’m just saying that there is clearly something wrong here and it’s easy to blame the app when there could be underlying issues that you may not be aware of.

If you’re willing to give up, then so be it. I can stop volunteering to help.

¯\_(ツ)_/¯

Best wishes!

1 Like
11

Of course you are right.

On the other side, whit the same drive, same usb stick and same specs of my PC, when I open Recordbox, it reads my whole library almost instantly, same with my usb sticks. I can work on it after idk… 30 seconds after I start it. It takes 2h to start using Engine DJ because of that.

I do not think this is my hardware simply because Recordbox is acting normally with that drive and my whole collection (and I didn’t mention, Serato too)

12

I know it’s easy to compare different DJ applications because they all kinda do a lot of the same stuff. However, the “how they are architected” can be wildly different.

When you open up engine DJ, look at task manager. Are you seeing high CPU utilization? Even though the 3700x is a 8core / 16thread CPU, it is 5 yrs old by this point. Should be plenty fast though.

Also, take a look at the disk performance charts. do you see anything out of whack when it’s “Aggregating”?

image
image1542×1168 97.1 KB

I found that string used twice in the binary executable and did google searches for it. Still don’t have a clue ■■■ that means.

13

Perfect example:

  • Engine DJ & Engine OS are made with Qt.
  • Rekordbox seems to be made with the Unity engine

Both of these have wildly different approaches to “single codebase, multiple deployment targets”.

14

When it starts aggregating Engine DJ taes from 6% to 10% of my CPU. In terms of drive activity it is 100-120MB/s for few seconds, then long long long time (10-15min) nothing, then again 100-120MB/s for few sec

15

I understand that and I know that those are completely different software and software languages.

But music is the same and here it is all about music and playing it.

When I need to wait to play or do something with my library it is frustrating, when you see a software on your desktop that is working (RB/Ser) as you want, but you have hardware that is using something that is not working as it should be.

It shouldn’t be like that. There are other smaller issues that are described here by lots of users that I’m also facing (this one is extremely frustrating) but developers are adding Bluetooth receiving ability (nobody knows why instead of Bluetooth sending) instead of bugfixing

16

I can’t begin to defend their strategy, though i will say most software is delivered half-baked today. It’s a sad thing for users for sure.

With less than 15,000 tracks, I wish i could help replicate your issue. I have an intel i7 from 2017 and some spinning rust drives to test with.

keen@CommanderKeen:DJSSD$ find . \( -iname "*.mp3" -o  -iname "*.wav" -o -iname "*.mp4" -o -iname "*.aiff" \) 2>/dev/null | wc -l
   14271
17

Thank you anyway for trying to help.

All good for you, kind sir!

1 Like
18

I’ll load up a very slow 2TB WD Green 5400RPM drive with tracks on my old 2017 Intel i7 and see if i can replicate this at all. I’d love to see what happens.

One question @Sparr , have you done the “Library Optimization” step in Engine DJ? I’m wondering if there are any opportunities to optimize the database a bit?

19

Im not completely versed inthe software, however I am quite sure there are some settings that you have made or not made yet that is taking this simple task to such an extremely long waiting time. I use a mac from 2012 (yes its old) However I can also presume that Engine has still to work trough that MASIVE 70k library you are using.

I used to have a similar problem with Traktor once upon a time, And it turned out to be the way I managed and stored my files…

So I don’t think its the gear you are using but more what Engine is analysing each point of start up.

You can disable and filter the search / aggregating functions in Traktor This solved it for me amongst actually deciding which music I was really going to play, then moving the other “Probably never gonna play ir play again” files to a separate folder and un-included that in my music libraries.

I would try some mangment of your files, And yes indeed record box, serato. traktor and engine file management is not the same… in regards to the way they each agregate or check-sum there library.default files and settings etc…

  1. Try modify Engine settings,
  2. Try cleaning up your library,
  3. Try A new faster 1TB ssd over larger Drives,
  4. If you like the way your players work, Then stick with it, there is always a solution.
  5. I use still use traktor for my music management,
  6. Just use record box if it works!

Best of luck, I would also like to see you resolve this annoyance.

20

Welcome to the community @dopeNL !

1 Like
21

Thank you…

Any idea how I can change my user name lol

Not really my DJ name :•) Cheers :beers:

22

In your profile and otherwise DM me what you want it to be. (Als ik dat kan overigens…)