Feature request "Live crates"

I keep my music very organized in folders on my laptop. What I love about Engine Dj is that it allows me to drag the root music folder from the hard drive straight into Engine Dj, and it will create the same structure using crates.

But: Whenever I download new music to my folders, to add them in Engine I have to first delete the “Music” crate that contains all the sub-crates/sub-folders, and then drag the folder again so it updates with the new folders and tracks I’ve added. The only small problem with this method is that it seems to create a lot of duplicates of the songs in the history section.

It would be cool to create a “Live crate” with the option to scan/rescan a particular folder and automatically import the folder structure + new songs inside Engine Dj.

This feature could run automatically at start-up (similar to how Traktor does it) or manually by right-clicking the “Live crate” and selecting “Rescan”

I hope it made sense :smiley:

6 Likes

See existing request for smart crates and add your user vote to it

I cant find the existing request. I really would like to user vote this as it’ll help me dramatically in my workflow*

(*my workflow may differ from yours, no disrespect intended)

2 Likes

Use the magnifying glass / spyglass for forum search

Here’s an example search to get you on the right track.

Remember a request is only a request regardless of how many votes it gets. It’ll your workflow cannot continue ti function if smart crates don’t get added then you might want to read up on some the the smart crate topics which search will find for you, and see that lexicon DJ , which is compatible with Engine, can create smart crates for Engine.

1 Like

I do use Lexicon DJ to create smart crates and sync them with Engine. And I’m all for smart crates being implemented in Engine DJ at some point.

But my idea is a bit different, it’s more like an updating “Mirror Crate” which mirrors the folder structure.

The idea is already half-implemented since in the current version of Engine dragging a root music folder inside the software will create the same structure. All it’s missing is an option to rescan and make updates.

4 Likes

If you organise your folders on your HDD in a way where they are identically reflected in your playlists, this would be a great feature to add. I’d take a “refresh source folder” option.

If I remember, this was something offered by Scratch Tools for Serato Scratch Live around 15 years ago.

It would have a monitored folder and auto scan and import that folder on start up so any new music in your HDD folder structure would get imported to the mirrored playlist structure. You just worried about adding your music to your HDD how you wanted it and Scratch Tools would import it to your playlists.

What you want is a watched folder which can “refresh” upon opening the software. This would be pretty good as I mirror my HDD folders and crates/playlists.

An option that basically states “If the particular folder is dragged and dropped into Engine DJ as a playlist, have Engine DJ watch that folder upon startup of Engine DJ and update it’s imported playlist for any freshly added media”.

3 Likes

What happens to the playlist when files are moved, renamed etc

I am experiencing a similar problem.

I put all the tracks I am interested onto a USB drive. So, I have a large collection of music, and I refine it by placing the tracks I want to mix onto a USB drive.

I can stick that USB drive into my laptop and then drag and drop the top-level directory into Engine Desktop “Collection”, which creates a new collection with a name like “USB Drive”.

Whenever I add music to the USB drive and drag and drop the top-level directory into Engine Desktop for a second time, I expect new tracks to be added to the “USB Drive” collection. However, it instead creates “USB Drive 2” collection and starts over completely.

I understand the want to create very specific playlists and ordering for some people making some sets. Unfortunately, I dont see any way to just load up tracks and mix from a large collection, matching BPM/Genre or whatever.

Honestly, it’s so bizarre to me that a device that can analyze tracks on its own and create STEMs on its own (Prime 4+) doesn’t allow me to insert a USB drive, search the drive for tracks, analyze the tracks and add it to a “Crate” which contains all the tracks on the USB, and then allows me to search through those tracks. I should be able to add more tracks to the USB, trigger a re-scan, and it only process/analyze and add new tracks. Sure, it might take twice as long as if I were to do it on my PC, but I’m really frustrated with the lack of this feature, and without the ability to even do this on my PC without re-creating the entire playlist/crate.

1 Like

I dont understand your workflow, are you not using the Engine desktop app?

Just download it for free, add your music into it, organise that music, sync to USB the fully analysed tracks then connect the USB to your device.

Thats how InMusic recommend you work, thats the best way to work. The device may be able to analyse tracks but thats not the preferred or optimal way to work, it’s merely a side measure for ‘non Denon’ Djs to connect their drives and mix.

1 Like

Thanks for your response.

My ideal workflow would be to put tracks onto a USB drive, insert the USB drive into the Prime 4+, and for the Prime 4+ to find all of the tracks, analyze them, and add them to a database relative to that USB drive. Treat all the files on the USB as a Crate, giving me search functionality to then create playlists after searching/filtering for tracks. That’s not available and I can look at Feature Requests in the future, but I’ve been told that’s not possible currently.

So, the next best thing for me would be to be able to use my USB drive as a source, use Engine Desktop to analyze, and then push that to the Prime 4+. The issue with doing it this way is that there is no “Scan for new files” or like functionality. If I drag and drop my USB drive into Engine Desktop, instead of updating the existing crate, it creates a new crate and add all the tracks to the new crate. Instead, I want any new tracks I’ve added to the USB to be added to the original crate.

Imagine you have 14TB of music (and growing), and you spend all day listening to it and pulling out the good tracks onto a USB drive. Now you want to mix with that USB drive. Well, I can’t. I have to load it into a PC, and even using a PC, I can’t update when I add new tracks. I have to start over each week that I find new tracks.

I want a single collection with all of this music. If I want to create a playlist of everything I found in October of 2022, cool, I can do that, but I can’t just add music to a directory and update my existing crates with newly added files.

Right now, the only way I can add new tracks is if I’ve been adding tracks from their parent directory, vs a top-level directory containing many directories and tracks. So, album by album, adding tracks, vs having 69 albums in a directory, adding that directory and then when I have 70 albums, just have it bring that new album in.

/Album1
/Album2

Adding stuff this way is not ideal (ie; Album1 then Album2). Instead…

/Music/Album1
/Music/Album2

I should be able to add “Music” and trigger re-scan when I add Album3. It notices Album3 is new and adds it to the existing crate.

1 Like

Is there any particular reason you have 14tb of music? Where are you getting it from and why aren’t you only buying tracks you want to DJ with? Sometimes quality over quantity is the key. To compare I have about 6000 tracks that I couldn’t possible ever find time to DJ with and it’s 110gb, you have 140 times that amount of storage used up.

As far as separating it out for music you want, why not just move stuff you want to use onto your PC then load it into playlists in Engine, then use a separate USB to export to your device (they are cheap enough). That way your music is organised and analysed correctly and you have it in searchable playlists.

It is what it is.

Searching a database requires …. A database

Databases are created by

• Analysing tracks in Engine DJ desktop • Playing tracks on the players.

Remember also that any removable usb drive will have its own database file stored on that particular usb drive.

If you absolutely couldn’t use a desktop PC to analyse files in bulk, (there’s an auto analyse feature), then you could set the deck to continuous play and play all the tracks in your usb stick / as each one loads, it analyses…. Which creates an entry in the database on that usb stick … which then means that tracks tags can be searched for

1 Like

So, that was an example, and there is plenty of great music in the world. Also, some of us have access to Labels catalogs and we search through those for tracks we want to play, or archive various genre’s of music, but only mix certain genres. There are plenty of reasons to have a large collection of music. A very simple example is you buy a CD and there is only one track on it you want to mix.

I think the point you’ve missed is I’ve already trim’d that down. Onto a USB. It’s only 128GB large. So, every single file on that USB is of value. It’s not like other collections where you may only want to mix some tracks, etc. I think that’s the disconnect.

Currently, Denon only considers workflows of “From this large collection of music, which tracks do you want, and how do you want it organized?”

What about those of us who take time to archive and store on our own, and those of us who don’t want to listen to music in Desktop Engine, but instead something like DeaDBeeF or foobar2k? I’ve already selected which tracks I want. They are all the ones in “this directory” or on “this USB drive”.

And when I update “this directory” or “this USB drive”, I want all the tracks I put there to be analyzed and added to a database for searchability.

Again, it seems really silly to me that a device with so much power cant see that there is currently no database on a USB but see there are tracks and ask you “Do you want to create a crate from this USB?” I say yes, it analyzes and adds all the tracks to a database that I can search. When I re-insert the USB, it see’s the previous database and asks if it should re-scan for new files. I say yes and it analyzes and adds all the NEW tracks to the database.

It’s the same when you get a USB from a label. I can’t just stick that USB into my Prime 4+, it creates a database and places it on the USB drive for next time. Instead, I have to copy all those files to my computer, format the drive, and then tell Engine Desktop to put those files back on the device. It just seems a bit redundant and wasteful.

It’s the same with this USB I’ve created. Every track on there is one I want to mix with. If I buy a new track that I like, I’ll put it on the USB by copying the file over.

I understand that Denon has deprecated Crates, and now all we have are Playlists, but I thought a USB drive would be a perfect example of a Crate. Anything I put in that USB is part of that Crate. And a USB Crate, just like one with my records in it, changes over time. I buy a new record, I add it to my crate, I don’t have to buy a new crate and put all of the previous records in a new crate with the new album.

Edit:

It seems that when I load files from my USB drive and analyze them, when I go to Sync to the USB, it doesn’t try to re-copy all the files, it just saves the database.

This is great and exactly what I was looking for! Thanks for everybody’s kind input and recommendations.

First of all, there’s indeed no automated rescan, directory scan or see changes functionality.

However:

Yes you can stick it in the player. It will create a database on that USB, but only the tracks loaded to a deck will go into that database. There is no bulk scan in EngineOS, you’d need Engine Desktop for that.

It does, unless I don’t follow you. Use the DRIVES section in Engine Desktop. Database will be on the drive. You could even use computer mode and stick it in the Prime device as removable storage.

1 Like

Thank you so much! You are correct! Desktop Engine confused me in the sync menu, showing it needed a bunch of GB space, but when I clicked through it was smart enough to just add the database because the files existed!

This is awesome!

1 Like

I’d still be tempted to have 2 different USB sticks for managing these tracks but it sounds like you have your problem solved.

Just consider you may not end up with a backup of this DB if you’re referencing the same USB you export to.

So what exactly is your workflow now? I feel like I’m one step behind you…

Hey,

So in regards to using the USB, I had all my favourite tunes on it. I was able to connect the USB drive to my computer and drag and drop the directory on the USB that contains the tracks into the “Collection” under “Playlists” within Engine DJ Desktop.

Engine DJ Desktop analyzed the files, and I was then able to use Sync onto the USB drive. Even though it’d say something like “Requires 108GB”, which made me think it was going to do something silly, when I Sync’d to it, it just put the database onto the USB and left all the files alone.

This was nice because it allowed me to pretty quickly use the controller and test it out. Honestly though, if I’d realized I could just connect to Engine DJ Desktop from Engine DJ OS if I allowed Engine DJ Desktop through the firewall, I might have just done that instead of worrying about the USB.


Previously, I used my own script to copy files to the USB. When I copied the files over, I would keep their directory structure from the source onto the USB. This would result in files such as:

/mnt/USB/Collection/Trust In Trance Records/Astral Projection/Ten [23152]/03 Astral Projection - Dancing Galaxy (Dynamic Remix).flac

However, even though I was able to get my tracks onto the USB the first time, I was not able to simply update the Engine DJ Desktop “Collection” with those new tracks. After the first time, any time I would drag and drop that same directory into Engine DJ Desktop, it would duplicate and create a new playlist of all of the files.

So, if I drag my Collection directory into “Collection” on Engine DJ Desktop, it creates the “Collection/Collection” playlist. If I drag and drop my Collection directory again, instead of updating the existing playlists, it will create “Collection/Collection 2” and re-import every track.

Because of this, even though it was cool I could put the database on my USB, it was pretty useless to me.


After learning the these limitations of Engine DJ Desktop, I had to take a different approach. I was pretty disappointed that it couldn’t detect new tracks by itself, and pretty frustrated that I couldn’t maintain my directory structure without Engine DJ Desktop creating playlists for every single directory, when I could just utilize the actual directory structure. Like, I had nowhere to put my own ACTUAL playlists because Engine DJ Desktop created thousands using my directory structure.

So now I modified the script that copies files to my USB drive to instead create relative, symbolic links to the source tracks into a flattened directory structure.

The way my script knows to pull a track is that I have tag’d it with a custom tag I call MIXIT. Within deadbeef, I have a Custom Shell Command called “Flag for Mix”:

metaflac --remove-tag=MIXIT --set-tag=MIXIT=$(date +%%Y-%%j) %F

Today, that’d result in a tag of MIXIT=2023-269. Honestly, I just wanted a boolean to mark tracks I wanted, but thankfully I’d talked to enough DJs and noticed that they all store their tracks in directories like “2023-09” or whatever, sorting their tracks by when they found them. I thought it would be smart for me to track that same information, so that’s why I apply the current date.

When scanning the files for those with this MIXIT tag, I pull out that value and find the year portion of the string, and I store each track link in a directory of that year. I also remove any track number prefix on the track when creating the link, so now the structure would look more like this:

/mnt/Music/MIXIT/2020/
/mnt/Music/MIXIT/2021/Astral Projection - Dancing Galaxy (Dynamic Remix).flac
/mnt/Music/MIXIT/2022/
/mnt/Music/MIXIT/2023/

Where, in this example, that file would actually be a relative, symbolic link to the following file:

/mnt/Music/Collection/Trust In Trance Records/Astral Projection/Ten [23152]/03 Astral Projection - Dancing Galaxy (Dynamic Remix).flac

The power of these being relative, symbolic links is that I can now share the /mnt/Music/ share via SSHFS in Linux or share /mnt/Music/MIXIT via SMB and access within Windows to work within Engine DJ Desktop. Even though Engine DJ Desktop arbitrarily refuses to add tracks from network drives, even when mapped to a drive letter, and even though the drive shows up in the software itself, you can use a workaround for that:

mklink /D "C:\Users\YOUR_NAME_HERE\Music\MIXIT" \\remote_server\MIXIT

The power of this directory structure is that I can go into the MIXIT/2023 directory, Select All (Ctrl+A), and drag and drop all the tracks into the “Collection/MIXIT/2023” playlist within Engine DJ Desktop, and it will add and analyze any new tracks without giving me an aneurysm.


Now, I can use this MIXIT share to export my tracks to the internal drive of the Prime 4+, as well as export to my USB drive to listen to tracks on the go, or plug in to another controller and have tracks ready to mix.

I appreciate this extremely thorough answer. It’s wild to go through ANY extra steps imo since this a ‘stand alone’ device. I look forward to the point where we can just drag and drop our file structure into the Prime 4+ SSD (or any other) and have it ‘analyze all’ without using Engine OS desktop. Cheers.

1 Like

My workflow isn’t that complex! Sorry I couldn’t read everything, but I’ll leave here how I deal with my music library on Engine DJ - SSDs on my SC6000, if it’s interesting for anyone!

1 - I do as already mentioned here: I drag my folder tree to the Engine, it creates the playlists following this scheme like a mirror of my PC!

2 - I create a new ROOT PLAYLIST: LIBRARY: Drag the playlist tree into this playlist.

3 - I create a new ROOT PLAYLIST: Live SETS: This one I will use to create new playlists of lives that I would do, so it is separate from the library playlists (The old CRATES already did this part, they didn’t need to have removed it)

4 - I create a new ROOT PLAYLIST: PARTY SETS: This will create playlists for events!

So I have 4 Large ROOT Playlists, which I access depending on the purpose. It’s more organized for my workflow.

Now when I download new tracks, that’s what takes work and this is where I defend the way TRAKTOR handles your COLLECTION and your PC’s own folders! Keeping them always up to date, just pressing a simple button “Add new tracks to the collection?”

Unfortunately, I do this update manually, I download and drag the new tracks to their respective playlists in the engine!

Then I go to the synchronization menu and synchronize with my 2 players (yes, I have 2 1TB SSDs in each! Redundancy!). My library is about 400gb!

When I want to use a simple pendrive, I create any playlist and only synchronize it to the pendrive! Simple!

The problem with all this is when I have to make changes, as happened to me recently. I had to format my PC and add more new storage units, I changed my library to an SSD, before they were on an HD, due to the formatting Windows changed the drive letters! This resulted in the Engine no longer finding tracks in the collection resulting in all tracks being red! There was no point using the relocate option, as it didn’t work!

In all this, I ended up losing the playlists I made manually for LIVE SETS and PARTY SETS! Because unfortunately, after everything is in red in the playlists you cannot export, and even if you do export, there is no function to import these playlists! The work is hard!