Good day @exwookie,
this should be no problem, I just tried on the Prime 4 in a running procedure as follows:
Go to the playlist with the track you want to remove
touch the symbol on top with pencil and lines (the circles at the tracks must appear)
touch the track, you want to remove, and hold the track till it fixed
move it to left and then you are requested to remove
go for it
at last, check in your collection, if the track is still available
btw: If you are not sure, how to proceed, take an separat USB-Stick as test environment
Good luck and brgds
BeatMaster