RAM or HD? How to load big projects

#1

I’ve tried the demo and really like the features, but I’ve been struggling to load larger projects as I run out of RAM (using MBP 2017, 16GB RAM). Is it entirely impossible to have clips stream from the SSD instead? Unfortunately it’s the nature of the beast that high quality HAP files are quite large, so RAM use adds up quickly when using multiple clips.

If the files have to be loaded into RAM, are there any tips on how to create a seamless long show whilst loading and unloading content to keep things moving and RAM use in check?

Thanks

#2

Hi Holmes
You should be able to stream multiple Hap videos from SSD without problems. 8 or more full HD (1080p) Hap videos should even be possible. Not only the speed of your drive is important, but also the power of the GPU (processor on the graphics card), since Hap videos are unpacked on the graphics card. What graphics card are you using?

#3

Hi Jeroen,
My laptop has a Radeon Pro 560 with 4GB. It’s not so much an issue with layers, but more with the amount of clips in a project. I’ve got around 250 clips for one project, which are roughly between 1 second and 20 seconds in length. As I add each clip to the Ableton Project I can see the RAM creeping down and I can’t load all of the clips, so I was assuming they were somehow streaming from the RAM instead of the HD. I’ve been able to load all the clips in Resolume though… That said, if I’m honest, I don’t really understand how Resolume or Ebo Suite/Max work with video files.

Any advice on how to manage multiple clips in larger project welcome.
Thanks.

#4

Hi Holmes, that’s an interesting question. We will run tests on different systems and try out different techniques. I will post the results here soon, probably Monday or Tuesday.

#5

Ok thanks for looking into it.

Just to clarify though, does Ebo Suite load all clips into RAM to playback (meaning there’s a theoretical limit to how many files you can have in a project based on the maximum RAM), or does it just stream content from the HD (albeit with a certain amount of RAM used to load a reference to the clip into Live)?

Thanks.

#6

HI Alex,
I hope you don’t mind if I chime in :grinning:, I am one of the developers of EboSuite.
First off: sorry to hear about your issue, let’s puzzle to see what is going on.

EboSuite consists of 2 parts, roughly divided like this:
A. the control and audio part in Ableton (using Max for live)
B. the video, playback and processing part in a native mac os x application

I’ve been struggling to load larger projects as I run out of RAM

What do you experience when you are running out of RAM? And how do you monitor your RAM usage?
Is it an issue in Ableton itself (due to A)?
Or an issue with the external EboSuite app (B)

If you load a new clip on a track with eClips then there is nothing loaded into RAM in A. However important frames are cached in B to prepare for a smooth playback so this will cost more if you use videos with very large dimensions such as 4k (and there is probably some memory overhead for the videoplayers themselves).
During playback the video is streamed from disk.

If you load a file in an eSampler then the situation is roughly the same with two notable differences

  1. if the file has an audiotrack (and is < 2min) then the audio is also loaded into RAM (in 32bit floating point data). If you load a file by dragging a clip from a clipslot it will only load the region from start to end marker, so this will help you reduce RAM usage.
  2. Additionally the eSampler has an option to load a full clip into ram for cases where disk speed is the bottleneck. Don’t use this if you are low on RAM.

Lastly: there is an internal maximum of 500 video files in 1 live-set, it should give an error if you are reaching that amount.

I hope this helps a bit
Cheers
Timo

#7

Hi Timo,

Many thanks for your detailed response! To be honest, it was a few months back that I was testing Ebo Suite, and my demo has now expired.

I have Memory Clean 2 running in my tool bar (OS X) that feeds back on overall free memory. The clips were around 1080x544 in size, and were quite large MB wise. I can’t remember whether I was running them with the audio built-in, or whether I had exported the audio as a Wav to load in separately.

I guess it’s possible that it has something to do with the way Ableton hold files. I’d don’t suppose it’s possible to reset my demo so that I can run some more tests…? Either that or I might just pull the trigger on the Black Friday deal and see how I get on.

The alternative at the moment is using OSC with Resolume, which comes with it’s own pros and cons. I was hoping that this laptop (2017 MBP) would be powerful enough to handle audio and video, but I think it may need some fine tuning to get the performance where I want it!

Cheers,
Alex

#8

HI Alex,

Thanks for the reply.
I am not familiar with Memory Clean 2, but I do know that it is complicated how Apple deals with memory.
What symptoms did you have caused by the memory pressure that stopped you from using all those files?

I will send you a new demo key (and also a lot has been improved under the hood)

Btw, how many videos did you stream at the same time? even with an ssd there is a limit (and now we support other codecs as well it is worth to try e.g. Prores).

Yes, indeed, what you describe should be possible with EboSuite I think, I hope the required fine tuning will be minimal.
Best
Timo

#9

Hi Timo,

Many thanks for the new demo key. That’s much appreciated. I think I need to do some more experimentation to find out exactly what’s going on. I don’t think there were more than 3 videos playing at once. My worry was that I would run out of memory completely, making Live stall and making it impossible to add additional processing and instruments to the project.

Out of interest, will there be much of a performance difference between using HAP and DXV codecs? Just because I have a bunch of files already formatted for Resolume.

I’ll do some more tests and see how it performs. I think the ideal situation would be to do most of the composition work using Ebo Suite in Live, but send layers to Resolume via Syphon to use the Resolume FX and Output slices. I probably need to walk before I can run though!

Thanks,
Alex

#10

HI Alex,

I understand your concern about memory, but Mac OS is also a bit creative and how it allocates and deallocates memory and sometimes it will free up memory for you at the moment you need it. So my advice is to try out what you want in stead of focussing only on the numbers.
We already offload all the video playback to our separate mac os application, so frees up Ableton for regular audio processing tasks.

DXV is similar to HAP, but… DXV is not open, only Resolume can make use of it’s hardware accelerated playback. So currently EboSuite does not support DXV files.

Yes, the option you mention to send multiple syphon streams to Resolume is a good one.
Let me know how it goes and what you experience along the way.

Best

Timo