OpenCL Textures exceed available single buffer allocation memory limit #51554

New Issue

Alma Talp · 2017-05-19T01:06:44+02:00

Alma Talp commented

2017-05-19 01:06:44 +02:00

Old problem, related to OpenCL.
I have a GTX 1070 and an RX480 (both 8GB).

The nVidia card renders all my huge scenes perfectly in the viewport, AMD RX480 generates 'OpenCL error 61 CL_invalid_buffer_size in CLcreatebuffer'

I made a limited test scene what reproduces the error.
I cannot share the original scene for obvious IP reasons, so the test scene contains just a few objects and textures and needs a little bit older Blender versions.

Here is the result of the test:

2.78a release (hash:e8299c8) : ERROR
2.78c (hash: e92f235283) : ERROR
2.78 nightly build (102394a): Renders the test scene, but ERROR on the real scene (far more materials, objects, etc, but almost similar memory needs).
Interesting thing: it uses far more memory for OpenCL (5,3 GB) while CUDA uses only 3,2 GB memory for the test scene.

File is here, available for 30 days: https://ufile.io/j6tc3

System: Win7 64 Pro

Thanks

A.

PS: This task is identical to a former one, but that was closed for some bad reason and I did not to want to reopen it without authorization, but as it still not resolved, I choose this way to remain correct.

Old problem, related to OpenCL. I have a GTX 1070 and an RX480 (both 8GB). The nVidia card renders all my huge scenes perfectly in the viewport, AMD RX480 generates 'OpenCL error 61 CL_invalid_buffer_size in CLcreatebuffer' I made a limited test scene what reproduces the error. I cannot share the original scene for obvious IP reasons, so the test scene contains just a few objects and textures and needs a little bit older Blender versions. Here is the result of the test: 2.78a release (hash:e8299c8) : ERROR 2.78c (hash: e92f235283) : ERROR 2.78 nightly build (102394a): Renders the test scene, but ERROR on the real scene (far more materials, objects, etc, but almost similar memory needs). Interesting thing: it uses far more memory for OpenCL (5,3 GB) while CUDA uses only 3,2 GB memory for the test scene. **File is here, available for 30 days:** https://ufile.io/j6tc3 System: Win7 64 Pro Thanks A. PS: This task is identical to a former one, but that was closed for some bad reason and I did not to want to reopen it without authorization, but as it still not resolved, I choose this way to remain correct.

Alma Talp commented

2017-05-19 01:06:44 +02:00

Changed status to: 'Open'

Alma Talp commented

2017-05-19 01:06:44 +02:00

Added subscriber: @AlmaTalp

Bastien Montagne commented

2017-05-19 09:00:59 +02:00

Added subscribers: @MaiLavelle, @mont29

Mai Lavelle was assigned by Bastien Montagne

2017-05-19 09:00:59 +02:00

Bastien Montagne commented

2017-05-19 09:00:59 +02:00

@MaiLavelle think this one is for you? Though given how big already is the demo file, could just be that it’s memory limitation I guess, file being 1.8Gb…

opencl-error-on-rx480.zip

@MaiLavelle think this one is for you? Though given how big already is the demo file, could just be that it’s memory limitation I guess, file being 1.8Gb… [opencl-error-on-rx480.zip](https://archive.blender.org/developer/F603518/opencl-error-on-rx480.zip)

Alma Talp commented

2017-05-19 16:01:22 +02:00

I you read the details it is not a memory limit. If not, please do so.
Do you want to upload the file here?

Thanks.

I you read the details it is not a memory limit. If not, please do so. Do you want to upload the file here? Thanks.

Mai Lavelle commented

2017-05-19 22:22:11 +02:00

I cant reproduce this on my hardware. My guess is the scene is too large, but we should probably have a better error message in that case. I'll try to put something together for that later.

Its also possible this is a driver bug. What might be helpful is if you could run from a terminal clinfo, and blender --debug-cycles and attach the output.

I cant reproduce this on my hardware. My guess is the scene is too large, but we should probably have a better error message in that case. I'll try to put something together for that later. Its also possible this is a driver bug. What might be helpful is if you could run from a terminal `clinfo`, and `blender --debug-cycles` and attach the output.

Alma Talp commented

2017-05-19 22:29:08 +02:00

terminal = Win 7 command prompt?

Dear Mai, would you please write me a step by step guide to do that? Then I could do that. I'm not a coder. Thanks.

terminal = Win 7 command prompt? Dear Mai, would you please write me a step by step guide to do that? Then I could do that. I'm not a coder. Thanks.

Alma Talp commented

2017-05-24 22:31:02 +02:00

BTW, have you tried with a former Blender version like 2.78a release (hash:e8299c8) on Win7 64 Pro and RX 480?

Wolfgang Faehnle commented

2017-06-18 16:27:12 +02:00

Added subscriber: @mib2berlin

Wolfgang Faehnle commented

2017-06-18 16:27:12 +02:00

Please close

Alma Talp commented

2017-06-18 21:04:47 +02:00

Why?

Wolfgang Faehnle commented

2017-06-18 22:35:26 +02:00

Hi Alma, thought no movement after + 3 weeks thread could close.
I am sorry if it is still under investigation, trying to clean up the bugtracker.
We have ~300 bugs still and many are old or already fixed.

Cheers, mib

Hi Alma, thought no movement after + 3 weeks thread could close. I am sorry if it is still under investigation, trying to clean up the bugtracker. We have ~300 bugs still and many are old or already fixed. Cheers, mib

Alma Talp commented

2017-06-21 03:31:07 +02:00

Hello Wolfgang,

My problem is that the case is still unsolved what means that I cannot use my RX 480 for anything, also I cannot sell it as that will simple not allow future testing for me. So if I do not push it, it might be never solved (it is the reborn of an old case), what means that AMD will be a no go for any real production in our pipeline (and possible for others). If you have idea how I could provide more info on it, please share with me (or contact me).

Hello Wolfgang, My problem is that the case is still unsolved what means that I cannot use my RX 480 for anything, also I cannot sell it as that will simple not allow future testing for me. So if I do not push it, it might be never solved (it is the reborn of an old case), what means that AMD will be a no go for any real production in our pipeline (and possible for others). If you have idea how I could provide more info on it, please share with me (or contact me).

mathieu menuet commented

2017-07-02 07:41:43 +02:00

Added subscriber: @bliblubli

mathieu menuet commented

2017-07-02 07:41:43 +02:00

The file is not available anymore. Please attach a file that really trigger the problem on latest master (we don't support 2.78 anymore). I render files that need 16GB of memory on the RX480 without problem, so it must come from something else.
Also update your drivers before testing further.

The file is not available anymore. Please attach a file that really trigger the problem on latest master (we don't support 2.78 anymore). I render files that need 16GB of memory on the RX480 without problem, so it must come from something else. Also update your drivers before testing further.

Alma Talp commented

2017-07-02 14:23:39 +02:00

Hello John; I'm on it; the last 2.79 I checked works on the example file, now I check it on the 'real' scenes. Some of them works (that is great), some results the error, but I have to examine what are differences, so I will need time for that. Thanks for the patience.

Mai Lavelle commented

2017-07-06 02:42:56 +02:00

Changed status from 'Open' to: 'Resolved'

Mai Lavelle closed this issue

2017-07-06 02:42:56 +02:00

Mai Lavelle commented

2017-07-06 02:42:56 +02:00

I've pushed a commit (222b96e5) that will give you a better error message if your scene is too large. Please test again with a build containing this commit and reopen this report if you get any errors aside from the new message ("Scene too complex to fit in available memory.") Thanks.

I've pushed a commit (222b96e5) that will give you a better error message if your scene is too large. Please test again with a build containing this commit and reopen this report if you get any errors aside from the new message ("Scene too complex to fit in available memory.") Thanks.

Alma Talp commented

2017-07-06 16:46:43 +02:00

Hello Mai, thanks for your effort!
When this commit will be included in a downloadable Blender version? I ran some tests; still not sure what makes the error, but it is clear that the same scene sometimes requires far more memory for OpenCL than CUDA.
Also I find it interesting that while final render time is comparable to CUDA, viewport render is far slower with it.

Hello Mai, thanks for your effort! When this commit will be included in a downloadable Blender version? I ran some tests; still not sure what makes the error, but it is clear that the same scene sometimes requires far more memory for OpenCL than CUDA. Also I find it interesting that while final render time is comparable to CUDA, viewport render is far slower with it.

Ray molenkamp commented

2017-07-06 16:51:28 +02:00

Added subscriber: @LazyDodo

Ray molenkamp commented

2017-07-06 16:51:28 +02:00

This is available right now in any 2.78 build from http://builder.blender.org

Alma Talp commented

2017-07-06 19:23:46 +02:00

Thanks, I'm on it.

Alma Talp commented

2017-07-06 20:01:13 +02:00

OK, quick test on real life scene:

Viewport render/Final render:

Same scene:

GTX 1070 8GB - CUDA renders well, memory usage: 4437 MB
RX 480 8GB - OpenCL: fails render: Scene is too complex to fit in available memory.

I think that is a clear nonsense. Any idea, guys?

At the weekend I will work on a new test scene what could be shared.

OK, quick test on real life scene: Viewport render/Final render: Same scene: GTX 1070 8GB - CUDA renders well, memory usage: 4437 MB RX 480 8GB - OpenCL: fails render: Scene is too complex to fit in available memory. I think that is a clear nonsense. Any idea, guys? At the weekend I will work on a new test scene what could be shared.

mathieu menuet commented

2017-07-06 20:45:10 +02:00

Did you update your driver to latest version
OpenCL can use your system memory, so there should be no memory limit other than your GPU+ available motherboard memory
Anyway, please share a file before adding more text.

- Did you update your driver to latest version - OpenCL can use your system memory, so there should be no memory limit other than your GPU+ available motherboard memory Anyway, please share a file before adding more text.

Alma Talp commented

2017-07-06 22:18:06 +02:00

Thanks, John.

Here is the new file: https://ufile.io/ivcut

On GTX 1070 8GB it renders, uses 4100 MB
ON RX 480 8GB it says it does not have enough memory. With hiding the marked plane it renders, using 5700 MB

System: Win 7 Pro 64 bit

Note: My AMD driver crashed during update, so I contacted AMD to solve it (their support page did not help and even AMD cleaner utility failed).
So the current AMD driver for the test was 17.10.1731.

BTW I contacted you through BA PM, so we could speed up the bug-hunting if you don't mind. THX.

Thanks, John. Here is the new file: https://ufile.io/ivcut On GTX 1070 8GB it renders, uses 4100 MB ON RX 480 8GB it says it does not have enough memory. With hiding the marked plane it renders, using 5700 MB System: Win 7 Pro 64 bit Note: My AMD driver crashed during update, so I contacted AMD to solve it (their support page did not help and even AMD cleaner utility failed). So the current AMD driver for the test was 17.10.1731. BTW I contacted you through BA PM, so we could speed up the bug-hunting if you don't mind. THX.

mathieu menuet commented

2017-07-07 09:12:07 +02:00

Changed status from 'Resolved' to: 'Open'

mathieu menuet reopened this issue

2017-07-07 09:12:07 +02:00

mathieu menuet commented

2017-07-07 09:12:07 +02:00

Added subscriber: @Stefan_Werner

mathieu menuet commented

2017-07-07 09:12:07 +02:00

ok, I can confirm it crashes although the scene is very simple. Only planes and some textures, but nothing fancy as far as I can tell. Only 64 x 8bit textures at 4096*4096, which is 64MB per texture with RGBA, which is 4288MB of textures at max.
It looks like all textures try to be allocated at once, instead of 4GB chuncks, which is crazy and would prevent using today's GPU memory with low poly scene with a lot of textures.
@Stefan_Werner you wrote the code for infinite textures, maybe you can have a look here on how to allocate chunks of 4GB max at once.
Limiting rendering to less than 70 x 8 bit textures in 4K in 2017 is way too low. Many displacement maps today are 16 or 32bit, HDRs are also very big.
As an element of comparison, Luxcore 1.6, which is now more than a year old takes 1100MB (1400MB used of which 300MB are Blender's UI) of memory to render it.

ok, I can confirm it crashes although the scene is very simple. Only planes and some textures, but nothing fancy as far as I can tell. Only 64 x 8bit textures at 4096*4096, which is 64MB per texture with RGBA, which is 4288MB of textures at max. It looks like all textures try to be allocated at once, instead of 4GB chuncks, which is crazy and would prevent using today's GPU memory with low poly scene with a lot of textures. @Stefan_Werner you wrote the code for infinite textures, maybe you can have a look here on how to allocate chunks of 4GB max at once. Limiting rendering to less than 70 x 8 bit textures in 4K in 2017 is way too low. Many displacement maps today are 16 or 32bit, HDRs are also very big. As an element of comparison, Luxcore 1.6, which is now more than a year old takes 1100MB (1400MB used of which 300MB are Blender's UI) of memory to render it.

mathieu menuet commented

2017-07-07 09:12:53 +02:00

Having BW textures stored as single channel would also help to save 3/4 of the memory.

mathieu menuet commented

2017-07-07 11:04:50 +02:00

I simplified the file as requested: 68text8bit.blend

I simplified the file as requested: [68text8bit.blend](https://archive.blender.org/developer/F654498/68text8bit.blend)

Mai Lavelle commented

2017-07-07 12:51:23 +02:00

This file is in fact too complex to render. OpenCL will only allow a certain amount of memory to be allocated at once (seems 4gb is most common). In Cycles each of geometry, BVH, textures, etc get allocated into their own buffer. If any one of these exceeds the limit for one buffer the allocation will fail and rendering will abort. So even tho your card may have more than enough total memory available to render the scene, the memory available for the individual buffer is insufficient.

A way to use the full device memory is in the works, so I'm going to mark this as a todo for now.

This file is in fact too complex to render. OpenCL will only allow a certain amount of memory to be allocated at once (seems 4gb is most common). In Cycles each of geometry, BVH, textures, etc get allocated into their own buffer. If any one of these exceeds the limit for one buffer the allocation will fail and rendering will abort. So even tho your card may have more than enough total memory available to render the scene, the memory available for the individual buffer is insufficient. A way to use the full device memory is in the works, so I'm going to mark this as a todo for now.

Mai Lavelle changed title from ~~OpenCL render error with example~~ to OpenCL Textures exceed available single buffer allocation memory limit

2017-07-07 12:53:03 +02:00

Alma Talp commented

2017-07-07 14:04:01 +02:00

Thanks, guys.
'Having BW textures stored as single channel would also help to save 3/4 of the memory.' As I had no idea what causes the crash, I wanted to use as big textures as possible (no BW conversion) except of some simulating mixed usage.

Thanks, guys. 'Having BW textures stored as single channel would also help to save 3/4 of the memory.' As I had no idea what causes the crash, I wanted to use as big textures as possible (no BW conversion) except of some simulating mixed usage.

Alma Talp commented

2017-07-12 20:02:21 +02:00

Hello Guys, question: what are the current texture limits in Cycles?
I know last time I asked for increasing it CUDA had about 96 and if I remember correctly that became 1024, also on CPU. Thanks.

Hello Guys, question: what are the current texture limits in Cycles? I know last time I asked for increasing it CUDA had about 96 and if I remember correctly that became 1024, also on CPU. Thanks.

Carlo Andreacchio commented

2017-07-13 03:09:44 +02:00

Added subscriber: @candreacchio

Carlo Andreacchio commented

2017-07-13 03:09:44 +02:00

Since this commit -- https://developer.blender.org/rBec25060a05e394560ec92388b347c45bd9eba0ed unlimited textures.

Limits for CUDA Fermi cards are still in place (~90-95 iirc). Any NVIDIA card newer than the 5xx series will be fine.

Since this commit -- https://developer.blender.org/rBec25060a05e394560ec92388b347c45bd9eba0ed unlimited textures. Limits for CUDA Fermi cards are still in place (~90-95 iirc). Any NVIDIA card newer than the 5xx series will be fine.

Marius Eckardt commented

2017-07-28 00:16:35 +02:00

Added subscriber: @Alirion-2

Marius Eckardt commented

2017-07-28 00:16:35 +02:00

The File: Bliblubli posted renders without error on Build: 57f937ad5a Blender 2.78.4 on an RX 480. Must be something that has changed after. Ill keep digging.

The File: Bliblubli posted renders without error on Build: 57f937ad5a Blender 2.78.4 on an RX 480. Must be something that has changed after. Ill keep digging.

Alma Talp commented

2017-07-29 17:17:18 +02:00

Hello Marius, could you offer me a link for that version? I would test it with the real scene. Thanks.

blender-admin commented

2017-08-08 13:12:04 +02:00

This issue was referenced by blender/cycles@2b0412aa67

This issue was referenced by blender/cycles@2b0412aa67100f9e64ca5d3744060cbab9da57d8

blender-admin commented

2017-08-08 13:12:04 +02:00

This issue was referenced by ec8ae4d5e9

This issue was referenced by ec8ae4d5e9f735ab5aeb149dea8aa47ab8f8f977

Mai Lavelle commented

2017-08-08 13:16:58 +02:00

Changed status from 'Open' to: 'Resolved'

Mai Lavelle closed this issue

2017-08-08 13:16:58 +02:00

Alma Talp commented

2017-08-09 09:13:10 +02:00

Thank you very much, I can't wait to test it.

Alma Talp commented

2017-08-09 21:20:18 +02:00

Thanks for all, I tested it and now - I waited for this so much - my scene renders on AMD. Great job!

I found an issue anyway: compared to CUDA (GTX 1070) the same scene uses 1.9-2.5 times more texture memory. Do you have any idea what could cause it? Thanks.

Thanks for all, I tested it and now - I waited for this so much - my scene renders on AMD. Great job! I found an issue anyway: compared to CUDA (GTX 1070) the same scene uses 1.9-2.5 times more texture memory. Do you have any idea what could cause it? Thanks.

Sign in to join this conversation.

No Label

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

OpenCL Textures exceed available single buffer allocation memory limit #51554