Load data between large cubes

[glaboria]

I've been using TM1py for many things, including loading data between cubes from different instances. It worked with low-sized cubes but, when encountering large cubes, I discovered the next problem.

TM1py.Exceptions.Exceptions.TM1pyRestException: Text: '{"error":{"code":"88","message":"TM1MemoryException: Temporary pool exceeded"}}' - Status Code: 500 - Reason: 'Internal Server Error' - Headers: {'Content-Length': '94', 'Connection': 'keep-alive', 'Content-Encoding': 'gzip', 'Cache-Control': 'no-cache', 'Content-Type': 'application/json; charset=utf-8', 'OData-Version': '4.0'}

When using MDX to retrieve data, I got this error saying that I exceeded the Temporary pool memory, I searched this error and found out that, to be able to use this on large cubes, I would need to edit the tm1s.cfg and add this parameter MaximumViewSize to increment the memory allowed on this process.

The problem I have right now is that I cannot edit that file configuration and restart the tm1 server because people are working on it and cannot be stopped. Is there another way to transfer data using TM1py that allows large data transfers without the problem mentioned?

If needed, I can share the python script I used for this process for more context.

[psegal]

I’m pretty sure MaximumViewSize is dynamic, so no need to restart.

[glaboria]

So I change the tm1s.cfg while the server is running and have the changes automatically applied without the need to restart the tm1server?

[MariusWirtz]

as @rclapp suggested, use_blob=True should resolve the issue. Alternatively, you can update the config settings on the fly before the retrieval. see #798 (comment)

[rclapp]

Also, if you use the UseBlob option you should be able to bypass this parameter. From: psegal ***@***.***> Reply-To: cubewise-code/tm1py ***@***.***> Date: Thursday, October 3, 2024 at 8:24 AM To: cubewise-code/tm1py ***@***.***> Cc: Subscribed ***@***.***> Subject: Re: [cubewise-code/tm1py] Load data between large cubes (Discussion #1156) I’m pretty sure MaximumViewSize is dynamic, so no need to restart. — Reply to this email directly, view it on GitHub<#1156 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AEK7GZWAYQY4V673ENBLPODZZVOSPAVCNFSM6AAAAABPJ542RSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAOBTGM2TONI>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[glaboria]

What's the UseBlob option?

[wimgielis]

That's the meaning of a dynamic setting in tm1s.cfg yes. Static settings require a restart.

…

------ Best regards / Beste groeten, Wim Gielis IBM Champion 2024 MS Excel MVP 2011-2014 https://www.wimgielis.com <http://www.wimgielis.be> Op do 3 okt 2024 om 18:47 schreef Guillem Labòria ***@***.***

: So I change the tm1s.cfg while the server is running and have the changes automatically applied without the need to restart the tm1server? — Reply to this email directly, view it on GitHub <#1156 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEDHULP77FSGU73TCA3YQP3ZZVYJXAVCNFSM6AAAAABPJ542RSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAOBTGQZTCMI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.*** com>

[glaboria]

Didn't know the tm1 servers had dynamic settings, thanks for the help!

[psegal]

There's a useful cheatsheet at https://bihints.com/tm1scfg-parameters-cheatsheet, which (among other things) specifies if a setting is dynamic or not.

[rclapp]

When running MDX you can enable that flag and it uses TI to extract the data. From: Guillem Labòria ***@***.***> Reply-To: cubewise-code/tm1py ***@***.***> Date: Thursday, October 3, 2024 at 9:50 AM To: cubewise-code/tm1py ***@***.***> Cc: "Clapp, Ryan" ***@***.***>, Comment ***@***.***> Subject: Re: [cubewise-code/tm1py] Load data between large cubes (Discussion #1156) What's the UseBlob option? — Reply to this email directly, view it on GitHub<#1156 (reply in thread)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AEK7GZTZ432DNJPFWUJOXNDZZVYQXAVCNFSM6AAAAABPJ542RSVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAOBTGQZTGMQ>. You are receiving this because you commented.Message ID: ***@***.***>