Bogus intermittent "too large" error from GET /documents/{docId}/content

Hi @Steve Strassmann, thank you for contributing to the Lucid for Developers Community and for reposting your question here! Since your ticket yesterday, I’ve had a chance to discuss this error with our Development team and we understand the confusion since this document size is quite small (360KB) yet we’re still getting this “Document is too large” error.

One explanation could be your document has many elements on the canvas (hundred or more) and has multiple pages with this many elements. In this case, the Document Content resource may fail to generate and the GET Doc Contents request returns this error.

Unfortunately our GET Document Contents endpoint doesn’t currently include a page parameter so the Document Content request will include all contents of a multi-page document, across all pages, and cannot be reduced to a page by page query. When you encounter this limit, the workaround will be to split the document up manually. 

Here are two methods for dividing larger multi-page Lucid documents into smaller documents with fewer pages:

Method 1: Duplicating then deleting pages*

1. Duplicate the document
   • Select File > Duplicate within the editor
   • Or select Duplicate from the three-dot menu next to the document thumbnail/title on your Documents page
2. Delete unwanted pages in the copy
3. Repeat as needed

* Keep the original document as an archive.

Method 2: Use Import Page Feature

1. Create a new Lucidchart document
2. Within the editor, go to Insert > Import Page, navigate and select the document and the specific page to import using the preview and breadcrumb arrows
3. Repeat as needed for other pages

Note: In case it is helpful, our Export Document endpoint supports single-page exporting (generates a document image) if that fits your use case.

Going forward

We recognize the potential for confusion and apologize for any inconvenience caused by the manual workaround. We have noted your feedback and are committed to improving our document contents export service. Currently, our API team is prioritizing other high-priority development projects for our API, so enhancements to this service may not occur soon but have been added to our backlog. It's important to understand that the document size limit for the GET Document Contents request does not consistently correlate with the memory size indicated (e.g. 360KB) in the document details. Therefore, larger documents with fewer elements may still successfully generate a Document Contents resource, often because the document’s memory consumption is attributed more to images than to the number of elements.

Please feel free to provide further questions or feedback in the comments below. We are eager to understand how you use our APIs and are here to assist troubleshoot or answer questions.

Hi Richard,

To be honest, I find this answer difficult to accept.

1. First, the problem is intermittent. I sometimes have no problems retrieving the document and sometimes I get this error. If it was “too big” as you say, then I would expect the problem to arise consistently.
    
2. Second, the error message is misleading. I think that short of actually fixing this bug, correcting this error message to avoid being misleading would be a step in the right direction.
    

3. Third, I disagree with your assessment that my LucidChart document has a lot of elements. After a successful retrieval, I counted the number of elements. Across the 6 pages, the counts are:
   [{'nshapes': 203, 'nlines': 122},

 {'nshapes': 183, 'nlines': 106},

 {'nshapes': 49, 'nlines': 44},

 {'nshapes': 177, 'nlines': 88},

 {'nshapes': 30, 'nlines': 26},

 {'nshapes': 261, 'nlines': 123}]   
   
   Again, given that the whole document is under 400KB in size, I find it very difficult to believe that a modern graph-drawing app like LucidChart is being crushed by the weight of a few hundred shapes and lines. If it is, I have some questions about how your code is being written.
   
   It seems to me that this is not a “too many objects” problem, but actually a bug in the LucidChart app.

Thanks for your response. While of course this is disappointing, I appreciate your straightforward reply.

We’ll find a way to work with this limitation and continue to hope your developers can fix this soon.

I have an additional feature request for the Document Contents endpoint. Currently the exported shapes and lines do not have positions (i.e. XY coordinates for endpoints or shape corners). I would find it extremely useful if the Document Contents endpoint could include position information. After all, that’s kind of the main point of drawing a graph!

@Steve Strassmann 

Thank you for your understanding with the bug! Also, thank you for the additional feedback! That's a great idea and I can see how position information would add value when building graphs with the Document Content resource! I’ve passed along your feedback directly to our Product team. 

We’d love to hear any further thoughts or details you have on how this feature would improve your use of Lucid’s products and our APIs.

@Steve Strassmann Our Development team has thoroughly investigated the issue and identified its cause. They are now diligently working on a solution, though due to the complexity of the problem, resolving it will take quite some time. We will provide updates in this thread as progress is made, but we cannot specify an exact timeline at this moment. Thank you for your patience.

Hi,

Is there any update on this issue? The initial report of the issue is already 3 monmths ago.

I’m having the same problem while retrieving document contents:

{"code":"badRequest","message":"Bad or malformed request","requestId":"ddb0c3e61307b6cd","details":{"error":"Document <document id> is too large to retrieve contents."}}

Hi renekreijveld, unfortunately we don't have any update about the issue with the endpoint at this time, and we don't yet have a timeline on its' completion. This issue has helped us find some deeper technical limitations we want to address, which will need to be properly planned and prioritized before we tackle it. Sorry about any inconvenience this causes!