hamburger menu
+ Create a topic
Solved

Bogus intermittent "too large" error from GET /documents/{docId}/content

  • 17 April 2024
  • 7 replies
  • 46 views
S
Badge +1

I've developed an app which connects to the Lucid API and calls GET https://api.lucid.co/documents/{doc_id}/contents to retrieve the contents of a document.

Most of the time, my request is successful: I get back a 200 response along with the JSON document.

However sometimes the Lucid API server fails to return the document. Instead I see this error.

Why is this happening? The document in question is only about 360KB, so it's not particularly large, and it is certainly not being modified by anyone.

Error: {"code":"badRequest",

        "message":"Bad or malformed request",

        "requestId":"8964e70358ce692e",

        "details":{"error":"Document 62563ead-aa4a-4d29-8f2d-858a0a163b0d is too large to retrieve contents."}}
 

Context

The scopes used to create the access token are
const SCOPES = ["lucidchart.document.content", "offline_access", "user.profile"];

The document request is

const LUCID_API_HOST = "https://api.lucid.co";

const url = `${LUCID_API_HOST}/documents/${docId}/contents`;

const headers = {

    "Lucid-Api-Version": "1",

    Authorization: `Bearer ${accessToken}`,

    "Content-Type": "application/json",

  };



fetch(url, { method: "get", headers })

icon

Best answer by Richard U 18 April 2024, 17:07

View original

Comments

Richard U
Userlevel 3
Badge +7

Hi @Steve Strassmann, thank you for contributing to the Lucid for Developers Community and for reposting your question here! Since your ticket yesterday, I’ve had a chance to discuss this error with our Development team and we understand the confusion since this document size is quite small (360KB) yet we’re still getting this “Document is too large” error.

One explanation could be your document has many elements on the canvas (hundred or more) and has multiple pages with this many elements. In this case, the Document Content resource may fail to generate and the GET Doc Contents request returns this error.

Unfortunately our GET Document Contents endpoint doesn’t currently include a page parameter so the Document Content request will include all contents of a multi-page document, across all pages, and cannot be reduced to a page by page query. When you encounter this limit, the workaround will be to split the document up manually. 

 

Here are two methods for dividing larger multi-page Lucid documents into smaller documents with fewer pages:

Method 1: Duplicating then deleting pages*

  1. Duplicate the document
    • Select File > Duplicate within the editor
    • Or select Duplicate from the three-dot menu next to the document thumbnail/title on your Documents page
  2. Delete unwanted pages in the copy
  3. Repeat as needed

* Keep the original document as an archive.

Method 2: Use Import Page Feature

  1. Create a new Lucidchart document
  2. Within the editor, go to Insert > Import Page, navigate and select the document and the specific page to import using the preview and breadcrumb arrows
  3. Repeat as needed for other pages

Note: In case it is helpful, our Export Document endpoint supports single-page exporting (generates a document image) if that fits your use case.

 

Going forward

We recognize the potential for confusion and apologize for any inconvenience caused by the manual workaround. We have noted your feedback and are committed to improving our document contents export service. Currently, our API team is prioritizing other high-priority development projects for our API, so enhancements to this service may not occur soon but have been added to our backlog. It's important to understand that the document size limit for the GET Document Contents request does not consistently correlate with the memory size indicated (e.g. 360KB) in the document details. Therefore, larger documents with fewer elements may still successfully generate a Document Contents resource, often because the document’s memory consumption is attributed more to images than to the number of elements.

Please feel free to provide further questions or feedback in the comments below. We are eager to understand how you use our APIs and are here to assist troubleshoot or answer questions.

S
Badge +1

Hi Richard,

To be honest, I find this answer difficult to accept.

  1. First, the problem is intermittent. I sometimes have no problems retrieving the document and sometimes I get this error. If it was “too big” as you say, then I would expect the problem to arise consistently.
     
  2. Second, the error message is misleading. I think that short of actually fixing this bug, correcting this error message to avoid being misleading would be a step in the right direction.
     

  3. Third, I disagree with your assessment that my LucidChart document has a lot of elements. After a successful retrieval, I counted the number of elements. Across the 6 pages, the counts are:
    [{'nshapes': 203, 'nlines': 122},

     {'nshapes': 183, 'nlines': 106},

     {'nshapes': 49, 'nlines': 44},

     {'nshapes': 177, 'nlines': 88},

     {'nshapes': 30, 'nlines': 26},

     {'nshapes': 261, 'nlines': 123}]       

    Again, given that the whole document is under 400KB in size, I find it very difficult to believe that a modern graph-drawing app like LucidChart is being crushed by the weight of a few hundred shapes and lines. If it is, I have some questions about how your code is being written.

    It seems to me that this is not a “too many objects” problem, but actually a bug in the LucidChart app.

Richard U
Userlevel 3
Badge +7

Thanks for your reply @Steve Strassmann! As you’ve correctly noticed, this limit is variable and our Development team is investigating this as part of their ongoing work to improve the API and our document contents export service. Thank you for providing the details regarding the number of elements on each page of your diagram and letting us know that it managed to export eventually, despite reaching the limit earlier. I shared these details with our developers this morning and confirmed you’re encountering a bug with our Document Contents endpoint. Until this can be fixed, unfortunately the only consistent solution will be to either split up the diagram or retry the export until it works. 

 

We understand this is less than ideal, though we are committed to improving this feature and will update you in this thread once that work is complete. Please let us know if there’s anything else we can do to support you in the meantime. 

S
Badge +1

Thanks for your response. While of course this is disappointing, I appreciate your straightforward reply.

We’ll find a way to work with this limitation and continue to hope your developers can fix this soon.

S
Badge +1

I have an additional feature request for the Document Contents endpoint. Currently the exported shapes and lines do not have positions (i.e. XY coordinates for endpoints or shape corners). I would find it extremely useful if the Document Contents endpoint could include position information. After all, that’s kind of the main point of drawing a graph!

Richard U
Userlevel 3
Badge +7

@Steve Strassmann 

Thank you for your understanding with the bug! Also, thank you for the additional feedback! That's a great idea and I can see how position information would add value when building graphs with the Document Content resource! I’ve passed along your feedback directly to our Product team. 

 

We’d love to hear any further thoughts or details you have on how this feature would improve your use of Lucid’s products and our APIs.

Richard U
Userlevel 3
Badge +7

Thanks for your response. While of course this is disappointing, I appreciate your straightforward reply.

We’ll find a way to work with this limitation and continue to hope your developers can fix this soon.

@Steve Strassmann Our Development team has thoroughly investigated the issue and identified its cause. They are now diligently working on a solution, though due to the complexity of the problem, resolving it will take quite some time. We will provide updates in this thread as progress is made, but we cannot specify an exact timeline at this moment. Thank you for your patience.

Reply


Lucid Software Inc. Supplemental Community Terms

Get Started

Products

Solutions

Resources

Company

Privacy Legal Cookie Policy Cookie Settings
©   2024  Lucid Software Inc.