In probing in to the chilipeppr codebase for cpu+memory optimization I quickly realized that lots of widgets all have their own copy of the gcode. This typically isn’t a problem, most gcode files for common tasks are going to be less than 200k, However, for my use cases, I’m getting gcode files >700k, which I’m having to break up in to smaller jobs to avoid everything grinding to a halt (even on my laptop).
Im proposing some time to sit and ponder a central gcode container, in which we adapt all components to utilize a shared workspace object instead of each maintaining their own copy. I understand that this would be quite an undertaking (I did just work on the 3d-viewer ), but it could gain us an incredible performance increase. Another thing that needs to be explored is an alternative to using local-storage to cache gcode.
I think that the way pubsub works in ChiliPeppr as a core method of widgets talking to each other has worked out to be an amazing way to scale. So I’d rather not break that to do something like create a global variable for the Gcode file. Rather, each widget needs to nicely release the memory if it doesn’t need it’s own copy. That is what I think the current design does. My guess would be that you’re looking at Gcode widget and 3D viewer widget as both having their own copies, and that is true. Beyond that though, no other widget I can think of has it’s own copy. Is that consistent with your analysis?
With the normal process of somebody dragging in a Gcode file, Eagle BRD wouldn’t have it’s own copy. In the process of them creating an Eagle BRD, then indeed it would have a copy, but in that case you’re really doing CAM work in ChiliPeppr and I think that’s appropriate. You could even possibly release that Gcode from memory in the Eagle widget, but it would save you almost no RAM.
I also think that having a 5MB Gcode file in memory is not where the problem lies. I think the crashing seen on massive Gcode files is the 3D viewer. I think each Gcode line converting to a 3D representation uses 10-fold the memory of the string itself. I’ve optimized this in the past, even moving to Three.js’s most efficient rendering method, yet it still isn’t good enough. Going further will be a challenge.
So, i’d rather focus on the area where the problem is, rather than optimize stuff where I really don’t think any issues are created.
@jlauer I totally agree with the pubsub comment. If the eagle brd stuff doesn’t store a chunk of gcode I guess that’s ok. I’m going to look further in to the viewer logic, but if there was a way that it didn’t have to store a copy after building the mesh, that would be ideal. Large files still seem to crap out with the viewer totally disabled, so I’ll spend some time on the gcode import code path. I don’t care if it’s slow, I just care that chromium keeps killing the page due to non-response (with viewer disabled).
), but it could gain us an incredible performance increase. Another thing that needs to be explored is an alternative to using local-storage to cache gcode.