loadTable on a very large data set causes a empty p5.Table to return and no errors

[m00nwtchr]

Hi I have a very large tsv file (over 2 000 000 lines) and loadTable refuses to work with it can someone help me? if doing this large oprations in p5js is not possible then can I do it in processing?

[limzykenneth]

Hi @lukas2005, that tsv sounds rather large and we probably never tried something that big before, is it possible to share the file so we can test it?

For Processing, see here

[m00nwtchr]

Here you go
https://1drv.ms/u/s!AocAtp8GO8YPoy6exjzdPogsiBpx

[m00nwtchr]

PS i succeded to import this tsv file on processing with memory max set to 4 GB

[limzykenneth]

@lukas2005 Sorry, I didn't manage to get back to you promptly. The link is no longer working. Actually, instead of sharing it, can you just let us know how big the file is? Both the file size and the number of columns would be great, we can probably generate some random file ourselves from those stats.

If you need 4GB of memory on Processing to import this file, the. I think you may not be able to do it with p5.js easily if at all but I still like to at least try.

[m00nwtchr]

it has 316 MB and has 10 columns

[TanviKumar]

#3127 solves this

[lmccart]

fixed with #3127