Hi,
I was wondering if it is possible to store sets or vectors as variables on the SPI flash memory on the whisper node for reading and writing to. I'd like to use the arduino IDE if possible.
The most likely datatype within the set or vector is going to be an unsigned short or unsigned integer if this changes the answer. I know the progmem library exists, but I was hoping to use the included storage that comes with the whisper node. Similarly if it would be possible to treat these like local variables that would be great, but if I have to call them specifically that shouldn't be a problem for my project.

Thanks,
Alex

Hi Alex,

At low-level you can only read/write bytes from different SPI Flash addresses. This happens via SPI commands sent to the SPI Flash, followed by memory address and content. There are some example functions in the Talk2 Library for reading and writing from flash, but many other Arduino Libraries will work as well.

You need to remember that SPI Flash is not the same as RAM. You can't simply store "variables" there.  You can have a variable in your code, which will live in the MCU RAM, and load the contents with data from the Flash Memory.

One approach that might work for you is defining some kind of data structure, which you can use to "locate" the correct memory address from the SPI Flash. For example, you can determine that each entry in your SPI Flash is 20 bytes long, and in those 20 bytes long, you create some standard that tells you the data type, the actual value, etc. After that, you develop functions to scan all those 20 bytes long entries and process them according to your needs.

Also, instead of using stand-alone variables, working with Struct types can make things easier as you could simply load a full structure sequentially, loading byte by byte from the SPI Flash memory into it.

Hi,
Thanks for your answer, it helps with my further planning.
Do you know if a structure is capable of handling a vector or set, and then saved on the SPI flash. As you may have seen from my other question, I'm likely going to save shorts or integers to serve as IDs for my project, so theoretically it should iterate 2 or 4 bytes per value with the structure.
I'd also like to try to reduce writes as much as possible to increase longevity of the device as much as possible and I hope to have the ability to remove data from my vector/set while "unallocating" the space. From what I've read, SPI Flash probably won't be able to achieve this as it doesn't have a full file system/index like a SD card with a FAT file system, and may be unnecessary depending on the eventual amount of data that I store for the lifetime of the device vs the endurance of your SPI chip.

Any insight into this, either in general or specific to the whisper node, would be greatly appreciated.

Thanks

Hi Alex, you can have something like: Basically, the variable mySensor, in the memory will be a byte string with all values in the Struct, one next to the other. You could create a function that reads byte by byte of the "mySensor" variable and stores it into Flash... or the opposite reads from Flash and stores in a variable type Sensor.

Similar to the "array" of Sensor, it will be a concatenation of all bytes of all sensors. You could read all, byte by byte from/to the SPI Flash.

This way when functions that read/write to the SPI flash don't really need to know what they are reading or writing... you just need to read the correct number of bytes, starting from the correct memory address.

Hi,
I was wondering if you were able to provide some example code for reading and writing sequentially to the SPI flash. I want to assume for now that my array will be too big to fit in the SRAM of the whisper node so only parts, i.e. one struct stored in the written array at a time, or would it be better to do something like: I know this probably isn't correct, but it hopefully gets my idea across. If this is what you meant from your previous reply, thanks.

I'm just having a difficult time looking online to read and write larger datasets to SPI flash and reading it back in an effective manner.

Thanks

AlexMH wrote: ↑

May 18, 2022

Hi,
I was wondering if you were able to provide some example code for reading and writing sequentially to the SPI flash. I want to assume for now that my array will be too big to fit in the SRAM of the whisper node so only parts, i.e. one struct stored in the written array at a time, or would it be better to do something like:

Code: Select all

while(receivingData()){
  address = 34; //memory address of SPI memory
  unsigned short data = getData();
  SPI.write(address, data);
  address += sizeOf(data);
}

I know this probably isn't correct, but it hopefully gets my idea across. If this is what you meant from your previous reply, thanks.

I'm just having a difficult time looking online to read and write larger datasets to SPI flash and reading it back in an effective manner.

Thanks

I've tried this code in different variations. It did not work correctly for me. Have you found other ways to solve your problem?