"SPIFFS Not Supported on Oak"

[postuma]

I've tried to get the SPIFFS file upload tool to work on Oak, both using the tool from https://github.com/digistump/OakCore/blob/master/doc/filesystem.md as well as the generic ESP8266 version. Both install fine, but return the same error when I select ESP8266 Sketch Data Upload from Tools in the IDE: "SPIFFS Not Supported on Oak".

I can use individual SPIFFS functions, but would rather not have to write include files with ASCII equivalents of hex codes to be written chunk by chunk into files. Text might be manageable, but too painful when starting with binary material.

Anyone have a fix?

[PeterF]

No fix, but I can reproduce that. I also wouldn't hold my breath... the esp upload tool mention in that doc is the 'official' ESP8266 Arduino one... so there is nothing 'special' about it, so unless someone else has work it out... we're sunk on that front! :-O

Damn... thought I was onto something... saw this mention of

Code: [Select]

oak.menu.FlashSize.OAK=OAK (4M/1M SPIFFS)
#this is in megabits - only needed for serial upload

... so thought maybe the SPIFFS upload would happen with a serial upload, but doesn't look like that was the case!

[postuma]

Plan B: write a sketch to copy files from the root directory of an SD card into SPIFFS, since SPIFFS file writes *do* work. *Should* be simple/straight-forward. Hope to get it done and tested over the next few days.

I can read directly from SD but this makes for a bulkier project, and the SD board uses 4 pins. It also draws about 200 mA at initialization - enough to mess with Oak's startup, on occasion, and it sometimes prevents flash updates from triggering, and even at 0.2 mA idle it can affect battery drain. Also, for projects that may experience the occasional shake, the actual card can come loose pretty easily ...

So SPIFFS does seem ideal - we got all this empty flash storage, free of charge.

Will post when I get it up and running.

[PeterF]

Good luck! Is there a particular reason the SD card has to *stay* part of the end unit? Why not load one sketch (or just a routine that triggers when you press and hold a button at power on, or issue a cloud command) so that you can connect the SD card when you need it, and have it disconnected the rest of the time... as you'll only need it when the SPIFFS file system needs to be reinitialized or whatever.

[postuma]

No absolute reason it can't stay; could power on/off by MOSFET. But it does use pins I'd rather keep free - I would have to use a port expander for one project, and it uses physical space I would rather not commit in another, very small project - plus the SD card itself could shake free in the second unless I glue it in place.

[postuma]

Oh yeah, and the other reason: my Oak often doesn't boot or flash if I've got the SD board connected. Confirmed again as I was testing my SD-to-SPIFFS code.

Which is working nicely, now. UPDATE: while it does upload the file to SPIFFS, and I can then detect it and read it from SPIFFS, both Oaks I've tested it on no longer work properly. Please don't use this code until we've verified there are no problems. However, I've left it in place so that others can see what we've tried so far, what the issues might be, and possibly how to solve it ...

One note: SD.h and FS.h (latter used for SPIFFS) have conflicting File definitions. An updated FS.h at https://github.com/esp8266/Arduino/blob/master/cores/esp8266/FS.h provides a solution: you'll need to include their #ifndef FS_NO_GLOBALS code into your FS.h file, then invoke #define FS_NO_GLOBALS before the FS.h include, as below:

Code: [Select]

#include <SPI.h>
#include <SD.h>

/* Note that the File definition in FS.h conflicts with that in SD.h. This is true for older versions of FS.h.
Either replace FS.h with a newer version, or copy the #ifndef FS_NO_GLOBALS code into yours, as found at
https://github.com/esp8266/Arduino/blob/master/cores/esp8266/FS.h */

#define FS_NO_GLOBALS
#include <FS.h>


int i;
char s[15];                                                       //character array for filename, SD uses 8.3 format
uint8_t filebuf[1024];

File fileSD;
fs::File fileSPI;                                                 //as above, library FS.h uses a different File definition from SD.h


void setup() {
  Particle.begin();

  bool result = SPIFFS.begin();
  if (!result) SPIFFS.format();

  if (!SD.begin(6)) {
    Particle.println("initialization failed");
    return;
  }

  File dir = SD.open("/");
  while (fileSD = dir.openNextFile()) {
    if (!fileSD.isDirectory()) {
      strcpy(s, "/");                                             //opening slash for files in SPIFFS
      strcat(s, fileSD.name());                                   //append file name on SD card
      Particle.print(s);

      fileSPI = SPIFFS.open(s, "w");                              //open in write mode
      if (!fileSPI) Particle.println("file creation failed");
      else while (i = fileSD.readBytes(filebuf, 1024)) fileSPI.write(filebuf, i);   //copy bytes from SD file to SPIFFS
    }
    fileSD.close();
  }
 
  delay(500);                                                     //delay causes Particle to report new output separately
  fs::Dir d = SPIFFS.openDir("/");                                //open SPIFFS 'directory'
  while (d.next()) {                                              //and read each file in sequence
    Particle.print(d.fileName());
    fileSPI = d.openFile("r");
    Particle.print(", Length: ");
    Particle.println(fileSPI.size());
    fileSPI.close();
  }
}


void loop() {
}

It works! Can read large files easily/quickly from SPIFFS now. No more PROGMEM!

[PeterF]

Nice! Glad it's working. I'll submit a pull request with necessary changes to FS.h so that they can be incorporated into the next update to the Oak files when Erik updates them next.

btw, I think you may have missed my point... I meant actually physically/completely remove the SD card (reader) once you've initialised the SPIFFS...  as once it's initialized and the necessary files loaded you won't need to format and upload files to it again will you (unless you want to reset it to match the SD card contents)? So program it, initialise it, and you've got your files in the SPIFFS and you'll get your free pins... and you'll only need the plug the SD module in if/when you need to re-initialise the SPIFFS.

[postuma]

  - you're right, I misunderstood what you were trying to say. That is exactly how I've done it. Now working on playing WAV files from SPIFFS via a MAX98357 board, using IIS. 

[PeterF]

One note: SD.h and FS.h (latter used for SPIFFS) have conflicting File definitions. An updated FS.h at https://github.com/esp8266/Arduino/blob/master/cores/esp8266/FS.h provides a solution: you'll need to include their #ifndef FS_NO_GLOBALS code into your FS.h file, then invoke #define FS_NO_GLOBALS before the FS.h include, as below:

I was simply going to replace FS.h in a pull request, but I see that some more bits were added to FS.cpp because of that, and also FSImpl.h, so I have submitted a pull request which brings all three up to date, as there is nothing existing code breaking in the changes.

Can I just confirm... all that is was needed was a replaced version of FS.h as linked, which then allows you to use #define FS_NO_GLOBALS to prevent a conflict between SD and FS? If so, I'll trigger the pull request.

[postuma]

Yeah, I noticed those. But when I implemented those other changes, I was getting even more errors. Since I did not need those other functions, I opted not to chase all their dependencies. Are you compiling OK since changing those files?

[PeterF]

Oh yuck... yeah, I missed that there was some virtual and override stuff... getting rid of that! I've rolled back the changes so it only includes the defines, as to do much more will mean needed to start refactoring a lot more of the underlying code. I think that will become one of my projects in the future though... to start refactoring the Oak code so we can take advantages of all the new stuff in the ESP8266 core... fingers crossed I get some time to do that

[postuma]

Kudos to you! I applaud your dedication & ambition - and thanks for all the help!

[postuma]

on a separate note - neither of the two Oaks I've tried this on, no longer work properly. The file did upload to SPIFFS, and I can read it from SPIFFS. On the downside, neither of these Oaks now flash properly. What actually happens is inconsistent. For example, using the BLINK sketch, when not in Config Mode, flashing message does not appear after "sending file", i.e.

Sending file to cloud, to flash Unnamed Device (Device ID: d957040015acac56745f580c)

There's no "Flashing ...." but eventually I do get an

Error : An error occurred while flashing the device:

Sometimes I do get the "Flashing.........." but the Oak light does not flash and it eventually times out. Reboot confirms flash did not work.

Only by shorting pin 1 to ground to enter Config Mode can I get it to flash. Then it sometimes flashes, sometimes stops flashing before the upload is complete. Even when the flash completes, it may reboot into Config Mode (nothing on pin 1), or into a do-nothing mode, or it may rarely do what the sketch intended (i.e. blink, or play a tone, or whatever), or it may revert to what the last sketch was, pre-flash.

Until I can figure this one out (or someone else does) I have to consider the possibility that either SPIFFS has borked my MCUs, or that my implementation of it has. So - beware. And if anyone figures this one out, please let us know.

[PeterF]

Just too enthusiastic to know any better... anyway, the more stuff the community can do, the quick the Oak can progress, and the less work for Erik

lol...you're not having much luck of late, are you! I'll find some time (and incentive) to hook up a SD card to an Oak, and see if I can break my test subject... or simply find out that postuma + oak = fail! :-P However, if config mode is going bonkers.... I'd think something else is going wrong, unless the firmware has somehow been corrupted. Try another flash when in config mode, but this time use http://build.particle.io and see if that takes. I have to use config mode with a TFT LCD screen setup all the time, and I just trigger it from OakTerm before the upload, and it completes most times.

[postuma]

Enthusiasm - not all bad.

Anyways, I've tested much more extensively and the above code *does* work, and my Oaks seem fine. The problem is one of flashing an Oak when connected to other hardware.

Observations:

1. Oak (often/usually) does not go into flashing mode if connected to a powered SD card board, with card inserted. These suck a great deal of power at startup; a documented problem with 5V Arduinos and presumably more of a problem with a 3.3V MCU. New SD cards especially are problematic. Possible solution: add a delay(2000) in setup to give time to either start a flash, or connect the power pin or push the SD card in all the way immediately after the Oak powers up

2. If connected to an SPI board, i.e. TFT screen, audio/amp board, etc., and the active sketch interfaces with these, an OTA flash often does not work. Shorting pin 1 to trigger Config Mode then does allow it to flash - usually

3. If step 2 fails, it will usually succeed if trying a few more times. If it doesn't, disconnect the SPI board and try again. Once the sketch uploads correctly, reconnect

I've tested the SD to SPIFFS code with as many as 8 files at once, as many as 612,080 bytes transferred to SPIFFS at once, and have verified that all files transferred cleanly. Additionally, the Oaks appear to be working properly, but will continue to test