The Oak by Digistump > Oak Support
Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates
postuma:
I've struggled enough with this I figured it deserved its own thread - and when I say struggled, I mean at least a half dozen hours to get it working.
So - why bother?
Despite the difficulties, it's already been worth it:
* Speed! Reliability! Approximately 30 seconds for a simple serial flash that works every time, and 8 seconds for an over-the-air flash ...
* Hours of fail-after-fail-after-fail Particle updates - more than I've invested in the fix thus far - gone now
* Previously, even when the code updated ... you couldn't always tell it failed. Not failing now
* I've had two Oaks that I've never been able to "claim" using Particle - they work fine now
* Code fixes that haven't made it to the Oak have long been implemented into the ESP8266 core
* Compatibility: newer libraries may not be compatible with the Oak, like the newer versions of the Adafruit GFX libraries. They are now!
* Serial output - Particle limited messages to 255 characters, a few times per second. No such limits on Serial output
* Getting code into SPIFFS required copying files to the SD card, then copy from SD card to SPIFFS - and the SD card frequently does not properly initialize - hours wasted here as well. With the change to ESP8266 core, the ESP8266 Sketch Data Upload tool becomes available, and directly and oh-so-easily transfers data to SPIFFS
I was a reluctant to make the change. What if things broke irrepairably? But Particle is so broken I felt I had no choice.
To his credit, Erik has acknowledged these problems - see https://www.kickstarter.com/projects/digistump/oak-by-digistump-wi-fi-for-all-things-arduino-comp/posts/1912567. He's also put a big disclaimer on the Digistump order page - caveat emptor. And I have no doubt he will address these issues in some way, at some time. But family comes first! and I've no doubt he's way over-extended (very excited about the BuildOne printer, BTW).
You'll need a USB to TTL adapter to program the Oak using a serial interface, found el cheapo on Ebay: http://www.ebay.com/itm/USB-to-TTL-CH340G-Converter-Module-Adapter-STC-Dowanloader-Programmer-3-3V-5V-/112215846564.
Follow the Brendan 'nog3' Halliday tutorial at http://nog3.net/2017/06/05/removing-particle-from-oak/. Things to note:
On Python:
* You have to have Python installed on your system (typically native on linux, not on Mac or Windows). Version 3.x may not work with the Arduino IDE; use 2.7. Use the installer
* Windows: make sure that Python is installed on the system PATH (use the installer; this is NOT selected by default)
On esptool.py:
* You'll need the esptool.py script - and the version in your Oak install may not work. No biggie. The one in the generic ESP8266 install does work
Alternately, the new-and-improved esptool.py can be had from https://github.com/espressif/esptool. Works nicely, as of this writing
The new install does not overwrite the Oak's original information; it creates new entries for the Oak-as-ESP8266. Don't hesitate to install the new package
Backing up your firmware after install: do it!
On the USB-to-TTL adapter:
* The USB-to-TTL serial adapter draws enough power from the USB port to power itself - but often not quite enough to power the Oak (especially using the 3.3V pin out; you may have luck with the 5V pin.) Best to power it separately using the micro-USB port, or using another power source for VIN.
* If powering the Oak independently, you don't need to connect either power-out pin from the adapter to your Oak. I don't - you'll only need to disconnect the power source to power-cycle your Oak, which you may do frequently. And if you're using more than 5V on VIN, you should not - the higher reverse voltage could damage the adapter
* On Windows: always use the same USB port for your serial adapter - if you happen to use another port by mistake, the COM port will change, but there won't be an error message that tells you so
* Windows - with the USB-to-TTL module plugged into the USB port you intend to use, click on Start > Settings > Devices and you should see it. Note the COM port used. With older version of Windows, you'll find it under Device Manager
* Serial port for Mac/linux is typically something like /dev/tty.SLAB_USBtoUART; dmesg | grep ttyUSB should get you the name
* To back up your current entire firmware, including SPIFFS, using Windows on COM4 would look something like
esptool.py --baud 115200 --port COM4 read_flash 0x000000 0x400000 flash_c70a.bin
(I use the last four hex values of the Oak's MAC address/Particle device ID to identify them/keep them straight - I write them on the back of the module in indelible marker. Here, I use it to uniquely identify the backup of each bin file as flash_xxxx.bin)
* Wiping flash on my Windows system, on COM4:
sptool.py --baud 115200 --port COM4 erase_flash
* If you regret your decision and want to restore, on Windows:
esptool.py --baud 115200 --port COM4 write_flash -fm dio 0x000000 flash_c70a.bin
Modifying the Arduino IDE:
* Your new entry in the variants folder is a sub-folder containing the file pins_arduino.h. These remap the ESP8266 pin definitions to our Oak equivalents, with a caveat: 1 still refers to the ESP8266 GPIO pin 1, 2 refers to the ESP8266 2, etc. The new versions are P1, P2, etc. What I like better - also defined are SDA, SCL, SS, MOSI, MISO, SCK, BUILTIN_LED/LED_BUILTIN, and A0. I no longer have to remember which maps to which!
* You may have to update libraries that are no longer compatible with the ESP8266 core. Example: my old Adafruit GFX and ILI9341 TFT libraries don't work with the new core
Using the Arduino IDE:
* Board selection: find esp8266 using Filter your search under Boards Manager, and load
* Traditionally the Oak uses 4 MB flash/1 MB SPIFFS; this is preferred if you intend to use OTA (as OTA updates actually store two sketches in memory). However, if updating only over serial then you're directly overwriting the old sketch with the new - 1 MB for program should be more than ample, and presumably 4M/3M SPIFFS is an option - lots of on-board storage! I haven't tested/confirmed this yet, however
* You need not disconnect pin 2 from ground after flashing to enter run-progam mode; the Oak will reboot automatically and run its program
* IF flashing again after a successful flash without first power-cycling the Oak, you'll get a lengthy, uninformative error message. It's rebooted into run-program mode, not bootloader mode. Disconnect both the adapter and your power source from the Oak (either one provides enough power to keep it turned on), and reconnect. Alternately, if you haven't connected the USB-TTL adapter power pin to your Oak, just disconnecting the Oak's power source will do
* Serial messages work much like Particle (only better). Remember to specify a baud rate when initializing, i.e. Serial.begin(115200). Also, to get useful info from the IDE's Serial Monitor, remember to set it to the same baud rate (mine defaulted to 9600)
* I've had javascript errors/upload fails that only resolved when I upgraded the Arduino IDE - but can't guarantee that this won't introduce new issues. Created a fresh backup before the upgrade just in case; hope not to need it
* If you're unwilling to take this step but still get the Javascript errors, or if this did not work, Sketch > Export compiled Binary also worked. I just had to manually upload the binary, i.e.
esptool.py --baud 115200 --port COM4 write_flash -fm dio 0x000000 blink.bin
Assuming all went well - and please reply if it didn't, let's figure out how to make it work consistently and well - click on Sketch > Upload to flash the new core and sketch to your Oak.
As mentioned, I was also (finally) able to load files directly to SPIFFS using the Sketch Data Upload tool. See https://github.com/esp8266/arduino-esp8266fs-plugin.
Anything pertinent that I've missed?
Now, on Serial over OTA:
Lovely, once you get it to work.
Notes thus far:
* Basic information/install info at http://esp8266.github.io/Arduino/versions/2.0.0/doc/ota_updates/ota_updates.html
* I've tried both Classic OTA and ArduinoOTA and don't know what the relative advantages are - but the former is deprecated; I'd go with the latter
* Find and load BasicOTA.ino from Sketch > Examples > ArduinoOTA
* For ssid and pass use the name and password for your local network
* If there's any chance of your Oak getting hacked (its WiFi range is respectable) I would also uncomment and set ArduinoOTA.setPassword((const char *)"somethingunique") to something unique. Not too hard, as you'll need to enter it for every OTA update
* If your Oak connects to the network, you'll see something like
Ready
IP address: 192.168.2.18
* Potential problem: you look under Tools > Port, and the expected OTA port does not show. From the command line, ping your IP - this should work. If it does, the problem is probably mDNS packets - either your router's blocking them, or your PC isn't processing them. On Windows systems, the problem is often that the Mac/Apple-specific Bonjour service, used for mDNS messaging, is not installed. Use Task Manager or msconfig to check that it is running. For me, downloading and installing Bonjour Print Services for Windows, then restarting the Arduino IDE, did the trick. (Also, please note that Bonjour may be blocked by some anti-viruses; firewalls will need to have UDP 5353 open for it to work properly)
* Don't worry if you don't see an “Upload Using: OTA” option: I've found that selecting the esp8266-xxxxxx network port is sufficient
* You may see a "missing 'runtime.pwd.192.168.2.18' configuration parameter" error message initially - my sketches upload fine!
* Make sure pin 2 is not connected to ground for an OTA upload - the Oak has to be up and running in program mode
* Finally, unless you embed the OTA update code into your new sketch, you'll get a "[ERROR]: No Answer" message on your next upload - the OTA protocol has to be installed to be ready for it
One thing I haven't figured out yet - whether I can get Serial.print feedback back to me, over the air? Anyone?
So - feedback very welcome. Other problems/solutions?
Paul
exeng:
Paul,
Thanks for this post and all the effort and detail that you obviously put into it. Also, thanks to Brendan (nog3) for his description of how to remove Oak from Particle.
While I have experienced the occasional fail OTAs and misrepresented fail/success of OTA's, I'm still able (after multiple retries) to get my Oak sketches OTA flashed. At a point where this becomes even less stable, I'll will most likely give this a go. I'm not dissing Particle but I never use their dev environment and only post a few bits of data there from my operational Oaks. Largely, this is due to the lack of (not sure if this is still true) a means to log data. I'm a fan of Thinkspeak. Others may have their personal favorites but I find the ability to keep historical data useful.
So again, thanks for bringing this solution to our attention.
exeng
PeterF:
Paul, I would also like to thank you for the writeup ;)
Like exeng, I'm still using Particle on my Oaks (bar one, a ESP8266 Arduino OTA guinea pig), but haven't programmed them in a while, so could quite quickly decide to jump ship should they start being 'lil s... ugars ;) I would like to add the following:
* Arduino OTA is indeed FAST! ;) You just have to remember to add the OTA.handle function, and give it the chance to do it's thing... it won't be 'always watching' like the Oak Particle firmware, unless you program it that way.
* True serial on the Oak was never limited AFAIK, just the Particle Serial implementation. You could always Serial.print() and monitor via a Serial UART, but it just wasn't as convenient.
* Unfortunately I don't think they've implemented any form of Serial over WiFi, so you won't be able to get that back in a simple fashion. Will probably have use some form of logging if you want that functionality, else, stick with hard-wired serial.
* Newer version of the docs (2.3.0 vs 2.0.0) is here (annoyingly the 'other versions' link at the top never worked, so you have to specifically look for the matching documentation for the current release) => http://esp8266.github.io/Arduino/versions/2.3.0/doc/ota_updates/readme.html
postuma:
Good observations - thanks for the additions!
DrJFM:
I also noted Erik's Kickstarter update acknowledging the issues with Particle. I also found that after a 2 to 3 month break from Oak use, I could not even log in to Particle Dashboard. Rather than spend time trying to recover my account at Particle, I decided to follow Brendan 'Nog3' Halliday's tutorial.
--- Quote ---To his credit, Erik has acknowledged these problems - see https://www.kickstarter.com/projects/digistump/oak-by-digistump-wi-fi-for-all-things-arduino-comp/posts/1912567. He's also put a big disclaimer on the Digistump order page - caveat emptor. And I have no doubt he will address these issues in some way, at some time. But family comes first! and I've no doubt he's way over-extended (very excited about the BuildOne printer, BTW).
You'll need a USB to TTL adapter to program the Oak using a serial interface, found el cheapo on Ebay: http://www.ebay.com/itm/USB-to-TTL-CH340G-Converter-Module-Adapter-STC-Dowanloader-Programmer-3-3V-5V-/112215846564.
Follow the Brendan 'nog3' Halliday tutorial at http://nog3.net/2017/06/05/removing-particle-from-oak/.
--- End quote ---
After a moderate amount of extra time getting a never previously powered up Oak turned into a ESP8266Oak, I decided to pop in here and see if others were also making the transition. Wish I had looked in here sooner -- might have saved some time! That said, I really just wanted to encourage others to make the switch. It is really not bad. If you have ever done a serial flash to restore the Oak, this is easier. Paul's further hints, directions and clarifications address some of the few issues that slowed the process for me and should help assure success.
I will likely move to the OTA loading just to say I have, but I see myself more than content to use the serial upload process for most of my hobby projects. I have been most satisfied using Blynk to monitor and control my Oaks and spent only about 1 hr (in part because I have not been using Arduino/Oak much lately) porting my Temp Humidity monitor to my newly configured ESP8266Oak. One of my older registered Oaks had been updating my Blynk app w Temp/RH data for a couple months, most likely with absolutely no Particle connection. I moved the level shifter shield to the ESP8266Oak and deleted a bunch of Particle overhead. A couple tweaks and back to knowing when my dehumidifier needed to be emptied using my Blynk Android app.
So some bottom line thought:
Do it, it is not a hard process
There is a robust ESP8266 community that has likely solved most of your future problems
Erik sounded to me like any Cloud server/services he will develop will be based on the ESP8266 Core
It is great to type in a new SSID and Password and upload over serial in about 30 sec
As long as the ESP8266Oak can easily transmit over WiFi, there are several non-particle systems to allow remote interactions
I did a fresh install of Arduino 1.8.3 before I started. The ESP8266 core is loaded by the Board Manager after adding in the link as described. No python flash needed! I will likely convert my last "new" Oak and maybe some of the registered ones without worrying about the python backup -- just the erase! The ESP8266 Core I installed was 2.3.0. In my Windows 10 system, the files from the board manager end up in my AppData folder ie
C:\Users\UserName\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.3.0\variants -not in my sketch folder. For anyone using Blynk, I added the most current Blynk Arduino library, which includes BlynkESP8266 version, not the one in the Oak download. I backed the Blynk Kickstart campaign so have lots of juice (ie $) on Blynk but would likely pay for the service as a very easy way to get data and control to you phone.
Finally, should we see if they will set up a ESP8266Oak branch of the forum?
Cheers.
James
Navigation
[0] Message Index
[#] Next page
Go to full version