Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates

[exeng]

You have got to be kidding me!.... Grabbed another Oak to de-Particle (since it had failed back to a ready to upload state and was not running the loaded sketch) and just for fun tried to upload the simple LED blink code using Particle. It took about 4 to 5 retires but damn it loaded the sketch. It's just maddening knowing that is probably some stupid but simple bug that is causing it to be unreliable.  Does anyone have a clue?  Erik is probably the best resource but he seems busy with other more profitable things.

UPDATE: Well the successful upload via Particle (after 4 to 5 retries) was just a one time event.  It's back to endless retries with no success, so that's it, time to de-Particle the rest of my Oak's and be done with it.  I'm probably the last one to do this.  I assume most, if not all, Oak owners have already moved on and removed their Oak's from Particle.

[exeng]

Just an update: Converted 4 of 5 built Oaks to non-Particle generic ESP8266s and couldn't be happier.  They are all up and running doing their job as monitors for weather, garage door, and an aggregate OLED display of all their feeds through Thingspeak and control via Blynk.  Gone are the fights and failures with Particle OTA. Now I can focus on developing again. The only reason the 5th one has not been converted is I need a temperature probe to complete a pool temp monitor. My old one bit the dust. Also, have 5 more new Oaks looking for something to do.

If you are still fighting Particle, convert. It's not that hard. Be sure to back up your Oaks and keep a record of the device IDs just in case the Particle stuff is fixed down the road.

exeng

[Rboyer]

Glad to see that the wiki is still active. I'm about to de-Particle my Oaks and get them to be useful. Thanks to all for the guidance - wish me luck!

[PeterF]

Glad to hear you're liking life post Particle... it is sad that things didn't work out with the Particle integration... but not point dwelling over it. I've not regretted moving mine over to the official ESP8266 Arduino core... get all the bells and whistles that come with all the improvements they've made to their core since Erik forked it for the Oak, and their OTA support is such much better now... and if you want to do remote upload you can still do that if you want.

[nirajbhatt7]

Is this still valid in 2020? I bought few oaks in kickstarter struggled at the beginning so they went in a box and now I have found them again and wanted to put them to good use.

[bratoff]

Yep, still valid for the most part.  Once you move away from the broken Particle environment, the Oak is basically just an ESP-12F module plus power supply regulation, and with a quasi-Digispark-compatible pin layout.  The ESP8266 Arduino core now supports Oak directly so you don't need to use the special core here - you just need a 3.3v serial interface to flash code into it.  Have fun!

[SHadowX]

I have converted one of my Oaks to Arduino ESP8266 Core a while back and it worked fine. But now when I reflashed it with Platform.IO and left it off overnight it would not boot anymore and upon inspection it just kept drawing about 500 mA at 3.3 V while cooking itself silly quite in a hurry.

I thought it was just bad luck but when the same happened to two of the other Oaks I was left completely baffled.

The last one was flashed multiple times, restarted and everything worked fine, but as soon as I completely removed it's supply voltage (not using the reset pin to restart it) it went into the same unresponsive power hungry state as the others.

None of them will respond on the serial port, even though I try to start them in serial programming mode by holding P2 to GND and they just sit there trying to crisp themselves with 450 to 800 mA of current at 3.3 V.

Does anyone have an idea if they are salvageable or should I just forget about them, toss them in the bin and get a more mainstream ESP board?
 

[StickyC]

Nog3's blog referenced in the first post has been wiped. It does live on in the Internet Archive though (bless their hearts - they've saved me countless times recovering old hack documentation - donate!)

I'll repost the text of it here... If Nog3 doesn't like it, he can bloody well restore his blog.
--------------
So a while back I backed the Digistump Oak Kickstarter. I thought these boards were great and I'd supported other Digistump projects by backing or purchasing Erik's dev boards.

One of the big drawcards was over the air updates via Particle and at the time the ESP8266 Arduino Core didn't have great OTA support so this was win-win. (It's now waaaay better.)

Now the boards are great and the Oak support for arduino and the firmware was solid. The one thing I never found consistantly solid however was the Particle OTA support. I'd be able to flash it successfully a few times, come back a few months later and then flashing would just randomly and annoyingly fail.

Each time it failed I'd spend days troubleshooting, manually wiping the firmware/bootloader and replacing it until it kinda worked. The Oaks would phone home properly to Particle and show up on the Dashboard but would either stop recieving data during a flash or just stop responding.

Eventually when I came back to work on a project I'd built on the Oak pinout I finally gave in as I spent all the time I'd intended to use for development on fault finding the Particle OTA.

At its core, the Oak is simply an ESP8266 12E with a 4mb flash on it. I've had plenty of experience flashing them using esptool.py and esptool-ck, so I did what any self respecting person did.

Firstly, I wired a Serial -> USB Adaptor to it:
https://digistump.com/wiki/oak/tutorials/serialfirmware was a good primer for how to wire it and install esptool.py.

Make sure you connect P2 to ground, or the ESP8266 will not boot into serial flash mode.
My setup looks like this. (missing pic)

Then I backed it up using esptool.py:

Code: [Select]

python esptool.py --baud 115200 --port /dev/tty.SLAB_USBtoUART read_flash 0x000000 0x400000 flash_devicenumber.bin
You will definitely need to adjust the --port part of this command to suit your circumstances (COM on windows, usually /dev/tty. on mac and similar on linux) and make sure you put the device number or a friendly name in for your.bin file so if you ever have to go back you're not totally screwed.

Then, I wiped it with esptool.py:

Code: [Select]

python esptool.py --baud 115200 --port /dev/tty.SLAB_USBtoUART erase_flash
Again, change the port to whatever port your serial-usb adaptor shows up as on your relvant OS.

Your Digispark Oak is now Particle free
But it's still not entirely usable. At this point you could:

Flash it using the Arduino ESP8266 Core and use ESP8266 OTA
Flash it with Nodemcu (Lua), Micropython, Espruino or half a dozen other projects.
I expect most people will want the first option as it means you can use local or HTTP based Over The Air updating.

But we need to tell the Arduino ESP8266 Core about the Digispark Oak as the pin numbers on the board don't match what pins are being used on the ESP8266.

Telling Arduino about Oak's pins.
First thing you'll need to do is to create a variant in the hardware folder for the ESP8266 core. These instructions are assuming you've already installed the core.

On Windows 10, this is in your user's Documents folder. The full address will look something like:

Code: [Select]

C:\Users\Username\Documents\ArduinoData\packages\esp8266\hardware\esp8266\2.3.0\variants
On Mac, it's in your user's Library folder, so the location is:

Code: [Select]

/Users/<USERNAME>/Library/Arduino15/packages/esp8266/hardware/esp8266/2.3.0/variants
The version numbers may change when new releases of the core come out, so you will need to double-check.

To begin, unzip this zip file in the variants folder. This should create a folder called DigiStumpOak with a file inside it called Pins_Arduino.h . For those of you who'd rather not download a zip file, here's a gist with it.

Then, we will need to add some lines to boards.txt, which is in the folder above the variants folder.

At the end of the boards.txt file, add the following:

Code: [Select]

##############################################################
oak.name=Digistump Oak

oak.upload.tool=esptool
oak.upload.speed=115200
oak.upload.resetmethod=ck
oak.upload.maximum_size=434160
oak.upload.maximum_data_size=81920
oak.upload.wait_for_upload_port=true
oak.serial.disableDTR=true
oak.serial.disableRTS=true

oak.build.mcu=esp8266
oak.build.f_cpu=80000000L
oak.build.board=ESP8266_OAK
oak.build.core=esp8266
oak.build.variant=DigiStumpOak
oak.build.flash_mode=qio
oak.build.spiffs_pagesize=256
oak.build.debug_port=
oak.build.debug_level=

oak.menu.CpuFrequency.80=80 MHz
oak.menu.CpuFrequency.80.build.f_cpu=80000000L
oak.menu.CpuFrequency.160=160 MHz
oak.menu.CpuFrequency.160.build.f_cpu=160000000L

oak.menu.FlashFreq.40=40MHz
oak.menu.FlashFreq.40.build.flash_freq=40
oak.menu.FlashFreq.80=80MHz
oak.menu.FlashFreq.80.build.flash_freq=80

oak.menu.FlashMode.dio=DIO
oak.menu.FlashMode.dio.build.flash_mode=dio
oak.menu.FlashMode.qio=QIO
oak.menu.FlashMode.qio.build.flash_mode=qio
oak.menu.FlashMode.dout=DOUT
oak.menu.FlashMode.dout.build.flash_mode=dout
oak.menu.FlashMode.qout=QOUT
oak.menu.FlashMode.qout.build.flash_mode=qout

oak.menu.UploadSpeed.115200=115200
oak.menu.UploadSpeed.115200.upload.speed=115200
oak.menu.UploadSpeed.9600=9600
oak.menu.UploadSpeed.9600.upload.speed=9600
oak.menu.UploadSpeed.57600=57600
oak.menu.UploadSpeed.57600.upload.speed=57600
oak.menu.UploadSpeed.256000.windows=256000
oak.menu.UploadSpeed.256000.upload.speed=256000
oak.menu.UploadSpeed.230400.linux=230400
oak.menu.UploadSpeed.230400.macosx=230400
oak.menu.UploadSpeed.230400.upload.speed=230400
oak.menu.UploadSpeed.460800.linux=460800
oak.menu.UploadSpeed.460800.macosx=460800
oak.menu.UploadSpeed.460800.upload.speed=460800
oak.menu.UploadSpeed.512000.windows=512000
oak.menu.UploadSpeed.512000.upload.speed=512000
oak.menu.UploadSpeed.921600=921600
oak.menu.UploadSpeed.921600.upload.speed=921600

oak.menu.FlashSize.4M1M=4M (1M SPIFFS)
oak.menu.FlashSize.4M1M.build.flash_size=4M
oak.menu.FlashSize.4M1M.build.flash_ld=eagle.flash.4m1m.ld
oak.menu.FlashSize.4M1M.build.spiffs_start=0x300000
oak.menu.FlashSize.4M1M.build.spiffs_end=0x3FB000
oak.menu.FlashSize.4M1M.build.spiffs_blocksize=8192
oak.menu.FlashSize.4M1M.build.spiffs_pagesize=256
oak.menu.FlashSize.4M1M.upload.maximum_size=1044464

oak.menu.FlashSize.4M3M=4M (3M SPIFFS)
oak.menu.FlashSize.4M3M.build.flash_size=4M
oak.menu.FlashSize.4M3M.build.flash_ld=eagle.flash.4m.ld
oak.menu.FlashSize.4M3M.build.spiffs_start=0x100000
oak.menu.FlashSize.4M3M.build.spiffs_end=0x3FB000
oak.menu.FlashSize.4M3M.build.spiffs_blocksize=8192
oak.menu.FlashSize.4M3M.upload.maximum_size=1044464

oak.menu.ResetMethod.ck=ck
oak.menu.ResetMethod.ck.upload.resetmethod=ck
oak.menu.ResetMethod.nodemcu=nodemcu
oak.menu.ResetMethod.nodemcu.upload.resetmethod=nodemcu

oak.menu.Debug.Disabled=Disabled
oak.menu.Debug.Disabled.build.debug_port=
oak.menu.Debug.Serial=Serial
oak.menu.Debug.Serial.build.debug_port=-DDEBUG_ESP_PORT=Serial
oak.menu.Debug.Serial1=Serial1
oak.menu.Debug.Serial1.build.debug_port=-DDEBUG_ESP_PORT=Serial1

oak.menu.DebugLevel.None____=None
oak.menu.DebugLevel.None____.build.debug_level=
oak.menu.DebugLevel.Core____=Core
oak.menu.DebugLevel.Core____.build.debug_level=-DDEBUG_ESP_CORE
oak.menu.DebugLevel.SSL_____=Core + SSL
oak.menu.DebugLevel.SSL_____.build.debug_level=-DDEBUG_ESP_CORE -DDEBUG_ESP_SSL
oak.menu.DebugLevel.SSL_MEM_=Core + SSL + TLS Mem
oak.menu.DebugLevel.SSL_MEM_.build.debug_level=-DDEBUG_ESP_CORE -DDEBUG_ESP_SSL -DDEBUG_TLS_MEM
oak.menu.DebugLevel.WiFic___=Core + WiFi
oak.menu.DebugLevel.WiFic___.build.debug_level=-DDEBUG_ESP_CORE -DDEBUG_ESP_WIFI
oak.menu.DebugLevel.WiFi____=WiFi
oak.menu.DebugLevel.WiFi____.build.debug_level=-DDEBUG_ESP_WIFI
oak.menu.DebugLevel.HTTPClient=HTTPClient
oak.menu.DebugLevel.HTTPClient.build.debug_level=-DDEBUG_ESP_HTTP_CLIENT
oak.menu.DebugLevel.HTTPClient2=HTTPClient + SSL
oak.menu.DebugLevel.HTTPClient2.build.debug_level=-DDEBUG_ESP_HTTP_CLIENT -DDEBUG_ESP_SSL
oak.menu.DebugLevel.HTTPUpdate=HTTPUpdate
oak.menu.DebugLevel.HTTPUpdate.build.debug_level=-DDEBUG_ESP_HTTP_UPDATE
oak.menu.DebugLevel.HTTPUpdate2=HTTPClient + HTTPUpdate
oak.menu.DebugLevel.HTTPUpdate2.build.debug_level=-DDEBUG_ESP_HTTP_UPDATE -DDEBUG_ESP_HTTP_UPDATE
oak.menu.DebugLevel.HTTPUpdate3=HTTPClient + HTTPUpdate + Updater
oak.menu.DebugLevel.HTTPUpdate3.build.debug_level=-DDEBUG_ESP_HTTP_UPDATE -DDEBUG_ESP_HTTP_UPDATE -DDEBUG_ESP_UPDATER
oak.menu.DebugLevel.HTTPServer=HTTPServer
oak.menu.DebugLevel.HTTPServer.build.debug_level=-DDEBUG_ESP_HTTP_SERVER
oak.menu.DebugLevel.UPDATER=Updater
oak.menu.DebugLevel.UPDATER.build.debug_level=-DDEBUG_ESP_UPDATER
oak.menu.DebugLevel.OTA_____=OTA
oak.menu.DebugLevel.OTA_____.build.debug_level=-DDEBUG_ESP_OTA
oak.menu.DebugLevel.OTA2____=OTA + Updater
oak.menu.DebugLevel.OTA2____.build.debug_level=-DDEBUG_ESP_OTA -DDEBUG_ESP_UPDATER
oak.menu.DebugLevel.all_____=All
oak.menu.DebugLevel.all_____.build.debug_level=-DDEBUG_ESP_CORE -DDEBUG_ESP_SSL -DDEBUG_ESP_WIFI -DDEBUG_ESP_HTTP_CLIENT -DDEBUG_ESP_HTTP_UPDATE -DDEBUG_ESP_HTTP_SERVER -DDEBUG_ESP_UPDATER -DDEBUG_ESP_OTA -DDEBUG_TLS_MEM
view rawaddtoboards.txt hosted with ❤ by GitHub
Congrats, you're done!
At this point, you can now reopen the Arduino IDE and you should end up seeing another board in your Boards menu under Tools.

I may submit this upstream as a new board definition for the Arduino ESP8266 core, but that will take a little bit to be added.

If you've found this useful, please let me know.