Digistump Forums

The Oak by Digistump => Oak Support => Topic started by: postuma on July 09, 2017, 05:05:54 pm

Title: Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates
Post by: postuma on July 09, 2017, 05:05:54 pm
I've struggled enough with this I figured it deserved its own thread - and when I say struggled, I mean at least a half dozen hours to get it working.

So - why bother?

Despite the difficulties, it's already been worth it:


I was a reluctant to make the change. What if things broke irrepairably? But Particle is so broken I felt I had no choice.

To his credit, Erik has acknowledged these problems - see https://www.kickstarter.com/projects/digistump/oak-by-digistump-wi-fi-for-all-things-arduino-comp/posts/1912567 (https://www.kickstarter.com/projects/digistump/oak-by-digistump-wi-fi-for-all-things-arduino-comp/posts/1912567). He's also put a big disclaimer on the Digistump order page - caveat emptor. And I have no doubt he will address these issues in some way, at some time. But family comes first! and I've no doubt he's way over-extended (very excited about the BuildOne printer, BTW).

You'll need a USB to TTL adapter to program the Oak using a serial interface, found el cheapo on Ebay: http://www.ebay.com/itm/USB-to-TTL-CH340G-Converter-Module-Adapter-STC-Dowanloader-Programmer-3-3V-5V-/112215846564 (http://Ebay: http://www.ebay.com/itm/USB-to-TTL-CH340G-Converter-Module-Adapter-STC-Dowanloader-Programmer-3-3V-5V-/112215846564).

Follow the Brendan 'nog3' Halliday tutorial at http://nog3.net/2017/06/05/removing-particle-from-oak/ (http://nog3.net/2017/06/05/removing-particle-from-oak/). Things to note:

On Python:


On esptool.py:


On the USB-to-TTL adapter:


Modifying the Arduino IDE:
     

Using the Arduino IDE:


Assuming all went well - and please reply if it didn't, let's figure out how to make it work consistently and well - click on Sketch > Upload to flash the new core and sketch to your Oak.

As mentioned, I was also (finally) able to load files directly to SPIFFS using the Sketch Data Upload tool. See https://github.com/esp8266/arduino-esp8266fs-plugin.

Anything pertinent that I've missed?

Now, on Serial over OTA:

Lovely, once you get it to work.

Notes thus far:


One thing I haven't figured out yet - whether I can get Serial.print feedback back to me, over the air? Anyone?

So - feedback very welcome. Other problems/solutions?

Paul
Title: Re: Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates
Post by: exeng on July 10, 2017, 09:32:49 am
Paul,

Thanks for this post and all the effort and detail that you obviously put into it. Also, thanks to Brendan (nog3) for his description of how to remove Oak from Particle.

While I have experienced the occasional fail OTAs and misrepresented fail/success of OTA's, I'm still able (after multiple retries) to get my Oak sketches OTA flashed. At a point where this becomes even less stable, I'll will most likely give this a go. I'm not dissing Particle but I never use their dev environment and only post a few bits of data there from my operational Oaks. Largely, this is due to the lack of (not sure if this is still true) a means to log data. I'm a fan of Thinkspeak. Others may have their personal favorites but I find the ability to keep historical data useful.

So again, thanks for bringing this solution to our attention.

exeng
Title: Re: Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates
Post by: PeterF on July 11, 2017, 05:24:22 pm
Paul, I would also like to thank you for the writeup ;)

Like exeng, I'm still using Particle on my Oaks (bar one, a ESP8266 Arduino OTA guinea pig), but haven't programmed them in a while, so could quite quickly decide to jump ship should they start being 'lil s... ugars ;) I would like to add the following:

* Arduino OTA is indeed FAST! ;) You just have to remember to add the OTA.handle function, and give it the chance to do it's thing... it won't be 'always watching' like the Oak Particle firmware, unless you program it that way.

* True serial on the Oak was never limited AFAIK, just the Particle Serial implementation. You could always Serial.print() and monitor via a Serial UART, but it just wasn't as convenient.

* Unfortunately I don't think they've implemented any form of Serial over WiFi, so you won't be able to get that back in a simple fashion. Will probably have use some form of logging if you want that functionality, else, stick with hard-wired serial.

* Newer version of the docs (2.3.0 vs 2.0.0) is here (annoyingly the 'other versions' link at the top never worked, so you have to specifically look for the matching documentation for the current release) => http://esp8266.github.io/Arduino/versions/2.3.0/doc/ota_updates/readme.html
Title: Re: Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates
Post by: postuma on July 12, 2017, 10:21:33 am
Good observations - thanks for the additions!
Title: Re: Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates
Post by: DrJFM on July 12, 2017, 09:38:10 pm
I also noted Erik's Kickstarter update acknowledging the issues with Particle.  I also found that after a 2 to 3 month break from Oak use, I could not even log in to Particle Dashboard.  Rather than spend time trying to recover my account at Particle, I decided to follow Brendan 'Nog3' Halliday's tutorial.

Quote
To his credit, Erik has acknowledged these problems - see https://www.kickstarter.com/projects/digistump/oak-by-digistump-wi-fi-for-all-things-arduino-comp/posts/1912567. He's also put a big disclaimer on the Digistump order page - caveat emptor. And I have no doubt he will address these issues in some way, at some time. But family comes first! and I've no doubt he's way over-extended (very excited about the BuildOne printer, BTW).

You'll need a USB to TTL adapter to program the Oak using a serial interface, found el cheapo on Ebay: http://www.ebay.com/itm/USB-to-TTL-CH340G-Converter-Module-Adapter-STC-Dowanloader-Programmer-3-3V-5V-/112215846564.

Follow the Brendan 'nog3' Halliday tutorial at http://nog3.net/2017/06/05/removing-particle-from-oak/.

After a moderate amount of extra time getting a never previously powered up Oak turned into a ESP8266Oak, I decided to pop in here and see if others were also making the transition. Wish I had looked in here sooner -- might have saved some time! That said, I really just wanted to encourage others to make the switch.  It is really not bad.  If you have ever done a serial flash to restore the Oak, this is easier.  Paul's further hints, directions and clarifications address some of the few issues that slowed the process for me and should help assure success. 

I will likely move to the OTA loading just to say I have, but I see myself more than content to use the serial upload process for most of my hobby projects.  I have been most satisfied using Blynk to monitor and control my Oaks and spent only about 1 hr (in part because I have not been using Arduino/Oak much lately) porting my Temp Humidity monitor to my newly configured ESP8266Oak.  One of my older registered Oaks had been updating my Blynk app w Temp/RH data for a couple months, most likely with absolutely no Particle connection.  I moved the level shifter shield to the ESP8266Oak and deleted a bunch of Particle overhead.  A couple tweaks and back to knowing when my dehumidifier needed to be emptied using my Blynk Android app.

So some bottom line thought:
I did a fresh install of Arduino 1.8.3 before I started.  The ESP8266 core is loaded by the Board Manager after adding in the link as described. No python flash needed!  I will likely convert my last "new" Oak and maybe some of the registered ones without worrying about the python backup -- just the erase!  The ESP8266 Core I installed was 2.3.0.  In my Windows 10 system, the files from the board manager end up in my AppData folder ie
C:\Users\UserName\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.3.0\variants   -not in my sketch folder. For anyone using Blynk, I added the most current Blynk Arduino library, which includes BlynkESP8266 version, not the one in the Oak download. I backed the Blynk Kickstart campaign so have lots of juice (ie $) on Blynk but would likely pay for the service as a very easy way to get data and control to you phone.

Finally, should we see if they will set up a ESP8266Oak branch of the forum?
Cheers. 
James
Title: Re: Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates
Post by: wickwire on July 26, 2017, 03:46:50 pm
Hi,


I was a backer for the Oak by Digistump kickstarter campaign, when it happened. I had already bought a Spark Core (now I guess it's called Particle Core) and hadn't done much with it, and for the price tag, once I found out about the Oak, I was on board.

I followed Erik's journey with the Oak and received my order when the time came: a bunch of Oaks, a bunch of Acorns, LCD displays, batteries, 3.3v relays, you name it.

By then, I was already fiddling with ESP8266-12 and ESP8266-01 modules and nodeMCU, which in spite of not having the cloud infra-structure set up, were really cheap to buy and as entertaining and a learning tool as any.

So I tried one Oak, had some troubles connecting it to the cloud, managed to get it going and since I already had stuff going on using MQTT with SSL on the ESP8266, was also going for that - which then lead to this github issue:

https://github.com/digistump/OakCore/issues/71

The Oak was set aside once more, I continued using the ESP8266-12 and nodeMCU, and then this week decided to give the Oak another shot.

After about an hour of failing to set it up with the Particle Cloud, I found an email message from Erik mentioning some possible issues with the particle registration process and the available alternative firmwares that could be loaded on the Oak.

The email contained nog3's instructions on how to set up the ESP8266 Arduino firmware, pretty much like the initial post in this thread, so I set everything up - since I also wanted to try out something other than nodeMCU.

I have to say that the ESP8266 Arduino project on the Oak has been a very positive experience so far, I got my Oak to register and publish timed messages to an MQTT broker online with SSL by updating the new sketch over the air fairly quickly and I'm willing to dedicate more of my time to this alternative. This is something that not even the original Oak firmware has allowed me to do, so far.

I consider myself to be an electronics newbie since my background is in app software development, however I do have an ESP8266 currently controlling on demand power sockets using solid state relays, also with MQTT and SSL but on nodeMCU - and it seems to me that sooner or later I'll be updating that setup code-wise by using ESP8266 Arduino + PubSubClient instead.

That said, and getting back to the Oak and Acorn, would it be fair to say that when compared to the ESP8266-12 dev kit or generic ESP8266-12 module respectively, it would just be extra cash for no real added value at the moment?

I must admit that I didn't check the differences between both hardware implementations, so please feel free to correct me on this and if you also feel that in spite of the noted issues with the cloud platform, this solution is still worth the money extra over the ESP, please note that too.

Finally, I'd like to mention that regardless of any possible issues with the OTA infra-structure with the Oak at the moment, I believe that it will get sorted out with time, as well.

Title: Re: Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates
Post by: PeterF on July 28, 2017, 02:22:43 am
Yeah, because of the age of the ESP8266 core and because we don't have reference point to determine where exactly Erik branched off the ESP8266 Arduino core (so we could work out how to bring as much of it as possible up to date), things can and will not work properly, and we just have to live with it.

Having said that, I'm liking the ESP8266 Arduino OTA stuff more and more, especially since it is actually faster than doing serial uploads... which I thought were reasonably fast already.

As far as hardware, all I can say is the Oak is a solid and reliable board. Beyond that, you need to make your own determination. The regulator is well spec'd, and is reliable. The ESP8266 ("Acorn") is also well made, and even though I haven't looked under the can, I suspect it is better quality than the cheap ESP-12F boards you can find online. You just need to work out if you want to spend ~US$11 for a single Oak (they do get cheaper if you want quantities) or ~US$7 for a D1 WeMos (http://www.ebay.com/itm/D1-Mini-NodeMcu-4M-bytes-Lua-WIFI-Development-Board-ESP8266-by-WeMos-/263093593303l) from eBay or (my preference) the ESP8266 Witty modules for ~US$4 on eBay (http://www.ebay.com/itm/ESP8266-serial-WIFI-Witty-cloud-Development-Board-ESP-12F-module-MINI-nodemcu-/401103611225). The reason I like the latter is not because they're cheaper, but because they have the option of the USB serial or not. So you have it when you want to program it, and when you're finished, or need it back in your project, you just remove the module, and there's no power loss from unused USB serial circuitry (which is why I don't like the nodemcu boards for low power ESP8266 stuff). This is where the Oak really shines... if you have it in deep sleep mode, it draws next to no power... as the regulator is a good low quiescent current one, and there is no extra stuff to draw power (you just cut the trace for the power led as provided for), and it's as low as you can get it.
Title: Re: Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates
Post by: wickwire on July 28, 2017, 02:33:26 am
I guess you are right, I have no hardware related complaints whatsoever on the Oak so far, whereas I've had some stresses with ESP8266-12 modules in the past. Granted, I fiddled a lot more with the ESP8266-12 than the Oak, so I guess it will still take me some time, effort and acquired knowledge to properly compare the two solutions.

That said, I'm very pleased with ESP8266 Arduino on the Oak, it's just like you said, it will even be slightly smaller than my ESP8266-12 + GPIO expansion boards and includes micro-USB for power, so having bought 5 Oaks and 10 Acorns on the kickstarter campaign, in no way I feel that the Oaks and Acorns will be shelved for long. Would I have to rely solely on the Particle Cloud and original Oak firmware though, I guess that my happen.

As for buying more modules, for now I'm good working with the ones I still have and will take your comments into account as well when the time comes where I'll have to choose which solution to go for.

I'm just a hobbyist wanting to have some fun with some DIY projects and if I could spend more of my time working on my designs and less time maintaining the modules themselves, that would be the best outcome for me - and by the looks of it, right now it looks like money aside, the Oak + ESP8266 Arduino seems to accomplish that just fine :)
Title: Re: Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates
Post by: PeterF on July 28, 2017, 03:08:32 am
Sorry, I submitted my post before I was finished, so I've finished it and reposted the edit above.
Title: Re: Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates
Post by: kathblair on August 12, 2017, 07:01:12 pm
I wanted to say thanks for this info. I  had claimed some oaks a while ago (successfully) on the particle cloud, but couldn't use any libraries. This info let me swap to generic ESP8266, which was awesome!

I also wanted to add a note about the OTA updating - I couldn't see my port, and it turned out that I had to enable IPv6 in my router to be able to see the mDNS sent out by the oak. Before I did that, I couldn't see it even with Bonjour Browser, and afterwards I can and it works to upload code - haven't tried adding additional code with the OTA sketch, but baby steps.

I was also able to upload binaries with the Example > ESP8266WebServer > WebUpdate example, even without mDNS working, if I went to [oak IP]/update and used exported bins, so that would have worked if I couldn't have updated router settings.

Just to avoid having to power on/off the oak when uploading new code, but eh, it's less confusing. I still have my serial port connected for debugging, but I don't need to futz with P2 low/high.
Title: Re: Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates
Post by: exeng on January 06, 2019, 06:38:31 am
It appears that the server hosting nog3's blog which describes how to remove the Oak from Particle is and has been down for awhile. I've reached out to nog3 via PM but no answer. Does anyone have a copy of his or your own instruction for switching the Oak to generic ESP8266?  My Oaks are all down and have continually failed to accept any Particle OTA updates. I'm ready to make the switch and abandon any Particle support.

Thanks,
exeng
Title: Re: Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates
Post by: JeffRand on January 06, 2019, 08:18:15 am
It appears that the server hosting nog3's blog which describes how to remove the Oak from Particle is and has been down for awhile. I've reached out to nog3 via PM but no answer. Does anyone have a copy of his or your own instruction for switching the Oak to generic ESP8266?  My Oaks are all down and have continually failed to accept any Particle OTA updates. I'm ready to make the switch and abandon any Particle support.

Thanks,
exeng
The Wayback Machine has the page archived. Here:
https://web.archive.org/web/20180223211918/http://nog3.net/2017/06/05/removing-particle-from-oak (https://web.archive.org/web/20180223211918/http://nog3.net/2017/06/05/removing-particle-from-oak)
Title: Re: Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates
Post by: exeng on January 06, 2019, 10:59:33 am
Jeff,

You are awesome. Thanks for the quick response and for providing a working link. The old link is referenced in many places, including an old update by Erik on Kickstarter.  Now there is a valid working link to this for anyone else that may be looking.

exeng
Title: Re: Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates
Post by: victagayun on January 10, 2019, 01:19:21 am
You may use this HW connections to use UART https://arduino-esp8266.readthedocs.io/en/2.5.0-beta2/boards.html#minimal-hardware-setup-for-bootloading-and-usage (https://arduino-esp8266.readthedocs.io/en/2.5.0-beta2/boards.html#minimal-hardware-setup-for-bootloading-and-usage) and still program and erase the OAK.
This will be like any normal ESP8266.
Title: Re: Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates
Post by: exeng on May 10, 2019, 09:08:12 pm
Finally found the time to convert one of my Oaks to using generic ESP8266 code with serial and OTA updates using nog3's archived blog.
Link: https://web.archive.org/web/20180223211918/http://nog3.net/2017/06/05/removing-particle-from-oak (https://web.archive.org/web/20180223211918/http://nog3.net/2017/06/05/removing-particle-from-oak)

Thank you JeffRand (again) for posting that. I can now upload via serial and OTA reliably.  Goodbye Particle.  Still have four more Oaks to convert that were in service but eventually failed to take new uploads via Particle.  Looking forward to not wasting anymore time hoping that a Particle upload will eventually work.

Another useful reference (which may be linked in the blog) was this one: https://arduino-esp8266.readthedocs.io/en/latest/ota_updates/readme.html#arduino-ide (https://arduino-esp8266.readthedocs.io/en/latest/ota_updates/readme.html#arduino-ide)

exeng
Title: Re: Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates
Post by: exeng on May 11, 2019, 04:49:32 pm
You have got to be kidding me!.... Grabbed another Oak to de-Particle (since it had failed back to a ready to upload state and was not running the loaded sketch) and just for fun tried to upload the simple LED blink code using Particle. It took about 4 to 5 retires but damn it loaded the sketch. It's just maddening knowing that is probably some stupid but simple bug that is causing it to be unreliable.  Does anyone have a clue?  Erik is probably the best resource but he seems busy with other more profitable things.

UPDATE: Well the successful upload via Particle (after 4 to 5 retries) was just a one time event.  It's back to endless retries with no success, so that's it, time to de-Particle the rest of my Oak's and be done with it.  I'm probably the last one to do this.  I assume most, if not all, Oak owners have already moved on and removed their Oak's from Particle.
Title: Re: Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates
Post by: exeng on May 21, 2019, 05:03:27 pm
Just an update: Converted 4 of 5 built Oaks to non-Particle generic ESP8266s and couldn't be happier.  They are all up and running doing their job as monitors for weather, garage door, and an aggregate OLED display of all their feeds through Thingspeak and control via Blynk.  Gone are the fights and failures with Particle OTA. Now I can focus on developing again. The only reason the 5th one has not been converted is I need a temperature probe to complete a pool temp monitor. My old one bit the dust. Also, have 5 more new Oaks looking for something to do.

If you are still fighting Particle, convert. It's not that hard. Be sure to back up your Oaks and keep a record of the device IDs just in case the Particle stuff is fixed down the road.

exeng
Title: Re: Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates
Post by: Rboyer on May 31, 2019, 08:04:04 am
Glad to see that the wiki is still active. I'm about to de-Particle my Oaks and get them to be useful. Thanks to all for the guidance - wish me luck!
Title: Re: Changing from Oak/Particle to generic ESP8266 code with serial and OTA updates
Post by: kaokub99 on July 04, 2019, 03:44:13 am
thank you bro By : เกมส์สล็อต (https://www.168slotxo.net/)