Digistump Forums

The Oak by Digistump => Oak Support => Topic started by: mab299108 on July 20, 2016, 11:59:57 pm

Title: Can oak get time from the cloud?
Post by: mab299108 on July 20, 2016, 11:59:57 pm
Looking at the particle reference document, I see that devices on the particle cloud should sync time with the server when they connect (https://docs.particle.io/reference/firmware/photon/#time). However, when I add Time.now() to my code I  get: 'Time' was not declared in this scope. I think that other particle devices have a built in RTC, so I am wondering if this this an ability that oak is missing, or is there another version of the firmware that I need to support getting the current time on oak?
Title: Re: Can oak get time from the cloud?
Post by: PeterF on July 22, 2016, 01:39:27 am
AFAIK, it is unfortunately a missing ability, probably due to the different architecture of the Oaks onboard processor. As you say, boards like the Particle Photon have an onboard RTC. The ESP8266 MCU in the Oak has a RTC as well, but it is for internal timing purposes, rather being used as a time clock.

Best bet is probably to use an external RTC such as the DS1307, and/or NTP time : http://www.esp8266.com/viewtopic.php?f=32&t=2881 (http://www.esp8266.com/viewtopic.php?f=32&t=2881)
Title: Re: Can oak get time from the cloud?
Post by: t4ce on July 29, 2016, 09:46:39 am
Are you looking for Particle.syncTime()?

https://docs.particle.io/reference/firmware/core/#particle-synctime-
Title: Re: Can oak get time from the cloud?
Post by: PeterF on July 29, 2016, 08:23:36 pm
That is indeed the correct call - however, as per the wiki and v1 firmware release notes, 'Particle.syncTime() - though time is not yet connected to anything'.

So the function is available (in as far as compiling without failing, it doesn't sync with anything, as the Oak doesn't have a user accessible RTC to sync the time with. And if you try to access any of the Time functions, it will fail with a 'Time not declared in this scope' error.

You could get the time via a HTTP request from one of the NIST time servers like in the code below.

Code: [Select]
#include <ESP8266WiFi.h>

// const char* host = "utcnist2.colorado.edu";
const char* host = "128.138.141.172";
String TimeDate = "";

void setup()
{
  Particle.begin();
  Particle.println("time test sketch");
}

void loop()
{
  // Use WiFiClient class to create TCP connections
  WiFiClient client;
  const int httpPort = 13;

  if (!client.connect(host, httpPort)) {
    Particle.println("connection failed");
    return;
  }

  // This will send the request to the server
  client.print("HEAD / HTTP/1.1\r\nAccept: */*\r\nUser-Agent: Mozilla/4.0 (compatible; ESP8266 Arduino Oak;)\r\n\r\n");

  delay(100);

  // Read all the lines of the reply from server and print them to Serial
  // expected line is like : Date: Thu, 01 Jan 2015 22:00:14 GMT
  char buffer[12];
  String dateTime = "";

  while (client.available())
  {
    String line = client.readStringUntil('\r');

    if (line.indexOf("Date") != -1)
    {
      Particle.print("=====>");
    }
    else
    {
      //Particle.print(line);

      TimeDate = line.substring(7);
      //Particle.println(TimeDate);

      // date starts at pos 7
      TimeDate = line.substring(7, 15);
      TimeDate.toCharArray(buffer, 10);
      Particle.print("UTC Date/Time: ");
      Particle.print(buffer);

      // time starts at pos 14
      TimeDate = line.substring(16, 24);
      TimeDate.toCharArray(buffer, 10);
      Particle.print(" ");
      Particle.println(buffer);
    }
  }

  delay(10000);

}

With a few modifications to the TimeNTP_ESP8266Wifi examples that come with the Time library, it is also possible to get the time via NTP, and in an easier to manipulate form.

Code: [Select]
/*
 * Time_NTP.pde
 * Example showing time sync to NTP time source
 *
 * This sketch uses the ESP8266WiFi library
 */
 
#include <TimeLib.h>
#include <ESP8266WiFi.h>
#include <WiFiUdp.h>

const char ssid[] = "*************";  //  your network SSID (name)
const char pass[] = "********";       // your network password

// NTP Servers:
IPAddress timeServer(132, 163, 4, 101); // time-a.timefreq.bldrdoc.gov
// IPAddress timeServer(132, 163, 4, 102); // time-b.timefreq.bldrdoc.gov
// IPAddress timeServer(132, 163, 4, 103); // time-c.timefreq.bldrdoc.gov


const int timeZone = 1;     // Central European Time
//const int timeZone = -5;  // Eastern Standard Time (USA)
//const int timeZone = -4;  // Eastern Daylight Time (USA)
//const int timeZone = -8;  // Pacific Standard Time (USA)
//const int timeZone = -7;  // Pacific Daylight Time (USA)


WiFiUDP Udp;
unsigned int localPort = 8888;  // local port to listen for UDP packets

void setup()
{
  Particle.begin();
//  while (!Serial) ; // Needed for Leonardo only
  delay(250);
  Particle.println("TimeNTP Example");
//  Particle.print("Connecting to ");
//  Serial.println(ssid);
//  WiFi.begin(ssid, pass);
 
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
//    Serial.print(".");
  }

 
//  Serial.print("IP number assigned by DHCP is ");
//  Serial.println(WiFi.localIP());
//  Serial.println("Starting UDP");
  Udp.begin(localPort);
//  Serial.print("Local port: ");
//  Serial.println(Udp.localPort());
  Particle.println("waiting for sync");
  setSyncProvider(getNtpTime);
}

time_t prevDisplay = 0; // when the digital clock was displayed

void loop()

  if (timeStatus() != timeNotSet) {
    if (now() != prevDisplay) { //update the display only if time has changed
      prevDisplay = now();
      digitalClockDisplay(); 
    }
  }
}

void digitalClockDisplay(){
  // digital clock display of the time
  Particle.print(hour());
  printDigits(minute());
  printDigits(second());
  Particle.print(" ");
  Particle.print(day());
  Particle.print(".");
  Particle.print(month());
  Particle.print(".");
  Particle.print(year());
  Particle.println();
}



void printDigits(int digits){
  // utility for digital clock display: prints preceding colon and leading 0
  Particle.print(":");
  if(digits < 10)
    Particle.print('0');
  Particle.print(digits);
}

/*-------- NTP code ----------*/

const int NTP_PACKET_SIZE = 48; // NTP time is in the first 48 bytes of message
byte packetBuffer[NTP_PACKET_SIZE]; //buffer to hold incoming & outgoing packets

time_t getNtpTime()
{
  while (Udp.parsePacket() > 0) ; // discard any previously received packets
  Particle.println("Transmit NTP Request");
  sendNTPpacket(timeServer);
  uint32_t beginWait = millis();
  while (millis() - beginWait < 1500) {
    int size = Udp.parsePacket();
    if (size >= NTP_PACKET_SIZE) {
      Particle.println("Receive NTP Response");
      Udp.read(packetBuffer, NTP_PACKET_SIZE);  // read packet into the buffer
      unsigned long secsSince1900;
      // convert four bytes starting at location 40 to a long integer
      secsSince1900 =  (unsigned long)packetBuffer[40] << 24;
      secsSince1900 |= (unsigned long)packetBuffer[41] << 16;
      secsSince1900 |= (unsigned long)packetBuffer[42] << 8;
      secsSince1900 |= (unsigned long)packetBuffer[43];
      return secsSince1900 - 2208988800UL + timeZone * SECS_PER_HOUR;
    }
  }
  Particle.println("No NTP Response :-(");
  return 0; // return 0 if unable to get the time
}

// send an NTP request to the time server at the given address
void sendNTPpacket(IPAddress &address)
{
  // set all bytes in the buffer to 0
  memset(packetBuffer, 0, NTP_PACKET_SIZE);
  // Initialize values needed to form NTP request
  // (see URL above for details on the packets)
  packetBuffer[0] = 0b11100011;   // LI, Version, Mode
  packetBuffer[1] = 0;     // Stratum, or type of clock
  packetBuffer[2] = 6;     // Polling Interval
  packetBuffer[3] = 0xEC;  // Peer Clock Precision
  // 8 bytes of zero for Root Delay & Root Dispersion
  packetBuffer[12]  = 49;
  packetBuffer[13]  = 0x4E;
  packetBuffer[14]  = 49;
  packetBuffer[15]  = 52;
  // all NTP fields have been given values, now
  // you can send a packet requesting a timestamp:                 
  Udp.beginPacket(address, 123); //NTP requests are to port 123
  Udp.write(packetBuffer, NTP_PACKET_SIZE);
  Udp.endPacket();
}
Title: Re: Can oak get time from the cloud?
Post by: brohan on July 29, 2016, 10:03:12 pm
I know that personally, I can't get the time from the cloud, but I can when the sun is shining. Perhaps the Oak can get the time if it's taller than the cloud?
Title: Re: Can oak get time from the cloud?
Post by: PeterF on July 29, 2016, 11:54:20 pm
Oh... you're a tough nut to crack aren't you! :D :D
Title: Re: Can oak get time from the cloud?
Post by: brohan on July 30, 2016, 08:37:20 am
No, but an Oak is.  8)
Title: Re: Can oak get time from the cloud?
Post by: emardee on July 30, 2016, 06:37:59 pm
Oh... you're a tough nut to crack aren't you! :D :D
No, but an Oak is.  8)

I think you are confusing and an Oak and an Acorn! (Acorns don't have the voltage regulator onboard)
Title: Re: Can oak get time from the cloud?
Post by: PeterF on July 30, 2016, 07:01:51 pm
lol... I couldn't come up with anything wittier at the time, so hoped no one would notice!  :o Ruin my fun why don't you... !!!  :'( :'(

So... which came first... the acorn or the oak?  ;D ;D ;D
Title: Re: Can oak get time from the cloud?
Post by: brohan on July 30, 2016, 10:17:14 pm
Sorry mab299108, we now return you to your regularly scheduled q & a.
Title: Re: Can oak get time from the cloud?
Post by: mab299108 on August 23, 2016, 09:17:15 am
Thanks PeterF. Your sample of pulling the data direct from the NIST server is a big help. Works like a charm and I was able to upload and test the first sample in a matter of seconds.
Title: Re: Can oak get time from the cloud?
Post by: crambo on November 24, 2016, 09:37:37 am
Hi All,

I love the humor here, but I have an issue when using the above sample NTP code. It works beautifully for a couple of hours and then the Oak seems to freeze up and become unresponsive until reboot. I am running some simple code for a few hours to see if it might be a hardware issue, but if not I am wondering if there might be some kind of memory leak.

Anyone running NTP on an Oak successfully? Just curious if it might be a bigger issue.

(Thank some higher being for these forums!)

-Cam
Title: Re: Can oak get time from the cloud?
Post by: crambo on November 24, 2016, 09:46:48 am
Huh. Well it already crashed running a simple pixel test. Might be a hardware thing.

Will post here with update.

Title: Re: Can oak get time from the cloud?
Post by: crambo on November 24, 2016, 12:05:37 pm
Ok, so running a simple neopixel test crashes both oaks I have. How would I begin troubleshooting such a thing? Could it be the neopixels I am using? I will get some more and work that angle as well....


Here is the code that crashes the neopixel:

#include <Adafruit_NeoPixel_Oak.h>

#define PIN      1
#define N_LEDS 21

Adafruit_NeoPixel strip = Adafruit_NeoPixel(N_LEDS, PIN, NEO_GRB + NEO_KHZ800);

void setup() {
  strip.begin();
}

void loop() {
  chaseup(strip.Color(255, 0, 0)); // Red
  chasedown(strip.Color(255, 0, 0)); // Red
  chaseup(strip.Color(0, 255, 0)); // Green
  chasedown(strip.Color(0, 255, 0)); // Green
  chaseup(strip.Color(0, 0, 255)); // Blue
  chasedown(strip.Color(0, 0, 255)); // Blue
}

static void chaseup(uint32_t c) {
  for (uint16_t i = 0; i < strip.numPixels() + 4; i++) {
    strip.setPixelColor(i  , c); // Draw new pixel
    strip.setPixelColor(i - 4, 0); // Erase pixel a few steps back
    strip.show();
    delay(5);
  }
}
static void chasedown(uint32_t c) {
  for (uint16_t i = strip.numPixels(); i > 0 + 4; i--) {
    strip.setPixelColor(i  , c); // Draw new pixel
    strip.setPixelColor(i + 4, 0); // Erase pixel a few steps back
    strip.show();
    delay(5);
  }
}
Title: Re: Can oak get time from the cloud?
Post by: emardee on November 24, 2016, 01:01:19 pm
I'd flick a few "yield()" commands in there, to see if it solves the crashing.

I'd personally start with too many and then see which I could remove later... so would put a "yield ()" before each "delay (5)". Note this might cause glitches in the animation, but at the moment we are troubleshooting to see if that is the problem or not, so that is acceptable for now.... clearly once diagnosed you might want to improve that later.

As I understanding it, yield lets the particle cloud stuff do its thing, but if there are no gaps, then the particle communication code seems to get to a point where it crashes. That was my experience with longer animations, or longer pixel strings. This is based on observation and reading these forums, rather than knowledge of how Oak actually works (especially with regard to the Particle "magic")... so what I think I've observed and what is really happening, might not be the same at all!
Title: Re: Can oak get time from the cloud?
Post by: PeterF on November 24, 2016, 05:13:17 pm
Same here... I didn't bother disabling the particle connection, and used the strandtest code, and the oak crashed/froze somewhere between 10 mintues and 30 minutes later.... So I think some yields() are needed to kick the wifi / particle tasks.

I think it could be an interaction of two things... (bear in mind that I haven't looked at this bit of code yet so what I say now could be complete rubbish!)

From fiddling with the pure ESP8266 Arduino core (and seeing forum/github stuff), there used to be (still is?) a problem where you didn't call yield() often enough, and the wifi link would drop or the ESP8266 would crash 'n burn. yield() does essentially what it says it does... it yields (to the background periodic processes that need to run). Then with the Oak I think you add the Particle integration on top, meaning not only could the wifi get b0rked, but also the Particle link. Hence yield() has to be run periodically. Now, every time the main loop() function runs it should trigger, as well as any background particle stuff. IIRC, the particle stuff is also triggered whenever a long enough delay() is used (hence why there is no longer a special Particle.delay() call). So if you have code (like standtest) that doesn't let the loop() run frequently enough, either or both the wifi and Particle code get cranky, and the Oak locks up (is that a  technical enough description?).

Moral of the story... be generous with your yields(). I don't know if Particle.process() is needed - it might be called in the yield() function also meaning it's not needed - although I know it is needed if you are in full manual Particle mode.

Title: Re: Can oak get time from the cloud?
Post by: crambo on November 25, 2016, 06:55:01 am
Nope. Sprinkled with yield() and it still crashes. Starting to think this platform was a mistake...

There are no Particle references in the code I shared...Is there something else you need to do to get Particle out of the way? I just need NTP access. Trying to make what I thought would be a simple clock....
Title: Re: Can oak get time from the cloud?
Post by: exeng on November 25, 2016, 10:31:12 am
crambo, Wish I could help but ran your Neopixel chase code on both UNO and Oak without a crash (unless it takes some time to occur).

Initially I thought these lines might be a problem...
Code: [Select]
   for (uint16_t i = 0; i < strip.numPixels() + 4; i++) {
    strip.setPixelColor(i  , c); // Draw new pixel
    strip.setPixelColor(i - 4, 0); // Erase pixel a few steps back
    strip.show();
    delay(5);

When i=0 you are referencing pixel -4. At i=1, pixel -3, but it doesn't seem to matter. Perhaps Neopixel lib has some bounds checking built in.
In any case I'm running you code on the Oak. No crash.

EDIT:
Also (not related to the crash question) your chasedown routine...

Code: [Select]
static void chasedown(uint32_t c) {
  for (uint16_t i = strip.numPixels(); i > 0 + 4; i--) {
    strip.setPixelColor(i  , c); // Draw new pixel
    strip.setPixelColor(i + 4, 0); // Erase pixel a few steps back
    strip.show();
    delay(5);
  }
}
... does not make it all the way back since you stop at i > 0 + 4 which is 5 and leaves out pixels 4 through 0. If you slow the chase down you'll see it.

Again... running it now with no crash.




Title: Re: Can oak get time from the cloud?
Post by: crambo on November 25, 2016, 11:34:17 am
The program will run anywhere from minutes to hours....But alas, it always crashes. Same code on plain old vanilla Uno, runs for days plus now... small bugs in program aside, which don't seem to impact the uno negatively.

Title: Re: Can oak get time from the cloud?
Post by: crambo on November 25, 2016, 11:40:53 am
Yup, even with changes in code. Still crashing on two different Oak's. ???
Title: Re: Can oak get time from the cloud?
Post by: crambo on November 25, 2016, 11:42:49 am
Can anyone explain how to get Particle out of the picture? For this project, it is unnecessary...
Title: Re: Can oak get time from the cloud?
Post by: PeterF on November 25, 2016, 05:15:09 pm
Just particle, or wifi? You will get the most stability with both disabled, but it should be more stable (time-sensitive-code wise) without Particle.

If you put SYSTEM_MODE(SEMI_AUTOMATIC) at the top of the your sketch (not in setup() - it goes outside of any functions - the Oak will still try to establish a wifi connection, but not Particle - you need to do that manually. SYSTEM_MODE(MANUAL) also prevents automatic particle connection, and requires that you use regular Particle.process() calls to maintain the Particle connection. I can't remember if the manual mode still connects to the wifi, or if that becomes a manual connect also. There is some documentation (http://digistump.com/wiki/oak/tutorials/api_reference) on this API / Particle stuff on the wiki.

Just a word of warning - as soon as you do either the SEMI_AUTOMATIC or MANUAL system modes, unless you have some code that establishes a particle connection in response to some condition (i.e. button attached to a pin), the only way you can do Particle over-the-air programming will be through the P1 - GND config/safe mode.

Something like this might be a starting point. I made one or two minor changes... I'm using a 24 pixel ring as a test subject and I haven't renamed the Oak neopixel library so Adafruit_NeoPixel_Oak didn't work for me (but these shouldn't change anything that would break stuff).  I added the strip.show() call after strip.begin() as that ensures all the pixels are off. Then there is the very dumb time-out to allow for OTA programming - you could also use config mode, or test for a pin being pulled high or low to halt startup and wait for programming. I'll leave that running for a few hours... but so far it is no longer randomly & periodically halting every 20-30 seconds whilst the particle processing happens.

Code: [Select]
SYSTEM_MODE(SEMI_AUTOMATIC)
#include <Adafruit_NeoPixel.h>

#define PIN      1
#define N_LEDS  24

Adafruit_NeoPixel strip = Adafruit_NeoPixel(N_LEDS, PIN, NEO_GRB + NEO_KHZ800);

void setup()
{
  strip.begin();
  strip.show();

  Particle.connect();
  Particle.publish("oak4/connect", "Giving Particle time to catch me for programming", PRIVATE);

  delay(15000);

  Particle.publish("oak4/disconnect", "I ain't no IoT device... I quit!", PRIVATE);
  Particle.disconnect();
}

void loop()
{
  chaseup(strip.Color(255, 0, 0)); // Red
  chasedown(strip.Color(255, 0, 0)); // Red
  chaseup(strip.Color(0, 255, 0)); // Green
  chasedown(strip.Color(0, 255, 0)); // Green
  chaseup(strip.Color(0, 0, 255)); // Blue
  chasedown(strip.Color(0, 0, 255)); // Blue
}

static void chaseup(uint32_t c)
{
  for (uint16_t i = 0; i < strip.numPixels() + 4; i++)
  {
    strip.setPixelColor(i  , c); // Draw new pixel
    strip.setPixelColor(i - 4, 0); // Erase pixel a few steps back
    strip.show();
    delay(5);
  }
}
static void chasedown(uint32_t c)
{
  for (uint16_t i = strip.numPixels(); i > 0 + 4; i--)
  {
    strip.setPixelColor(i  , c); // Draw new pixel
    strip.setPixelColor(i + 4, 0); // Erase pixel a few steps back
    strip.show();
    delay(5);
  }
}

Title: Re: Can oak get time from the cloud?
Post by: crambo on November 25, 2016, 06:02:03 pm
PeterF,

I appreciate your both validating what I am seeing (thought I was crazy) and your input and expertise. I too will try these changes. I assume Wifi will still work, as ultimately, I will need it for the NTP example you contributed early in this thread.

THANK YOU!!!!

Cameron
Title: Re: Can oak get time from the cloud?
Post by: crambo on November 25, 2016, 06:11:56 pm
PeterF,

I too am running code you posted. I am now hoping that this will work, and that by doing so, NTP/Wifi will keep working, as that will be necessary for the clock code.

Best,
Cameron
Title: Re: Can oak get time from the cloud?
Post by: PeterF on November 25, 2016, 07:15:57 pm
No worries Cameron. This 'lil sucker has been running for more than two hours now and hasn't even hickup'd once so far... which is starting to get annoying as it's a bit bright on the desk in front of me!  ;D  :o

Anyway, the wifi backend stuff should still be working, just not doing much since Particle isn't needing to establish a connection. I suspect what you'll need to do is do your NTP time sync, and just live with the fact that while that is happening the pixels can't be controlled - this is strictly a one task at a time affair. The easiest way to cheat that is to use non-blocking delays to do stuff. But that is the topic for a another thread ;0

Pete
Title: Re: Can oak get time from the cloud?
Post by: crambo on November 25, 2016, 07:48:59 pm
Oh no! It froze on me! Arrrrggghhh!!! Is it still running for you? I ran for more than an hour!
Title: Re: Can oak get time from the cloud?
Post by: crambo on November 25, 2016, 09:54:01 pm
Ugh. I give up. I'm gonna just use an Uno with a shield. Would have been col tho!

Pete, thanks for all your help. I imagine the Oak will eventually be a great little platform and I will hold on to my five for that moment!

Title: Re: Can oak get time from the cloud?
Post by: PeterF on November 25, 2016, 10:39:33 pm
It ran for about five hours then the battery went flat... so I'll get it off the hook on that point! :D Edit: It looks like I may have spoken too soon... about 3.5 hours later, and running from USB power this time... and it froze! So something doesn't want to co-operate. :-(

hm... I dunno what's going on here... when it locks up... is the P1 led doing a triple blink pattern (indicating it is in safe mode)? And how are you powering the oak... maybe it's not getting quite enough power? The only other thing is which version of the board package you are using... the latest (1.0.6)?
Title: Re: Can oak get time from the cloud?
Post by: crambo on November 30, 2016, 09:38:34 pm
I have now run this on a plain old ESP8266 for multiple days. Clearly there is some inefficiency (bug) deep in the particle support. That is the only thing I can identify as different....

Is there any way for me to convert my Oaks to plain old 8266 units? At least I won't be out $50.

Title: Re: Can oak get time from the cloud?
Post by: PeterF on December 01, 2016, 12:42:24 am
It's also possible that there is a bug that was kept over from an earlier version of the ESP8266 core, as the Oak one is a bit behind... but I would be more inclined to point the finger at something to do with the particle integration.

AFAIK, all you need to do is a) save your device ID once you've claimed your Oaks on Particle so that you can re-Oakify them later if you wish; and then b) just serial flash them with a compiled ESP8266 sketch when they are in serial programming mode (P2 to GND)... that should be all there is to it. Then have a look at the Oak to ESP8266 pin mapping (http://digistump.com/wiki/oak/tutorials/pinout#oak_pin_to_esp8266_gpio_mapping) so you can work out what pin is now which in ESP8266 Arduino parlance. And they also have enough on-board memory that you should be able to play with OTA programming ESP8266 Arduino style.
Title: Re: Can oak get time from the cloud?
Post by: crambo on December 01, 2016, 08:15:55 pm
Thanks Peter!

I did finally get the ESP8266 to crash! I wonder if there is a lower level issue within the 8266 core... Hum. I am surprised that stability issues haven't presented more. I will convert an Oak and see if stability is any better.
Title: Re: Can oak get time from the cloud?
Post by: PeterF on December 02, 2016, 12:33:23 am
Please let us know how you go... in both the Oak -> Arduino ESP8266 process, and if you track down that stability issue.

I amended my prior response to say "an earlier version of the ESP8266 core"... as what I was meaning was that a bug in an earlier Arduino ESP8266 core which has been subsequently fixed could still be present in the Oak's core. However, it doesn't really surprise me that the ESP8266 finally crashed... since it is playing double-duty as both the wifi management and application MCU... something has to give if the dual-roles aren't properly managed. However, it will be interesting to see how stable it can be made.