Author Topic: Streaming data over WiFi (Read 6261 times)

digi_guy · « **on:** January 14, 2015, 02:11:50 pm »

Here his a chunk of code that takes the wifiserver example and introduces the ability to stream data, in this case reading analog data from pins 5 and 9.

The first thing to note is that this is EXTREMELY temperamental, and there are a million little ways in which is can fail. The second thing to note is that I've got the baud rate set to 19200. I'm going to write a followup post explaining some of the inner workings when I get a chance.

In theory you should be able to pop this into your pro, connect something to pins 5 and 9, and then go to the what ever default page you use. This code will then read the pins once every second and update the webpage, for a minute. Then it basically goes to sleep for about a minute before waking back up. I'll talk more about that later. Enjoy!

Code: [Select]

int i=0;
char ii;
String msg2 = "temp";
String text2;
int j=0;

void setup() {

  pinMode(5,INPUT);
  pinMode(9,INPUT);  
  delay(1000);//wait for wifi to connect
  Serial.begin(19200); //open connection to wifi module
}


void loop() {
  
  if(serverRequest()){

    String path = getRequestPath();
    
    if(path == F("/stream")){

        sendResponseStartStream();
 
       for (int k=0; k<60; k++){ 
 // you can change this loop to make it run longer or shorter or change it to while (true) and have it run forever

          msg2=String(analogRead(A5));
          sendStream("pin5", "Pin 5: "+msg2);
          msg2=String(analogRead(A9));
          sendStream("pin9", "Pin 9: "+msg2);
          i++;
          
          while (Serial.read() !=-1); // empty the buffer
          delay(1000); // change this to update faster or slower
       }
        sendResponseEnd();

    }
    else{
      sendResponseStart();
      sendResponseChunk(F("<!DOCTYPE html><html><body>"));
      sendResponseChunk(F("<h1>Getting server updates</h1>"));
      sendResponseChunk(F("<div id=\"pin5\">pin5 goes here</div>"));
      sendResponseChunk(F("<div id=\"pin9\">pin9 goes here</div>"));      
      sendResponseChunk(F("<div id=\"result\"></div>")); 
      sendResponseChunk(F("<div id=\"onclose\"></div>")); 
      sendResponseChunk(F("<div id=\"onopen\"></div>"));       
      sendResponseChunk(F("<script>"));

    
      sendResponseChunk(F("if(typeof(EventSource) !== \"undefined\") {"));
      sendResponseChunk(F("var source = new EventSource(\"/stream\");"));

      sendResponseChunk(F("source.onopen = function(event) {"));
      sendResponseChunk(F("var today=new Date();"));       
      sendResponseChunk(F(" document.getElementById(\"onopen\").innerHTML = \"opened: \"+today+ \"<br>\"; }; "));      
      
      sendResponseChunk(F("source.onerror = function(event) {"));
      sendResponseChunk(F("var today=new Date();"));       
      sendResponseChunk(F(" document.getElementById(\"onclose\").innerHTML = \"closed: \"+today+ \"<br>\"; }; "));      
      
      sendResponseChunk(F("source.addEventListener(\'pin5\', function(e) {document.getElementById(\"pin5\").innerHTML = e.data;}, false);"));
      sendResponseChunk(F("source.addEventListener(\'pin9\', function(e) {document.getElementById(\"pin9\").innerHTML = e.data;}, false);"));
      sendResponseChunk(F("} else {document.getElementById(\"result\").innerHTML = \"fail\";}"));
      sendResponseChunk(F("</script></body></html>"));
      sendResponseEnd();
    }
  }
  
  
}

bool serverRequest(){
  if(Serial.available()>4){
    return Serial.find("GET ");
  }
  return false;
}

String getRequestPath(){
    String path = Serial.readStringUntil(' ');
    while(Serial.read() != -1); //clear read buffer
    return path;
}
void sendResponse(String response){
    sendResponseStart();
    sendResponseChunk(response);
    sendResponseEnd();
}


void sendResponseStart(){
    //sends a chunked response
    Serial.println(F("HTTP/1.1 200 OK"));
    Serial.println(F("Content-Type: text/html"));
    Serial.println(F("Connection: close"));
    Serial.println(F("Transfer-Encoding: chunked"));
    Serial.println();
}
void sendResponseChunk(String response){
    Serial.println(response.length()+2,HEX);
    Serial.println(response);
    Serial.println();
}

void sendResponseEnd(){
    Serial.println(F("0"));
    Serial.println();
}

void sendResponseStartStream(){
    //sends the start of the stream response
    Serial.println(F("HTTP/1.1 200 OK"));
    Serial.println(F("Content-Type: text/event-stream"));
    Serial.println(F("Cache-Control: no-cache"));
    Serial.println(F("Connection: keep-alive"));
    Serial.println();
}


void sendStream(String event, String text) {
    //sends the data of the stream response
  String msg = "event: " + event + "\ndata: " + text + "\n\n";
  Serial.print(msg);                   
}

Edit 1: I've only tested this on chrome, on my desktop.

digi_guy · « **Reply #1 on:** January 14, 2015, 06:00:26 pm »

As promised here is my attempt at a tutorial for anyone interested in adding this to their projects. The key feature here is this tiny bit of javascript:

Code: [Select]

var source = new EventSource("/link");Normally the client (web browser) sends a request to a server, the server sends data back, then a connection is closed. This feature of HTML5 creates a connection that is going to stay open, and even if something closes it the default is to retry the connection after 3 seconds.

While that connection is open the browser is going to listen for events from the server, which we can use to execute a function:

Code: [Select]

source.addEventListener("event_name", function(e) {
// do something, in my case we update the webpage with the data
document.getElementById("pin9").innerHTML = e.data;
}, false);

That's really it for the client side. Once the page loads it will start listening for data sent from the server, and like I said if something goes wrong it will close the connection, wait 3 seconds, then try again.

The real challenge is dealing with the server side of things and by that I mean our little Digispark Pro. Normally this stuff is handled by a server specific scripting language like php that does a lot of the wrapping and packaging. The browser is going to send a GET request to the "link" you provided. If you are using the wifiserver example that will show up in the path variable. In my example the EventSource asks for "/stream" which is handled by a simple if statement.

Code: [Select]

if(path == F("/stream")){
// this section will process and send the stream data
}

Now for the first of two major challenges, the server needs to respond with a very specific header that should look something like:

Code: [Select]

    Serial.println(F("  HTTP/1.1 200 OK  "));
    Serial.println(F("  Content-Type: text/event-stream  "));
    Serial.println(F("  Cache-Control: no-cache  "));
    Serial.println(F("  Connection: keep-alive  "));

The second line is crucial, and what tells the browser to stay connected and listen for data. I should point out that I don't really know what I'm doing, but this chunk seemed to work. If we were using PHP most of that would happen for us.

The next part is a massive pain in the ass. The browser is waiting to hear a VERY specific message, sent in a very specific way:

Code: [Select]

event: name_of_event \n
data: data_to_send \n\n

The colons, spacing, and line returns all have to line up precisely or the browser will reject everything. For our purposes I set up a function that will stitch it all together and send it. I then followed it all with the standard termination "0" and blank line but I don't know if that's needed.

Code: [Select]

void sendStream(String event, String text) {
    //sends the data of the stream response
  String msg = "event: " + event + "\ndata: " + text + "\n\n";
  Serial.print(msg);                   
}

So the browser sends a request, the spark then sends the special text/event-stream header, and now the browser is listening for data. If you write "event: pin5 \ndata: Pin 5 is 12 \n\n" the browser will trigger the function associated with "pin5" and pass a variable event.data="Pin 5 is 12". You can then update your browser to display that info.

The browser is still listening and will do so for about 2min. If you don't send anything else it will close the connection, wait 3 seconds, and then launch "/stream" again. What you'll want to do is set up a loop in your sketch that sends as much data as fast as you want. In my case I have it looping 60 times, with a 1 sec delay. My next goal will be to make the loop continuous and have something trigger it to break. Might be a physical push button, but more likely it will be a button on the browser that sends a "quit" message.

This is about all I know on the subject. If you read through the references they talk about attaching an id to each message, which is nice if timing is important, or you worry about missing bits of data and want things to line up. There is also a slightly easier way to do this but limits you to just one event called "message."

References:
http://www.w3schools.com/html/html5_serversentevents.asp
http://www.developerdrive.com/2012/03/pushing-updates-to-the-web-page-with-html5-server-sent-events/
http://www.w3.org/TR/2012/WD-eventsource-20120426/

Digistump Forums

Author Topic: Streaming data over WiFi (Read 6261 times)

digi_guy

Streaming data over WiFi

digi_guy

Re: Streaming data over WiFi